evaluating-llms-harness

Item: evaluating-llms-harness
Rating: 4.9
Author: Implexa

Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting…

view source

installs

stars

karma

SkillRank score ↗

4.9/ 10

evaluated by implexa, claude-haiku-4-5 · 2026-06-01

evaluating-llms-harness provides a command-line interface to benchmark language models against 60+ academic tasks like mmlu and gsm8k. lacks guidance on interpreting results, comparing models, or handling common failure modes.

structure

3.0

trigger phrases

4.0

procedure

6.0

edge cases

2.0

documentation

4.0

strengths

SKILL.md

lm-evaluation-harness - LLM Benchmarking

Quick start

lm-evaluation-harness evaluates LLMs across 60+ academic benchmarks using standardized prompts and metrics.

Installation:

pip install lm-eval

Evaluate any HuggingFace model:

lm_eval --model hf \
  --model_args pretrained=meta-llama/Llama-2-7b-hf \
  --tasks mmlu,gsm8k,hellaswag \
  --device cuda:0 \
  --batch_size 8

related skills

semantically similar in the cross-vendor index

skills.sh

65% match

llm-evaluation

Implement comprehensive evaluation strategies for LLM applications using automated metrics, human feedback, and benchmarking. Use when testing LLM performance,…

don't have the plugin yet? install it then click "run inline in claude" again.