evaluating-llms-harness
orchestra-research912 installs
Compare language models on academic benchmarks.
What it does
Produces standardized benchmark results for language models, including model comparisons and training-progress evaluations.
When to use it
Use for academic benchmarking, model-quality comparisons, training-progress tracking, and reproducible evaluation.
How to use it
Give it a model or checkpoint to evaluate; it runs benchmarks and saves the results as JSON files.
What you provide
- a model or checkpoint
Uses
LE
Good to know
- cannot be undone: OpenAI; Anthropic
Latest review
What you need · 4
Lists lm-eval as a dependency and provides an installation step before evaluation.
Lists transformers as a dependency.
Access · 6
This skill
OpenAI
Write
Sends paid benchmark requests (irreversible)
Anthropic
Write
Sends paid benchmark requests (irreversible)