huggingface-community-evals
huggingface1.7k installs
Test models locally, starting with a smoke test.
What it does
Evaluates Hugging Face Hub models locally and helps choose evaluation frameworks and inference backends.
When to use it
Use for backend selection, local GPU evaluations, and choosing between vLLM, Transformers, and accelerate.
How to use it
Give it a model ID and evaluation tasks; it runs evaluations and saves results on the local computer.
What you provide
- a Hugging Face model ID
- evaluation tasks
Uses
Latest review
What you need · 9
The evaluation scripts require Python 3.10 or newer.
The recommended local execution path uses uv run and checks that uv is available.
Access · 2
This skill
Hugging Face Hub
Read
Loads models from Hugging Face
Hugging Face model repository code
Execute
Custom model repository code