Compare language model performance on Jetson.
What it does
Measures model-serving latency and throughput on Jetson devices and returns structured results for comparing deployments and tuning changes.
When to use it
Use to benchmark or compare models on Jetson, quantify a deployment, or check whether tuning improved performance.
How to use it
Give it a runtime and model identifier or local model path; it returns a verified wrapper command or, when execution is requested or the setup is confirmed, summarized benchmark JSON.
What you provide
- the serving runtime
- the model ID or GGUF path
Uses
Good to know
- works out of sight
What you need · 3
Benchmark execution must run on the Jetson device hosting the model runtime.
The wrappers invoke Python 3 to parse and emit benchmark JSON.
Access · 4
This skill
Ollama
Sends benchmark prompts to Ollama
vLLM
VLLM benchmark containers
Sends benchmark requests to vLLM