Check cost and routing claims against real cases.
What it does
Measures benchmark performance against structural and adversarial cases and saves results that substantiate cost and routing claims.
When to use it
Use before releases, after expanding the corpus, when auditing claimed results, or when comparing model costs.
How to use it
Give it a benchmark request in the project; it runs the corpus benchmark and saves per-case and summary results.
What you provide
- An existing project
Uses
What you need · 6
Requires Node.js to run the benchmark script; no runtime version is specified.
Requires agent-booster to resolve when the benchmark runs from v3/.
For the optional Gemini baseline, the key is automatically retrieved through gcloud secrets; BENCH_LLM_API_KEY is an alternative override.
For the optional Anthropic baseline, the key is automatically retrieved through gcloud secrets; BENCH_ANTHROPIC_API_KEY is an alternative override.
Access · 0
No additional actions listed.