Tests that show whether your agent skills help.
What it does
Creates validated evaluation specs for skills, agents, and workflow packages while guarding against overfitting.
When to use it
Use when creating evaluation specs, adding stimuli, sizing evaluations, repairing fixtures, or reviewing graders and rubrics for overfitting.
How to use it
Give it an existing target, plugin name, and scenario hypothesis in the repository; it creates and validates evaluation specs and fixtures.
What you provide
- a skill or agent name
- the plugin name
- an expected improvement or invariant
- An existing project
Uses
Good to know
- cannot be undone: GitHub
What you need · 3
Requires the dotnet runtime and SDK tooling to run the repository's skill-validator during validation.
Requires Python to run the repository's evaluation-quality checker.
Requires a GitHub account able to submit a pull-request review for the official evaluation run.
Access · 1
This skill
GitHub
Posts a GitHub PR review (irreversible)