AI quality checks tested against human labels.
What it does
Creates evaluators and CI quality checks for AI applications, including LLM judges validated against human labels.
When to use it
Use when an application has a recurring issue that cannot be resolved with a prompt change.
How to use it
Give it an AI application to evaluate; it builds and validates evaluators for batch evaluation, Phoenix experiments, or CI checks.
What you provide
- An existing project
Uses
Latest review
What you need · 2
Requires a Phoenix server.
Required together with base_url for remote or cloud Phoenix access; the files do not specify where the key comes from.
Access · 1
This skill
Phoenix
Write
Adds failure labels to spans