Compare AI responses with repeatable prompt tests.
What it does
Creates and runs prompt evaluations that compare model outputs using custom assertions and LLM grading.
When to use it
Use when setting up prompt tests, evaluation configurations, custom Python assertions, LLM grading, or few-shot examples.
How to use it
Give it a prompt-testing request; it creates evaluation configuration and assertion files and runs model comparisons.
Uses
Good to know
- cannot be undone: Anthropic; OpenAI
What you need · 3
Provides Anthropic authentication through an environment variable; OPENAI_API_KEY is the credential for the separate OpenAI provider.
Provides OpenAI authentication through an environment variable; ANTHROPIC_API_KEY is the credential for the separate Anthropic provider.
Access · 3
This skill
Anthropic
Sends evaluation prompts to Anthropic (irreversible)
OpenAI
Sends evaluation prompts to OpenAI (irreversible)