error-discovery
ai-evals-course943 installs
Find failure patterns in your AI output data.
What it does
Discovers and organizes failure modes in LLM outputs or traces through an interactive, human-reviewed sampling workflow.
When to use it
Use for interactive sessions where a user wants to review a dataset of LLM outputs or traces to discover failure modes.
How to use it
Give it a dataset of LLM outputs or traces; it builds a review app and works with you to organize discovered failures.
What you provide
- Your files
Good to know
- starts one helper agent per item, so the count depends on the work
Latest review
What you need · 1
Requires Python to run the review app's HTTP server, which uses the standard library without additional dependencies.
Access · 3
This skill
Python HTTP server
Execute
Starts a background review server
annotation monitor
Execute
Starts background annotation monitoring
localStorage
Write
Saves a browser annotation backup