Judge prompts for failures code cannot catch.
Creates a binary LLM judge prompt that evaluates one specific failure mode.
When to use it
Use when a failure requires interpretation and code-based checks cannot evaluate it.
Give it an identified failure mode and human-labeled traces with at least 20 Pass and 20 Fail examples; it returns a binary judge prompt.
What you provide
No setup requirements listed.
No additional actions listed.