ai-system-testing
Catch quality and safety regressions in AI features.
What it does
Guides the creation and verification of regression, quality, grounding, and safety tests for AI-powered product features.
When to use it
Use when testing product LLM features, prompt changes, hallucinations, RAG grounding, safety, or agents that read untrusted content.
How to use it
Give it a project with AI features to test; it produces and verifies evaluation suites, attack fixtures, and quality gates.
What you provide
- An existing project
Good to know
- cannot be undone: LLM APIs
What you need · 2
Requires Python 3 to run the bundled injection detector, which uses only the standard library.
Requires Garak for the pre-launch red-team scan; probe names must be checked against the installed version.
Access · 3
This skill
LLM APIs
Sends paid model evaluation requests (irreversible)
URLs in model responses
Checks URLs cited by models