Compare RAG quality and diagnose weak results.
What it does
Evaluates filesystem-based RAG datasets and helps compare retrieval and generation configurations using RAGAS quality results.
When to use it
Use for NVIDIA RAG Blueprint quality benchmarks based on corpus/ and train.json datasets.
How to use it
Give it dataset directories and a RAG server host and port in the repository; it runs quality evaluation and produces JSON results.
What you provide
- RAG server host and port
- An existing project
- Your files
Uses
NR
Latest review
What you need · 6
Requires Python 3.11 or newer.
Requires uv to install the evaluation dependencies and run the evaluator from the repository root.
Access · 5
This skill
ingestor
Write
Uploads corpus documents to the ingestor
RAG server
Write
Sends benchmark queries to RAG