langsmith-evaluator
langchain-ai4.6k installs
Build LangSmith evaluators from real agent outputs.
What it does
Builds LangSmith evaluation pipelines with evaluators, run functions, and local or uploaded evaluation runs.
When to use it
Use when building evaluation pipelines for LangSmith.
How to use it
Give it an agent project; it creates evaluators and run functions for local or uploaded evaluation.
What you provide
- An existing project
Uses
Good to know
- cannot be undone: LangSmith evaluator rules; LangSmith Cloud feedback
- acts without asking
Latest review
What you need · 2
Requires access to LangSmith through a saved CLI profile or API key.
An OpenAI account is optional and enables locally executed OpenAI judges.
Access · 6
This skill
LangSmith evaluator rules
Write
Changes LangSmith evaluator rules (irreversible)
LangSmith Cloud feedback
Write
Sends feedback to LangSmith Cloud (irreversible)