langsmith-evaluator
langchain-aiBuild LangSmith evaluators from real agent outputs.
AI evaluation
What it does
Builds LangSmith evaluation pipelines with evaluators, run functions, and local or uploaded evaluation runs.
When to use it
Use when building evaluation pipelines for LangSmith.
How to use it
Give it an agent project; it creates evaluators and run functions for local or uploaded evaluation.
What you provide
- An existing project
Uses
Access · 6
This skill
LangSmith evaluator rules
Write
Changes LangSmith evaluator rules (irreversible)
LangSmith Cloud feedback
Write
Sends feedback to LangSmith Cloud (irreversible)
LangSmith
Read
Reads your LangSmith data
LangSmith evaluations
Execute
LangSmith evaluation jobs
Good to know
- cannot be undone: LangSmith evaluator rules; LangSmith Cloud feedback
- acts without asking
What you need · 2
Requires access to LangSmith through a saved CLI profile or API key.
An OpenAI account is optional and enables locally executed OpenAI judges.
About this skill
- Visibility
- Public
- Repository
- langchain-ai/langsmith-skills
- Created
- Oct 8, 2026
- Updated
- Oct 8, 2026
- Files
- 1