langsmith-online-eval-engineering
langchain-aiTurn your traces into verified online evaluators.
AI evaluation
What it does
Creates and verifies one LangSmith online evaluator at a time through trace inspection and user-guided iteration.
When to use it
Use when creating online evaluators for LangSmith rather than Harbor-style evaluations.
How to use it
Give it a LangSmith project name and quality concerns; it proposes options, gets approval, and creates and attaches the selected evaluator.
What you provide
- LangSmith project name
- quality concerns
Uses
Access · 2
This skill
LangSmith
Write
Creates evaluators and run rules
LangSmith run rule
Execute
Enables evaluation of new traces
What you need · 3
Requires access to a LangSmith tracing project whose traces can be inspected.
Uses Python to define and run evaluator functions and API workflows.
Requires httpx to create LangSmith run rules through the REST API.
About this skill
- Visibility
- Public
- Repository
- langchain-ai/langchain-skills
- Created
- Oct 8, 2026
- Updated
- Oct 8, 2026
- Files
- 4