langsmith-online-eval-engineering
langchain-ai3.2k installs
Turn your traces into verified online evaluators.
What it does
Creates and verifies one LangSmith online evaluator at a time through trace inspection and user-guided iteration.
When to use it
Use when creating online evaluators for LangSmith rather than Harbor-style evaluations.
How to use it
Give it a LangSmith project name and quality concerns; it proposes options, gets approval, and creates and attaches the selected evaluator.
What you provide
- LangSmith project name
- quality concerns
Uses
Latest review
What you need · 3
Requires access to a LangSmith tracing project whose traces can be inspected.
Uses Python to define and run evaluator functions and API workflows.
Access · 2
This skill
LangSmith
Write
Creates evaluators and run rules
LangSmith run rule
Execute
Enables evaluation of new traces