Lightlines
By wshobson

eval-harness-first

wshobson2.1k installs

Know what improves before you fine-tune.

What it does

Builds a fine-tuning evaluation harness with labeled goldens, failure-specific graders, a drift suite, and a measured base-model baseline.

When to use it

Use when starting fine-tuning, turning traces into an evaluation set, or calibrating a judge against human labels.

How to use it

Give it production traces or a task specification; it builds an evaluation harness in the project's eval/ directory.

Latest review

What you need · 0

No setup requirements listed.


Access · 0

No additional actions listed.