Train smaller language models with larger teachers.
What it does
Guides training smaller language models to imitate larger teachers using distillation losses and teacher-generated responses.
When to use it
Use when compressing language models, transferring teacher capabilities, reducing inference costs, or specializing smaller models.
How to use it
Give it a language-model distillation task; it provides training code that trains and saves a smaller student model.
Latest review
What you need · 2
The training examples run as Python code.
The basic quick start places the student model on the CUDA device cuda:0.
Access · 8
This skill
meta-llama/Llama-2-70b-hf
Read
Loads the teacher model
meta-llama/Llama-2-7b-hf
Read
Loads the student model