model-pruning
orchestra-research888 installs
Compress language models without retraining.
What it does
Guides the creation of pruned language models using one-shot and structured pruning methods.
When to use it
Use when reducing model size, compressing without retraining, or accelerating inference with hardware-friendly sparsity.
How to use it
Give it a model to prune; it provides a pruning pipeline that saves the pruned model and tokenizer in the project.
What you provide
- the model to prune
Uses
LL
Latest review
What you need · 2
The practical pruning procedure runs a Python script; no Python version is specified.
The Wanda quick-start requires a CUDA-capable GPU because it loads the model onto CUDA.
Access · 8
This skill
meta-llama/Llama-2-7b-hf
Read
Downloads model weights and tokenizer
wikitext
Read
Loads WikiText calibration data