gguf-quantization
orchestra-research924 installs
Smaller language models for local inference.
What it does
Converts and quantizes language models into GGUF files for local inference with llama.cpp.
When to use it
Use when deploying models on consumer hardware, running on Apple Silicon, or needing CPU inference and flexible quantization.
How to use it
Give it language model files; it creates a quantized GGUF model and tests local inference.
What you provide
- Your files
Uses
Latest review
What you need · 1
Requires Python to run the model conversion script; no Python version is specified.
Access · 5
This skill
GitHub
Read
Downloads llama.cpp from GitHub
HuggingFace
Read
Downloads a language model
llama.cpp
Execute
Builds llama.cpp inference tools