Serve large language models with less GPU memory.
What it does
Deploys and tunes LLM inference services and batch generation with vLLM.
When to use it
Use when deploying production LLM APIs, improving inference performance, or fitting large models into limited GPU memory.
How to use it
Give it deployment requirements; it configures, tests, and deploys a vLLM inference service.
Uses
Latest review
What you need · 6
Uses Python to run inference code and query the local server.
Declares vllm as a dependency and supplies an installation command.
Access · 4
This skill
HuggingFace
Read
Finds and downloads quantized models
vllm
Execute
Installs and starts vLLM
Read
Reads local server metrics