Run a language model on your AMD Instinct GPU.
What it does
Launches and verifies a vLLM language-model endpoint on AMD Instinct GPUs.
When to use it
Use when asked to serve a language model on AMD Instinct or run vLLM on ROCm; excludes NVIDIA, consumer AMD GPUs, Ryzen AI, NPU, MI250X, and MI100.
How to use it
Give it a request to serve a model on AMD Instinct; it confirms the configuration with you before launching the endpoint.
Uses
Good to know
- cannot be undone: Docker; /proc/sys/kernel/numa_balancing
- acts without asking
What you need · 8
Requires supported AMD Instinct GPU hardware, including MI300X, MI325X, MI350X, or MI355X; excludes consumer AMD GPUs, MI250X, and MI100.
The GPU host must have the ROCm driver installed, with /dev/kfd and /dev/dri present.
Access · 8
This skill
Docker
Removes containers using the port (irreversible)
/proc/sys/kernel/numa_balancing
Disables host NUMA balancing (irreversible)