Get an LLM endpoint running on your EPYC CPU.
What it does
Launches and verifies a hardware-sized LLM endpoint on one AMD EPYC CPU socket using vLLM and zentorch.
When to use it
Use when the requested LLM endpoint should run on an EPYC CPU, even if the host also contains AMD Instinct GPUs.
How to use it
Ask it to serve an LLM on an EPYC host; it obtains confirmation of the launch plan, starts the server, and returns a verified endpoint.
Uses
Good to know
- cannot be undone: vllm-epyc
Latest review
What you need · 5
The validated hardware targets are EPYC 9000-series Genoa, Turin, and Venice; AVX-512 alone does not establish support for other EPYC parts.
The supplied machine configuration specifies Linux.
Access · 7
This skill
vllm-epyc
Write
Removes the existing vllm-epyc container (irreversible)
HuggingFace
Read
Reads model configuration and templates