Find a lower-memory way to serve models on Jetson.
What it does
Recommends a serving runtime and memory-related launch flags for a Jetson workload using a current device audit.
When to use it
Use when a model needs to fit in memory, a server risks running out of memory, or a lower-memory serving stack is wanted.
How to use it
Give it the intended serving workload; it audits the Jetson and returns a recommended runtime and exact launch flags without launching the server.
What you provide
- LLM, VLM, embedding, or RAG workload
Uses
NJ
Latest review
What you need · 3
Requires access to the target Jetson to obtain a current memory audit.
The recommendation script uses Python 3.
Requires the audit skill to capture the device baseline before recommendations and repeat the audit for verification.
Access · 1
This skill
NVIDIA Jetson
Read
Reads Jetson memory state