optimizing-attention-flash
Faster transformer attention with less GPU memory.
What it does
Guides transformer attention optimization to reduce GPU memory use and improve speed while checking accuracy against a baseline.
When to use it
Use for transformer training with sequences over 512 tokens, long-context inference, or GPU memory constraints.
How to use it
It changes how the agent implements transformer attention by enabling Flash Attention and verifying performance and baseline accuracy.
What you provide
- An existing project
What you need · 4
Uses Python to check the installed PyTorch version and run the attention code.
Uses PyTorch for the native integration path; checks for version 2.2 or newer and instructs upgrading older installations.
The hardware requirements specify NVIDIA Ampere+ or AMD MI200+ GPUs; an earlier troubleshooting section also lists Turing T4 as supported.
Access · 1
This skill
torch
Upgrades your PyTorch installation