Tune expert-parallel overlap with controlled benchmarks.
What it does
Configure and validate MoE expert-parallel overlap to improve training throughput while preserving the baseline loss trajectory.
When to use it
Use when tuning throughput for an MoE model with EP greater than one, meaningful dispatch/combine communication costs, and memory headroom.
How to use it
It changes how the agent configures and benchmarks MoE expert-parallel overlap, testing plain overlap before adding delayed weight-gradient computation.
What you provide
- An existing project
What you need · 2
EP overlap requires PyTorch version 2.6.0 or later.
For optional delayed weight-gradient computation, TE 2.7.0 or later is required with gradient-reduction overlap or gradient-accumulation fusion; the CUDA graph attention-scope combination requires TE 2.12.0 or later, gradient-accumulation fusion, and no attention bias.
Access · 0
No additional actions listed.