Size parallelism and recompute for long-context MoE training.
Guides long-context MoE training configuration to balance memory use, throughput, and parallelism in Megatron Bridge.
When to use it
Use when training MoE models at long sequence lengths or investigating commits that caused long-context OOM or reduced throughput.
Ask for long-context MoE training guidance; it returns configuration instructions and analysis in Markdown with inline code blocks.
No setup requirements listed.
No additional actions listed.