Lightlines
By nvidia

nemo-mbridge-perf-moe-long-context

nvidia1.9k installs

Size parallelism and recompute for long-context MoE training.

What it does

Guides long-context MoE training configuration to balance memory use, throughput, and parallelism in Megatron Bridge.

When to use it

Use when training MoE models at long sequence lengths or investigating commits that caused long-context OOM or reduced throughput.

How to use it

Ask for long-context MoE training guidance; it returns configuration instructions and analysis in Markdown with inline code blocks.

Latest review

What you need · 0

No setup requirements listed.


Access · 0

No additional actions listed.