Configure hierarchical parallelism and check it is active.
What it does
Provides configuration settings, constraints, and verification checks for enabling hierarchical context parallelism in Megatron-Bridge.
When to use it
Use when scaling context parallelism beyond KV heads or investigating CP configuration changes that caused OOM or regressions.
How to use it
Give it a Megatron-Bridge project to configure or diagnose; it returns configuration instructions, shell commands, and analysis.
What you provide
- An existing project
Latest review
What you need · 3
Requires Transformer Engine version 1.12.0 or later for a2a+p2p attention with hierarchical groups.
Python runs the instructed decentralized-PG unit test.
The instructed unit-test verification uses pytest.
Access · 0
No additional actions listed.