training-llms-megatron
orchestra-research894 installs
Train large language models across NVIDIA GPUs.
What it does
Creates distributed language-model training configurations and guides their launch and performance tuning with Megatron-Core.
When to use it
Use for large-scale training above 10B parameters when GPU efficiency and fine-grained parallelism control matter.
How to use it
Give it a language-model training task in a project; it configures parallelism and hyperparameters, launches training, and guides performance monitoring.
What you provide
- An existing project
Latest review
What you need · 7
Lists megatron-core as a dependency for the training workflows.
Lists torch as a training dependency.
Lists apex as a training dependency.
Lists transformer-engine as a training dependency.
Access · 0
No additional actions listed.