distributed-llm-pretraining-torchtitan

Guides distributed language-model pretraining with TorchTitan, including parallelism configuration, Float8 training, and checkpoints.

When to use it

Use when pretraining language models from scratch and needing PyTorch-native, composable distributed parallelism.

Give it a model-training project; it configures and launches distributed pretraining with monitoring and checkpoints.