Nikolay Blagoev

Go With The Flow: Churn-Tolerant Decentralized Training of Large Language Models

2025 · arXiv (Cornell University)

Motivated by the emergence of large language models (LLMs) and the importance of democratizing their training, we propose GWTF, the first crash tolerant practical decentralized training framework for LLMs. Differently from existing distributed and federated …