Lie He

RelaySum for Decentralized Deep Learning on Heterogeneous Data

2021 · arXiv (Cornell University)

In decentralized machine learning, workers compute model updates on their local data. Because the workers only communicate with few neighbors without central coordination, these updates propagate progressively over the network. This paradigm enables distributed training …