Bicheng Ying

Exponential Graph is Provably Efficient for Decentralized Deep Training

2021 · arXiv (Cornell University)

Decentralized SGD is an emerging training method for deep learning known for its much less (thus faster) communication per iteration, which relaxes the averaging step in parallel SGD to inexact averaging. The less exact the …