LingFei Dai

A Distributed SGD Algorithm with Global Sketching for Deep Learning Training Acceleration

2021 · arXiv (Cornell University)

Distributed training is an effective way to accelerate the training process of large-scale deep learning models. However, the parameter exchange and synchronization of distributed stochastic gradient descent introduce a large amount of communication overhead. Gradient …