Negar Foroutan Eghlidi

Sparse Communication for Training Deep Networks

2020 · arXiv (Cornell University)

Synchronous stochastic gradient descent (SGD) is the most common method used for distributed training of deep learning models. In this algorithm, each worker shares its local gradients with others and updates the parameters using the …