Bao Xin Chen

Elastic Bulk Synchronous Parallel Model for Distributed Deep Learning

2020 · arXiv (Cornell University)

The bulk synchronous parallel (BSP) is a celebrated synchronization model for general-purpose parallel computing that has successfully been employed for distributed training of machine learning models. A prevalent shortcoming of the BSP is that it …