Zhize Li

MARINA: Faster Non-Convex Distributed Learning with Compression

2021 · arXiv (Cornell University)

We develop and analyze MARINA: a new communication efficient method for non-convex distributed learning over heterogeneous datasets. MARINA employs a novel communication compression strategy based on the compression of gradient differences that is reminiscent of …