Distributed Gradient Methods for Nonconvex Optimization: Local and\n Global Convergence Guarantees

The article discusses distributed gradient-descent algorithms for computing\nlocal and global minima in nonconvex optimization. For local optimization, we\nfocus on distributed stochastic gradient descent (D-SGD)--a simple\nnetwork-based variant of classical SGD. We discuss local minima convergence\nguarantees and explore the simple but critical role of the stable-manifold\ntheorem in analyzing saddle-point avoidance. For global optimization, we\ndiscuss annealing-based methods in which slowly decaying noise is added to\nD-SGD. Conditions are discussed under which convergence to global minima is\nguaranteed. Numerical examples illustrate the key concepts in the paper.\n