Qingcan Wang

Global Convergence of Gradient Descent for Deep Linear Residual Networks

2019 · arXiv (Cornell University)

We analyze the global convergence of gradient descent for deep linear residual networks by proposing a new initialization: zero-asymmetric (ZAS) initialization. It is motivated by avoiding stable manifolds of saddle points. We prove that under …