Yuxiang Zhao

$ε$-rank and the Staircase Phenomenon: New Insights into Neural Network Training Dynamics

2024 · arXiv (Cornell University)

Understanding the training dynamics of deep neural networks (DNNs), particularly how they evolve low-dimensional features from high-dimensional data, remains a central challenge in deep learning theory. In this work, we introduce the concept of $ε$-rank, …