Jianfeng Yao

Towards Quantifying the Hessian Structure of Neural Networks

2025 · arXiv (Cornell University)

Empirical studies reported that the Hessian matrix of neural networks (NNs) exhibits a near-block-diagonal structure, yet its theoretical foundation remains unclear. In this work, we reveal that the reported Hessian structure comes from a mixture …