Thomas R. Harvey

Sven: Singular Value Descent as a Computationally Efficient Natural Gradient Method

2026 · HAL (Le Centre pour la Communication Scientifique Directe)

We introduce Sven (Singular Value dEsceNt), a new optimization algorithm for neural networks that exploits the natural decomposition of loss functions into a sum over individual data points, rather than reducing the full loss to …