E Weinan
3 papers in the PaperMetrix corpus
Papers by this author
-
On the emergence of tetrahedral symmetry in the final and penultimate layers of neural network classifiers.
2020 · arXiv (Cornell University)
A recent numerical study observed that neural network classifiers enjoy a large degree of symmetry in the penultimate layer. Namely, if $h(x) = Af(x) +b$ where $A$ is a linear map and $f$ is the …
-
On the emergence of simplex symmetry in the final and penultimate layers of neural network classifiers
2020 · arXiv (Cornell University)
A recent numerical study observed that neural network classifiers enjoy a large degree of symmetry in the penultimate layer. Namely, if $h(x) = Af(x) +b$ where $A$ is a linear map and $f$ is the …
-
GradPower: Powering Gradients for Faster Language Model Pre-Training
2025 · ArXiv.org
We propose GradPower, a lightweight gradient-transformation technique for accelerating language model pre-training. Given a gradient vector $g=(g_i)_i$, GradPower first applies the elementwise sign-power transformation: $φ_p(g)=({\rm sign}(g_i)|g_i|^p)_{i}$ for a fixed $p>0$, and then feeds the transformed …