Samuel S. Schoenholz
3 papers in the PaperMetrix corpus
Papers by this author
-
Mean Field Residual Networks: On the Edge of Chaos
2017 · arXiv (Cornell University)
We study randomly initialized residual networks using mean field theory and the theory of difference equations. Classical feedforward neural networks, such as those with tanh activations, exhibit exponential behavior on the average when propagating inputs …
-
Dynamical Isometry and a Mean Field Theory of RNNs: Gating Enables Signal Propagation in Recurrent Neural Networks
2018 · arXiv (Cornell University)
Recurrent neural networks have gained widespread use in modeling sequence data across various domains. While many successful recurrent architectures employ a notion of gating, the exact mechanism that enables such remarkable performance is not well …
-
Disentangling Trainability and Generalization in Deep Neural Networks
2019 · arXiv (Cornell University)
A longstanding goal in the theory of deep learning is to characterize the conditions under which a given neural network architecture will be trainable, and if so, how well it might generalize to unseen data. …