Behnam Neyshabur
5 papers in the PaperMetrix corpus
Papers by this author
-
Path-Normalized Optimization of Recurrent Neural Networks with ReLU Activations
2016 · arXiv (Cornell University)
We investigate the parameter-space geometry of recurrent neural networks (RNNs), and develop an adaptation of path-SGD optimization method, attuned to this geometry, that can learn plain RNNs with ReLU activations. On several datasets that require …
-
Extreme Memorization via Scale of Initialization
2021 · arXiv (Cornell University)
We construct an experimental setup in which changing the scale of initialization strongly impacts the implicit regularization induced by SGD, interpolating from good generalization performance to completely memorizing the training set while making little progress …
-
Deep Learning Through the Lens of Example Difficulty
2021 · arXiv (Cornell University)
Existing work on understanding deep learning often employs measures that compress all data-dependent information into a few numbers. In this work, we adopt a perspective based on the role of individual examples. We introduce a …
-
Leveraging Unlabeled Data to Predict Out-of-Distribution Performance
2022 · arXiv (Cornell University)
Real-world machine learning deployments are characterized by mismatches between the source (training) and target (test) distributions that may cause performance drops. In this work, we investigate methods for predicting the target domain accuracy using only …
-
Solving Quantitative Reasoning Problems with Language Models
2022 · arXiv (Cornell University)
Language models have achieved remarkable performance on a wide range of tasks that require natural language understanding. Nevertheless, state-of-the-art models have generally struggled with tasks that require quantitative reasoning, such as solving mathematics, science, and …