Nathan Srebro

Path-Normalized Optimization of Recurrent Neural Networks with ReLU Activations

2016 · arXiv (Cornell University)

We investigate the parameter-space geometry of recurrent neural networks (RNNs), and develop an adaptation of path-SGD optimization method, attuned to this geometry, that can learn plain RNNs with ReLU activations. On several datasets that require …

VC Classes are Adversarially Robustly Learnable, but Only Improperly

2019 · arXiv (Cornell University)

We study the question of learning an adversarially robust predictor. We show that any hypothesis class $\mathcal{H}$ with finite VC dimension is robustly PAC learnable with an improper learning rule. The requirement of being improper …

Computational Complexity of Learning Neural Networks: Smoothness and Degeneracy

2023 · arXiv (Cornell University)

Understanding when neural networks can be learned efficiently is a fundamental question in learning theory. Existing hardness results suggest that assumptions on both the input distribution and the network's weights are necessary for obtaining efficient …

Noisy Interpolation Learning with Shallow Univariate ReLU Networks

2023 · arXiv (Cornell University)

Understanding how overparameterized neural networks generalize despite perfect interpolation of noisy training data is a fundamental question. Mallinar et. al. 2022 noted that neural networks seem to often exhibit ``tempered overfitting'', wherein the population risk …