Yaodong Yu

Learning One-hidden-layer ReLU Networks via Gradient Descent

2018 · arXiv (Cornell University)

We study the problem of learning one-hidden-layer neural networks with Rectified Linear Unit (ReLU) activation function, where the inputs are sampled from standard Gaussian distribution and the outputs are generated from a noisy teacher network. …

Learning Diverse and Discriminative Representations via the Principle of Maximal Coding Rate Reduction

2020 · arXiv (Cornell University)

To learn intrinsic low-dimensional structures from high-dimensional data that most discriminate between classes, we propose the principle of Maximal Coding Rate Reduction ($\text{MCR}^2$), an information-theoretic measure that maximizes the coding rate difference between the whole …

Robust Calibration with Multi-domain Temperature Scaling

2022 · arXiv (Cornell University)

Uncertainty quantification is essential for the reliable deployment of machine learning models to high-stakes application domains. Uncertainty quantification is all the more challenging when training distribution and test distribution are different, even the distribution shifts …

Token Statistics Transformer: Linear-Time Attention via Variational Rate Reduction

2024 · arXiv (Cornell University)

The attention operator is arguably the key distinguishing factor of transformer architectures, which have demonstrated state-of-the-art performance on a variety of tasks. However, transformer attention operators often impose a significant computational burden, with the computational …