Roger Grosse

Eigenvalue Corrected Noisy Natural Gradient

2018 · arXiv (Cornell University)

Variational Bayesian neural networks combine the flexibility of deep learning with Bayesian uncertainty estimation. However, inference procedures for flexible variational posteriors are computationally expensive. A recently proposed method, noisy natural gradient, is a surprisingly simple …

Functional Variational Bayesian Neural Networks

2019 · arXiv (Cornell University)

Variational Bayesian neural networks (BNNs) perform variational inference over weights, but it is difficult to specify meaningful priors and approximate posteriors in a high-dimensional weight space. We introduce functional variational Bayesian neural networks (fBNNs), which …

Discovering Language Model Behaviors with Model-Written Evaluations

2022 · arXiv (Cornell University)

As language models (LMs) scale, they develop many novel behaviors, good and bad, exacerbating the need to evaluate how they behave. Prior work creates evaluations with crowdwork (which is time-consuming and expensive) or existing data …

Connecting the Dots: LLMs can Infer and Verbalize Latent Structure from Disparate Training Data

2024 · arXiv (Cornell University)

One way to address safety risks from large language models (LLMs) is to censor dangerous knowledge from their training data. While this removes the explicit information, implicit information can remain scattered across various training documents. …