Juho Lee

Scale Mixtures of Neural Network Gaussian Processes

2021 · arXiv (Cornell University)

Recent works have revealed that infinitely-wide feed-forward or recurrent neural networks of any architecture correspond to Gaussian processes referred to as Neural Network Gaussian Processes (NNGPs). While these works have extended the class of neural …

Deep neural networks with dependent weights: Gaussian Process mixture limit, heavy tails, sparsity and compressibility

2022 · arXiv (Cornell University)

This article studies the infinite-width limit of deep feedforward neural networks whose weights are dependent, and modelled via a mixture of Gaussian distributions. Each hidden node of the network is assigned a nonnegative random variable …