Stephen Roberts

Marginalising over Stationary Kernels with Bayesian Quadrature

2021 · arXiv (Cornell University)

Marginalising over families of Gaussian Process kernels produces flexible model classes with well-calibrated uncertainty estimates. Existing approaches require likelihood evaluations of many kernels, rendering them prohibitively expensive for larger datasets. We propose a Bayesian Quadrature …

Learning Rates as a Function of Batch Size: A Random Matrix Theory\n Approach to Neural Network Training

2020 · arXiv (Cornell University)

We study the effect of mini-batching on the loss landscape of deep neural\nnetworks using spiked, field-dependent random matrix theory. We demonstrate\nthat the magnitude of the extremal values of the batch Hessian are larger than\nthose of …