Enabling scalable stochastic gradient-based inference for Gaussian\n processes by employing the Unbiased LInear System SolvEr (ULISSE)

In applications of Gaussian processes where quantification of uncertainty is\nof primary interest, it is necessary to accurately characterize the posterior\ndistribution over covariance parameters. This paper proposes an adaptation of\nthe Stochastic Gradient Langevin Dynamics algorithm to draw samples from the\nposterior distribution over covariance parameters with negligible bias and\nwithout the need to compute the marginal likelihood. In Gaussian process\nregression, this has the enormous advantage that stochastic gradients can be\ncomputed by solving linear systems only. A novel unbiased linear systems solver\nbased on parallelizable covariance matrix-vector products is developed to\naccelerate the unbiased estimation of gradients. The results demonstrate the\npossibility to enable scalable and exact (in a Monte Carlo sense)\nquantification of uncertainty in Gaussian processes without imposing any\nspecial structure on the covariance or reducing the number of input vectors.\n