Dougal Maclaurin

Gradient-based Hyperparameter Optimization through Reversible Learning

2015 · arXiv (Cornell University)

Tuning hyperparameters of learning algorithms is hard because gradients are usually unavailable. We compute exact gradients of cross-validation performance with respect to all hyperparameters by chaining derivatives backwards through the entire training procedure. These gradients …