Arnaud Doucet

On the Selection of Initialization and Activation Function for Deep Neural Networks

2018 · arXiv (Cornell University)

The weight initialization and the activation function of deep neural networks have a crucial impact on the performance of the training procedure. An inappropriate selection can lead to the loss of information of the input …

Generalisation under gradient descent via deterministic PAC-Bayes

2022 · arXiv (Cornell University)

We establish disintegrated PAC-Bayesian generalisation bounds for models trained with gradient descent methods or continuous gradient flows. Contrary to standard practice in the PAC-Bayesian setting, our result applies to optimisation algorithms that are deterministic, without …