Nicolas Ballas

Recurrent Batch Normalization

2016 · arXiv (Cornell University)

We propose a reparameterization of LSTM that brings the benefits of batch normalization to recurrent neural networks. Whereas previous works only apply batch normalization to the input-to-hidden transformation of RNNs, we demonstrate that it is …

Needles in Haystacks: On Classifying Tiny Objects in Large Images

2019 · arXiv (Cornell University)

In some important computer vision domains, such as medical or hyperspectral imaging, we care about the classification of tiny objects in large images. However, most Convolutional Neural Networks (CNNs) for image classification were developed using …

A Closer Look at Codistillation for Distributed Training

2020 · arXiv (Cornell University)

Codistillation has been proposed as a mechanism to share knowledge among concurrently trained models by encouraging them to represent the same function through an auxiliary loss. This contrasts with the more commonly used fully-synchronous data-parallel …

Zoneout: Regularizing RNNs by Randomly Preserving Hidden Activations

2016 · PolyPublie (École Polytechnique de Montréal)

We propose zoneout, a novel method for regularizing RNNs. At each timestep, zoneout stochastically forces some hidden units to maintain their previous values. Like dropout, zoneout uses random noise to train a pseudo-ensemble, improving generalization. …