Chris Pal

Sparse Attentive Backtracking: Long-Range Credit Assignment in Recurrent Networks

2017 · arXiv (Cornell University)

A major drawback of backpropagation through time (BPTT) is the difficulty of learning long-term dependencies, coming from having to propagate credit information backwards through every single step of the forward computation. This makes BPTT both …

Towards Deep Conversational Recommendations

2018 · PolyPublie (École Polytechnique de Montréal)

There has been growing interest in using neural networks and deep learning techniques to create dialogue systems. Conversational recommendation is an interesting setting for the scientific exploration of dialogue with natural language as the associated …

Zoneout: Regularizing RNNs by Randomly Preserving Hidden Activations

2016 · PolyPublie (École Polytechnique de Montréal)

We propose zoneout, a novel method for regularizing RNNs. At each timestep, zoneout stochastically forces some hidden units to maintain their previous values. Like dropout, zoneout uses random noise to train a pseudo-ensemble, improving generalization. …