Jonathan Binas

Sparse Attentive Backtracking: Long-Range Credit Assignment in Recurrent Networks

2017 · arXiv (Cornell University)

A major drawback of backpropagation through time (BPTT) is the difficulty of learning long-term dependencies, coming from having to propagate credit information backwards through every single step of the forward computation. This makes BPTT both …