Shaojie Bai

An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling

2018 · arXiv (Cornell University)

For most deep learning practitioners, sequence modeling is synonymous with recurrent networks. Yet recent results indicate that convolutional architectures can outperform recurrent networks on tasks such as audio synthesis and machine translation. Given a new …

Deep Equilibrium Models

2019 · arXiv (Cornell University)

We present a new approach to modeling sequential data: the deep equilibrium model (DEQ). Motivated by an observation that the hidden layers of many existing deep sequence models converge towards some fixed point, we propose …