Stephen Merity

Pointer Sentinel Mixture Models

2016 · arXiv (Cornell University)

Recent neural network sequence models with softmax classifiers have achieved their best language modeling performance only with very large hidden states and large vocabularies. Even then they struggle to predict rare or unseen words even …

An Analysis of Neural Language Modeling at Multiple Scales

2018 · arXiv (Cornell University)

Many of the leading approaches in language modeling introduce novel, complex and specialized architectures. We take existing state-of-the-art word level language models based on LSTMs and QRNNs and extend them to both larger vocabularies as …

Quasi-Recurrent Neural Networks

2016 · arXiv (Cornell University)

Recurrent neural networks are a powerful tool for modeling sequential data, but the dependence of each timestep's computation on the previous timestep's output limits parallelism and makes RNNs unwieldy for very long sequences. We introduce …