Mohammad Rastegari

DeFINE: DEep Factorized INput Token Embeddings for Neural Sequence Modeling

2019 · arXiv (Cornell University)

For sequence models with large vocabularies, a majority of network parameters lie in the input and output layers. In this work, we describe a new method, DeFINE, for learning deep token representations efficiently. Our architecture …