Aran Komatsuzaki

Current Limitations of Language Models: What You Need is Retrieval

2020 · arXiv (Cornell University)

We classify and re-examine some of the current approaches to improve the performance-computes trade-off of language models, including (1) non-causal models (such as masked language models), (2) extension of batch length with efficient attention, (3) …