Elena Gribovskaya

Mind the Gap: Assessing Temporal Generalization in Neural Language Models

2021 · arXiv (Cornell University)

Our world is open-ended, non-stationary, and constantly evolving; thus what we talk about and how we talk about it change over time. This inherent dynamic nature of language contrasts with the current static language modelling …

Scaling Language Models: Methods, Analysis & Insights from Training Gopher

2021 · arXiv (Cornell University)

Language modelling provides a step towards intelligent communication systems by harnessing large repositories of written human knowledge to better predict and understand the world. In this paper, we present an analysis of Transformer-based language model …