Cyprien de Masson d’Autume
4 papers in the PaperMetrix corpus
Papers by this author
-
Learning and Evaluating General Linguistic Intelligence
2019 · arXiv (Cornell University)
We define general linguistic intelligence as the ability to reuse previously acquired knowledge about a language's lexicon, syntax, semantics, and pragmatic conventions to adapt to new tasks quickly. Using this definition, we analyze state-of-the-art natural …
-
A Mutual Information Maximization Perspective of Language Representation Learning
2020 · arXiv (Cornell University)
We show state-of-the-art word representation learning methods maximize an objective function that is a lower bound on the mutual information between different parts of a word sequence (i.e., a sentence). Our formulation provides an alternative …
-
Mind the Gap: Assessing Temporal Generalization in Neural Language Models
2021 · arXiv (Cornell University)
Our world is open-ended, non-stationary, and constantly evolving; thus what we talk about and how we talk about it change over time. This inherent dynamic nature of language contrasts with the current static language modelling …
-
Scaling Language Models: Methods, Analysis & Insights from Training Gopher
2021 · arXiv (Cornell University)
Language modelling provides a step towards intelligent communication systems by harnessing large repositories of written human knowledge to better predict and understand the world. In this paper, we present an analysis of Transformer-based language model …