Rebecca Marvin

Targeted Syntactic Evaluation of Language Models

2018

We present a dataset for evaluating the grammaticality of the predictions of a language model. We automatically construct a large number of minimally different pairs of English sentences, each consisting of a grammatical and an …