Robin Jia

Masked Language Modeling and the Distributional Hypothesis: Order Word\n Matters Pre-training for Little

2021 · arXiv (Cornell University)

A possible explanation for the impressive performance of masked language\nmodel (MLM) pre-training is that such models have learned to represent the\nsyntactic structures prevalent in classical NLP pipelines. In this paper, we\npropose a different explanation: MLMs …

Benchmarking Long-tail Generalization with Likelihood Splits

2022 · arXiv (Cornell University)

In order to reliably process natural language, NLP systems must generalize to the long tail of rare utterances. We propose a method to create challenging benchmarks that require generalizing to the tail of the distribution …

On the Robustness of Reading Comprehension Models to Entity Renaming

2021 · arXiv (Cornell University)

We study the robustness of machine reading comprehension (MRC) models to entity renaming -- do models make more wrong predictions when the same questions are asked about an entity whose name has been changed? Such …

Adversarial Examples for Evaluating Reading Comprehension Systems

2017 · arXiv (Cornell University)

Standard accuracy metrics indicate that reading comprehension systems are making rapid progress, but the extent to which these systems truly understand language remains unclear. To reward systems with real language understanding abilities, we propose an …

Know What You Don't Know: Unanswerable Questions for SQuAD

2018 · arXiv (Cornell University)

Extractive reading comprehension systems can often locate the correct answer to a question in a context document, but they also tend to make unreliable guesses on questions for which the correct answer is not stated …

Document-Level N-ary Relation Extraction with Multiscale Representation Learning

2019

Robin Jia, Cliff Wong, Hoifung Poon. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 2019.

Data Recombination for Neural Semantic Parsing

2016

Modeling crisp logical regularities is crucial in semantic parsing, making it difficult for neural models with no task-specific prior knowledge to achieve good results. In this paper, we introduce data recombination, a novel framework for …

Certified Robustness to Adversarial Word Substitutions

2019

Robin Jia, Aditi Raghunathan, Kerem Göksel, Percy Liang. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.

Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little

2021 · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing

A possible explanation for the impressive performance of masked language model (MLM) pre-training is that such models have learned to represent the syntactic structures prevalent in classical NLP pipelines. In this paper, we propose a …