Shauli Ravfogel

Amnesic Probing: Behavioral Explanation with Amnesic Counterfactuals

2020 · arXiv (Cornell University)

A growing body of work makes use of probing to investigate the working of neural models, often considered black boxes. Recently, an ongoing debate emerged surrounding the limitations of the probing paradigm. In this work, …

BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models

2022

We introduce BitFit, a sparse-finetuning method where only the bias-terms of the model (or a subset of them) are being modified. We show that with small-to-medium training data, applying BitFit on pre-trained BERT models is …

Measuring and Improving Consistency in Pretrained Language Models

2021 · Transactions of the Association for Computational Linguistics

Abstract Consistency of a model—that is, the invariance of its behavior under meaning-preserving alternations in its input—is a highly desirable property in natural language processing. In this paper we study the question: Are Pretrained Language …