Tiago Pimentel

Information-Theoretic Probing for Linguistic Structure

2020

The success of neural networks on a diverse set of NLP tasks has led researchers to question how much these networks actually "know" about natural language. Probes are a natural way of assessing this. When …

Naturalistic Causal Probing for Morpho-Syntax

2023 · Transactions of the Association for Computational Linguistics

Abstract Probing has become a go-to methodology for interpreting and analyzing deep neural models in natural language processing. However, there is still a lack of understanding of the limitations and weaknesses of various types of …

The Architectural Bottleneck Principle

2022

In this paper, we seek to measure how much information a component in a neural network could extract from the representations fed into it. Our work stands in contrast to prior probing work, most of …