Josef Valvoda

Information-Theoretic Probing for Linguistic Structure

2020

The success of neural networks on a diverse set of NLP tasks has led researchers to question how much these networks actually "know" about natural language. Probes are a natural way of assessing this. When …

The Architectural Bottleneck Principle

2022

In this paper, we seek to measure how much information a component in a neural network could extract from the representations fed into it. Our work stands in contrast to prior probing work, most of …