Benjamin Hoover

exBERT: A Visual Analysis Tool to Explore Learned Representations in Transformer Models

2020

Large Transformer-based language models can route and reshape complex information via their multi-headed attention mechanism. Although the attention never receives explicit supervision, it can exhibit recognizable patterns following linguistic or positional information. Analyzing the learned …