Joe Benton

Polysemanticity and Capacity in Neural Networks

2022 · arXiv (Cornell University)

Individual neurons in neural networks often represent a mixture of unrelated features. This phenomenon, called polysemanticity, can make interpreting neural networks more difficult and so we aim to understand its causes. We propose doing so …