Joan Bruna

Distributional Associations vs In-Context Reasoning: A Study of Feed-forward and Attention Layers

2024 · arXiv (Cornell University)

Large language models have been successful at tasks involving basic forms of in-context reasoning, such as generating coherent language, as well as storing vast amounts of knowledge. At the core of the Transformer architecture behind …