Cyril Letrouit

Token Sample Complexity of Attention

2025 · arXiv (Cornell University)

As context windows in large language models continue to expand, it is essential to characterize how attention behaves at extreme sequence lengths. We introduce token sample complexity: the rate at which attention computed on $n$ …