Vladimir Khomyakov

Information-Geometric Context Window Governance and the Probabilistic Theory of Long-Context Collapse in Large Language Models

2026 · Zenodo (CERN European Organization for Nuclear Research)

This work develops a unified theoretical framework for long-context degradation in transformer-based large language models, combining information geometry and extreme value theory. The central quantity is the observer entropy S_obs(p_θ, ε) defined via Kullback–Leibler divergence …