Information-Geometric Context Window Governance and the Probabilistic Theory of Long-Context Collapse in Large Language Models

This work develops a unified theoretical framework for long-context degradation in transformer-based large language models, combining information geometry and extreme value theory. The central quantity is the observer entropy S_obs(p_θ, ε) defined via Kullback–Leibler divergence under coarse-graining. The main result (Bridge Theorem) establishes the quadratic scaling law S_obs = ½ ε² v(θ)^⊤ I(θ) v(θ) + O(ε³), showing that information loss is governed at leading order by the Fisher information matrix. On the probabilistic side, attention collapse is analysed using extreme value theory for weakly dependent logit maxima. This yields a closed-form probabilistic risk law and leads to a Fundamental Impossibility Theorem: for any finite signal strength, observer entropy vanishes in the long-context limit, implying that full information retention is impossible under softmax attention. These results are connected through bounds of the form S_obs(L) ≤ c₂ e^{μ_L}/L, where μ_L = μ_s − σ√(2 log L_eff), providing a unified information-theoretic characterization of long-context collapse. A control-theoretic response is formulated via the CPL 4.0 phase-aware governor, which enforces a hard context cap, guarantees entropy contraction, and achieves sub-linear fragmentation bounds. The paper includes formal statements, proofs (complete or conditional where explicitly stated), numerical verification, and a proposed experimental protocol for validation on real LLM systems. Note: This Zenodo version omits non-scientific funding information present in the public DPID release. The scientific content is identical. v4.2: Added the niah_experiment.py script code and its verification console output log.