Curvature as a Precursor Signal: Geometric Instability in LLM Reasoning Dynamics

Title Curvature as a Precursor Signal: Geometric Instability in LLM Reasoning Dynamics Abstract This paper introduces a novel paradigm for detecting hallucinations in Large Language Models (LLMs) by reframing the reasoning process as a geometric trajectory of hidden states. While existing methods rely on post-hoc output uncertainty (LogProb), we identify Discrete Trajectory Curvature ($\kappa_{t}$) as a robust precursor signal that manifests before a model confirms a factual error. Our research demonstrates that hallucination is not merely a probabilistic event but a dynamical system failure characterized by geometric instability. Key Findings The "Close but Sharply Bending" Pattern: Prior to a hallucination, hidden-state trajectories exhibit a unique signature of short displacement coupled with abrupt directional change. Precursor Crossover at $t=-3$: In experiments with Llama-3-8B, we measured for the first time a crossover effect where $\kappa_{t}$ achieves superior discriminative performance ($AUC=0.772$) over LogProb ($AUC=0.755$) three tokens before the failure is finalized. The Smoothing Effect: We identified that the initial "wobble" of a hallucination occurs in shallow layers (e.g., Layer 7 in Llama-3) and is subsequently "smoothed" into a geometrically plausible lie by deeper self-attention mechanisms. Cross-Model Universality: Validation across Llama-3-8B, Mistral-7B, and Qwen2-7B confirms the universal advantage of second-order differential (curvature) over first-order drift ($\Delta AUC > +0.10, p < 10^{-10}$). Methodological Highlights Teacher Forcing Extraction: Utilized a controlled comparison protocol to isolate geometric properties from sampling randomness. Scale-Invariant Curvature: Defined a normalized curvature metric $\kappa_{t}$ that is robust against the dramatic activation scale variations across different layers. Multi-Model Analysis: Established three distinct forms of smoothing dynamics: Early Concealment (Llama), Balanced Management (Qwen), and Late Manifestation (Mistral). Practical Implications $\kappa_{t}$ can be computed in real-time via forward hooks without modifying model weights. This enables the development of monitoring systems capable of early detection and proactive intervention—such as prompt reconstruction or early stopping—during the reasoning process itself.