Alpha-Criticality: A Thermodynamic Framework for Neural Information Processing

Neural networks exhibit a previously undiscovered thermodynamic phase transition at the critical boundary where the activation function parameter α = 0. Through rigorous empirical analysis and information-theoretic measurements, we demonstrate that this critical boundary separates two distinct processing regimes with dramatically different properties. The negative α regime (α < 0) favors energy-efficient, ordered information processing, while the positive α regime (α > 0) exhibits higher information preservation but at greater energy cost. Most significantly, the critical point itself (α = 0) displays unique information amplification properties not present in either regime. This paper presents mathematical characterization of this phenomenon, identifies entropy change as the order parameter governing the transition, and establishes different critical exponents for each regime, confirming their classification as distinct universality classes. These findings provide a thermodynamic foundation for neural computation that connects to Landauer's principle and offers practical implications for designing energy-efficient neural architectures.