Deterministic and Auditable Routing for Artificial Intelligence in Regulated Environments

# Deterministic and Auditable Routing for Artificial Intelligence in Regulated Environments **Felippe Barcelos** Independent Researcher felippe.barcelos10@gmail.com *Preprint — submitted to Zenodo. Version 1.0.0 — Protocol P55.0 — 2026-06-03* --- ## Abstract The deployment of Large Language Models (LLMs) in regulated industries — banking, healthcare, and government — demands audit trails that satisfy strict reproducibility requirements under the EU AI Act (2024), the General Data Protection Regulation (GDPR), and sector-specific frameworks such as HIPAA and Basel III/SR 11-7. Existing ML-based routing systems, including RouteLLM and FrugalGPT, achieve significant cost reductions but produce non-reproducible routing decisions whose outputs change with model retraining, rendering them incompatible with formal compliance auditing. We present the **TEIA Cognitive Router**, a compliance-first LLM routing system based on a fixed six-axis semantic entropy formula that produces deterministic routing decisions without neural weights, training data, or external dependencies. The system guarantees the *Write==Read invariant*: identical input text always yields an identical routing decision, an identical canonical JSON representation, and an identical SHA-256 audit seal. Routing decisions are organized in an append-only Merkle time-anchor chain and can be notarized by any RFC 3161-compliant Trusted Timestamp Authority (TSA) for legally binding external proof of existence. Evaluation on a 100-prompt simulation aligned with MT-Bench benchmark categories demonstrates **99.6% quality retention with 16.3% cost reduction** in compliance-safe mode, and **73.8% quality retention with 98.4% cost reduction** in max-savings mode. We provide a formal compliance mapping to EU AI Act Articles 12, 13, and Annex IV; GDPR Article 22; SOC 2 CC7; and HIPAA §164.312(b). The system is released under Apache 2.0 and is available on PyPI as `teia-cognitive-router`. **Keywords:** LLM routing, compliance, determinism, audit trail, EU AI Act, GDPR, SHA-256, Merkle chain, RFC 3161, semantic entropy **arXiv classifications:** cs.CR (Cryptography and Security) · cs.AI (Artificial Intelligence) · cs.LG (Machine Learning) --- ## 1. Introduction The deployment of Large Language Models (LLMs) across regulated industries has accelerated markedly since the public release of instruction-tuned models beginning in 2022. Financial institutions leverage LLMs for risk analysis, contract review, and fraud detection. Healthcare providers deploy them for clinical documentation, drug-interaction queries, and patient communication. Government agencies apply them to policy analysis, benefits adjudication, and citizen services. This acceleration has collided with a fundamental regulatory barrier: the *audit reproducibility problem*. Modern regulatory frameworks — the European Union AI Act (2024), GDPR (2018), HIPAA (1996), and the Federal Reserve's SR 11-7 model risk guidance — require that automated decision-making systems be *auditable*, *reproducible*, and *explainable*. An organization must be able to answer, for any past automated decision: "Why did this happen? Can you prove it happened exactly this way? Has the record been tampered with?" LLM deployments routinely employ multi-tier model architectures to optimize cost: a small, fast local model handles simple tasks; a large, expensive cloud model handles complex tasks. The routing decision — which tier receives each request — is itself an AI-driven automation. Yet this decision is typically made by an opaque ML classifier (RouteLLM [1]), a learned cascade (FrugalGPT [2]), or an informal heuristic with no formal audit trail. When a compliance officer asks for proof of how a specific routing decision was made six months ago, the organization cannot provide a mathematically verifiable answer. This paper makes the following contributions: 1. **TEIA Cognitive Router**: A fixed arithmetic routing formula that produces deterministic, verifiable routing decisions without neural weights, training data, or external calls, with a formal proof of the Write==Read invariant (Section 4). 2. **Three-Layer Cryptographic Audit Architecture**: SHA-256 body seal + append-only Merkle time-anchor chain + RFC 3161 trusted timestamp, with formal tamper-detection proofs (Section 5). 3. **Compliance-Safe Mode**: A conservative threshold (E(t) < 0.20) that achieves 99.6% quality retention with a risk-free 16.3% cost reduction (Section 6). 4. **Policy Engine**: A JSON-specified rule layer that applies hard compliance overrides (HIPAA, GDPR data locality) on top of the entropy verdict, with its own deterministic policy_seal (Section 4.4). 5. **Regulatory Mapping**: A formal analysis of TEIA's compliance properties against EU AI Act, GDPR, SOC 2, and HIPAA (Section 7). The remainder of this paper is structured as follows. Section 2 reviews background and related work. Section 3 analyzes the compliance failures of ML-based routers. Section 4 presents the TEIA system design. Section 5 describes the cryptographic audit architecture. Section 6 reports empirical evaluation. Section 7 provides regulatory analysis. Section 8 concludes. --- ## 2. Background and Related Work ### 2.1 LLM Cost Optimization via Routing The cost of frontier LLM inference is substantial and highly variable by task. GPT-4-class models charge approximately $0.010–0.030 per 1,000 tokens, while locally hosted 7–13B parameter models incur only compute costs of approximately $0.0004–0.001 per 1,000 tokens on commodity hardware. For organizations processing millions of requests daily, routing decisions have direct revenue implications. **RouteLLM** (Ong et al., 2024) [1] frames routing as binary classification: given a prompt, should it route to a "strong" or "weak" model? They train classifiers — including a causal LLM preference model and a matrix factorization model — on preference data from human feedback, achieving 40–50% cost reduction relative to a strong-model baseline on MT-Bench while maintaining quality above the 95th percentile. **FrugalGPT** (Chen et al., 2023) [2] employs a learned cascade: each model in a chain has a learned scoring function and prompts halt when the score exceeds a learned confidence threshold, achieving 98% quality at 40% of cloud cost on representative workloads. Both systems represent significant engineering achievements in LLM cost optimization. However, Section 3 argues that their reliance on learned parameters renders them incompatible with formal compliance auditing. ### 2.2 Merkle Trees and Cryptographic Audit Logs Merkle (1988) [7] introduced hash trees as a structure for efficient and secure verification of content in large datasets. Certificate Transparency (Laurie et al., 2013) demonstrated that append-only Merkle logs can serve as tamper-evident public audit logs for TLS certificates. The TEIA temporal chain applies this principle to LLM routing audit records. ### 2.3 RFC 3161 Trusted Timestamping RFC 3161 (Adams et al., 2001) [5] defines the Internet X.509 Public Key Infrastructure Time Stamp Protocol. A Trusted Timestamp Authority (TSA) cryptographically signs a hash along with a trusted time, producing a TimeStampToken that constitutes legally binding proof of existence under eIDAS Article 41 (qualified timestamps) and analogous national legislation. ### 2.4 Compliance Frameworks The **EU AI Act** (Regulation 2024/1689) establishes requirements for high-risk AI systems, including comprehensive logging (Art. 12), transparency (Art. 13), and technical documentation (Annex IV). **GDPR** Article 22 restricts automated decision-making with legal or similarly significant effects, requiring explainability and the right to human review. **HIPAA** §164.312(b) mandates audit controls for systems handling protected health information. **Basel III/SR 11-7** requires model risk management including validation, documentation, and drift monitoring. --- ## 3. The Compliance Failure of ML-Based Routing ### 3.1 The Reproducibility Requirement Formal compliance auditing requires *reproducibility*: given a historical record of a decision, an independent auditor must verify that the decision was made as claimed. **Definition 1 (Reproducible Routing).** A routing function f is *reproducible* if:- (i) f(P) is a pure function of prompt P alone, with no dependence on mutable external state;- (ii) there exists a deterministic verification function verify(P, R) that returns True if and only if R = f(P); and- (iii) a collision-resistant commitment C(P, R) binds the stored record to the computation. ### 3.2 Why Learned Routers Fail This Requirement **Non-reproducibility via weight drift.** RouteLLM trains a classifier with parameters θ such that the routing decision is d = argmax_c P_θ(c | P). The decision is a function of both P and θ. After routine retraining (required for performance maintenance and concept drift), the updated parameters θ' may produce d'(P) ≠ d(P) for the same input P. An auditor attempting to reproduce a past routing decision from the input alone cannot recover the historical weights, making reproduction impossible in practice. The organization faces a choice between storing immutable historical model weights forever (operationally impractical) or being unable to prove past routing behavior (compliance failure). **Hash non-determinism.** Because the routing function f_θ(P) is parameterized by mutable weights, the SHA-256 hash H(canonical(P || d)) cannot serve as a verifiable commitment to a reproducible computation. The same prompt P may produce different decisions at different times. A stored hash proves only that P produced d at *some* point, not that P *deterministically implies* d. **Training data provenance.** ML-based routers learn from examples of model performance on historical prompts. Under GDPR Article 22 and Article 13, automated decisions must be explainable. A routing model trai