[2603.00824] A Gauge Theory of Superposition: Toward a Sheaf-Theoretic Atlas of Neural Representations

Computer Science > Machine Learning arXiv:2603.00824 (cs) [Submitted on 28 Feb 2026] Title:A Gauge Theory of Superposition: Toward a Sheaf-Theoretic Atlas of Neural Representations Authors:Hossein Javidnia View a PDF of the paper titled A Gauge Theory of Superposition: Toward a Sheaf-Theoretic Atlas of Neural Representations, by Hossein Javidnia View PDF HTML (experimental) Abstract:We develop a discrete gauge-theoretic framework for superposition in large language models (LLMs) that replaces the single-global-dictionary premise with a sheaf-theoretic atlas of local semantic charts. Contexts are clustered into a stratified context complex; each chart carries a local feature space and a local information-geometric metric (Fisher/Gauss--Newton) identifying predictively consequential feature interactions. This yields a Fisher-weighted interference energy and three measurable obstructions to global interpretability: (O1) local jamming (active load exceeds Fisher bandwidth), (O2) proxy shearing (mismatch between geometric transport and a fixed correspondence proxy), and (O3) nontrivial holonomy (path-dependent transport around loops). We prove and instantiate four results on a frozen open LLM (Llama~3.2~3B Instruct) using WikiText-103, a C4-derived English web-text subset, and \texttt{the-stack-smol}. (A) After constructive gauge fixing on a spanning tree, each chord residual equals the holonomy of its fundamental cycle, making holonomy computable and gauge-invariant. (B) Shear...