[2603.20406] Thinking in Different Spaces: Domain-Specific Latent Geometry Survives Cross-Architecture Translation

Computer Science > Machine Learning arXiv:2603.20406 (cs) [Submitted on 20 Mar 2026] Title:Thinking in Different Spaces: Domain-Specific Latent Geometry Survives Cross-Architecture Translation Authors:Marcus Armstrong, Navid Ayoobi, Arjun Mukherjee View a PDF of the paper titled Thinking in Different Spaces: Domain-Specific Latent Geometry Survives Cross-Architecture Translation, by Marcus Armstrong and 2 other authors View PDF HTML (experimental) Abstract:We investigate whether independently trained language models converge to geometrically compatible latent representations, and whether this compatibility can be exploited to correct model behavior at inference time without any weight updates. We learn a linear projection matrix that maps activation vectors from a large teacher model into the coordinate system of a smaller student model, then intervene on the student's residual stream during generation by substituting its internal state with the translated teacher representation. Across a fully crossed experimental matrix of 20 heterogeneous teacher-student pairings spanning mixture-of-experts, dense, code-specialized, and synthetically trained architectures, the Ridge projection consistently achieves R^2 = 0.50 on verbal reasoning and R^2 = 0.40 on mathematical reasoning, collapsing to R^2 = -0.22 under permutation control and R^2 = 0.01 under L_1 regularization. Behavioral correction rates range from 14.0% to 50.0% on TruthfulQA (mean 25.2%) and from 8.5% to 43.3% on GSM...