[2512.22227] Geometric Organization of Cognitive States in Transformer Embedding Spaces

Computer Science > Computation and Language arXiv:2512.22227 (cs) [Submitted on 23 Dec 2025 (v1), last revised 6 Apr 2026 (this version, v2)] Title:Geometric Organization of Cognitive States in Transformer Embedding Spaces Authors:Sophie Zhao View a PDF of the paper titled Geometric Organization of Cognitive States in Transformer Embedding Spaces, by Sophie Zhao View PDF HTML (experimental) Abstract:Recent work has shown that transformer-based language models learn rich geometric structure in their embedding spaces. In this work, we investigate whether sentence embeddings exhibit structured geometric organization aligned with human-interpretable cognitive or psychological attributes. We construct a dataset of 480 natural-language sentences annotated with both continuous energy scores (ranging from -5 to +5) and discrete tier labels spanning seven ordered cognitive annotation tiers, intended to capture a graded progression from highly constricted or reactive expressions toward more coherent and integrative cognitive states. Using fixed sentence embeddings from multiple transformer models, we evaluate the recoverability of these annotations via linear and shallow nonlinear probes. Across models, both continuous energy scores and tier labels are reliably decodable, with linear probes already capturing substantial structure. To assess statistical significance, we conduct nonparametric permutation tests that randomize labels, showing that probe performance exceeds chance under ...