![[2602.00628] From Associations to Activations: Comparing Behavioral and Hidden-State Semantic Geometry in LLMs](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.00628] From Associations to Activations: Comparing Behavioral and Hidden-State Semantic Geometry in LLMs
Summary
This paper examines the relationship between behavioral and hidden-state semantic geometry in large language models (LLMs) through psycholinguistic experiments, revealing insights into how behavior can predict internal cognitive states.
Why It Matters
Understanding the connection between behavioral outputs and internal representations in LLMs can enhance our grasp of their cognitive processes and improve model design. This research provides a framework for evaluating LLMs' semantic understanding and their potential applications in NLP.
Key Takeaways
- Behavioral similarity, especially from forced-choice tasks, aligns closely with hidden-state geometry in LLMs.
- The study involved extensive data collection, with over 17.5 million trials across various models.
- Behavior-only measurements can reveal significant information about internal semantic structures.
- Forced-choice paradigms are more effective than free association in predicting hidden-state similarities.
- Insights from this research may inform future psycholinguistic and cognitive modeling studies.
Computer Science > Machine Learning arXiv:2602.00628 (cs) [Submitted on 31 Jan 2026 (v1), last revised 16 Feb 2026 (this version, v2)] Title:From Associations to Activations: Comparing Behavioral and Hidden-State Semantic Geometry in LLMs Authors:Louis Schiekiera, Max Zimmer, Christophe Roux, Sebastian Pokutta, Fritz Günther View a PDF of the paper titled From Associations to Activations: Comparing Behavioral and Hidden-State Semantic Geometry in LLMs, by Louis Schiekiera and 4 other authors View PDF HTML (experimental) Abstract:We investigate the extent to which an LLM's hidden-state geometry can be recovered from its behavior in psycholinguistic experiments. Across eight instruction-tuned transformer models, we run two experimental paradigms -- similarity-based forced choice and free association -- over a shared 5,000-word vocabulary, collecting 17.5M+ trials to build behavior-based similarity matrices. Using representational similarity analysis, we compare behavioral geometries to layerwise hidden-state similarity and benchmark against FastText, BERT, and cross-model consensus. We find that forced-choice behavior aligns substantially more with hidden-state geometry than free association. In a held-out-words regression, behavioral similarity (especially forced choice) predicts unseen hidden-state similarities beyond lexical baselines and cross-model consensus, indicating that behavior-only measurements retain recoverable information about internal semantic geometry. Fina...