![[2602.19115] How Do LLMs Encode Scientific Quality? An Empirical Study Using Monosemantic Features from Sparse Autoencoders](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.19115] How Do LLMs Encode Scientific Quality? An Empirical Study Using Monosemantic Features from Sparse Autoencoders
Summary
This paper investigates how large language models (LLMs) encode scientific quality using monosemantic features from sparse autoencoders, revealing key dimensions of research quality.
Why It Matters
Understanding how LLMs assess scientific quality is crucial as their use in research evaluation grows. This study sheds light on the internal mechanisms of LLMs, potentially guiding future improvements in AI-assisted research assessment and generation.
Key Takeaways
- LLMs can evaluate research quality based on specific features.
- Four key feature types related to scientific quality were identified.
- The study provides insights into how LLMs encapsulate complex research concepts.
Computer Science > Computation and Language arXiv:2602.19115 (cs) [Submitted on 22 Feb 2026] Title:How Do LLMs Encode Scientific Quality? An Empirical Study Using Monosemantic Features from Sparse Autoencoders Authors:Michael McCoubrey, Angelo Salatino, Francesco Osborne, Enrico Motta View a PDF of the paper titled How Do LLMs Encode Scientific Quality? An Empirical Study Using Monosemantic Features from Sparse Autoencoders, by Michael McCoubrey and 3 other authors View PDF HTML (experimental) Abstract:In recent years, there has been a growing use of generative AI, and large language models (LLMs) in particular, to support both the assessment and generation of scientific work. Although some studies have shown that LLMs can, to a certain extent, evaluate research according to perceived quality, our understanding of the internal mechanisms that enable this capability remains limited. This paper presents the first study that investigates how LLMs encode the concept of scientific quality through relevant monosemantic features extracted using sparse autoencoders. We derive such features under different experimental settings and assess their ability to serve as predictors across three tasks related to research quality: predicting citation count, journal SJR, and journal h-index. The results indicate that LLMs encode features associated with multiple dimensions of scientific quality. In particular, we identify four recurring types of features that capture key aspects of how resea...