![[2602.15848] Can LLMs Assess Personality? Validating Conversational AI for Trait Profiling](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.15848] Can LLMs Assess Personality? Validating Conversational AI for Trait Profiling
Summary
This study evaluates the effectiveness of Large Language Models (LLMs) in assessing personality traits compared to traditional questionnaires, revealing moderate validity in trait profiling.
Why It Matters
As conversational AI becomes more prevalent, understanding its capabilities in psychological assessments is crucial. This research provides insights into the potential of LLMs to offer a dynamic alternative to conventional methods, which could revolutionize psychometrics and enhance user experience in personality assessments.
Key Takeaways
- LLMs show moderate convergent validity with traditional personality assessments.
- Conscientiousness, Openness, and Neuroticism scores align closely between methods.
- Significant differences in Agreeableness and Extraversion indicate the need for trait-specific calibration.
- Participants rated LLM-generated profiles as equally accurate as traditional methods.
- The findings suggest LLMs could transform psychometric evaluations.
Computer Science > Computation and Language arXiv:2602.15848 (cs) [Submitted on 23 Jan 2026] Title:Can LLMs Assess Personality? Validating Conversational AI for Trait Profiling Authors:Andrius Matšenas, Anet Lello, Tõnis Lees, Hans Peep, Kim Lilii Tamm View a PDF of the paper titled Can LLMs Assess Personality? Validating Conversational AI for Trait Profiling, by Andrius Mat\v{s}enas and 4 other authors View PDF HTML (experimental) Abstract:This study validates Large Language Models (LLMs) as a dynamic alternative to questionnaire-based personality assessment. Using a within-subjects experiment (N=33), we compared Big Five personality scores derived from guided LLM conversations against the gold-standard IPIP-50 questionnaire, while also measuring user-perceived accuracy. Results indicate moderate convergent validity (r=0.38-0.58), with Conscientiousness, Openness, and Neuroticism scores statistically equivalent between methods. Agreeableness and Extraversion showed significant differences, suggesting trait-specific calibration is needed. Notably, participants rated LLM-generated profiles as equally accurate as traditional questionnaire results. These findings suggest conversational AI offers a promising new approach to traditional psychometrics. Comments: Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI) ACM classes: I.2.4; I.2.1; J.4 Cite as: arXiv:2602.15848 [cs.CL] (or arXiv:2602.15848v1 [cs.CL] for this version) https://doi.org/10.48550/a...
