![[2602.15832] What Persona Are We Missing? Identifying Unknown Relevant Personas for Faithful User Simulation](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.15832] What Persona Are We Missing? Identifying Unknown Relevant Personas for Faithful User Simulation
Summary
This article explores the identification of unknown user personas in simulations, introducing the PICQ dataset and evaluating leading LLMs on fidelity and influence in user interactions.
Why It Matters
Understanding user personas is crucial for creating accurate user simulations in AI. This research addresses gaps in current methodologies, enhancing the reliability of simulations and informing future developments in human-computer interaction.
Key Takeaways
- Introduces the PICQ dataset for context-aware user persona identification.
- Highlights a 'Fidelity vs. Insight' dilemma in LLM performance based on model size.
- Establishes a benchmark for evaluating user simulations in AI.
- Explores cognitive differences affecting simulation fidelity.
- Provides insights for improving user interaction models in AI.
Computer Science > Human-Computer Interaction arXiv:2602.15832 (cs) [Submitted on 3 Jan 2026] Title:What Persona Are We Missing? Identifying Unknown Relevant Personas for Faithful User Simulation Authors:Weiwen Su, Yuhan Zhou, Zihan Wang, Naoki Yoshinaga, Masashi Toyoda View a PDF of the paper titled What Persona Are We Missing? Identifying Unknown Relevant Personas for Faithful User Simulation, by Weiwen Su and 4 other authors View PDF HTML (experimental) Abstract:Existing user simulations, where models generate user-like responses in dialogue, often lack verification that sufficient user personas are provided, questioning the validity of the simulations. To address this core concern, this work explores the task of identifying relevant but unknown personas of the simulation target for a given simulation context. We introduce PICQ, a novel dataset of context-aware choice questions, annotated with unknown personas (e.g., ''Is the user price-sensitive?'') that may influence user choices, and propose a multi-faceted evaluation scheme assessing fidelity, influence, and inaccessibility. Our benchmark of leading LLMs reveals a complex ''Fidelity vs. Insight'' dilemma governed by model scale: while influence generally scales with model size, fidelity to human patterns follows an inverted U-shaped curve. We trace this phenomenon to cognitive differences, particularly the human tendency for ''cognitive economy.'' Our work provides the first comprehensive benchmark for this crucial ...
