[2510.00177] PrefDisco: Benchmarking Proactive Personalized Reasoning

Computer Science > Computation and Language arXiv:2510.00177 (cs) [Submitted on 30 Sep 2025 (v1), last revised 4 Mar 2026 (this version, v2)] Title:PrefDisco: Benchmarking Proactive Personalized Reasoning Authors:Shuyue Stella Li, Avinandan Bose, Faeze Brahman, Simon Shaolei Du, Pang Wei Koh, Maryam Fazel, Yulia Tsvetkov View a PDF of the paper titled PrefDisco: Benchmarking Proactive Personalized Reasoning, by Shuyue Stella Li and 6 other authors View PDF HTML (experimental) Abstract:Current large language model (LLM) development treats task-solving and preference-alignment as separate challenges, optimizing first for objective correctness, then for alignment to aggregated human preferences. This paradigm fails in human-facing applications where solving a problem correctly is insufficient if the response mismatches the user's needs. This challenge intensifies in just-in-time scenarios where no prior user interaction history exists due to cold-start conditions or privacy constraints. LLMs need to proactively identify what they don't know about the user, strategically elicit preference values through questioning, then adapt their reasoning processes and responses accordingly -- a complicated chain of cognitive processes which we term personalized reasoning. We introduce PrefDisco, an evaluation methodology that transforms static benchmarks into interactive personalization tasks using psychologically-grounded personas with sparse, context-dependent preferences, and define Pr...