[2603.03295] Language Model Goal Selection Differs from Humans' in an Open-Ended Task

Computer Science > Computation and Language arXiv:2603.03295 (cs) [Submitted on 6 Feb 2026] Title:Language Model Goal Selection Differs from Humans' in an Open-Ended Task Authors:Gaia Molinaro, Dave August, Danielle Perszyk, Anne G. E. Collins View a PDF of the paper titled Language Model Goal Selection Differs from Humans' in an Open-Ended Task, by Gaia Molinaro and 3 other authors View PDF Abstract:As large language models (LLMs) get integrated into human decision-making, they are increasingly choosing goals autonomously rather than only completing human-defined ones, assuming they will reflect human preferences. However, human-LLM similarity in goal selection remains largely untested. We directly assess the validity of LLMs as proxies for human goal selection in a controlled, open-ended learning task borrowed from cognitive science. Across four state-of-the-art models (GPT-5, Gemini 2.5 Pro, Claude Sonnet 4.5, and Centaur), we find substantial divergence from human behavior. While people gradually explore and learn to achieve goals with diversity across individuals, most models exploit a single identified solution (reward hacking) or show surprisingly low performance, with distinct patterns across models and little variability across instances of the same model. Even Centaur, explicitly trained to emulate humans in experimental settings, poorly captures people's goal selection. Chain-of-thought reasoning and persona steering provide limited improvements. These findings ...