[2603.22813] Learning What Matters Now: Dynamic Preference Inference under Contextual Shifts

Computer Science > Artificial Intelligence arXiv:2603.22813 (cs) [Submitted on 24 Mar 2026] Title:Learning What Matters Now: Dynamic Preference Inference under Contextual Shifts Authors:Xianwei Cao, Dou Quan, Zhenliang Zhang, Shuang Wang View a PDF of the paper titled Learning What Matters Now: Dynamic Preference Inference under Contextual Shifts, by Xianwei Cao and 2 other authors View PDF HTML (experimental) Abstract:Humans often juggle multiple, sometimes conflicting objectives and shift their priorities as circumstances change, rather than following a fixed objective function. In contrast, most computational decision-making and multi-objective RL methods assume static preference weights or a known scalar reward. In this work, we study sequential decision-making problem when these preference weights are unobserved latent variables that drift with context. Specifically, we propose Dynamic Preference Inference (DPI), a cognitively inspired framework in which an agent maintains a probabilistic belief over preference weights, updates this belief from recent interaction, and conditions its policy on inferred preferences. We instantiate DPI as a variational preference inference module trained jointly with a preference-conditioned actor-critic, using vector-valued returns as evidence about latent trade-offs. In queueing, maze, and multi-objective continuous-control environments with event-driven changes in objectives, DPI adapts its inferred preferences to new regimes and achi...