[2603.00374] Conservative Equilibrium Discovery in Offline Game-Theoretic Multiagent Reinforcement Learning

Computer Science > Artificial Intelligence arXiv:2603.00374 (cs) [Submitted on 27 Feb 2026] Title:Conservative Equilibrium Discovery in Offline Game-Theoretic Multiagent Reinforcement Learning Authors:Austin A. Nguyen, Michael P. Wellman View a PDF of the paper titled Conservative Equilibrium Discovery in Offline Game-Theoretic Multiagent Reinforcement Learning, by Austin A. Nguyen and 1 other authors View PDF Abstract:Offline learning of strategies takes data efficiency to its extreme by restricting algorithms to a fixed dataset of state-action trajectories. We consider the problem in a mixed-motive multiagent setting, where the goal is to solve a game under the offline learning constraint. We first frame this problem in terms of selecting among candidate equilibria. Since datasets may inform only a small fraction of game dynamics, it is generally infeasible in offline game-solving to even verify a proposed solution is a true equilibrium. Therefore, we consider the relative probability of low regret (i.e., closeness to equilibrium) across candidates based on the information available. Specifically, we extend Policy Space Response Oracles (PSRO), an online game-solving approach, by quantifying game dynamics uncertainty and modifying the RL objective to skew towards solutions more likely to have low regret in the true game. We further propose a novel meta-strategy solver, tailored for the offline setting, to guide strategy exploration in PSRO. Our incorporation of Conservat...