[2502.13731] Robust Counterfactual Inference in Markov Decision Processes

Computer Science > Artificial Intelligence arXiv:2502.13731 (cs) [Submitted on 19 Feb 2025 (v1), last revised 3 Mar 2026 (this version, v4)] Title:Robust Counterfactual Inference in Markov Decision Processes Authors:Jessica Lally, Milad Kazemi, Nicola Paoletti View a PDF of the paper titled Robust Counterfactual Inference in Markov Decision Processes, by Jessica Lally and 2 other authors View PDF Abstract:This paper addresses a key limitation in existing counterfactual inference methods for Markov Decision Processes (MDPs). Current approaches assume a specific causal model to make counterfactuals identifiable. However, there are usually many causal models that align with the observational and interventional distributions of an MDP, each yielding different counterfactual distributions, so fixing a particular causal model limits the validity (and usefulness) of counterfactual inference. We propose a novel non-parametric approach that computes tight bounds on counterfactual transition probabilities across all compatible causal models. Unlike previous methods that require solving prohibitively large optimisation problems (with variables that grow exponentially in the size of the MDP), our approach provides closed-form expressions for these bounds, making computation highly efficient and scalable for non-trivial MDPs. Once such an interval counterfactual MDP is constructed, our method identifies robust counterfactual policies that optimise the worst-case reward w.r.t. the uncer...