[2509.22981] MDP modeling for multi-stage stochastic programs

Computer Science > Machine Learning arXiv:2509.22981 (cs) [Submitted on 26 Sep 2025 (v1), last revised 7 Apr 2026 (this version, v2)] Title:MDP modeling for multi-stage stochastic programs Authors:David P. Morton, Oscar Dowson, Bernardo K. Pagnoncelli View a PDF of the paper titled MDP modeling for multi-stage stochastic programs, by David P. Morton and 2 other authors View PDF HTML (experimental) Abstract:We study a class of multi-stage stochastic programs, which incorporate modeling features from Markov decision processes (MDPs). This class includes structured MDPs with continuous action and state spaces. We extend policy graphs to include decision-dependent uncertainty for one-step transition probabilities as well as a limited form of statistical learning. We focus on the expressiveness of our modeling approach, illustrating ideas with a series of examples of increasing complexity. As a solution method, we develop new variants of stochastic dual dynamic programming, including approximations to handle non-convexities. Subjects: Machine Learning (cs.LG); Optimization and Control (math.OC) MSC classes: 90C15, 90C40 Cite as: arXiv:2509.22981 [cs.LG]   (or arXiv:2509.22981v2 [cs.LG] for this version)   https://doi.org/10.48550/arXiv.2509.22981 Focus to learn more arXiv-issued DOI via DataCite Submission history From: David Morton [view email] [v1] Fri, 26 Sep 2025 22:25:16 UTC (131 KB) [v2] Tue, 7 Apr 2026 22:30:08 UTC (138 KB) Full-text links: Access Paper: View a PDF of th...