[2604.05185] Cross-fitted Proximal Learning for Model-Based Reinforcement Learning

Computer Science > Machine Learning arXiv:2604.05185 (cs) [Submitted on 6 Apr 2026] Title:Cross-fitted Proximal Learning for Model-Based Reinforcement Learning Authors:Nishanth Venkatesh, Andreas A. Malikopoulos View a PDF of the paper titled Cross-fitted Proximal Learning for Model-Based Reinforcement Learning, by Nishanth Venkatesh and 1 other authors View PDF HTML (experimental) Abstract:Model-based reinforcement learning is attractive for sequential decision-making because it explicitly estimates reward and transition models and then supports planning through simulated rollouts. In offline settings with hidden confounding, however, models learned directly from observational data may be biased. This challenge is especially pronounced in partially observable systems, where latent factors may jointly affect actions, rewards, and future observations. Recent work has shown that policy evaluation in such confounded partially observable Markov decision processes (POMDPs) can be reduced to estimating reward-emission and observation-transition bridge functions satisfying conditional moment restrictions (CMRs). In this paper, we study the statistical estimation of these bridge functions. We formulate bridge learning as a CMR problem with nuisance objects given by a conditional mean embedding and a conditional density. We then develop a $K$-fold cross-fitted extension of the existing two-stage bridge estimator. The proposed procedure preserves the original bridge-based identifica...