[2603.21546] What Do World Models Learn in RL? Probing Latent Representations in Learned Environment Simulators

Computer Science > Machine Learning arXiv:2603.21546 (cs) [Submitted on 23 Mar 2026] Title:What Do World Models Learn in RL? Probing Latent Representations in Learned Environment Simulators Authors:Xinyu Zhang View a PDF of the paper titled What Do World Models Learn in RL? Probing Latent Representations in Learned Environment Simulators, by Xinyu Zhang View PDF HTML (experimental) Abstract:World models learn to simulate environment dynamics from experience, enabling sample-efficient reinforcement learning. But what do these models actually represent internally? We apply interpretability techniques--including linear and nonlinear probing, causal interventions, and attention analysis--to two architecturally distinct world models: IRIS (discrete token transformer) and DIAMOND (continuous diffusion UNet), trained on Atari Breakout and Pong. Using linear probes, we find that both models develop linearly decodable representations of game state variables (object positions, scores), with MLP probes yielding only marginally higher R^2, confirming that these representations are approximately linear. Causal interventions--shifting hidden states along probe-derived directions--produce correlated changes in model predictions, providing evidence that representations are functionally used rather than merely correlated. Analysis of IRIS attention heads reveals spatial specialization: specific heads attend preferentially to tokens overlapping with game objects. Multi-baseline token ablati...