[2603.05296] Latent Policy Steering through One-Step Flow Policies

Computer Science > Robotics arXiv:2603.05296 (cs) [Submitted on 5 Mar 2026] Title:Latent Policy Steering through One-Step Flow Policies Authors:Hokyun Im, Andrey Kolobov, Jianlong Fu, Youngwoon Lee View a PDF of the paper titled Latent Policy Steering through One-Step Flow Policies, by Hokyun Im and 3 other authors View PDF HTML (experimental) Abstract:Offline reinforcement learning (RL) allows robots to learn from offline datasets without risky exploration. Yet, offline RL's performance often hinges on a brittle trade-off between (1) return maximization, which can push policies outside the dataset support, and (2) behavioral constraints, which typically require sensitive hyperparameter tuning. Latent steering offers a structural way to stay within the dataset support during RL, but existing offline adaptations commonly approximate action values using latent-space critics learned via indirect distillation, which can lose information and hinder convergence. We propose Latent Policy Steering (LPS), which enables high-fidelity latent policy improvement by backpropagating original-action-space Q-gradients through a differentiable one-step MeanFlow policy to update a latent-action-space actor. By eliminating proxy latent critics, LPS allows an original-action-space critic to guide end-to-end latent-space optimization, while the one-step MeanFlow policy serves as a behavior-constrained generative prior. This decoupling yields a robust method that works out-of-the-box with minima...