[2603.21872] Manifold-Aware Exploration for Reinforcement Learning in Video Generation

Computer Science > Computer Vision and Pattern Recognition arXiv:2603.21872 (cs) [Submitted on 23 Mar 2026] Title:Manifold-Aware Exploration for Reinforcement Learning in Video Generation Authors:Mingzhe Zheng, Weijie Kong, Yue Wu, Dengyang Jiang, Yue Ma, Xuanhua He, Bin Lin, Kaixiong Gong, Zhao Zhong, Liefeng Bo, Qifeng Chen, Harry Yang View a PDF of the paper titled Manifold-Aware Exploration for Reinforcement Learning in Video Generation, by Mingzhe Zheng and 11 other authors View PDF HTML (experimental) Abstract:Group Relative Policy Optimization (GRPO) methods for video generation like FlowGRPO remain far less reliable than their counterparts for language models and images. This gap arises because video generation has a complex solution space, and the ODE-to-SDE conversion used for exploration can inject excess noise, lowering rollout quality and making reward estimates less reliable, which destabilizes post-training alignment. To address this problem, we view the pre-trained model as defining a valid video data manifold and formulate the core problem as constraining exploration within the vicinity of this manifold, ensuring that rollout quality is preserved and reward estimates remain reliable. We propose SAGE-GRPO (Stable Alignment via Exploration), which applies constraints at both micro and macro levels. At the micro level, we derive a precise manifold-aware SDE with a logarithmic curvature correction and introduce a gradient norm equalizer to stabilize sampling a...