[2510.00430] PromptLoop: Plug-and-Play Prompt Refinement via Latent Feedback for Diffusion Model Alignment

Computer Science > Machine Learning arXiv:2510.00430 (cs) [Submitted on 1 Oct 2025 (v1), last revised 25 Mar 2026 (this version, v2)] Title:PromptLoop: Plug-and-Play Prompt Refinement via Latent Feedback for Diffusion Model Alignment Authors:Suhyeon Lee, Jong Chul Ye View a PDF of the paper titled PromptLoop: Plug-and-Play Prompt Refinement via Latent Feedback for Diffusion Model Alignment, by Suhyeon Lee and 1 other authors View PDF HTML (experimental) Abstract:Despite recent progress, reinforcement learning (RL)-based fine-tuning of diffusion models often struggles with generalization, composability, and robustness against reward hacking. Recent studies have explored prompt refinement as a modular alternative, but most adopt a feed-forward approach that applies a single refined prompt throughout the entire sampling trajectory, thereby failing to fully leverage the sequential nature of reinforcement learning. To address this, we introduce PromptLoop, a plug-and-play RL framework that incorporates latent feedback into step-wise prompt refinement. Rather than modifying diffusion model weights, a multimodal large language model (MLLM) is trained with RL to iteratively update prompts based on intermediate latent states of diffusion models. This design achieves a structural analogy to the Diffusion RL approach, while retaining the flexibility and generality of prompt-based alignment. Extensive experiments across diverse reward functions and diffusion backbones demonstrate that...