[2602.11590] Learn from Your Mistakes: Self-Correcting Masked Diffusion Models

Computer Science > Machine Learning arXiv:2602.11590 (cs) [Submitted on 12 Feb 2026 (v1), last revised 5 Mar 2026 (this version, v2)] Title:Learn from Your Mistakes: Self-Correcting Masked Diffusion Models Authors:Yair Schiff, Omer Belhasin, Roy Uziel, Guanghan Wang, Marianne Arriola, Gilad Turok, Michael Elad, Volodymyr Kuleshov View a PDF of the paper titled Learn from Your Mistakes: Self-Correcting Masked Diffusion Models, by Yair Schiff and 7 other authors View PDF HTML (experimental) Abstract:Masked diffusion models (MDMs) have emerged as a promising alternative to autoregressive models, enabling parallel token generation while achieving competitive performance. Despite these advantages, MDMs face a fundamental limitation: once tokens are unmasked, they remain fixed, leading to error accumulation and ultimately degrading sample quality. We address this by proposing a framework that trains a model to perform both unmasking and correction. By reusing outputs from the MDM denoising network as inputs for corrector training, we train a model to recover from potential mistakes. During generation we apply additional corrective refinement steps between unmasking ones in order to change decoded tokens and improve outputs. We name our training and sampling method Progressive Self-Correction (ProSeCo) for its unique ability to iteratively refine an entire sequence, including already generated tokens. We conduct extensive experimental validation across multiple conditional and un...