[2509.22459] Universal Inverse Distillation for Matching Models with Real-Data Supervision (No GANs)

Statistics > Machine Learning arXiv:2509.22459 (stat) [Submitted on 26 Sep 2025 (v1), last revised 2 Mar 2026 (this version, v2)] Title:Universal Inverse Distillation for Matching Models with Real-Data Supervision (No GANs) Authors:Nikita Kornilov, David Li, Tikhon Mavrin, Aleksei Leonov, Nikita Gushchin, Evgeny Burnaev, Iaroslav Koshelev, Alexander Korotin View a PDF of the paper titled Universal Inverse Distillation for Matching Models with Real-Data Supervision (No GANs), by Nikita Kornilov and 7 other authors View PDF Abstract:While achieving exceptional generative quality, modern diffusion, flow, and other matching models suffer from slow inference, as they require many steps of iterative generation. Recent distillation methods address this by training efficient one-step generators under the guidance of a pre-trained teacher model. However, these methods are often constrained to only one specific framework, e.g., only to diffusion or only to flow models. Furthermore, these methods are naturally data-free, and to benefit from the usage of real data, it is required to use an additional complex adversarial training with an extra discriminator model. In this paper, we present RealUID, a universal distillation framework for all matching models that seamlessly incorporates real data into the distillation procedure without GANs. Our RealUID approach offers a simple theoretical foundation that covers previous distillation methods for Flow Matching and Diffusion models, and is...