[2603.20210] CRoCoDiL: Continuous and Robust Conditioned Diffusion for Language

Computer Science > Computation and Language arXiv:2603.20210 (cs) [Submitted on 2 Mar 2026] Title:CRoCoDiL: Continuous and Robust Conditioned Diffusion for Language Authors:Roy Uziel, Omer Belhasin, Itay Levi, Akhiad Bercovich, Ran El-Yaniv, Ran Zilberstein, Michael Elad View a PDF of the paper titled CRoCoDiL: Continuous and Robust Conditioned Diffusion for Language, by Roy Uziel and 6 other authors View PDF HTML (experimental) Abstract:Masked Diffusion Models (MDMs) provide an efficient non-causal alternative to autoregressive generation but often struggle with token dependencies and semantic incoherence due to their reliance on discrete marginal distributions. We address these limitations by shifting the diffusion process into a continuous sentence-level semantic space. We propose CRoCoDiL (Continuous and Robust Conditioned Diffusion for Language), a unified fine-tuning approach that jointly trains an encoder-demasker architecture, grounding the MDM demasking in continuous latent representations. This leads to the formation of a novel autoencoder in which decoding is obtained by an MDM algorithm. Relying on the same framework, we introduce two unconditional text synthesis algorithms: Continuous-Then-Discrete (ConThenDisc), a hybrid-diffusion approach that first generates latent representations in continuous space and then decodes these to tokens via an MDM, and Continuous-Within-Discrete (ConWithinDisc), a multi-diffusion strategy that refines latent representations thr...