[2603.01367] DUEL: Exact Likelihood for Masked Diffusion via Deterministic Unmasking

Computer Science > Machine Learning arXiv:2603.01367 (cs) [Submitted on 2 Mar 2026] Title:DUEL: Exact Likelihood for Masked Diffusion via Deterministic Unmasking Authors:Gilad Turok, Chris De Sa, Volodymyr Kuleshov View a PDF of the paper titled DUEL: Exact Likelihood for Masked Diffusion via Deterministic Unmasking, by Gilad Turok and 2 other authors View PDF HTML (experimental) Abstract:Masked diffusion models (MDMs) generate text by iteratively selecting positions to unmask and then predicting tokens at those positions. Yet MDMs lack proper perplexity evaluation: the ELBO is a loose bound on likelihood under the training distribution, not the test-time distribution, while generative perplexity requires a biased external model and ignores diversity. To address this, we introduce the \textsc{DUEL} framework, which formalizes \emph{deterministic} position selection, unifying leading MDM sampling strategies. We prove \textbf{\textsc{DUEL} admits \emph{exact} likelihood computation} via a simple algorithm, evaluated under the same position selection used at test time. This \textbf{gives MDMs proper perplexity for the first time} -- the natural analogue of autoregressive perplexity. With proper perplexity in hand, we revisit key questions about MDMs. \textbf{MDMs are substantially better than previously thought}: the MDM-autoregressive perplexity gap shrinks by up to 32\% on in-domain data and 82\% on zero-shot benchmarks. \textsc{DUEL} enables the first principled comparison...