[2603.20216] Locally Coherent Parallel Decoding in Diffusion Language Models

Computer Science > Computation and Language arXiv:2603.20216 (cs) [Submitted on 3 Mar 2026] Title:Locally Coherent Parallel Decoding in Diffusion Language Models Authors:Michael Hersche, Nicolas Menet, Ronan Tanios, Abbas Rahimi View a PDF of the paper titled Locally Coherent Parallel Decoding in Diffusion Language Models, by Michael Hersche and 3 other authors View PDF HTML (experimental) Abstract:Diffusion language models (DLMs) have emerged as a promising alternative to autoregressive (AR) models, offering sub-linear generation latency and bidirectional capabilities that are particularly appealing for code generation and editing. Achieving sub-linear latency in discrete DLMs requires predicting multiple tokens in parallel. However, standard DLMs sample tokens independently from conditional marginal distributions, failing to capture the joint dependencies among concurrently generated tokens. As a result, they often lead to syntactic inconsistencies and break multi-token structures. In this work, we introduce CoDiLA (Coherent Diffusion with Local Autoregression), a method that reconciles parallel sampling with local dependency modeling. Rather than forcing the DLM to resolve fine-grained syntax, CoDiLA delegates local decoding to a small, auxiliary AR model operating on the diffusion latents. This design allows for parallel block generation while ensuring sequential validity within each block and maintaining core DLM capabilities, including bidirectional modeling across b...