![[2602.16813] One-step Language Modeling via Continuous Denoising](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.16813] One-step Language Modeling via Continuous Denoising
Summary
The paper presents a novel flow-based language model (FLM) that utilizes continuous denoising to achieve superior generation quality and speed compared to traditional discrete diffusion models, challenging existing assumptions in generative modeling.
Why It Matters
This research is significant as it addresses the limitations of current language models in generating high-quality outputs quickly. By demonstrating that flow-based methods can outperform discrete diffusion, it opens new avenues for efficient language modeling, which is crucial for applications in natural language processing and AI.
Key Takeaways
- Flow-based language models can outperform discrete diffusion models in both quality and speed.
- The proposed model achieves high-quality generation with fewer steps, challenging existing paradigms.
- A new time reparameterization technique enhances training stability and generation quality.
- The distilled flow map language model (FMLM) shows significant improvements in few-step generation.
- This research paves the way for more efficient and scalable language modeling techniques.
Computer Science > Computation and Language arXiv:2602.16813 (cs) [Submitted on 18 Feb 2026] Title:One-step Language Modeling via Continuous Denoising Authors:Chanhyuk Lee, Jaehoon Yoo, Manan Agarwal, Sheel Shah, Jerry Huang, Aditi Raghunathan, Seunghoon Hong, Nicholas M. Boffi, Jinwoo Kim View a PDF of the paper titled One-step Language Modeling via Continuous Denoising, by Chanhyuk Lee and 8 other authors View PDF HTML (experimental) Abstract:Language models based on discrete diffusion have attracted widespread interest for their potential to provide faster generation than autoregressive models. In practice, however, they exhibit a sharp degradation of sample quality in the few-step regime, failing to realize this promise. Here we show that language models leveraging flow-based continuous denoising can outperform discrete diffusion in both quality and speed. By revisiting the fundamentals of flows over discrete modalities, we build a flow-based language model (FLM) that performs Euclidean denoising over one-hot token encodings. We show that the model can be trained by predicting the clean data via a cross entropy objective, where we introduce a simple time reparameterization that greatly improves training stability and generation quality. By distilling FLM into its associated flow map, we obtain a distilled flow map language model (FMLM) capable of few-step generation. On the LM1B and OWT language datasets, FLM attains generation quality matching state-of-the-art discret...
