[2510.04573] LaDiR: Latent Diffusion Enhances LLMs for Text Reasoning

Computer Science > Machine Learning arXiv:2510.04573 (cs) [Submitted on 6 Oct 2025 (v1), last revised 3 Mar 2026 (this version, v5)] Title:LaDiR: Latent Diffusion Enhances LLMs for Text Reasoning Authors:Haoqiang Kang, Yizhe Zhang, Nikki Lijing Kuang, Nicklas Majamaki, Navdeep Jaitly, Yi-An Ma, Lianhui Qin View a PDF of the paper titled LaDiR: Latent Diffusion Enhances LLMs for Text Reasoning, by Haoqiang Kang and 6 other authors View PDF HTML (experimental) Abstract:Large Language Models (LLMs) demonstrate their reasoning ability through chain-of-thought (CoT) generation. However, LLM's autoregressive decoding may limit the ability to revisit and refine earlier tokens in a holistic manner, which can also lead to inefficient exploration for diverse solutions. In this paper, we propose LaDiR (Latent Diffusion Reasoner), a novel reasoning framework that unifies the expressiveness of continuous latent representation with the iterative refinement capabilities of latent diffusion models for an existing LLM. We first construct a structured latent reasoning space using a Variational Autoencoder (VAE) that encodes text reasoning steps into blocks of thought tokens, preserving semantic information and interpretability while offering compact but expressive representations. Subsequently, we utilize a latent diffusion model that learns to denoise a block of latent thought tokens with a blockwise bidirectional attention mask, enabling longer horizon and iterative refinement with adapti...