[2509.23405] Planner Aware Path Learning in Diffusion Language Models Training

Computer Science > Machine Learning arXiv:2509.23405 (cs) [Submitted on 27 Sep 2025 (v1), last revised 4 Mar 2026 (this version, v2)] Title:Planner Aware Path Learning in Diffusion Language Models Training Authors:Fred Zhangzhi Peng, Zachary Bezemek, Jarrid Rector-Brooks, Shuibai Zhang, Anru R. Zhang, Michael Bronstein, Avishek Joey Bose, Alexander Tong View a PDF of the paper titled Planner Aware Path Learning in Diffusion Language Models Training, by Fred Zhangzhi Peng and 7 other authors View PDF Abstract:Diffusion language models have emerged as a powerful alternative to autoregressive models, enabling fast inference through more flexible and parallel generation paths. This flexibility of sampling is unlocked by new engineered sampling strategies, or planners, that select more favorable generation paths by iteratively planning - versus uniformly at random - where to denoise along the sequence. However, by modifying the reverse paths via planning, planners create an irrevocable mismatch between the uniformly random denoising paths assumed during training and planning-based inference. In this paper, we systematically investigate the mismatch of discrete diffusion training and inference under planning and theoretically prove that the standard discrete diffusion training evidence lower bound (ELBO) does not accurately describe a denoiser that uses a non-uniform planner. To address this gap, we derive a new planned evidence lower bound (P-ELBO) that incorporates planner-bas...