[2603.04893] Free Lunch for Pass@$k$? Low Cost Diverse Sampling for Diffusion Language Models

Computer Science > Computation and Language arXiv:2603.04893 (cs) [Submitted on 5 Mar 2026] Title:Free Lunch for Pass@$k$? Low Cost Diverse Sampling for Diffusion Language Models Authors:Sean Lamont, Christian Walder, Paul Montague, Amir Dezfouli, Michael Norrish View a PDF of the paper titled Free Lunch for Pass@$k$? Low Cost Diverse Sampling for Diffusion Language Models, by Sean Lamont and 4 other authors View PDF HTML (experimental) Abstract:Diverse outputs in text generation are necessary for effective exploration in complex reasoning tasks, such as code generation and mathematical problem solving. Such Pass@$k$ problems benefit from distinct candidates covering the solution space. However, traditional sampling approaches often waste computational resources on repetitive failure modes. While Diffusion Language Models have emerged as a competitive alternative to the prevailing Autoregressive paradigm, they remain susceptible to this redundancy, with independent samples frequently collapsing into similar modes. To address this, we propose a training free, low cost intervention to enhance generative diversity in Diffusion Language Models. Our approach modifies intermediate samples in a batch sequentially, where each sample is repelled from the feature space of previous samples, actively penalising redundancy. Unlike prior methods that require retraining or beam search, our strategy incurs negligible computational overhead, while ensuring that each sample contributes a un...