![[2602.12586] Can I Have Your Order? Monte-Carlo Tree Search for Slot Filling Ordering in Diffusion Language Models](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.12586] Can I Have Your Order? Monte-Carlo Tree Search for Slot Filling Ordering in Diffusion Language Models
Summary
This paper introduces McDiffuSE, a Monte Carlo Tree Search framework aimed at optimizing slot filling orders in Masked Diffusion Models, demonstrating improved performance in mathematical and code reasoning tasks.
Why It Matters
The research addresses the critical issue of output variance in diffusion models, providing a novel approach to enhance generation quality. By optimizing slot infilling order, it contributes to advancements in AI-generated reasoning, which is vital for applications in natural language processing and beyond.
Key Takeaways
- McDiffuSE improves slot filling order optimization using Monte Carlo Tree Search.
- The framework shows an average performance improvement of 3.2% over autoregressive models.
- Non-sequential generation is crucial for maximizing performance in slot filling tasks.
- Larger exploration constants help overcome model confidence biases.
- The findings highlight the effectiveness of MCTS-based planning in enhancing generation quality.
Computer Science > Artificial Intelligence arXiv:2602.12586 (cs) [Submitted on 13 Feb 2026] Title:Can I Have Your Order? Monte-Carlo Tree Search for Slot Filling Ordering in Diffusion Language Models Authors:Joshua Ong Jun Leang, Yu Zhao, Mihaela Cătălina Stoian, Wenda Li, Shay B. Cohen, Eleonora Giunchiglia View a PDF of the paper titled Can I Have Your Order? Monte-Carlo Tree Search for Slot Filling Ordering in Diffusion Language Models, by Joshua Ong Jun Leang and 5 other authors View PDF HTML (experimental) Abstract:While plan-and-infill decoding in Masked Diffusion Models (MDMs) shows promise for mathematical and code reasoning, performance remains highly sensitive to slot infilling order, often yielding substantial output variance. We introduce McDiffuSE, a framework that formulates slot selection as decision making and optimises infilling orders through Monte Carlo Tree Search (MCTS). McDiffuSE uses look-ahead simulations to evaluate partial completions before commitment, systematically exploring the combinatorial space of generation orders. Experiments show an average improvement of 3.2% over autoregressive baselines and 8.0% over baseline plan-and-infill, with notable gains of 19.5% on MBPP and 4.9% on MATH500. Our analysis reveals that while McDiffuSE predominantly follows sequential ordering, incorporating non-sequential generation is essential for maximising performance. We observe that larger exploration constants, rather than increased simulations, are necess...
