[2603.20589] Generating from Discrete Distributions Using Diffusions: Insights from Random Constraint Satisfaction Problems

Computer Science > Machine Learning arXiv:2603.20589 (cs) [Submitted on 21 Mar 2026] Title:Generating from Discrete Distributions Using Diffusions: Insights from Random Constraint Satisfaction Problems Authors:Alankrita Bhatt, Mukur Gupta, Germain Kolossov, Andrea Montanari View a PDF of the paper titled Generating from Discrete Distributions Using Diffusions: Insights from Random Constraint Satisfaction Problems, by Alankrita Bhatt and 3 other authors View PDF HTML (experimental) Abstract:Generating data from discrete distributions is important for a number of application domains including text, tabular data, and genomic data. Several groups have recently used random $k$-satisfiability ($k$-SAT) as a synthetic benchmark for new generative techniques. In this paper, we show that fundamental insights from the theory of random constraint satisfaction problems have observable implications (sometime contradicting intuition) on the behavior of generative techniques on such benchmarks. More precisely, we study the problem of generating a uniformly random solution of a given (random) $k$-SAT or $k$-XORSAT formula. Among other findings, we observe that: $(i)$~Continuous diffusions outperform masked discrete diffusions; $(ii)$~Learned diffusions can match the theoretical `ideal' accuracy; $(iii)$~Smart ordering of the variables can significantly improve accuracy, although not following popular heuristics. Comments: Subjects: Machine Learning (cs.LG) Cite as: arXiv:2603.20589 [cs.LG...