![[2602.15925] Robust Stochastic Gradient Posterior Sampling with Lattice Based Discretisation](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.15925] Robust Stochastic Gradient Posterior Sampling with Lattice Based Discretisation
Summary
The paper presents a novel method, Stochastic Gradient Lattice Random Walk (SGLRW), to enhance Bayesian posterior sampling by improving robustness against minibatch size and gradient noise.
Why It Matters
This research addresses critical challenges in scalable Bayesian methods, particularly the sensitivity of stochastic-gradient MCMC techniques to noise and minibatch size. By proposing SGLRW, the authors contribute to more stable and efficient Bayesian inference, which is essential for advancing machine learning applications.
Key Takeaways
- SGLRW improves robustness to minibatch size and gradient noise.
- The method retains asymptotic correctness while enhancing stability.
- Experimental results show SGLRW outperforms SGLD in challenging scenarios.
Statistics > Machine Learning arXiv:2602.15925 (stat) [Submitted on 17 Feb 2026] Title:Robust Stochastic Gradient Posterior Sampling with Lattice Based Discretisation Authors:Zier Mensch, Lars Holdijk, Samuel Duffield, Maxwell Aifer, Patrick J. Coles, Max Welling, Miranda C. N. Cheng View a PDF of the paper titled Robust Stochastic Gradient Posterior Sampling with Lattice Based Discretisation, by Zier Mensch and 6 other authors View PDF HTML (experimental) Abstract:Stochastic-gradient MCMC methods enable scalable Bayesian posterior sampling but often suffer from sensitivity to minibatch size and gradient noise. To address this, we propose Stochastic Gradient Lattice Random Walk (SGLRW), an extension of the Lattice Random Walk discretization. Unlike conventional Stochastic Gradient Langevin Dynamics (SGLD), SGLRW introduces stochastic noise only through the off-diagonal elements of the update covariance; this yields greater robustness to minibatch size while retaining asymptotic correctness. Furthermore, as comparison we analyze a natural analogue of SGLD utilizing gradient clipping. Experimental validation on Bayesian regression and classification demonstrates that SGLRW remains stable in regimes where SGLD fails, including in the presence of heavy-tailed gradient noise, and matches or improves predictive performance. Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG) Cite as: arXiv:2602.15925 [stat.ML] (or arXiv:2602.15925v1 [stat.ML] for this version) htt...
