![[2602.12923] Annealing in variational inference mitigates mode collapse: A theoretical study on Gaussian mixtures](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.12923] Annealing in variational inference mitigates mode collapse: A theoretical study on Gaussian mixtures
Summary
This article presents a theoretical analysis of how annealing strategies can mitigate mode collapse in variational inference, particularly in Gaussian mixtures.
Why It Matters
Mode collapse is a significant issue in variational inference, affecting the ability to capture multimodal distributions. This research provides a mathematical framework and practical guidance for improving inference models, which is crucial for advancements in machine learning applications.
Key Takeaways
- Mode collapse is a central challenge in variational inference.
- The interplay between initial temperature and annealing rate is critical.
- An appropriately designed annealing scheme can prevent mode collapse.
- The findings extend to neural network models and RealNVP normalizing flows.
- This research offers practical guidance for designing effective variational inference strategies.
Statistics > Machine Learning arXiv:2602.12923 (stat) [Submitted on 13 Feb 2026] Title:Annealing in variational inference mitigates mode collapse: A theoretical study on Gaussian mixtures Authors:Luigi Fogliani, Bruno Loureiro, Marylou Gabrié View a PDF of the paper titled Annealing in variational inference mitigates mode collapse: A theoretical study on Gaussian mixtures, by Luigi Fogliani and 2 other authors View PDF HTML (experimental) Abstract:Mode collapse, the failure to capture one or more modes when targetting a multimodal distribution, is a central challenge in modern variational inference. In this work, we provide a mathematical analysis of annealing based strategies for mitigating mode collapse in a tractable setting: learning a Gaussian mixture, where mode collapse is known to arise. Leveraging a low dimensional summary statistics description, we precisely characterize the interplay between the initial temperature and the annealing rate, and derive a sharp formula for the probability of mode collapse. Our analysis shows that an appropriately chosen annealing scheme can robustly prevent mode collapse. Finally, we present numerical evidence that these theoretical tradeoffs qualitatively extend to neural network based models, RealNVP normalizing flows, providing guidance for designing annealing strategies mitigating mode collapse in practical variational inference pipelines. Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG) Cite as: arXiv:2602.12923 [...
