![[2602.18825] Bayesian Lottery Ticket Hypothesis](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.18825] Bayesian Lottery Ticket Hypothesis
Summary
The paper explores the Bayesian Lottery Ticket Hypothesis, demonstrating that sparse subnetworks in Bayesian neural networks can achieve comparable or superior accuracy to dense networks, potentially reducing computational demands.
Why It Matters
This research is significant as it bridges the gap between Bayesian neural networks and the Lottery Ticket Hypothesis, offering insights that could lead to more efficient training algorithms and resource management in machine learning applications, particularly in computer vision.
Key Takeaways
- Bayesian neural networks can utilize sparse subnetworks effectively.
- The Lottery Ticket Hypothesis holds true in Bayesian settings.
- Pruning strategies should prioritize magnitude and standard deviation.
- Winning tickets can achieve matching or surpassing accuracy.
- Mask structure and weight initialization influence model performance.
Computer Science > Machine Learning arXiv:2602.18825 (cs) [Submitted on 21 Feb 2026] Title:Bayesian Lottery Ticket Hypothesis Authors:Nicholas Kuhn, Arvid Weyrauch, Lars Heyen, Achim Streit, Markus Götz, Charlotte Debus View a PDF of the paper titled Bayesian Lottery Ticket Hypothesis, by Nicholas Kuhn and 5 other authors View PDF HTML (experimental) Abstract:Bayesian neural networks (BNNs) are a useful tool for uncertainty quantification, but require substantially more computational resources than conventional neural networks. For non-Bayesian networks, the Lottery Ticket Hypothesis (LTH) posits the existence of sparse subnetworks that can train to the same or even surpassing accuracy as the original dense network. Such sparse networks can lower the demand for computational resources at inference, and during training. The existence of the LTH and corresponding sparse subnetworks in BNNs could motivate the development of sparse training algorithms and provide valuable insights into the underlying training process. Towards this end, we translate the LTH experiments to a Bayesian setting using common computer vision models. We investigate the defining characteristics of Bayesian lottery tickets, and extend our study towards a transplantation method connecting BNNs with deterministic Lottery Tickets. We generally find that the LTH holds in BNNs, and winning tickets of matching and surpassing accuracy are present independent of model size, with degradation at very high sparsit...
