[2601.07325] Robust Bayesian Inference via Variational Approximations of Generalized Rho-Posteriors

Statistics > Machine Learning arXiv:2601.07325 (stat) [Submitted on 12 Jan 2026 (v1), last revised 26 Mar 2026 (this version, v2)] Title:Robust Bayesian Inference via Variational Approximations of Generalized Rho-Posteriors Authors:EL Mahdi Khribch, Pierre Alquier View a PDF of the paper titled Robust Bayesian Inference via Variational Approximations of Generalized Rho-Posteriors, by EL Mahdi Khribch and Pierre Alquier View PDF HTML (experimental) Abstract:We introduce the $\widetilde{\rho}$-posterior, a modified version of the $\rho$-posterior, obtained by replacing the supremum over competitor parameters with a softmax aggregation. This modification allows a PAC-Bayesian analysis of the $\widetilde{\rho}$-posterior. This yields finite-sample oracle inequalities with explicit convergence rates that inherit the key robustness properties of the original framework, in particular, graceful degradation under model misspecification and data contamination. Crucially, the PAC-Bayesian oracle inequalities extend to variational approximations of the $\widetilde{\rho}$-posterior, providing theoretical guarantees for tractable inference. Numerical experiments on exponential families, regression, and real-world datasets confirm that the resulting variational procedures achieve robustness competitive with theoretical predictions at computational cost comparable to standard variational Bayes. Comments: Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Statistics Theory (ma...