![[2602.19126] Robust Predictive Uncertainty and Double Descent in Contaminated Bayesian Random Features](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.19126] Robust Predictive Uncertainty and Double Descent in Contaminated Bayesian Random Features
Summary
This paper presents a robust Bayesian approach to random feature regression, addressing prior and likelihood misspecification through Huber-style contamination sets, and establishes bounds for predictive uncertainty.
Why It Matters
Understanding predictive uncertainty in machine learning models is crucial for improving their reliability, especially in real-world applications where data may be contaminated. This research enhances Bayesian inference methods by providing robust solutions to common issues of misspecification, which can lead to better decision-making in uncertain environments.
Key Takeaways
- Introduces a robust Bayesian framework for random feature regression.
- Addresses prior and likelihood misspecification using contamination sets.
- Establishes bounds for predictive uncertainty, enhancing model reliability.
- Demonstrates that predictive uncertainty remains tractable under contamination.
- Implements an Imprecise Highest Density Region for robust uncertainty quantification.
Computer Science > Machine Learning arXiv:2602.19126 (cs) [Submitted on 22 Feb 2026] Title:Robust Predictive Uncertainty and Double Descent in Contaminated Bayesian Random Features Authors:Michele Caprio, Katerina Papagiannouli, Siu Lun Chau, Sayan Mukherjee View a PDF of the paper titled Robust Predictive Uncertainty and Double Descent in Contaminated Bayesian Random Features, by Michele Caprio and Katerina Papagiannouli and Siu Lun Chau and Sayan Mukherjee View PDF HTML (experimental) Abstract:We propose a robust Bayesian formulation of random feature (RF) regression that accounts explicitly for prior and likelihood misspecification via Huber-style contamination sets. Starting from the classical equivalence between ridge-regularized RF training and Bayesian inference with Gaussian priors and likelihoods, we replace the single prior and likelihood with $\epsilon$- and $\eta$-contaminated credal sets, respectively, and perform inference using pessimistic generalized Bayesian updating. We derive explicit and tractable bounds for the resulting lower and upper posterior predictive densities. These bounds show that, when contamination is moderate, prior and likelihood ambiguity effectively acts as a direct contamination of the posterior predictive distribution, yielding uncertainty envelopes around the classical Gaussian predictive. We introduce an Imprecise Highest Density Region (IHDR) for robust predictive uncertainty quantification and show that it admits an efficient oute...
