![[2602.12680] A Regularization-Sharpness Tradeoff for Linear Interpolators](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.12680] A Regularization-Sharpness Tradeoff for Linear Interpolators
Summary
This paper introduces a regularization-sharpness tradeoff for linear interpolators in overparameterized settings, challenging traditional bias-variance tradeoff notions.
Why It Matters
Understanding the regularization-sharpness tradeoff is crucial for improving model performance in overparameterized machine learning contexts. This research provides a new framework that can help practitioners better select models and regularization techniques, potentially leading to more effective predictive models.
Key Takeaways
- Proposes a new regularization-sharpness tradeoff for linear regression.
- Challenges the traditional bias-variance tradeoff in overparameterized models.
- Introduces a framework that decomposes selection penalties into regularization and sharpness terms.
- Validates the theory with empirical results on real-world datasets.
- Extends prior analyses to include LASSO interpolators with stronger sparsity.
Statistics > Machine Learning arXiv:2602.12680 (stat) [Submitted on 13 Feb 2026] Title:A Regularization-Sharpness Tradeoff for Linear Interpolators Authors:Qingyi Hu, Liam Hodgkinson View a PDF of the paper titled A Regularization-Sharpness Tradeoff for Linear Interpolators, by Qingyi Hu and 1 other authors View PDF HTML (experimental) Abstract:The rule of thumb regarding the relationship between the bias-variance tradeoff and model size plays a key role in classical machine learning, but is now well-known to break down in the overparameterized setting as per the double descent curve. In particular, minimum-norm interpolating estimators can perform well, suggesting the need for new tradeoff in these settings. Accordingly, we propose a regularization-sharpness tradeoff for overparameterized linear regression with an $\ell^p$ penalty. Inspired by the interpolating information criterion, our framework decomposes the selection penalty into a regularization term (quantifying the alignment of the regularizer and the interpolator) and a geometric sharpness term on the interpolating manifold (quantifying the effect of local perturbations), yielding a tradeoff analogous to bias-variance. Building on prior analyses that established this information criterion for ridge regularizers, this work first provides a general expression of the interpolating information criterion for $\ell^p$ regularizers where $p \ge 2$. Subsequently, we extend this to the LASSO interpolator with $\ell^1$ reg...
