![[2602.13910] Sufficient Conditions for Stability of Minimum-Norm Interpolating Deep ReLU Networks](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.13910] Sufficient Conditions for Stability of Minimum-Norm Interpolating Deep ReLU Networks
Summary
This paper explores the stability of minimum-norm interpolating deep ReLU networks, identifying conditions under which these networks maintain stability despite perturbations in training data.
Why It Matters
Understanding the stability of deep learning models is crucial for ensuring their reliability and generalization capabilities. This research provides insights into how network architecture influences stability, which can guide future model design and training practices in machine learning.
Key Takeaways
- Minimum-norm interpolating networks can achieve stability under specific conditions.
- The presence of a stable sub-network is essential for overall network stability.
- Low-rank weight matrices in subsequent layers contribute to stability.
- The findings challenge assumptions about stability in deep neural networks.
- This research could influence future developments in overparameterized models.
Computer Science > Machine Learning arXiv:2602.13910 (cs) [Submitted on 14 Feb 2026] Title:Sufficient Conditions for Stability of Minimum-Norm Interpolating Deep ReLU Networks Authors:Ouns El Harzli, Yoonsoo Nam, Ilja Kuzborskij, Bernardo Cuenca Grau, Ard A. Louis View a PDF of the paper titled Sufficient Conditions for Stability of Minimum-Norm Interpolating Deep ReLU Networks, by Ouns El Harzli and 4 other authors View PDF Abstract:Algorithmic stability is a classical framework for analyzing the generalization error of learning algorithms. It predicts that an algorithm has small generalization error if it is insensitive to small perturbations in the training set such as the removal or replacement of a training point. While stability has been demonstrated for numerous well-known algorithms, this framework has had limited success in analyses of deep neural networks. In this paper we study the algorithmic stability of deep ReLU homogeneous neural networks that achieve zero training error using parameters with the smallest $L_2$ norm, also known as the minimum-norm interpolation, a phenomenon that can be observed in overparameterized models trained by gradient-based algorithms. We investigate sufficient conditions for such networks to be stable. We find that 1) such networks are stable when they contain a (possibly small) stable sub-network, followed by a layer with a low-rank weight matrix, and 2) such networks are not guaranteed to be stable even when they contain a stable...
