[2509.20721] Scaling Laws are Redundancy Laws

Computer Science > Machine Learning arXiv:2509.20721 (cs) This paper has been withdrawn by Yuda Bi [Submitted on 25 Sep 2025 (v1), last revised 22 Mar 2026 (this version, v2)] Title:Scaling Laws are Redundancy Laws Authors:Yuda Bi, Vince D Calhoun View a PDF of the paper titled Scaling Laws are Redundancy Laws, by Yuda Bi and 1 other authors No PDF available, click to view other formats Abstract:Scaling laws, a defining feature of deep learning, reveal a striking power-law improvement in model performance with increasing dataset and model size. Yet, their mathematical origins, especially the scaling exponent, have remained elusive. In this work, we show that scaling laws can be formally explained as redundancy laws. Using kernel regression, we show that a polynomial tail in the data covariance spectrum yields an excess risk power law with exponent alpha = 2s / (2s + 1/beta), where beta controls the spectral tail and 1/beta measures redundancy. This reveals that the learning curve's slope is not universal but depends on data redundancy, with steeper spectra accelerating returns to scale. We establish the law's universality across boundedly invertible transformations, multi-modal mixtures, finite-width approximations, and Transformer architectures in both linearized (NTK) and feature-learning regimes. This work delivers the first rigorous mathematical explanation of scaling laws as finite-sample redundancy laws, unifying empirical observations with theoretical foundations. C...