[2510.00253] DReS: Dual Reconstruction Smoothing for Functional Regularization

Computer Science > Machine Learning arXiv:2510.00253 (cs) [Submitted on 30 Sep 2025 (v1), last revised 8 May 2026 (this version, v2)] Title:DReS: Dual Reconstruction Smoothing for Functional Regularization Authors:Parsa Moradi, Tayyebeh Jahaninezhad, Hanzaleh Akbarinodehi, Mohammad Ali Maddah-Ali View a PDF of the paper titled DReS: Dual Reconstruction Smoothing for Functional Regularization, by Parsa Moradi and 3 other authors View PDF HTML (experimental) Abstract:Smoothness is a key inductive bias in machine learning and is closely related to generalization. Existing smoothness-inducing methods typically rely either on explicit gradient regularization, which often incurs substantial computational and memory overhead, or on data-mixing strategies, which are less naturally applicable to unsupervised and self-supervised settings. In this work, we propose $\textit{Dual Reconstruction Smoothing}$ (DReS), a nonparametric regularization framework that induces smoothness through a spline-based auxiliary branch with shared model parameters. The method introduces no additional trainable parameters and can be applied to arbitrary submodules, making it suitable for unsupervised, self-supervised, and supervised regimes. We show theoretically that the discrepancy between the target function and its DReS approximation is controlled by higher-order smoothness quantities of the function, establishing the method as an implicit higher-order smoothness regularizer. Empirically, DReS improve...