[2603.22824] Towards The Implicit Bias on Multiclass Separable Data Under Norm Constraints

Computer Science > Machine Learning arXiv:2603.22824 (cs) [Submitted on 24 Mar 2026] Title:Towards The Implicit Bias on Multiclass Separable Data Under Norm Constraints Authors:Shengping Xie, Zekun Wu, Quan Chen, Kaixu Tang View a PDF of the paper titled Towards The Implicit Bias on Multiclass Separable Data Under Norm Constraints, by Shengping Xie and 3 other authors View PDF HTML (experimental) Abstract:Implicit bias induced by gradient-based algorithms is essential to the generalization of overparameterized models, yet its mechanisms can be subtle. This work leverages the Normalized Steepest Descent} (NSD) framework to investigate how optimization geometry shapes solutions on multiclass separable data. We introduce NucGD, a geometry-aware optimizer designed to enforce low rank structures through nuclear norm constraints. Beyond the algorithm itself, we connect NucGD with emerging low-rank projection methods, providing a unified perspective. To enable scalable training, we derive an efficient SVD-free update rule via asynchronous power iteration. Furthermore, we empirically dissect the impact of stochastic optimization dynamics, characterizing how varying levels of gradient noise induced by mini-batch sampling and momentum modulate the convergence toward the expected maximum margin this http URL code is accessible at: this https URL. Subjects: Machine Learning (cs.LG); Optimization and Control (math.OC); Machine Learning (stat.ML) Cite as: arXiv:2603.22824 [cs.LG]   (or ...