[2603.02622] Implicit Bias in Deep Linear Discriminant Analysis

Computer Science > Machine Learning arXiv:2603.02622 (cs) [Submitted on 3 Mar 2026] Title:Implicit Bias in Deep Linear Discriminant Analysis Authors:Jiawen Li View a PDF of the paper titled Implicit Bias in Deep Linear Discriminant Analysis, by Jiawen Li View PDF HTML (experimental) Abstract:While the Implicit Bias(or Implicit Regularization) of standard loss functions has been studied, the optimization geometry induced by discriminative metric-learning objectives remains largely this http URL the best of our knowledge, this paper presents an initial theoretical analysis of the implicit regularization induced by the Deep LDA,a scale invariant objective designed to minimize intraclass variance and maximize interclass distance. By analyzing the gradient flow of the loss on a L-layer diagonal linear network, we prove that under balanced initialization, the network architecture transforms standard additive gradient updates into multiplicative weight updates, which demonstrates an automatic conservation of the (2/L) quasi-norm. Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML) Cite as: arXiv:2603.02622 [cs.LG]   (or arXiv:2603.02622v1 [cs.LG] for this version)   https://doi.org/10.48550/arXiv.2603.02622 Focus to learn more arXiv-issued DOI via DataCite (pending registration) Submission history From: Jiawen Li [view email] [v1] Tue, 3 Mar 2026 05:49:24 UTC (434 KB) Full-text links: Access Paper: View a PDF of the paper titled Implicit Bias in Deep Linear Discriminan...