![[2602.23336] Differentiable Zero-One Loss via Hypersimplex Projections](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.23336] Differentiable Zero-One Loss via Hypersimplex Projections
Summary
This paper presents a novel differentiable approximation to the zero-one loss, enhancing gradient-based optimization in machine learning through a new operator called Soft-Binary-Argmax.
Why It Matters
The introduction of a differentiable zero-one loss is significant as it addresses the limitations of traditional classification metrics in gradient-based optimization. This advancement could lead to improved model performance, especially in scenarios involving large-batch training, thus impacting various applications in machine learning.
Key Takeaways
- The paper introduces a differentiable approximation to the zero-one loss, enhancing optimization.
- The Soft-Binary-Argmax operator improves gradient-based learning for classification tasks.
- Empirical results show significant improvements in generalization with large-batch training.
- The method imposes geometric consistency constraints on output logits.
- This work aligns structured optimization with task-specific objectives in machine learning.
Computer Science > Machine Learning arXiv:2602.23336 (cs) [Submitted on 26 Feb 2026] Title:Differentiable Zero-One Loss via Hypersimplex Projections Authors:Camilo Gomez, Pengyang Wang, Liansheng Tang View a PDF of the paper titled Differentiable Zero-One Loss via Hypersimplex Projections, by Camilo Gomez and 2 other authors View PDF HTML (experimental) Abstract:Recent advances in machine learning have emphasized the integration of structured optimization components into end-to-end differentiable models, enabling richer inductive biases and tighter alignment with task-specific objectives. In this work, we introduce a novel differentiable approximation to the zero-one loss-long considered the gold standard for classification performance, yet incompatible with gradient-based optimization due to its non-differentiability. Our method constructs a smooth, order-preserving projection onto the n,k-dimensional hypersimplex through a constrained optimization framework, leading to a new operator we term Soft-Binary-Argmax. After deriving its mathematical properties, we show how its Jacobian can be efficiently computed and integrated into binary and multiclass learning systems. Empirically, our approach achieves significant improvements in generalization under large-batch training by imposing geometric consistency constraints on the output logits, thereby narrowing the performance gap traditionally observed in large-batch training. Comments: Subjects: Machine Learning (cs.LG); Machin...
