[2603.25976] Second-Order, First-Class: A Composable Stack for Curvature-Aware Training

Computer Science > Machine Learning arXiv:2603.25976 (cs) [Submitted on 26 Mar 2026] Title:Second-Order, First-Class: A Composable Stack for Curvature-Aware Training Authors:Mikalai Korbit, Mario Zanon View a PDF of the paper titled Second-Order, First-Class: A Composable Stack for Curvature-Aware Training, by Mikalai Korbit and 1 other authors View PDF HTML (experimental) Abstract:Second-order methods promise improved stability and faster convergence, yet they remain underused due to implementation overhead, tuning brittleness, and the lack of composable APIs. We introduce Somax, a composable Optax-native stack that treats curvature-aware training as a single JIT-compiled step governed by a static plan. Somax exposes first-class modules -- curvature operators, estimators, linear solvers, preconditioners, and damping policies -- behind a single step interface and composes with Optax by applying standard gradient transformations (e.g., momentum, weight decay, schedules) to the computed direction. This design makes typically hidden choices explicit and swappable. Somax separates planning from execution: it derives a static plan (including cadences) from module requirements, then runs the step through a specialized execution path that reuses intermediate results across modules. We report system-oriented ablations showing that (i) composition choices materially affect scaling behavior and time-to-accuracy, and (ii) planning reduces per-step overhead relative to unplanned compo...