![[2602.19893] Generalized Random Direction Newton Algorithms for Stochastic Optimization](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.19893] Generalized Random Direction Newton Algorithms for Stochastic Optimization
Summary
This paper introduces generalized Hessian estimators for stochastic optimization using random direction stochastic approximation, demonstrating lower-order bias with increased function measurements and providing convergence analyses.
Why It Matters
The research addresses challenges in stochastic optimization, which is crucial for various machine learning applications. By improving Hessian estimators, the findings could enhance algorithm performance in noisy environments, making them more applicable in real-world scenarios.
Key Takeaways
- Introduces generalized Hessian estimators for stochastic optimization.
- Demonstrates that more function measurements reduce estimation bias.
- Provides asymptotic and non-asymptotic convergence analyses.
- Validates theoretical findings through numerical experiments.
- Enhances understanding of stochastic Newton methods.
Computer Science > Machine Learning arXiv:2602.19893 (cs) [Submitted on 23 Feb 2026] Title:Generalized Random Direction Newton Algorithms for Stochastic Optimization Authors:Soumen Pachal, Prashanth L.A., Shalabh Bhatnagar, Avinash Achar View a PDF of the paper titled Generalized Random Direction Newton Algorithms for Stochastic Optimization, by Soumen Pachal and 3 other authors View PDF HTML (experimental) Abstract:We present a family of generalized Hessian estimators of the objective using random direction stochastic approximation (RDSA) by utilizing only noisy function measurements. The form of each estimator and the order of the bias depend on the number of function measurements. In particular, we demonstrate that estimators with more function measurements exhibit lower-order estimation bias. We show the asymptotic unbiasedness of the estimators. We also perform asymptotic and non-asymptotic convergence analyses for stochastic Newton methods that incorporate our generalized Hessian estimators. Finally, we perform numerical experiments to validate our theoretical findings. Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML) Cite as: arXiv:2602.19893 [cs.LG] (or arXiv:2602.19893v1 [cs.LG] for this version) https://doi.org/10.48550/arXiv.2602.19893 Focus to learn more arXiv-issued DOI via DataCite (pending registration) Submission history From: Soumen Pachal [view email] [v1] Mon, 23 Feb 2026 14:33:39 UTC (35 KB) Full-text links: Access Paper: View a PDF of...