![[2602.17876] Interactive Learning of Single-Index Models via Stochastic Gradient Descent](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.17876] Interactive Learning of Single-Index Models via Stochastic Gradient Descent
Summary
This article discusses the application of Stochastic Gradient Descent (SGD) in learning single-index models, revealing its effectiveness in high-dimensional optimization and adaptive data settings.
Why It Matters
Understanding the dynamics of SGD in learning single-index models is crucial for advancing machine learning techniques, particularly in high-dimensional spaces. This research provides insights that can enhance model performance and efficiency, making it relevant for both academic and practical applications in machine learning.
Key Takeaways
- SGD is effective for learning single-index models in high-dimensional settings.
- The learning process involves a 'burn-in' phase followed by a 'learning' phase.
- An optimal learning rate schedule can improve sample complexity and regret guarantees.
Statistics > Machine Learning arXiv:2602.17876 (stat) [Submitted on 19 Feb 2026] Title:Interactive Learning of Single-Index Models via Stochastic Gradient Descent Authors:Nived Rajaraman, Yanjun Han View a PDF of the paper titled Interactive Learning of Single-Index Models via Stochastic Gradient Descent, by Nived Rajaraman and 1 other authors View PDF HTML (experimental) Abstract:Stochastic gradient descent (SGD) is a cornerstone algorithm for high-dimensional optimization, renowned for its empirical successes. Recent theoretical advances have provided a deep understanding of how SGD enables feature learning in high-dimensional nonlinear models, most notably the \textit{single-index model} with i.i.d. data. In this work, we study the sequential learning problem for single-index models, also known as generalized linear bandits or ridge bandits, where SGD is a simple and natural solution, yet its learning dynamics remain largely unexplored. We show that, similar to the optimal interactive learner, SGD undergoes a distinct ``burn-in'' phase before entering the ``learning'' phase in this setting. Moreover, with an appropriately chosen learning rate schedule, a single SGD procedure simultaneously achieves near-optimal (or best-known) sample complexity and regret guarantees across both phases, for a broad class of link functions. Our results demonstrate that SGD remains highly competitive for learning single-index models under adaptive data. Comments: Subjects: Machine Learning...