[2509.03417] Initialization Schemes for Kolmogorov-Arnold Networks: An Empirical Study

Computer Science > Machine Learning arXiv:2509.03417 (cs) [Submitted on 3 Sep 2025 (v1), last revised 30 Mar 2026 (this version, v3)] Title:Initialization Schemes for Kolmogorov-Arnold Networks: An Empirical Study Authors:Spyros Rigas, Dhruv Verma, Georgios Alexandridis, Yixuan Wang View a PDF of the paper titled Initialization Schemes for Kolmogorov-Arnold Networks: An Empirical Study, by Spyros Rigas and 3 other authors View PDF HTML (experimental) Abstract:Kolmogorov-Arnold Networks (KANs) are a recently introduced neural architecture that replace fixed nonlinearities with trainable activation functions, offering enhanced flexibility and interpretability. While KANs have been applied successfully across scientific and machine learning tasks, their initialization strategies remain largely unexplored. In this work, we study initialization schemes for spline-based KANs, proposing two theory-driven approaches inspired by LeCun and Glorot, as well as an empirical power-law family with tunable exponents. Our evaluation combines large-scale grid searches on function fitting and forward PDE benchmarks, an analysis of training dynamics through the lens of the Neural Tangent Kernel, and evaluations on a subset of the Feynman dataset. Our findings indicate that the Glorot-inspired initialization significantly outperforms the baseline in parameter-rich models, while power-law initialization achieves the strongest performance overall, both across tasks and for architectures of varyi...