[2510.03101] AdaBet: Gradient-free Layer Selection for Efficient Training of Deep Neural Networks

Computer Science > Machine Learning arXiv:2510.03101 (cs) [Submitted on 3 Oct 2025 (v1), last revised 2 Mar 2026 (this version, v2)] Title:AdaBet: Gradient-free Layer Selection for Efficient Training of Deep Neural Networks Authors:Irene Tenison, Soumyajit Chatterjee, Fahim Kawsar, Mohammad Malekzadeh View a PDF of the paper titled AdaBet: Gradient-free Layer Selection for Efficient Training of Deep Neural Networks, by Irene Tenison and 3 other authors View PDF HTML (experimental) Abstract:To utilize pre-trained neural networks on edge and mobile devices, we often require efficient adaptation to user-specific runtime data distributions while operating under limited compute and memory resources. On-device retraining with a target dataset can facilitate such adaptations; however, it remains impractical due to the increasing depth of modern neural nets, as well as the computational overhead associated with gradient-based optimization across all layers. Current approaches reduce training cost by selecting a subset of layers for retraining, however, they rely on labeled data, at least one full-model backpropagation, or server-side meta-training; limiting their suitability for constrained devices. We introduce AdaBet, a gradient-free layer selection approach to rank important layers by analyzing topological features of their activation spaces through Betti Numbers and using forward passes alone. AdaBet allows selecting layers with high learning capacity, which are important for ...