[2603.20280] Mix-and-Match Pruning: Globally Guided Layer-Wise Sparsification of DNNs

Computer Science > Computer Vision and Pattern Recognition arXiv:2603.20280 (cs) [Submitted on 17 Mar 2026] Title:Mix-and-Match Pruning: Globally Guided Layer-Wise Sparsification of DNNs Authors:Danial Monachan, Samira Nazari, Mahdi Taheri, Ali Azarpeyvand, Milos Krstic, Michael Huebner, Christian Herglotz View a PDF of the paper titled Mix-and-Match Pruning: Globally Guided Layer-Wise Sparsification of DNNs, by Danial Monachan and 6 other authors View PDF HTML (experimental) Abstract:Deploying deep neural networks (DNNs) on edge devices requires strong compression with minimal accuracy loss. This paper introduces Mix-and-Match Pruning, a globally guided, layer-wise sparsification framework that leverages sensitivity scores and simple architectural rules to generate diverse, high-quality pruning configurations. The framework addresses a key limitation that different layers and architectures respond differently to pruning, making single-strategy approaches suboptimal. Mix-and-Match derives architecture-aware sparsity ranges, e.g., preserving normalization layers while pruning classifiers more aggressively, and systematically samples these ranges to produce ten strategies per sensitivity signal (magnitude, gradient, or their combination). This eliminates repeated pruning runs while offering deployment-ready accuracy-sparsity trade-offs. Experiments on CNNs and Vision Transformers demonstrate Pareto-optimal results, with Mix-and-Match reducing accuracy degradation on Swin-Tin...