[2603.18492] AIMER: Calibration-Free Task-Agnostic MoE Pruning

Computer Science > Machine Learning arXiv:2603.18492 (cs) [Submitted on 19 Mar 2026 (v1), last revised 13 Apr 2026 (this version, v2)] Title:AIMER: Calibration-Free Task-Agnostic MoE Pruning Authors:Zongfang Liu, Shengkun Tang, Yifan Shen, Huan Wang, Xin Yuan View a PDF of the paper titled AIMER: Calibration-Free Task-Agnostic MoE Pruning, by Zongfang Liu and 4 other authors View PDF HTML (experimental) Abstract:Mixture-of-Experts (MoE) language models increase parameter capacity without proportional per-token compute, but the deployment still requires storing all experts, making expert pruning important for reducing memory and serving overhead. Existing task-agnostic expert pruning methods are typically calibration-dependent: they estimate expert importance from routing or activation statistics on a calibration set, which makes pruning outcomes sensitive to the choice of calibration set and adds substantial preprocessing cost. We introduce AIMER (\textbf{A}bsolute mean over root mean square \textbf{IM}portance for \textbf{E}xpert \textbf{R}anking), a simple calibration-free criterion that yields clear within-layer score separation and distinct expert stratification. Across 7B to 30B MoE language models at 25\% and 50\% pruning ratios over 16 benchmarks, AIMER consistently delivers competitive or stronger overall performance against state-of-the-art calibration-based expert pruning baselines with only 0.22--1.27 seconds for scoring the experts. Subjects: Machine Learning (...