[2511.11743] Uncertainty Makes It Stable: Curiosity-Driven Quantized Mixture-of-Experts

Computer Science > Machine Learning arXiv:2511.11743 (cs) [Submitted on 13 Nov 2025 (v1), last revised 24 Mar 2026 (this version, v3)] Title:Uncertainty Makes It Stable: Curiosity-Driven Quantized Mixture-of-Experts Authors:Sebastián Andrés Cajas Ordóñez, Luis Fernando Torres Torres, Mackenzie J. Meni, Carlos Andrés Duran Paredes, Eric Arazo, Cristian Bosch, Ricardo Simon Carbajo, Yuan Lai, Leo Anthony Celi View a PDF of the paper titled Uncertainty Makes It Stable: Curiosity-Driven Quantized Mixture-of-Experts, by Sebasti\'an Andr\'es Cajas Ord\'o\~nez and 8 other authors View PDF HTML (experimental) Abstract:Deploying deep neural networks on resource-constrained devices faces two critical challenges: maintaining accuracy under aggressive quantization while ensuring predictable inference latency. We present a curiosity-driven quantized Mixture-of-Experts framework that addresses both through Bayesian epistemic uncertainty-based routing across heterogeneous experts (BitNet ternary, 1-16 bit BitLinear, post-training quantization). Evaluated on audio classification benchmarks (ESC-50, Quinn, UrbanSound8K), our 4-bit quantization maintains 99.9 percent of full-precision F1 (0.858 vs 0.859) with 4x compression and 31 percent energy savings versus 8-bit, while both achieve statistical parity with full precision (p > 0.05). Crucially, curiosity-driven routing simultaneously improves accuracy and stability: on Quinn, F1 increases from 0.802 to 0.809 while cross-fold variance drop...