![[2602.20191] MoBiQuant: Mixture-of-Bits Quantization for Token-Adaptive Elastic LLMs](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.20191] MoBiQuant: Mixture-of-Bits Quantization for Token-Adaptive Elastic LLMs
Summary
The paper presents MoBiQuant, a novel quantization framework for elastic large language models (LLMs) that adapts weight precision based on token sensitivity, enhancing performance without repeated calibration.
Why It Matters
As LLMs are increasingly deployed on diverse computational platforms, optimizing their performance through adaptive quantization is crucial. MoBiQuant addresses the challenges of precision switching and calibration, making it relevant for developers and researchers focused on efficient AI model deployment.
Key Takeaways
- MoBiQuant allows dynamic adjustment of weight precision based on token sensitivity.
- The framework improves generalization for token outliers without needing repeated calibration.
- Experimental results show MoBiQuant matches the performance of traditional methods on LLaMA3-8B.
Computer Science > Machine Learning arXiv:2602.20191 (cs) [Submitted on 21 Feb 2026] Title:MoBiQuant: Mixture-of-Bits Quantization for Token-Adaptive Elastic LLMs Authors:Dongwei Wang, Jinhee Kim, Seokho Han, Denis Gudovskiy, Yohei Nakata, Tomoyuki Okuno, KhayTze Peong, Kang Eun Jeon, Jong Hwan Ko, Yiran Chen, Huanrui Yang View a PDF of the paper titled MoBiQuant: Mixture-of-Bits Quantization for Token-Adaptive Elastic LLMs, by Dongwei Wang and 10 other authors View PDF HTML (experimental) Abstract:Changing runtime complexity on cloud and edge devices necessitates elastic large language model (LLM) deployment, where an LLM can be inferred with various quantization precisions based on available computational resources. However, it has been observed that the calibration parameters for quantization are typically linked to specific precisions, which presents challenges during elastic-precision calibration and precision switching at runtime. In this work, we attribute the source of varying calibration parameters to the varying token-level sensitivity caused by a precision-dependent outlier migration this http URL by this observation, we propose \texttt{MoBiQuant}, a novel Mixture-of-Bits quantization framework that adjusts weight precision for elastic LLM inference based on token sensitivity. Specifically, we propose the many-in-one recursive residual quantization that can iteratively reconstruct higher-precision weights and the token-aware router to dynamically select the numb...