![[2602.17698] ScaleBITS: Scalable Bitwidth Search for Hardware-Aligned Mixed-Precision LLMs](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.17698] ScaleBITS: Scalable Bitwidth Search for Hardware-Aligned Mixed-Precision LLMs
Summary
The paper presents ScaleBITS, a mixed-precision quantization framework designed to optimize bitwidth allocation in large language models, enhancing efficiency without runtime overhead.
Why It Matters
As large language models (LLMs) become increasingly prevalent, optimizing their performance while managing resource constraints is crucial. ScaleBITS addresses the challenge of low-bit quantization, which is essential for deploying LLMs in resource-limited environments, thus contributing to advancements in AI efficiency and accessibility.
Key Takeaways
- ScaleBITS enables automated, fine-grained bitwidth allocation for LLMs.
- The framework improves performance by up to 36% over uniform-precision quantization.
- It outperforms existing sensitivity-aware methods by up to 13% in ultra-low-bit regimes.
- The method preserves hardware efficiency without adding runtime overhead.
- Introduces a new sensitivity analysis and block-wise weight partitioning scheme.
Computer Science > Machine Learning arXiv:2602.17698 (cs) [Submitted on 6 Feb 2026] Title:ScaleBITS: Scalable Bitwidth Search for Hardware-Aligned Mixed-Precision LLMs Authors:Xinlin Li, Timothy Chou, Josh Fromm, Zichang Liu, Yunjie Pan, Christina Fragouli View a PDF of the paper titled ScaleBITS: Scalable Bitwidth Search for Hardware-Aligned Mixed-Precision LLMs, by Xinlin Li and 5 other authors View PDF HTML (experimental) Abstract:Post-training weight quantization is crucial for reducing the memory and inference cost of large language models (LLMs), yet pushing the average precision below 4 bits remains challenging due to highly non-uniform weight sensitivity and the lack of principled precision allocation. Existing solutions use irregular fine-grained mixed-precision with high runtime overhead or rely on heuristics or highly constrained precision allocation strategies. In this work, we propose ScaleBITS, a mixed-precision quantization framework that enables automated, fine-grained bitwidth allocation under a memory budget while preserving hardware efficiency. Guided by a new sensitivity analysis, we introduce a hardware-aligned, block-wise weight partitioning scheme, powered by bi-directional channel reordering. We formulate global bitwidth allocation as a constrained optimization problem and develop a scalable approximation to the greedy algorithm, enabling end-to-end principled allocation. Experiments show that ScaleBITS significantly improves over uniform-precision ...