![[2601.19320] StableQAT: Stable Quantization-Aware Training at Ultra-Low Bitwidths](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2601.19320] StableQAT: Stable Quantization-Aware Training at Ultra-Low Bitwidths
Summary
The paper presents StableQAT, a novel framework for quantization-aware training (QAT) that enhances stability and efficiency at ultra-low bitwidths, addressing challenges in gradient mismatch and computational overhead.
Why It Matters
As machine learning models grow larger, deploying them efficiently under strict memory and latency constraints becomes crucial. StableQAT offers a solution to improve training stability and performance at low bitwidths, making it relevant for developers and researchers in AI and machine learning.
Key Takeaways
- StableQAT improves quantization-aware training stability at 2-4 bitwidths.
- It provides a novel surrogate for backpropagation, enhancing gradient performance.
- The framework generalizes existing techniques, offering better robustness and efficiency.
- Experiments show negligible training overhead compared to standard QAT methods.
- Code for StableQAT is publicly available, promoting further research and application.
Computer Science > Machine Learning arXiv:2601.19320 (cs) [Submitted on 27 Jan 2026 (v1), last revised 18 Feb 2026 (this version, v2)] Title:StableQAT: Stable Quantization-Aware Training at Ultra-Low Bitwidths Authors:Tianyi Chen, Sihan Chen, Xiaoyi Qu, Dan Zhao, Ruomei Yan, Jongwoo Ko, Luming Liang, Pashmina Cameron View a PDF of the paper titled StableQAT: Stable Quantization-Aware Training at Ultra-Low Bitwidths, by Tianyi Chen and 7 other authors View PDF Abstract:Quantization-aware training (QAT) is essential for deploying large models under strict memory and latency constraints, yet achieving stable and robust optimization at ultra-low bitwidths remains challenging. Common approaches based on the straight-through estimator (STE) or soft quantizers often suffer from gradient mismatch, instability, or high computational overhead. As such, we propose StableQAT, a unified and efficient QAT framework that stabilizes training in ultra low-bit settings via a novel, lightweight, and theoretically grounded surrogate for backpropagation derived from a discrete Fourier analysis of the rounding operator. StableQAT strictly generalizes STE as the latter arises as a special case of our more expressive surrogate family, yielding smooth, bounded, and inexpensive gradients that improve QAT training performance and stability across various hyperparameter choices. In experiments, StableQAT exhibits stable and efficient QAT at 2-4 bit regimes, demonstrating improved training stability, ...
