[2506.02939] QKV Projections Require a Fraction of Their Memory

Computer Science > Machine Learning arXiv:2506.02939 (cs) [Submitted on 3 Jun 2025 (v1), last revised 1 Mar 2026 (this version, v3)] Title:QKV Projections Require a Fraction of Their Memory Authors:Malik Khalaf, Yara Shamshoum, Nitzan Hodos, Yuval Sieradzki, Assaf Schuster View a PDF of the paper titled QKV Projections Require a Fraction of Their Memory, by Malik Khalaf and 4 other authors View PDF HTML (experimental) Abstract:The Multi-Head Attention mechanism is central to LLM operation, and multiple works target its compute and memory efficiency during training. While most works focus on approximating the scaled dot product, the memory consumption of the linear projections that compute the $Q$, $K$, and $V$ tensors from the input $x$ is often overlooked. To address this, we propose Point-Approximate Matrix Multiplication (PAMM), a novel tensor compression technique that compresses the activations of the $Q,K,V$ projections in attention layers by a factor of up to $\times 512$, effectively erasing their memory footprint, while achieving similar or better final perplexity. PAMM is fully composable with efficient attention techniques such as FlashAttention, making it a practical and complementary method for memory-efficient LLM training. Comments: Subjects: Machine Learning (cs.LG) Cite as: arXiv:2506.02939 [cs.LG]   (or arXiv:2506.02939v3 [cs.LG] for this version)   https://doi.org/10.48550/arXiv.2506.02939 Focus to learn more arXiv-issued DOI via DataCite Submission hist...