[2502.01941] Semantic Integrity Matters: Benchmarking and Preserving High-Density Reasoning in KV Cache Compression

Computer Science > Computation and Language arXiv:2502.01941 (cs) [Submitted on 4 Feb 2025 (v1), last revised 8 May 2026 (this version, v3)] Title:Semantic Integrity Matters: Benchmarking and Preserving High-Density Reasoning in KV Cache Compression Authors:Xiang Liu, Zhenheng Tang, Hong Chen, Peijie Dong, Zeyu Li, Xiuze Zhou, Bo Li, Xuming Hu, Xiaowen Chu View a PDF of the paper titled Semantic Integrity Matters: Benchmarking and Preserving High-Density Reasoning in KV Cache Compression, by Xiang Liu and 8 other authors View PDF HTML (experimental) Abstract:While Key-Value (KV) cache compression is essential for efficient LLM inference, current evaluations disproportionately focus on sparse retrieval tasks, potentially masking the degradation of High-Density Reasoning where Chain-of-Thought (CoT) coherence is critical. We introduce KVFundaBench to systematically evaluate this gap, revealing a sharp dichotomy: while retrieval tasks remain robust, reasoning tasks exhibit severe Task-Dependent Degradation under aggressive compression due to disrupted CoT links. Extending our analysis to the DeepSeek-R1 model, we uncover that its specialized attention patterns offer unique insights into the fragility of reasoning chains. Guided by these findings -- specifically the necessity of preserving few-shot examples as indivisible Semantic Units -- we propose ShotKV. This approach explicitly separates prefill and decoding phases to prioritize semantic integrity. Empirical results demon...