[2604.08585] QCFuse: Query-Centric Cache Fusion for Efficient RAG Inference

Computer Science > Databases arXiv:2604.08585 (cs) [Submitted on 30 Mar 2026] Title:QCFuse: Query-Centric Cache Fusion for Efficient RAG Inference Authors:Jianxin Yan, Zeheng Qian, Wangze Ni, Zhitao Shen, Zhiping Wang, Haoyang Li, Jia Zhu, Lei Chen, Kui Ren View a PDF of the paper titled QCFuse: Query-Centric Cache Fusion for Efficient RAG Inference, by Jianxin Yan and 7 other authors View PDF HTML (experimental) Abstract:Cache fusion accelerates generation process of LLMs equipped with RAG through KV caching and selective token recomputation, thereby reducing computational costs and improving efficiency. However, existing methods primarily rely on local perspectives for token selection and lack global awareness from the user query. Utilizing this global awareness is challenging due to the high cost of obtaining context-aware query representations and the strict pipeline constraints required for efficient attention analysis. Thus, this demonstration introduces QCFuse, an innovative KV cache fusion system centered on the user query. QCFuse leverages semantic summary anchors to enhance query representations and selectively recomputes query-related tokens to improve accuracy, updating tokens based on the attention distribution of the most critical Transformer layer to preserve the high efficiency of the pipeline structure. Evaluations on real-world datasets demonstrate that QCFuse significantly improves the response efficiency of LLMs by 40\% while maintaining equivalent accu...