[2603.02153] Scaling Retrieval Augmented Generation with RAG Fusion: Lessons from an Industry Deployment

Computer Science > Information Retrieval arXiv:2603.02153 (cs) [Submitted on 2 Mar 2026] Title:Scaling Retrieval Augmented Generation with RAG Fusion: Lessons from an Industry Deployment Authors:Luigi Medrano, Arush Verma, Mukul Chhabra View a PDF of the paper titled Scaling Retrieval Augmented Generation with RAG Fusion: Lessons from an Industry Deployment, by Luigi Medrano and 2 other authors View PDF HTML (experimental) Abstract:Retrieval-Augmented Generation (RAG) systems commonly adopt retrieval fusion techniques such as multi-query retrieval and reciprocal rank fusion (RRF) to increase document recall, under the assumption that higher recall leads to better answer quality. While these methods show consistent gains in isolated retrieval benchmarks, their effectiveness under realistic production constraints remains underexplored. In this work, we evaluate retrieval fusion in a production-style RAG pipeline operating over an enterprise knowledge base, with fixed retrieval depth, re-ranking budgets, and latency constraints. Across multiple fusion configurations, we find that retrieval fusion does increase raw recall, but these gains are largely neutralized after re-ranking and truncation. In our setting, fusion variants fail to outperform single-query baselines on KB-level Top-$k$ accuracy, with Hit@10 decreasing from $0.51$ to $0.48$ in several configurations. Moreover, fusion introduces additional latency overhead due to query rewriting and larger candidate sets, witho...