![[2509.21865] Beyond RAG vs. Long-Context: Learning Distraction-Aware Retrieval for Efficient Knowledge Grounding](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2509.21865] Beyond RAG vs. Long-Context: Learning Distraction-Aware Retrieval for Efficient Knowledge Grounding
Summary
This paper introduces LDAR, a new retrieval method that enhances the efficiency of knowledge grounding in Large Language Models (LLMs) by mitigating distractions from irrelevant passages.
Why It Matters
As LLMs evolve, effective knowledge grounding becomes crucial for improving their performance. LDAR addresses the limitations of existing methods by optimizing context retrieval, which can lead to more accurate and relevant outputs in various applications, making it a significant advancement in the field of machine learning.
Key Takeaways
- LDAR improves context retrieval by reducing distractions from irrelevant passages.
- The method demonstrates higher performance with less token usage compared to traditional long-context approaches.
- Extensive experiments validate LDAR's effectiveness across multiple LLM architectures and benchmarks.
Computer Science > Machine Learning arXiv:2509.21865 (cs) [Submitted on 26 Sep 2025 (v1), last revised 25 Feb 2026 (this version, v2)] Title:Beyond RAG vs. Long-Context: Learning Distraction-Aware Retrieval for Efficient Knowledge Grounding Authors:Seongwoong Shim, Myunsoo Kim, Jae Hyeon Cho, Byung-Jun Lee View a PDF of the paper titled Beyond RAG vs. Long-Context: Learning Distraction-Aware Retrieval for Efficient Knowledge Grounding, by Seongwoong Shim and 3 other authors View PDF HTML (experimental) Abstract:Retrieval-Augmented Generation (RAG) is a framework for grounding Large Language Models (LLMs) in external, up-to-date information. However, recent advancements in context window size allow LLMs to process inputs of up to 128K tokens or more, offering an alternative strategy: supplying the full document context directly to the model, rather than relying on RAG to retrieve a subset of contexts. Nevertheless, this emerging alternative strategy has notable limitations: (i) it is token-inefficient to handle large and potentially redundant contexts; (ii) it exacerbates the `lost in the middle' phenomenon; and (iii) under limited model capacity, it amplifies distraction, ultimately degrading LLM output quality. In this paper, we propose LDAR (Learning Distraction-Aware Retrieval), an adaptive retriever that learns to retrieve contexts in a way that mitigates interference from distracting passages, thereby achieving significantly higher performance with reduced token usage...
