[2512.01183] TempPerturb-Eval: On the Joint Effects of Internal Temperature and External Perturbations in RAG Robustness

Computer Science > Computation and Language arXiv:2512.01183 (cs) [Submitted on 1 Dec 2025 (v1), last revised 20 Mar 2026 (this version, v2)] Title:TempPerturb-Eval: On the Joint Effects of Internal Temperature and External Perturbations in RAG Robustness Authors:Yongxin Zhou, Philippe Mulhem, Didier Schwab View a PDF of the paper titled TempPerturb-Eval: On the Joint Effects of Internal Temperature and External Perturbations in RAG Robustness, by Yongxin Zhou and 2 other authors View PDF HTML (experimental) Abstract:The evaluation of Retrieval-Augmented Generation (RAG) systems typically examines retrieval quality and generation parameters like temperature in isolation, overlooking their interaction. This work presents a systematic investigation of how text perturbations (simulating noisy retrieval) interact with temperature settings across multiple LLM runs. We propose a comprehensive RAG Perturbation-Temperature Analysis Framework that subjects retrieved documents to three distinct perturbation types across varying temperature settings. Through extensive experiments on HotpotQA with both open-source and proprietary LLMs, we demonstrate that performance degradation follows distinct patterns: high-temperature settings consistently amplify vulnerability to perturbations, while certain perturbation types exhibit non-linear sensitivity across the temperature range. Our work yields three key contributions: (1) a diagnostic benchmark for assessing RAG robustness, (2) an analyt...