[2603.22608] Understanding LLM Performance Degradation in Multi-Instance Processing: The Roles of Instance Count and Context Length

Computer Science > Artificial Intelligence arXiv:2603.22608 (cs) [Submitted on 23 Mar 2026] Title:Understanding LLM Performance Degradation in Multi-Instance Processing: The Roles of Instance Count and Context Length Authors:Jingxuan Chen, Mohammad Taher Pilehvar, Jose Camacho-Collados View a PDF of the paper titled Understanding LLM Performance Degradation in Multi-Instance Processing: The Roles of Instance Count and Context Length, by Jingxuan Chen and 2 other authors View PDF Abstract:Users often rely on Large Language Models (LLMs) for processing multiple documents or performing analysis over a number of instances. For example, analysing the overall sentiment of a number of movie reviews requires an LLM to process the sentiment of each review individually in order to provide a final aggregated answer. While LLM performance on such individual tasks is generally high, there has been little research on how LLMs perform when dealing with multi-instance inputs. In this paper, we perform a comprehensive evaluation of the multi-instance processing (MIP) ability of LLMs for tasks in which they excel individually. The results show that all LLMs follow a pattern of slight performance degradation for small numbers of instances (approximately 20-100), followed by a performance collapse on larger instance counts. Crucially, our analysis shows that while context length is associated with this degradation, the number of instances has a stronger effect on the final results. This findi...