[2603.20034] CoverageBench: Evaluating Information Coverage across Tasks and Domains

Computer Science > Information Retrieval arXiv:2603.20034 (cs) [Submitted on 20 Mar 2026] Title:CoverageBench: Evaluating Information Coverage across Tasks and Domains Authors:Saron Samuel, Andrew Yates, Dawn Lawrie, Ian Soboroff, Trevor Adriaanse, Benjamin Van Durme, Eugene Yang View a PDF of the paper titled CoverageBench: Evaluating Information Coverage across Tasks and Domains, by Saron Samuel and 6 other authors View PDF HTML (experimental) Abstract:We wish to measure the information coverage of an ad hoc retrieval algorithm, that is, how much of the range of available relevant information is covered by the search results. Information coverage is a central aspect for retrieval, especially when the retrieval system is integrated with generative models in a retrieval-augmented generation (RAG) system. The classic metrics for ad hoc retrieval, precision and recall, reward a system as more and more relevant documents are retrieved. However, since relevance in ad hoc test collections is defined for a document without any relation to other documents that might contain the same information, high recall is sufficient but not necessary to ensure coverage. The same is true for other metrics such as rank-biased precision (RBP), normalized discounted cumulative gain (nDCG), and mean average precision (MAP). Test collections developed around the notion of diversity ranking in web search incorporate multiple aspects that support a concept of coverage in the web domain. In this work...