![[2602.14080] Empty Shelves or Lost Keys? Recall Is the Bottleneck for Parametric Factuality](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.14080] Empty Shelves or Lost Keys? Recall Is the Bottleneck for Parametric Factuality
Summary
The paper explores the limitations of factuality evaluations in large language models (LLMs), identifying recall as a key bottleneck in accessing encoded knowledge, rather than mere knowledge absence.
Why It Matters
Understanding the recall limitations in LLMs is crucial for improving their factual accuracy and performance. This research highlights the need for better methodologies that enhance how models utilize existing knowledge, which can lead to more reliable AI applications.
Key Takeaways
- Recall issues in LLMs often stem from access limitations rather than missing knowledge.
- A new benchmark, WikiProfile, helps profile factual knowledge in LLMs.
- Thinking processes can significantly improve recall and reduce errors.
- Current LLMs encode a high percentage of facts, but accessing them remains challenging.
- Future improvements may focus more on recall methods than on scaling model size.
Computer Science > Computation and Language arXiv:2602.14080 (cs) [Submitted on 15 Feb 2026] Title:Empty Shelves or Lost Keys? Recall Is the Bottleneck for Parametric Factuality Authors:Nitay Calderon, Eyal Ben-David, Zorik Gekhman, Eran Ofek, Gal Yona View a PDF of the paper titled Empty Shelves or Lost Keys? Recall Is the Bottleneck for Parametric Factuality, by Nitay Calderon and 4 other authors View PDF Abstract:Standard factuality evaluations of LLMs treat all errors alike, obscuring whether failures arise from missing knowledge (empty shelves) or from limited access to encoded facts (lost keys). We propose a behavioral framework that profiles factual knowledge at the level of facts rather than questions, characterizing each fact by whether it is encoded, and then by how accessible it is: cannot be recalled, can be directly recalled, or can only be recalled with inference-time computation (thinking). To support such profiling, we introduce WikiProfile, a new benchmark constructed via an automated pipeline with a prompted LLM grounded in web search. Across 4 million responses from 13 LLMs, we find that encoding is nearly saturated in frontier models on our benchmark, with GPT-5 and Gemini-3 encoding 95--98% of facts. However, recall remains a major bottleneck: many errors previously attributed to missing knowledge instead stem from failures to access it. These failures are systematic and disproportionately affect long-tail facts and reverse questions. Finally, we show ...
