[2604.16909] PRISM: Probing Reasoning, Instruction, and Source Memory in LLM Hallucinations

Computer Science > Computation and Language arXiv:2604.16909 (cs) [Submitted on 18 Apr 2026 (v1), last revised 26 Apr 2026 (this version, v2)] Title:PRISM: Probing Reasoning, Instruction, and Source Memory in LLM Hallucinations Authors:Yuhe Wu, Guangyu Wang, Yuran Chen, Jiatong Zhang, Yutong Zhang, Yujie Chen, Jiaming Shang, Guang Zhang, Zhuang Liu View a PDF of the paper titled PRISM: Probing Reasoning, Instruction, and Source Memory in LLM Hallucinations, by Yuhe Wu and 8 other authors View PDF HTML (experimental) Abstract:As large language models (LLMs) evolve from conversational assistants into agents capable of handling complex tasks, they are increasingly deployed in high-risk domains. However, existing benchmarks largely rely on mixed queries and posterior evaluation, output-level scoring, which quantifies hallucination severity but offers limited insight into where and why hallucinations arise in the generation pipeline. We therefore reformulate hallucination evaluation as a diagnostic problem and propose PRISM, a controlled benchmark that disentangles hallucinations into four dimensions: knowledge missing, knowledge errors, reasoning errors, and instruction-following errors, grounded in three stages of generation (memory, instruction, and reasoning). PRISM contains 9,448 instances across 65 tasks and supports fine-grained, stage-aware diagnostic evaluation. Evaluating 24 mainstream open-source and proprietary LLMs, we uncover consistent trade-offs across instructi...