[2603.00105] LIDS: LLM Summary Inference Under the Layered Lens

Computer Science > Machine Learning arXiv:2603.00105 (cs) [Submitted on 18 Feb 2026] Title:LIDS: LLM Summary Inference Under the Layered Lens Authors:Dylan Park, Yingying Fan, Jinchi Lv View a PDF of the paper titled LIDS: LLM Summary Inference Under the Layered Lens, by Dylan Park and 2 other authors View PDF HTML (experimental) Abstract:Large language models (LLMs) have gained significant attention by many researchers and practitioners in natural language processing (NLP) since the introduction of ChatGPT in 2022. One notable feature of ChatGPT is its ability to generate summaries based on prompts. Yet evaluating the quality of these summaries remains challenging due to the complexity of language. To this end, in this paper we suggest a new method of LLM summary inference with BERT-SVD-based direction metric and SOFARI (LIDS) that assesses the summary accuracy equipped with interpretable key words for layered themes. The LIDS uses a latent SVD-based direction metric to measure the similarity between the summaries and original text, leveraging the BERT embeddings and repeated prompts to quantify the statistical uncertainty. As a result, LIDS gives a natural embedding of each summary for large text reduction. We further exploit SOFARI to uncover important key words associated with each latent theme in the summary with controlled false discovery rate (FDR). Comprehensive empirical studies demonstrate the practical utility and robustness of LIDS through human verification an...