![[2602.14002] The Sufficiency-Conciseness Trade-off in LLM Self-Explanation from an Information Bottleneck Perspective](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.14002] The Sufficiency-Conciseness Trade-off in LLM Self-Explanation from an Information Bottleneck Perspective
Summary
This paper explores the trade-off between sufficiency and conciseness in self-explanations provided by large language models (LLMs), emphasizing the balance needed for effective multi-step reasoning.
Why It Matters
Understanding the sufficiency-conciseness trade-off is crucial for optimizing LLM performance in tasks requiring explanations. This research can inform the development of more efficient AI systems that maintain accuracy while reducing computational costs, which is vital in resource-constrained environments.
Key Takeaways
- Concise explanations can maintain accuracy in LLM outputs.
- Excessive compression of explanations can degrade performance.
- The study introduces an evaluation pipeline for assessing explanation sufficiency.
- Research extends to both English and Persian languages, highlighting cross-linguistic implications.
- Findings can guide future LLM designs to balance explanation length and effectiveness.
Computer Science > Computation and Language arXiv:2602.14002 (cs) [Submitted on 15 Feb 2026] Title:The Sufficiency-Conciseness Trade-off in LLM Self-Explanation from an Information Bottleneck Perspective Authors:Ali Zahedzadeh, Behnam Bahrak View a PDF of the paper titled The Sufficiency-Conciseness Trade-off in LLM Self-Explanation from an Information Bottleneck Perspective, by Ali Zahedzadeh and 1 other authors View PDF HTML (experimental) Abstract:Large Language Models increasingly rely on self-explanations, such as chain of thought reasoning, to improve performance on multi step question answering. While these explanations enhance accuracy, they are often verbose and costly to generate, raising the question of how much explanation is truly necessary. In this paper, we examine the trade-off between sufficiency, defined as the ability of an explanation to justify the correct answer, and conciseness, defined as the reduction in explanation length. Building on the information bottleneck principle, we conceptualize explanations as compressed representations that retain only the information essential for producing correct this http URL operationalize this view, we introduce an evaluation pipeline that constrains explanation length and assesses sufficiency using multiple language models on the ARC Challenge dataset. To broaden the scope, we conduct experiments in both English, using the original dataset, and Persian, as a resource-limited language through translation. Our exp...
