![[2602.14626] Concepts' Information Bottleneck Models](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.14626] Concepts' Information Bottleneck Models
Summary
This article presents the Concepts' Information Bottleneck Models, which enhance the interpretability of predictions in machine learning by introducing a regularizer that improves predictive performance and concept reliability.
Why It Matters
The research addresses the challenge of balancing interpretability and accuracy in machine learning models. By proposing an Information Bottleneck regularizer, it offers a theoretically grounded method to enhance the reliability of concept-based predictions, which is crucial for applications requiring transparency and trust.
Key Takeaways
- Introduces an Information Bottleneck regularizer to improve Concept Bottleneck Models.
- Enhances predictive performance while maintaining interpretability.
- Demonstrates robust gains across multiple model families and datasets.
- Addresses prior evaluation inconsistencies in concept-based models.
- Offers a theoretically grounded approach that is architecture-agnostic.
Computer Science > Machine Learning arXiv:2602.14626 (cs) [Submitted on 16 Feb 2026] Title:Concepts' Information Bottleneck Models Authors:Karim Galliamov, Syed M Ahsan Kazmi, Adil Khan, Adín Ramírez Rivera View a PDF of the paper titled Concepts' Information Bottleneck Models, by Karim Galliamov and Syed M Ahsan Kazmi and Adil Khan and Ad\'in Ram\'irez Rivera View PDF Abstract:Concept Bottleneck Models (CBMs) aim to deliver interpretable predictions by routing decisions through a human-understandable concept layer, yet they often suffer reduced accuracy and concept leakage that undermines faithfulness. We introduce an explicit Information Bottleneck regularizer on the concept layer that penalizes $I(X;C)$ while preserving task-relevant information in $I(C;Y)$, encouraging minimal-sufficient concept representations. We derive two practical variants (a variational objective and an entropy-based surrogate) and integrate them into standard CBM training without architectural changes or additional supervision. Evaluated across six CBM families and three benchmarks, the IB-regularized models consistently outperform their vanilla counterparts. Information-plane analyses further corroborate the intended behavior. These results indicate that enforcing a minimal-sufficient concept bottleneck improves both predictive performance and the reliability of concept-level interventions. The proposed regularizer offers a theoretic-grounded, architecture-agnostic path to more faithful and int...
