[2602.11448] Hierarchical Concept Embedding & Pursuit for Interpretable Image Classification

Computer Science > Machine Learning arXiv:2602.11448 (cs) [Submitted on 11 Feb 2026 (v1), last revised 30 Mar 2026 (this version, v3)] Title:Hierarchical Concept Embedding & Pursuit for Interpretable Image Classification Authors:Nghia Nguyen, Tianjiao Ding, René Vidal View a PDF of the paper titled Hierarchical Concept Embedding & Pursuit for Interpretable Image Classification, by Nghia Nguyen and 2 other authors View PDF HTML (experimental) Abstract:Interpretable-by-design models are gaining traction in computer vision because they provide faithful explanations for their predictions. In image classification, these models typically recover human-interpretable concepts from an image and use them for classification. Sparse concept recovery methods leverage the latent space of vision-language models to represent image embeddings as sparse combinations of concept embeddings. However, by ignoring the hierarchical structure of semantic concepts, these methods may produce correct predictions with explanations that are inconsistent with the hierarchy. In this work, we propose Hierarchical Concept Embedding & Pursuit (HCEP), a framework that induces a hierarchy of concept embeddings in the latent space and performs hierarchical sparse coding to recover the concepts present in an image. Given a hierarchy of semantic concepts, we introduce a geometric construction for the corresponding hierarchy of embeddings. Under the assumption that the true concepts form a rooted path in the hier...