[2603.00431] Taxonomy-Aware Representation Alignment for Hierarchical Visual Recognition with Large Multimodal Models

Computer Science > Computer Vision and Pattern Recognition arXiv:2603.00431 (cs) [Submitted on 28 Feb 2026] Title:Taxonomy-Aware Representation Alignment for Hierarchical Visual Recognition with Large Multimodal Models Authors:Hulingxiao He, Zhi Tan, Yuxin Peng View a PDF of the paper titled Taxonomy-Aware Representation Alignment for Hierarchical Visual Recognition with Large Multimodal Models, by Hulingxiao He and 2 other authors View PDF HTML (experimental) Abstract:A high-performing, general-purpose visual understanding model should map visual inputs to a taxonomic tree of labels, identify novel categories beyond the training set for which few or no publicly available images exist. Large Multimodal Models (LMMs) have achieved remarkable progress in fine-grained visual recognition (FGVR) for known categories. However, they remain limited in hierarchical visual recognition (HVR) that aims at predicting consistent label paths from coarse to fine categories, especially for novel categories. To tackle these challenges, we propose Taxonomy-Aware Representation Alignment (TARA), a simple yet effective strategy to inject taxonomic knowledge into LMMs. TARA leverages representations from biology foundation models (BFMs) that encode rich biological relationships through hierarchical contrastive learning. By aligning the intermediate representations of visual features with those of BFMs, LMMs are encouraged to extract discriminative visual cues well structured in the taxonomy tre...