![[2511.06450] Countering Multi-modal Representation Collapse through Rank-targeted Fusion](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2511.06450] Countering Multi-modal Representation Collapse through Rank-targeted Fusion
Summary
This paper presents a novel framework, Rank-enhancing Token Fuser, to address multi-modal representation collapse in machine learning, enhancing feature and modality integration for improved action anticipation.
Why It Matters
As multi-modal systems become increasingly prevalent in applications like action recognition, addressing representation collapse is crucial for improving model performance. This research provides a unified approach to enhance the effectiveness of multi-modal fusion, which can lead to better outcomes in various AI applications.
Key Takeaways
- Introduces Rank-enhancing Token Fuser to counter representation collapse.
- Demonstrates how effective rank can quantify and mitigate feature and modality collapse.
- Validates the approach through extensive experiments, outperforming existing methods by up to 3.74%.
Computer Science > Computer Vision and Pattern Recognition arXiv:2511.06450 (cs) [Submitted on 9 Nov 2025 (v1), last revised 23 Feb 2026 (this version, v2)] Title:Countering Multi-modal Representation Collapse through Rank-targeted Fusion Authors:Seulgi Kim, Kiran Kokilepersaud, Mohit Prabhushankar, Ghassan AlRegib View a PDF of the paper titled Countering Multi-modal Representation Collapse through Rank-targeted Fusion, by Seulgi Kim and 3 other authors View PDF HTML (experimental) Abstract:Multi-modal fusion methods often suffer from two types of representation collapse: feature collapse where individual dimensions lose their discriminative power (as measured by eigenspectra), and modality collapse where one dominant modality overwhelms the other. Applications like human action anticipation that require fusing multifarious sensor data are hindered by both feature and modality collapse. However, existing methods attempt to counter feature collapse and modality collapse separately. This is because there is no unifying framework that efficiently addresses feature and modality collapse in conjunction. In this paper, we posit the utility of effective rank as an informative measure that can be utilized to quantify and counter both the representation collapses. We propose \textit{Rank-enhancing Token Fuser}, a theoretically grounded fusion framework that selectively blends less informative features from one modality with complementary features from another modality. We show tha...
