![[2602.17584] Canonicalizing Multimodal Contrastive Representation Learning](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.17584] Canonicalizing Multimodal Contrastive Representation Learning
Summary
This article explores the geometric relationships between independently trained multimodal contrastive models, revealing that an orthogonal map can align their embedding spaces effectively.
Why It Matters
Understanding the geometric relationships between multimodal models is crucial for improving model interoperability and efficiency. This research offers insights that can facilitate backward-compatible upgrades and enhance privacy in learned representations, which are significant for advancing AI applications.
Key Takeaways
- Multimodal models can be aligned using an orthogonal map, improving interoperability.
- The same geometric relationship applies to both image and text encoders across models.
- Establishing explicit correspondence between representation spaces enhances model consistency.
- This research supports backward-compatible upgrades, reducing the need for costly re-embedding.
- Implications for privacy in learned representations are significant, impacting data security.
Computer Science > Machine Learning arXiv:2602.17584 (cs) [Submitted on 19 Feb 2026] Title:Canonicalizing Multimodal Contrastive Representation Learning Authors:Sharut Gupta, Sanyam Kansal, Stefanie Jegelka, Phillip Isola, Vikas Garg View a PDF of the paper titled Canonicalizing Multimodal Contrastive Representation Learning, by Sharut Gupta and 4 other authors View PDF Abstract:As models and data scale, independently trained networks often induce analogous notions of similarity. But, matching similarities is weaker than establishing an explicit correspondence between the representation spaces, especially for multimodal models, where consistency must hold not only within each modality, but also for the learned image-text coupling. We therefore ask: given two independently trained multimodal contrastive models (with encoders $(f, g)$ and $(\widetilde{f},\widetilde{g})$) -- trained on different distributions and with different architectures -- does a systematic geometric relationship exist between their embedding spaces? If so, what form does it take, and does it hold uniformly across modalities? In this work, we show that across model families such as CLIP, SigLIP, and FLAVA, this geometric relationship is well approximated by an orthogonal map (up to a global mean shift), i.e., there exists an orthogonal map $Q$ where $Q^\top Q = I$ such that $\widetilde{f}(x)\approx Q f(x)$ for paired images $x$. Strikingly, the same $Q$ simultaneously aligns the text encoders i.e., $\wid...
