[2603.02767] ITO: Images and Texts as One via Synergizing Multiple Alignment and Training-Time Fusion

Computer Science > Computer Vision and Pattern Recognition arXiv:2603.02767 (cs) [Submitted on 3 Mar 2026] Title:ITO: Images and Texts as One via Synergizing Multiple Alignment and Training-Time Fusion Authors:HanZpeng Liu, Yaqian Li, Zidan Wang, Shuoxi Zhang, Zonglin Zhao, Zihao Bo, Rinyoichi Takezoe, Kaiwen Long, Kun He View a PDF of the paper titled ITO: Images and Texts as One via Synergizing Multiple Alignment and Training-Time Fusion, by HanZpeng Liu and 8 other authors View PDF HTML (experimental) Abstract:Image-text contrastive pretraining has become a dominant paradigm for visual representation learning, yet existing methods often yield representations that remain partially organized by modality. We propose ITO, a framework addressing this limitation through two synergistic mechanisms. Multimodal multiple alignment enriches supervision by mining diverse image-text correspondences, while a lightweight training-time multimodal fusion module enforces structured cross-modal interaction. Crucially, the fusion module is discarded at inference, preserving the efficiency of standard dual-encoder architectures. Extensive experiments show that ITO consistently outperforms strong baselines across classification, retrieval, and multimodal benchmarks. Our analysis reveals that while multiple alignment drives discriminative power, training-time fusion acts as a critical structural regularizer -- eliminating the modality gap and stabilizing training dynamics to prevent the early...