[2604.04999] PRIME: Prototype-Driven Multimodal Pretraining for Cancer Prognosis with Missing Modalities

Computer Science > Machine Learning arXiv:2604.04999 (cs) [Submitted on 5 Apr 2026] Title:PRIME: Prototype-Driven Multimodal Pretraining for Cancer Prognosis with Missing Modalities Authors:Kai Yu, Shuang Zhou, Yiran Song, Zaifu Zhan, Jie Peng, Kaixiong Zhou, Tianlong Chen, Feng Xie, Meng Wang, Huazhu Fu, Mingquan Lin, Rui Zhang View a PDF of the paper titled PRIME: Prototype-Driven Multimodal Pretraining for Cancer Prognosis with Missing Modalities, by Kai Yu and 11 other authors View PDF HTML (experimental) Abstract:Multimodal self-supervised pretraining offers a promising route to cancer prognosis by integrating histopathology whole-slide images, gene expression, and pathology reports, yet most existing approaches require fully paired and complete inputs. In practice, clinical cohorts are fragmented and often miss one or more modalities, limiting both supervised fusion and scalable multimodal pretraining. We propose PRIME, a missing-aware multimodal self-supervised pretraining framework that learns robust and transferable representations from partially observed cohorts. PRIME maps heterogeneous modality embeddings into a unified token space and introduces a shared prototype memory bank for latent-space semantic imputation via patient-level consensus retrieval, producing structurally aligned tokens without reconstructing raw signals. Two complementary pretraining objectives: inter-modality alignment and post-fusion consistency under structured missingness augmentation, j...