[2604.01833] Language-Pretraining-Induced Bias: A Strong Foundation for General Vision Tasks

Computer Science > Computer Vision and Pattern Recognition arXiv:2604.01833 (cs) [Submitted on 2 Apr 2026] Title:Language-Pretraining-Induced Bias: A Strong Foundation for General Vision Tasks Authors:Yaxin Luo, Zhiqiang Shen View a PDF of the paper titled Language-Pretraining-Induced Bias: A Strong Foundation for General Vision Tasks, by Yaxin Luo and 1 other authors View PDF HTML (experimental) Abstract:The ratio of outlier parameters in language pre-training models and vision pre-training models differs significantly, making cross-modality (language and vision) inherently more challenging than cross-domain adaptation. As a result, many prior studies have focused on cross-domain transfer rather than attempting to bridge language and vision modalities, assuming that language pre-trained models are unsuitable for downstream visual tasks due to disparate parameter spaces. Contrary to this assumption, we show that adding a bridge training stage as a modality adaptation learner can effectively align Large Language Model (LLM) parameters with vision tasks. Specifically, we propose a simple yet powerful solution random label bridge training that requires no manual labeling and helps LLM parameters adapt to vision foundation tasks. Moreover, our findings reveal that partial bridge training is often advantageous, as certain layers in LLMs exhibit strong foundational properties that remain beneficial even without fine-tuning for visual tasks. This surprising discovery opens up new...