![[2602.22278] RETLLM: Training and Data-Free MLLMs for Multimodal Information Retrieval](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.22278] RETLLM: Training and Data-Free MLLMs for Multimodal Information Retrieval
Summary
The paper presents RETLLM, a novel framework for multimodal information retrieval (MMIR) that operates without the need for training or large datasets, enhancing retrieval performance through a unique querying approach.
Why It Matters
This research addresses the limitations of existing multimodal large language models (MLLMs) that require extensive pre-training and large datasets. By demonstrating effective MMIR capabilities without these requirements, it opens new avenues for efficient information retrieval in various applications, making it relevant for researchers and practitioners in AI and machine learning.
Key Takeaways
- RETLLM enables multimodal information retrieval without training or large datasets.
- The framework uses a coarse-then-fine pipeline for effective querying.
- A visual enhancement module improves retrieval by re-picking forgotten visuals.
- Extensive experiments show RETLLM outperforms traditional fine-tuned models.
- The research highlights the inherent multimodal reasoning ability of MLLMs.
Computer Science > Information Retrieval arXiv:2602.22278 (cs) [Submitted on 25 Feb 2026] Title:RETLLM: Training and Data-Free MLLMs for Multimodal Information Retrieval Authors:Dawei Su, Dongsheng Wang View a PDF of the paper titled RETLLM: Training and Data-Free MLLMs for Multimodal Information Retrieval, by Dawei Su and 1 other authors View PDF HTML (experimental) Abstract:Multimodal information retrieval (MMIR) has gained attention for its flexibility in handling text, images, or mixed queries and candidates. Recent breakthroughs in multimodal large language models (MLLMs) boost MMIR performance by incorporating MLLM knowledge under the contrastive finetuning framework. However, they suffer from pre-training inconsistency and require large datasets. In this work, we introduce a novel framework, RetLLM, designed to query MLLMs for MMIR in a training- and data-free manner. Specifically, we formulate MMIR as a similarity score generation task and prompt MLLMs to directly predict retrieval scores in a coarse-then-fine pipeline. At the coarse stage, a top-k filtering strategy builds a small yet high-quality candidate pool for each query, enabling MLLMs to focus on semantically relevant candidates. Subsequently, the retrieval score is predicted by feeding both the query and candidate into MLLMs at the fine stage. Importantly, we propose a visual enhancement module during reasoning to help MLLMs re-pick forgotten visuals, improving retrieval. Extensive experiments on MMIR ben...