[2603.20839] Dodgersort: Uncertainty-Aware VLM-Guided Human-in-the-Loop Pairwise Ranking

Computer Science > Computer Vision and Pattern Recognition arXiv:2603.20839 (cs) [Submitted on 21 Mar 2026] Title:Dodgersort: Uncertainty-Aware VLM-Guided Human-in-the-Loop Pairwise Ranking Authors:Yujin Park, Haejun Chung, Ikbeom Jang View a PDF of the paper titled Dodgersort: Uncertainty-Aware VLM-Guided Human-in-the-Loop Pairwise Ranking, by Yujin Park and Haejun Chung and Ikbeom Jang View PDF HTML (experimental) Abstract:Pairwise comparison labeling is emerging as it yields higher inter-rater reliability than conventional classification labeling, but exhaustive comparisons require quadratic cost. We propose Dodgersort, which leverages CLIP-based hierarchical pre-ordering, a neural ranking head and probabilistic ensemble (Elo, BTL, GP), epistemic--aleatoric uncertainty decomposition, and information-theoretic pair selection. It reduces human comparisons while improving the reliability of the rankings. In visual ranking tasks in medical imaging, historical dating, and aesthetics, Dodgersort achieves a 11--16\% annotation reduction while improving inter-rater reliability. Cross-domain ablations across four datasets show that neural adaptation and ensemble uncertainty are key to this gain. In FG-NET with ground-truth ages, the framework extracts 5--20$\times$ more ranking information per comparison than baselines, yielding Pareto-optimal accuracy--efficiency trade-offs. Comments: Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Hu...