[2510.03721] Person-Centric Annotations of LAION-400M: Auditing Bias and Its Transfer to Models

Computer Science > Computer Vision and Pattern Recognition arXiv:2510.03721 (cs) [Submitted on 4 Oct 2025 (v1), last revised 29 Mar 2026 (this version, v2)] Title:Person-Centric Annotations of LAION-400M: Auditing Bias and Its Transfer to Models Authors:Leander Girrbach, Stephan Alaniz, Genevieve Smith, Trevor Darrell, Zeynep Akata View a PDF of the paper titled Person-Centric Annotations of LAION-400M: Auditing Bias and Its Transfer to Models, by Leander Girrbach and Stephan Alaniz and Genevieve Smith and Trevor Darrell and Zeynep Akata View PDF HTML (experimental) Abstract:Vision-language models trained on large-scale multimodal datasets show strong demographic biases, but the role of training data in producing these biases remains unclear. A major barrier has been the lack of demographic annotations in web-scale datasets such as LAION-400M. We address this gap by creating person-centric annotations for the full dataset, including over 276 million bounding boxes, perceived gender and race/ethnicity labels, and automatically generated captions. These annotations are produced through validated automatic labeling pipelines combining object detection, multimodal captioning, and finetuned classifiers. Using them, we uncover demographic imbalances and harmful associations, such as the disproportionate linking of men and individuals perceived as Black or Middle Eastern with crime-related and negative content. We also show that a linear fit predicts 60-70% of gender bias in CLIP...