[2603.28605] Unsafe2Safe: Controllable Image Anonymization for Downstream Utility

Computer Science > Computer Vision and Pattern Recognition arXiv:2603.28605 (cs) [Submitted on 30 Mar 2026] Title:Unsafe2Safe: Controllable Image Anonymization for Downstream Utility Authors:Mih Dinh, SouYoung Jin View a PDF of the paper titled Unsafe2Safe: Controllable Image Anonymization for Downstream Utility, by Mih Dinh and 1 other authors View PDF HTML (experimental) Abstract:Large-scale image datasets frequently contain identifiable or sensitive content, raising privacy risks when training models that may memorize and leak such information. We present Unsafe2Safe, a fully automated pipeline that detects privacy-prone images and rewrites only their sensitive regions using multimodally guided diffusion editing. Unsafe2Safe operates in two stages. Stage 1 uses a vision-language model to (i) inspect images for privacy risks, (ii) generate paired private and public captions that respectively include and omit sensitive attributes, and (iii) prompt a large language model to produce structured, identity-neutral edit instructions conditioned on the public caption. Stage 2 employs instruction-driven diffusion editors to apply these dual textual prompts, producing privacy-safe images that preserve global structure and task-relevant semantics while neutralizing private content. To measure anonymization quality, we introduce a unified evaluation suite covering Quality, Cheating, Privacy, and Utility dimensions. Across MS-COCO, Caltech101, and MIT Indoor67, Unsafe2Safe reduces fa...