[2603.04325] Scalable Evaluation of the Realism of Synthetic Environmental Augmentations in Images

Computer Science > Computer Vision and Pattern Recognition arXiv:2603.04325 (cs) [Submitted on 4 Mar 2026] Title:Scalable Evaluation of the Realism of Synthetic Environmental Augmentations in Images Authors:Damian J. Ruck, Paul Vautravers, Oliver Chalkley, Jake Thomas View a PDF of the paper titled Scalable Evaluation of the Realism of Synthetic Environmental Augmentations in Images, by Damian J. Ruck and 3 other authors View PDF HTML (experimental) Abstract:Evaluation of AI systems often requires synthetic test cases, particularly for rare or safety-critical conditions that are difficult to observe in operational data. Generative AI offers a promising approach for producing such data through controllable image editing, but its usefulness depends on whether the resulting images are sufficiently realistic to support meaningful evaluation. We present a scalable framework for assessing the realism of synthetic image-editing methods and apply it to the task of adding environmental conditions-fog, rain, snow, and nighttime-to car-mounted camera images. Using 40 clear-day images, we compare rule-based augmentation libraries with generative AI image-editing models. Realism is evaluated using two complementary automated metrics: a vision-language model (VLM) jury for perceptual realism assessment, and embedding-based distributional analysis to measure similarity to genuine adverse-condition imagery. Generative AI methods substantially outperform rule-based approaches, with the bes...