[2603.25499] Knowledge-Guided Failure Prediction: Detecting When Object Detectors Miss Safety-Critical Objects

Computer Science > Computer Vision and Pattern Recognition arXiv:2603.25499 (cs) [Submitted on 26 Mar 2026] Title:Knowledge-Guided Failure Prediction: Detecting When Object Detectors Miss Safety-Critical Objects Authors:Jakob Paul Zimmermann, Gerrit Holzbach, David Lerch View a PDF of the paper titled Knowledge-Guided Failure Prediction: Detecting When Object Detectors Miss Safety-Critical Objects, by Jakob Paul Zimmermann and 2 other authors View PDF HTML (experimental) Abstract:Object detectors deployed in safety-critical environments can fail silently, e.g. missing pedestrians, workers, or other safety-critical objects without emitting any warning. Traditional Out Of Distribution (OOD) detection methods focus on identifying unfamiliar inputs, but do not directly predict functional failures of the detector itself. We introduce Knowledge Guided Failure Prediction (KGFP), a representation-based monitoring framework that treats missed safety-critical detections as anomalies to be detected at runtime. KGFP measures semantic misalignment between internal object detector features and visual foundation model embeddings using a dual-encoder architecture with an angular distance metric. A key property is that when either the detector is operating outside its competence or the visual foundation model itself encounters novel inputs, the two embeddings diverge, producing a high-angle signal that reliably flags unsafe images. We compare our novel KGFS method to baseline OOD detection...