[2604.07034] KITE: Keyframe-Indexed Tokenized Evidence for VLM-Based Robot Failure Analysis

Computer Science > Robotics arXiv:2604.07034 (cs) [Submitted on 8 Apr 2026] Title:KITE: Keyframe-Indexed Tokenized Evidence for VLM-Based Robot Failure Analysis Authors:Mehdi Hosseinzadeh, King Hang Wong, Feras Dayoub View a PDF of the paper titled KITE: Keyframe-Indexed Tokenized Evidence for VLM-Based Robot Failure Analysis, by Mehdi Hosseinzadeh and 2 other authors View PDF HTML (experimental) Abstract:We present KITE, a training-free, keyframe-anchored, layout-grounded front-end that converts long robot-execution videos into compact, interpretable tokenized evidence for vision-language models (VLMs). KITE distills each trajectory into a small set of motion-salient keyframes with open-vocabulary detections and pairs each keyframe with a schematic bird's-eye-view (BEV) representation that encodes relative object layout, axes, timestamps, and detection confidence. These visual cues are serialized with robot-profile and scene-context tokens into a unified prompt, allowing the same front-end to support failure detection, identification, localization, explanation, and correction with an off-the-shelf VLM. On the RoboFAC benchmark, KITE with Qwen2.5-VL substantially improves over vanilla Qwen2.5-VL in the training-free setting, with especially large gains on simulation failure detection, identification, and localization, while remaining competitive with a RoboFAC-tuned baseline. A small QLoRA fine-tune further improves explanation and correction quality. We also report qualit...