[2406.17297] Towards Camera Open-set 3D Object Detection for Autonomous Driving Scenarios

Computer Science > Computer Vision and Pattern Recognition arXiv:2406.17297 (cs) [Submitted on 25 Jun 2024 (v1), last revised 1 Mar 2026 (this version, v3)] Title:Towards Camera Open-set 3D Object Detection for Autonomous Driving Scenarios Authors:Zhuolin He, Xinrun Li, Jiacheng Tang, Shoumeng Qiu, Wenfu Wang, Xiangyang Xue, Jian Pu View a PDF of the paper titled Towards Camera Open-set 3D Object Detection for Autonomous Driving Scenarios, by Zhuolin He and 6 other authors View PDF HTML (experimental) Abstract:Conventional camera-based 3D object detectors in autonomous driving are limited to recognizing a predefined set of objects, which poses a safety risk when encountering novel or unseen objects in real-world scenarios. To address this limitation, we present OS-Det3D, a two-stage training framework designed for camera-based open-set 3D object detection. In the first stage, our proposed 3D object discovery network (ODN3D) uses geometric cues from LiDAR point clouds to generate class-agnostic 3D object proposals, each of which are assigned a 3D objectness score. This approach allows the network to discover objects beyond known categories, allowing for the detection of unfamiliar objects. However, due to the absence of class constraints, ODN3D-generated proposals may include noisy data, particularly in cluttered or dynamic scenes. To mitigate this issue, we introduce a joint selection (JS) module in the second stage. The JS module uses both camera bird's eye view (BEV) fea...