[2602.24138] Multimodal Optimal Transport for Unsupervised Temporal Segmentation in Surgical Robotics

Computer Science > Computer Vision and Pattern Recognition arXiv:2602.24138 (cs) [Submitted on 27 Feb 2026] Title:Multimodal Optimal Transport for Unsupervised Temporal Segmentation in Surgical Robotics Authors:Omar Mohamed, Edoardo Fazzari, Ayah Al-Naji, Hamdan Alhadhrami, Khalfan Hableel, Saif Alkindi, Cesare Stefanini View a PDF of the paper titled Multimodal Optimal Transport for Unsupervised Temporal Segmentation in Surgical Robotics, by Omar Mohamed and 6 other authors View PDF HTML (experimental) Abstract:Recognizing surgical phases and steps from video is a fundamental problem in computer-assisted interventions. Recent approaches increasingly rely on large-scale pre-training on thousands of labeled surgical videos, followed by zero-shot transfer to specific procedures. While effective, this strategy incurs substantial computational and data collection costs. In this work, we question whether such heavy pre-training is truly necessary. We propose Text-Augmented Action Segmentation Optimal Transport (TASOT), an unsupervised method for surgical phase and step recognition that extends Action Segmentation Optimal Transport (ASOT) by incorporating textual information generated directly from the videos. TASOT formulates temporal action segmentation as a multimodal optimal transport problem, where the matching cost is defined as a weighted combination of visual and text-based costs. The visual term captures frame-level appearance similarity, while the text term provides co...