[2603.26804] The Language of Touch: Translating Vibrations into Text with Dual-Branch Learning

Computer Science > Computer Vision and Pattern Recognition arXiv:2603.26804 (cs) [Submitted on 26 Mar 2026] Title:The Language of Touch: Translating Vibrations into Text with Dual-Branch Learning Authors:Jin Chen, Yifeng Lin, Chao Zeng, Si Wu, Tiesong Zhao View a PDF of the paper titled The Language of Touch: Translating Vibrations into Text with Dual-Branch Learning, by Jin Chen and 4 other authors View PDF HTML (experimental) Abstract:The standardization of vibrotactile data by IEEE P1918.1 workgroup has greatly advanced its applications in virtual reality, human-computer interaction and embodied artificial intelligence. Despite these efforts, the semantic interpretation and understanding of vibrotactile signals remain an unresolved challenge. In this paper, we make the first attempt to address vibrotactile captioning, {\it i.e.}, generating natural language descriptions from vibrotactile signals. We propose Vibrotactile Periodic-Aperiodic Captioning (ViPAC), a method designed to handle the intrinsic properties of vibrotactile data, including hybrid periodic-aperiodic structures and the lack of spatial semantics. Specifically, ViPAC employs a dual-branch strategy to disentangle periodic and aperiodic components, combined with a dynamic fusion mechanism that adaptively integrates signal features. It also introduces an orthogonality constraint and weighting regularization to ensure feature complementarity and fusion consistency. Additionally, we construct LMT108-CAP, the f...