[2507.11539] Streaming 4D Visual Geometry Transformer

Computer Science > Computer Vision and Pattern Recognition arXiv:2507.11539 (cs) [Submitted on 15 Jul 2025 (v1), last revised 31 Mar 2026 (this version, v2)] Title:Streaming 4D Visual Geometry Transformer Authors:Dong Zhuo, Wenzhao Zheng, Jiahe Guo, Yuqi Wu, Jie Zhou, Jiwen Lu View a PDF of the paper titled Streaming 4D Visual Geometry Transformer, by Dong Zhuo and 5 other authors View PDF HTML (experimental) Abstract:Perceiving and reconstructing 3D geometry from videos is a fundamental yet challenging computer vision task. To facilitate interactive and low-latency applications, we propose a streaming visual geometry transformer that shares a similar philosophy with autoregressive large language models. We explore a simple and efficient design and employ a causal transformer architecture to process the input sequence in an online manner. We use temporal causal attention and cache the historical keys and values as implicit memory to enable efficient streaming long-term 3D reconstruction. This design can handle low-latency 3D reconstruction by incrementally integrating historical information while maintaining high-quality spatial consistency. For efficient training, we propose to distill knowledge from the dense bidirectional visual geometry grounded transformer (VGGT) to our causal model. For inference, our model supports the migration of optimized efficient attention operators (e.g., FlashAttention) from large language models. Extensive experiments on various 3D geometry ...