[2603.19643] OmniDiT: Extending Diffusion Transformer to Omni-VTON Framework

Computer Science > Computer Vision and Pattern Recognition arXiv:2603.19643 (cs) [Submitted on 20 Mar 2026] Title:OmniDiT: Extending Diffusion Transformer to Omni-VTON Framework Authors:Weixuan Zeng, Pengcheng Wei, Huaiqing Wang, Boheng Zhang, Jia Sun, Dewen Fan, Lin HE, Long Chen, Qianqian Gan, Fan Yang, Tingting Gao View a PDF of the paper titled OmniDiT: Extending Diffusion Transformer to Omni-VTON Framework, by Weixuan Zeng and 10 other authors View PDF HTML (experimental) Abstract:Despite the rapid advancement of Virtual Try-On (VTON) and Try-Off (VTOFF) technologies, existing VTON methods face challenges with fine-grained detail preservation, generalization to complex scenes, complicated pipeline, and efficient inference. To tackle these problems, we propose OmniDiT, an omni Virtual Try-On framework based on the Diffusion Transformer, which combines try-on and try-off tasks into one unified model. Specifically, we first establish a self-evolving data curation pipeline to continuously produce data, and construct a large VTON dataset Omni-TryOn, which contains over 380k diverse and high-quality garment-model-tryon image pairs and detailed text prompts. Then, we employ the token concatenation and design an adaptive position encoding to effectively incorporate multiple reference conditions. To relieve the bottleneck of long sequence computation, we are the first to introduce Shifted Window Attention into the diffusion model, thus achieving a linear complexity. To remedy ...