![[2602.21472] The Design Space of Tri-Modal Masked Diffusion Models](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.21472] The Design Space of Tri-Modal Masked Diffusion Models
Summary
This paper introduces the first tri-modal masked diffusion model, pretrained on text, image-text, and audio-text data, analyzing its performance and optimization strategies.
Why It Matters
The research addresses the growing demand for advanced multimodal AI models, providing insights into scaling behaviors and optimization techniques that can enhance the performance of generative models across various modalities. This work is significant for researchers and practitioners in machine learning and AI development.
Key Takeaways
- Introduces a tri-modal masked diffusion model for text, image-text, and audio-text data.
- Analyzes multimodal scaling laws and provides optimized inference sampling defaults.
- Presents a novel stochastic differential equation-based reparameterization for batch size optimization.
- Demonstrates strong performance in text generation, text-to-image, and text-to-speech tasks.
- Represents a large-scale systematic study of multimodal discrete diffusion models.
Computer Science > Machine Learning arXiv:2602.21472 (cs) [Submitted on 25 Feb 2026] Title:The Design Space of Tri-Modal Masked Diffusion Models Authors:Louis Bethune, Victor Turrisi, Bruno Kacper Mlodozeniec, Pau Rodriguez Lopez, Lokesh Boominathan, Nikhil Bhendawade, Amitis Shidani, Joris Pelemans, Theo X. Olausson, Devon Hjelm, Paul Dixon, Joao Monteiro, Pierre Ablin, Vishnu Banna, Arno Blaas, Nick Henderson, Kari Noriy, Dan Busbridge, Josh Susskind, Marco Cuturi, Irina Belousova, Luca Zappella, Russ Webb, Jason Ramapuram View a PDF of the paper titled The Design Space of Tri-Modal Masked Diffusion Models, by Louis Bethune and 22 other authors View PDF HTML (experimental) Abstract:Discrete diffusion models have emerged as strong alternatives to autoregressive language models, with recent work initializing and fine-tuning a base unimodal model for bimodal generation. Diverging from previous approaches, we introduce the first tri-modal masked diffusion model pretrained from scratch on text, image-text, and audio-text data. We systematically analyze multimodal scaling laws, modality mixing ratios, noise schedules, and batch-size effects, and we provide optimized inference sampling defaults. Our batch-size analysis yields a novel stochastic differential equation (SDE)-based reparameterization that eliminates the need for tuning the optimal batch size as reported in recent work. This reparameterization decouples the physical batch size, often chosen based on compute constrai...
