[2603.20987] Interpreting the Synchronization Gap: The Hidden Mechanism Inside Diffusion Transformers

Computer Science > Machine Learning arXiv:2603.20987 (cs) [Submitted on 22 Mar 2026] Title:Interpreting the Synchronization Gap: The Hidden Mechanism Inside Diffusion Transformers Authors:Emil Albrychiewicz, Andrés Franco Valiente, Li-Ching Chen, Viola Zixin Zhao View a PDF of the paper titled Interpreting the Synchronization Gap: The Hidden Mechanism Inside Diffusion Transformers, by Emil Albrychiewicz and 3 other authors View PDF HTML (experimental) Abstract:Recent theoretical models of diffusion processes, conceptualized as coupled Ornstein-Uhlenbeck systems, predict a hierarchy of interaction timescales, and consequently, the existence of a synchronization gap between modes that commit at different stages of the reverse process. However, because these predictions rely on continuous time and analytically tractable score functions, it remains unclear how this phenomenology manifests in the deep, discrete architectures deployed in practice. In this work, we investigate how the synchronization gap is mechanistically realized within pretrained Diffusion Transformers (DiTs). We construct an explicit architectural realization of replica coupling by embedding two generative trajectories into a joint token sequence, modulated by a symmetric cross attention gate with variable coupling strength g. Through a linearized analysis of the attention difference, we show that the replica interaction decomposes mechanistically. We empirically validate our theoretical framework on a pretra...