![[2601.04205] STaRR: Spatial-Temporal Token-Dynamics-Aware Responsive Remasking for Diffusion Language Models](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2601.04205] STaRR: Spatial-Temporal Token-Dynamics-Aware Responsive Remasking for Diffusion Language Models
Summary
The paper presents STaRR, a novel framework for responsive remasking in diffusion language models that adapts remasking decisions based on token confidence dynamics, achieving significant speed improvements without sacrificing accuracy.
Why It Matters
As diffusion language models become increasingly prevalent in AI applications, optimizing their performance is crucial. STaRR addresses the limitations of static remasking strategies by introducing a dynamic approach that enhances both speed and accuracy, making it relevant for researchers and practitioners in the field of machine learning and natural language processing.
Key Takeaways
- STaRR introduces a dynamic remasking strategy based on token confidence evolution.
- The framework achieves an average speedup of 4.1 times and up to 8.9 times in processing.
- Two new metrics, temporal variance and spatial deviance, guide the remasking process.
- The approach is training-free, enhancing scalability and robustness.
- Maintains comparable accuracy while improving inference speed.
Computer Science > Computation and Language arXiv:2601.04205 (cs) [Submitted on 7 Dec 2025 (v1), last revised 21 Feb 2026 (this version, v2)] Title:STaRR: Spatial-Temporal Token-Dynamics-Aware Responsive Remasking for Diffusion Language Models Authors:Xinhao Sun, Huaijin Zhao, Maoliang Li, Zihao Zheng, Jiayu Chen, Yun Liang, Xiang Chen View a PDF of the paper titled STaRR: Spatial-Temporal Token-Dynamics-Aware Responsive Remasking for Diffusion Language Models, by Xinhao Sun and 5 other authors View PDF HTML (experimental) Abstract:Diffusion Language Models (DLMs) enable parallel decoding via iterative denoising, where remasking strategies play a critical role in balancing inference speed and output quality. Existing methods predominantly rely on static confidence thresholds, overlooking the spatial-temporal dynamics of token confidence, causing unnecessary remasking. We propose Spatial-Temporal Token-Dynamics-Aware Responsive Remasking (STaRR), a training-free framework that dynamically adapts remasking decisions based on token confidence evolution. STaRR introduces two metrics, temporal variance and spatial deviance, to guide fine-grained, step-wise dynamic thresholding. We further introduce a step-wise dynamic thresholding strategy, further enhanced with responsiveness optimizations for scalability and robustness. Experiments show that STaRR achieves an average speedup of 4.1 and up to 8.9 while maintaining comparable accuracy. Subjects: Computation and Language (cs.CL)...
