[2602.24245] Chunk-wise Attention Transducers for Fast and Accurate Streaming Speech-to-Text

Computer Science > Machine Learning arXiv:2602.24245 (cs) [Submitted on 27 Feb 2026] Title:Chunk-wise Attention Transducers for Fast and Accurate Streaming Speech-to-Text Authors:Hainan Xu, Vladimir Bataev, Travis M. Bartley, Jagadeesh Balam View a PDF of the paper titled Chunk-wise Attention Transducers for Fast and Accurate Streaming Speech-to-Text, by Hainan Xu and 3 other authors View PDF HTML (experimental) Abstract:We propose Chunk-wise Attention Transducer (CHAT), a novel extension to RNN-T models that processes audio in fixed-size chunks while employing cross-attention within each chunk. This hybrid approach maintains RNN-T's streaming capability while introducing controlled flexibility for local alignment modeling. CHAT significantly reduces the temporal dimension that RNN-T must handle, yielding substantial efficiency improvements: up to 46.2% reduction in peak training memory, up to 1.36X faster training, and up to 1.69X faster inference. Alongside these efficiency gains, CHAT achieves consistent accuracy improvements over RNN-T across multiple languages and tasks -- up to 6.3% relative WER reduction for speech recognition and up to 18.0% BLEU improvement for speech translation. The method proves particularly effective for speech translation, where RNN-T's strict monotonic alignment hurts performance. Our results demonstrate that the CHAT model offers a practical solution for deploying more capable streaming speech models without sacrificing real-time constraint...