![[2602.18473] Decentralized Attention Fails Centralized Signals: Rethinking Transformers for Medical Time Series](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.18473] Decentralized Attention Fails Centralized Signals: Rethinking Transformers for Medical Time Series
Summary
This article presents a novel approach to analyzing medical time series data using a centralized attention mechanism, addressing limitations of traditional Transformer models in capturing channel dependencies.
Why It Matters
As healthcare increasingly relies on accurate analysis of medical time series data for diagnostics, improving the efficiency and effectiveness of these analyses is crucial. The proposed CoTAR method enhances model performance while reducing computational demands, making it significant for both researchers and practitioners in the field.
Key Takeaways
- Traditional Transformer models struggle with channel dependencies in medical time series data.
- The CoTAR method introduces a centralized aggregation strategy to improve model performance.
- Experiments show a significant improvement in effectiveness and efficiency over previous models.
- The new approach reduces computational complexity from quadratic to linear.
- Code and training scripts are publicly available for further research and application.
Computer Science > Machine Learning arXiv:2602.18473 (cs) [Submitted on 9 Feb 2026] Title:Decentralized Attention Fails Centralized Signals: Rethinking Transformers for Medical Time Series Authors:Guoqi Yu, Juncheng Wang, Chen Yang, Jing Qin, Angelica I. Aviles-Rivero, Shujun Wang View a PDF of the paper titled Decentralized Attention Fails Centralized Signals: Rethinking Transformers for Medical Time Series, by Guoqi Yu and 5 other authors View PDF HTML (experimental) Abstract:Accurate analysis of medical time series (MedTS) data, such as electroencephalography (EEG) and electrocardiography (ECG), plays a pivotal role in healthcare applications, including the diagnosis of brain and heart diseases. MedTS data typically exhibit two critical patterns: temporal dependencies within individual channels and channel dependencies across multiple channels. While recent advances in deep learning have leveraged Transformer-based models to effectively capture temporal dependencies, they often struggle with modeling channel dependencies. This limitation stems from a structural mismatch: MedTS signals are inherently centralized, whereas the Transformer's attention mechanism is decentralized, making it less effective at capturing global synchronization and unified waveform patterns. To address this mismatch, we propose CoTAR (Core Token Aggregation-Redistribution), a centralized MLP-based module designed to replace decentralized attention. Instead of allowing all tokens to interact direc...
