[2603.20246] Decoding the decoder: Contextual sequence-to-sequence modeling for intracortical speech decoding

Computer Science > Computation and Language arXiv:2603.20246 (cs) [Submitted on 10 Mar 2026] Title:Decoding the decoder: Contextual sequence-to-sequence modeling for intracortical speech decoding Authors:Michal Olak, Tommaso Boccato, Matteo Ferrante View a PDF of the paper titled Decoding the decoder: Contextual sequence-to-sequence modeling for intracortical speech decoding, by Michal Olak and 2 other authors View PDF HTML (experimental) Abstract:Speech brain--computer interfaces require decoders that translate intracortical activity into linguistic output while remaining robust to limited data and day-to-day variability. While prior high-performing systems have largely relied on framewise phoneme decoding combined with downstream language models, it remains unclear what contextual sequence-to-sequence decoding contributes to sublexical neural readout, robustness, and interpretability. We evaluated a multitask Transformer-based sequence-to-sequence model for attempted speech decoding from area 6v intracortical recordings. The model jointly predicts phoneme sequences, word sequences, and auxiliary acoustic features. To address day-to-day nonstationarity, we introduced the Neural Hammer Scalpel (NHS) calibration module, which combines global alignment with feature-wise modulation. We further analyzed held-out-day generalization and attention patterns in the encoder and decoders. On the Willett et al. dataset, the proposed model achieved a state-of-the-art phoneme error rate...