[2603.03318] Quantum-Inspired Self-Attention in a Large Language Model

Computer Science > Computation and Language arXiv:2603.03318 (cs) [Submitted on 9 Feb 2026] Title:Quantum-Inspired Self-Attention in a Large Language Model Authors:Nikita Kuznetsov, Niyaz Ismagilov, Ernesto Campos View a PDF of the paper titled Quantum-Inspired Self-Attention in a Large Language Model, by Nikita Kuznetsov and 2 other authors View PDF HTML (experimental) Abstract:Recent advances in Natural Language Processing have been predominantly driven by transformer-based architectures, which rely heavily on self-attention mechanisms to model relationships between tokens in a sequence. Similarly, the field of Quantum Natural Language Processing, which seeks to leverage quantum principles to address challenges in language understanding and generation tasks, has seen the recent development of quantum self-attention mechanisms. We propose a classical quantum-inspired self-attention (QISA) mechanism and integrate it into the full autoregressive language modeling pipeline of GPT-1. To the best of our knowledge, this is the first integration of this kind, as previous quantum self-attention mechanisms have been primarily tested on text classification. In our experiments, QISA achieves better performance when compared to standard self-attention on the metrics character error rate ($15.5\times$ better), word error rate ($4.7 \times $) and cross-entropy loss ($13 \times$). This is achieved while only requiring a $ 2.6\times$ longer inference time. Comments: Subjects: Computation...