[2603.03389] Towards Improved Sentence Representations using Token Graphs

Computer Science > Machine Learning arXiv:2603.03389 (cs) [Submitted on 3 Mar 2026] Title:Towards Improved Sentence Representations using Token Graphs Authors:Krishna Sri Ipsit Mantri, Carola-Bibiane Schönlieb, Zorah Lähner, Moshe Eliasof View a PDF of the paper titled Towards Improved Sentence Representations using Token Graphs, by Krishna Sri Ipsit Mantri and 3 other authors View PDF HTML (experimental) Abstract:Obtaining a single-vector representation from a Large Language Model's (LLM) token-level outputs is a critical step for nearly all sentence-level tasks. However, standard pooling methods like mean or max aggregation treat tokens as an independent set, discarding the rich relational structure captured by the model's self-attention layers and making them susceptible to signal dilution. To address this, we introduce GLOT, a lightweight, structure-aware pooling module that reframes pooling as relational learning followed by aggregation. Operating on the outputs of a frozen LLM, GLOT first constructs a latent token-similarity graph, then refines token representations with a graph neural network, and finally aggregates them using a readout layer. Experimentally, our approach is remarkably robust and efficient: on a diagnostic stress test where 90% of tokens are random distractors, GLOT maintains over 97% accuracy while baseline methods collapse. Furthermore, it is competitive with state-of-the-art techniques on benchmarks like GLUE and MTEB with 20x fewer trainable par...