[2604.00342] Is One Token All It Takes? Graph Pooling Tokens for LLM-based GraphQA

Computer Science > Machine Learning arXiv:2604.00342 (cs) [Submitted on 1 Apr 2026] Title:Is One Token All It Takes? Graph Pooling Tokens for LLM-based GraphQA Authors:Ankit Grover, Lodovico Giaretta, Rémi Bourgerie, Sarunas Girdzijauskas View a PDF of the paper titled Is One Token All It Takes? Graph Pooling Tokens for LLM-based GraphQA, by Ankit Grover and 3 other authors View PDF HTML (experimental) Abstract:The integration of Graph Neural Networks (GNNs) with Large Language Models (LLMs) has emerged as a promising paradigm for Graph Question Answering (GraphQA). However, effective methods for encoding complex structural information into the LLM's latent space remain an open challenge. Current state-of-the-art architectures, such as G-Retriever, typically rely on standard GNNs and aggressive mean pooling to compress entire graph substructures into a single token, creating a severe information bottleneck. This work mitigates this bottleneck by investigating two orthogonal strategies: (1) increasing the bandwidth of the graph-to-LLM interface via multi-token pooling, and (2) enhancing the semantic quality of the graph encoder via global attention mechanisms. We evaluate a suite of hierarchical pruning and clustering-based pooling operators including Top-k, SAGPool, DiffPool, MinCutPool, and Virtual Node Pooling (VNPool) to project graph data into multiple learnable tokens. Empirically, we demonstrate that while pooling introduces significant instability during soft prompt...