[2603.29232] Long-Document QA with Chain-of-Structured-Thought and Fine-Tuned SLMs

Computer Science > Computation and Language arXiv:2603.29232 (cs) [Submitted on 31 Mar 2026] Title:Long-Document QA with Chain-of-Structured-Thought and Fine-Tuned SLMs Authors:Zhuowen Liang, Xiaotian Lin, Zhengxuan Zhang, Yuyu Luo, Haixun Wang, Nan Tang View a PDF of the paper titled Long-Document QA with Chain-of-Structured-Thought and Fine-Tuned SLMs, by Zhuowen Liang and 5 other authors View PDF HTML (experimental) Abstract:Large language models (LLMs) are widely applied to data analytics over documents, yet direct reasoning over long, noisy documents remains brittle and error-prone. Hence, we study document question answering (QA) that consolidates dispersed evidence into a structured output (e.g., a table, graph, or chunks) to support reliable, verifiable QA. We propose a two-pillar framework, LiteCoST, to achieve both high accuracy and low latency with small language models (SLMs). Pillar 1: Chain-of-Structured-Thought (CoST). We introduce a CoST template, a schema-aware instruction that guides a strong LLM to produce both a step-wise CoST trace and the corresponding structured output. The process induces a minimal structure, normalizes entities/units, aligns records, serializes the output, and verifies/refines it, yielding auditable supervision. Pillar 2: SLM fine-tuning. The compact models are trained on LLM-generated CoST data in two stages: Supervised Fine-Tuning for structural alignment, followed by Group Relative Policy Optimization (GRPO) incorporating triple...