![[2602.20130] To Reason or Not to: Selective Chain-of-Thought in Medical Question Answering](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.20130] To Reason or Not to: Selective Chain-of-Thought in Medical Question Answering
Summary
The paper presents Selective Chain-of-Thought (Selective CoT), a method to enhance medical question answering efficiency using large language models by selectively applying reasoning based on question complexity.
Why It Matters
This research addresses the challenge of balancing reasoning depth and computational efficiency in medical question answering systems. By optimizing when to apply reasoning, it enhances the practicality of LLMs in clinical settings, potentially improving patient care through faster and more accurate responses.
Key Takeaways
- Selective CoT reduces inference time by 13-45% and token usage by 8-47%.
- It maintains accuracy with minimal loss (≤4%) compared to traditional methods.
- The approach is model-agnostic and cost-effective, enhancing real-world deployability.
- Selective reasoning improves efficiency for recall-type questions.
- The method aligns reasoning efforts with question complexity, optimizing resource use.
Computer Science > Computation and Language arXiv:2602.20130 (cs) [Submitted on 23 Feb 2026] Title:To Reason or Not to: Selective Chain-of-Thought in Medical Question Answering Authors:Zaifu Zhan, Min Zeng, Shuang Zhou, Yiran Song, Xiaoyi Chen, Yu Hou, Yifan Wu, Yang Ruan, Rui Zhang View a PDF of the paper titled To Reason or Not to: Selective Chain-of-Thought in Medical Question Answering, by Zaifu Zhan and 8 other authors View PDF HTML (experimental) Abstract:Objective: To improve the efficiency of medical question answering (MedQA) with large language models (LLMs) by avoiding unnecessary reasoning while maintaining accuracy. Methods: We propose Selective Chain-of-Thought (Selective CoT), an inference-time strategy that first predicts whether a question requires reasoning and generates a rationale only when needed. Two open-source LLMs (Llama-3.1-8B and Qwen-2.5-7B) were evaluated on four biomedical QA benchmarks-HeadQA, MedQA-USMLE, MedMCQA, and PubMedQA. Metrics included accuracy, total generated tokens, and inference time. Results: Selective CoT reduced inference time by 13-45% and token usage by 8-47% with minimal accuracy loss ($\leq$4\%). In some model-task pairs, it achieved both higher accuracy and greater efficiency than standard CoT. Compared with fixed-length CoT, Selective CoT reached similar or superior accuracy at substantially lower computational cost. Discussion: Selective CoT dynamically balances reasoning depth and efficiency by invoking explicit reaso...
