[2602.11199] When and What to Ask: AskBench and Rubric-Guided RLVR for LLM Clarification

Computer Science > Computation and Language arXiv:2602.11199 (cs) [Submitted on 4 Feb 2026 (v1), last revised 20 Apr 2026 (this version, v2)] Title:When and What to Ask: AskBench and Rubric-Guided RLVR for LLM Clarification Authors:Jiale Zhao, Ke Fang, Lu Cheng View a PDF of the paper titled When and What to Ask: AskBench and Rubric-Guided RLVR for LLM Clarification, by Jiale Zhao and 2 other authors View PDF HTML (experimental) Abstract:Large language models (LLMs) often respond even when prompts omit critical details or include misleading information, leading to hallucinations or reinforced misconceptions. We study how to evaluate and improve LLMs' ability to decide when and what to ask for clarification without sacrificing task performance. We introduce AskBench, an interactive benchmark that converts standard QA pairs into multi-turn interactions with explicit checkpoints. A unified judge loop evaluates final answers and simulates user responses as needed. AskBench covers two settings: AskMind, with intent-deficient queries requiring clarification, and AskOverconfidence, with queries containing false premises that must be identified and corrected. We further propose rubric-guided reinforcement learning with verifier-based rewards (RLVR), which uses structured rubrics to encourage targeted clarification. Experiments show consistent improvements in accuracy, rubric adherence, and interaction efficiency, with strong generalization to unseen domains. Subjects: Computation ...