[2601.08258] Diagnosing and Mitigating Sycophancy and Skepticism in LLM Causal Judgment

Computer Science > Artificial Intelligence arXiv:2601.08258 (cs) [Submitted on 13 Jan 2026 (v1), last revised 8 Apr 2026 (this version, v3)] Title:Diagnosing and Mitigating Sycophancy and Skepticism in LLM Causal Judgment Authors:Edward Y. Chang View a PDF of the paper titled Diagnosing and Mitigating Sycophancy and Skepticism in LLM Causal Judgment, by Edward Y. Chang View PDF HTML (experimental) Abstract:Large language models increasingly fail in a way that scalar accuracy cannot diagnose: they produce a sound reasoning trace and then abandon it under social pressure or an authoritative hint. We argue that this is a control failure, not a knowledge failure, and that it requires an evaluation surface richer than a single accuracy number. We introduce CAUSALT3, a 454 instance expert curated benchmark for causal reasoning across all three rungs of Pearl's ladder, and a three axis evaluation that decomposes performance into Utility (sensitivity to valid causal claims), Safety (specificity against invalid ones), and Wise Refusal (calibrated abstention on genuinely underdetermined items). On this surface we document three reproducible pathologies: a Skepticism Trap at L1 where capable models over refuse sound links, a Sycophancy Trap at L2 where confident user pressure flips correct answers, and a Scaling Paradox at L3 where a frontier model underperforms an older one on counterfactual Safety by 55 points. To mitigate these failures without retraining, we propose Regulated Cau...