![[2510.08091] Everything is Plausible: Investigating the Impact of LLM Rationales on Human Notions of Plausibility](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2510.08091] Everything is Plausible: Investigating the Impact of LLM Rationales on Human Notions of Plausibility
Summary
This article explores how rationales generated by large language models (LLMs) influence human judgments of plausibility in commonsense reasoning tasks, revealing significant impacts on belief formation.
Why It Matters
Understanding the influence of LLM-generated rationales on human cognition is crucial as it highlights potential biases in decision-making processes. This research raises awareness about the persuasive power of AI in domains where humans consider themselves experts, prompting discussions on AI ethics and safety.
Key Takeaways
- LLM-generated rationales significantly affect human plausibility judgments.
- The study collected over 16,000 plausibility judgments from humans and LLMs.
- Findings suggest that even expert domains are susceptible to LLM influence.
- The research opens avenues for further exploration of human cognition with AI.
- Practical implications include the need for caution in AI applications affecting human beliefs.
Computer Science > Computation and Language arXiv:2510.08091 (cs) [Submitted on 9 Oct 2025 (v1), last revised 23 Feb 2026 (this version, v2)] Title:Everything is Plausible: Investigating the Impact of LLM Rationales on Human Notions of Plausibility Authors:Shramay Palta, Peter Rankel, Sarah Wiegreffe, Rachel Rudinger View a PDF of the paper titled Everything is Plausible: Investigating the Impact of LLM Rationales on Human Notions of Plausibility, by Shramay Palta and 3 other authors View PDF HTML (experimental) Abstract:We investigate the degree to which human plausibility judgments of multiple-choice commonsense benchmark answers are subject to influence by (im)plausibility arguments for or against an answer, in particular, using rationales generated by LLMs. We collect 3,000 plausibility judgments from humans and another 13,600 judgments from LLMs. Overall, we observe increases and decreases in mean human plausibility ratings in the presence of LLM-generated PRO and CON rationales, respectively, suggesting that, on the whole, human judges find these rationales convincing. Experiments with LLMs reveal similar patterns of influence. Our findings demonstrate a novel use of LLMs for studying aspects of human cognition, while also raising practical concerns that, even in domains where humans are ``experts'' (i.e., common sense), LLMs have the potential to exert considerable influence on people's beliefs. Comments: Subjects: Computation and Language (cs.CL); Artificial Intell...