[2512.08937] When AI Gives Advice: Evaluating AI and Human Responses to Online Advice-Seeking for Well-Being

Computer Science > Human-Computer Interaction arXiv:2512.08937 (cs) [Submitted on 24 Oct 2025 (v1), last revised 28 Feb 2026 (this version, v2)] Title:When AI Gives Advice: Evaluating AI and Human Responses to Online Advice-Seeking for Well-Being Authors:Harsh Kumar, Jasmine Chahal, Yinuo Zhao, Zeling Zhang, Annika Wei, Louis Tay, Ashton Anderson View a PDF of the paper titled When AI Gives Advice: Evaluating AI and Human Responses to Online Advice-Seeking for Well-Being, by Harsh Kumar and 6 other authors View PDF HTML (experimental) Abstract:Seeking advice is a core human behavior that the internet has reinvented twice: first through forums and Q&A communities that crowdsource public guidance, and now through large language models (LLMs). Yet the quality of this LLM advice for everyday well-being scenarios remains unclear. How does it compare, not only against human comments, but against the wisdom of the online crowd? We ran two studies (N=210) in which experts compared top-voted Reddit advice with LLM-generated advice. LLMs ranked significantly higher overall and on effectiveness, warmth, and willingness to seek advice again. GPT-4o beat GPT-5 on all metrics except sycophancy, suggesting that benchmark gains need not improve advice-giving. In Study-2, we examined how human and algorithmic advice could be combined, and found that human advice can be unobtrusively polished to compete with AI-generated comments. We conclude with design implications for advice-giving agent...