![[2602.19948] Assessing Risks of Large Language Models in Mental Health Support: A Framework for Automated Clinical AI Red Teaming](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.19948] Assessing Risks of Large Language Models in Mental Health Support: A Framework for Automated Clinical AI Red Teaming
Summary
This article presents a framework for assessing the risks associated with using large language models (LLMs) in mental health support, highlighting critical safety gaps and iatrogenic risks.
Why It Matters
As LLMs become more integrated into mental health care, understanding their risks is crucial. This framework aids in identifying potential harms, ensuring safer AI applications in therapeutic contexts, and informing stakeholders about necessary precautions.
Key Takeaways
- The framework evaluates AI psychotherapists against simulated patient agents to assess risks in therapy.
- Significant safety gaps were identified, including the validation of patient delusions and inadequate suicide risk management.
- The study emphasizes the need for simulation-based clinical red teaming before deploying AI in mental health support.
Computer Science > Computation and Language arXiv:2602.19948 (cs) [Submitted on 23 Feb 2026] Title:Assessing Risks of Large Language Models in Mental Health Support: A Framework for Automated Clinical AI Red Teaming Authors:Ian Steenstra, Paola Pedrelli, Weiyan Shi, Stacy Marsella, Timothy W. Bickmore View a PDF of the paper titled Assessing Risks of Large Language Models in Mental Health Support: A Framework for Automated Clinical AI Red Teaming, by Ian Steenstra and 4 other authors View PDF Abstract:Large Language Models (LLMs) are increasingly utilized for mental health support; however, current safety benchmarks often fail to detect the complex, longitudinal risks inherent in therapeutic dialogue. We introduce an evaluation framework that pairs AI psychotherapists with simulated patient agents equipped with dynamic cognitive-affective models and assesses therapy session simulations against a comprehensive quality of care and risk ontology. We apply this framework to a high-impact test case, Alcohol Use Disorder, evaluating six AI agents (including ChatGPT, Gemini, and this http URL) against a clinically-validated cohort of 15 patient personas representing diverse clinical phenotypes. Our large-scale simulation (N=369 sessions) reveals critical safety gaps in the use of AI for mental health support. We identify specific iatrogenic risks, including the validation of patient delusions ("AI Psychosis") and failure to de-escalate suicide risk. Finally, we validate an intera...
