[2603.03633] Goal-Driven Risk Assessment for LLM-Powered Systems: A Healthcare Case Study

Computer Science > Cryptography and Security arXiv:2603.03633 (cs) [Submitted on 4 Mar 2026] Title:Goal-Driven Risk Assessment for LLM-Powered Systems: A Healthcare Case Study Authors:Neha Nagaraja, Hayretdin Bahsi View a PDF of the paper titled Goal-Driven Risk Assessment for LLM-Powered Systems: A Healthcare Case Study, by Neha Nagaraja and 1 other authors View PDF HTML (experimental) Abstract:While incorporating LLMs into systems offers significant benefits in critical application areas such as healthcare, new security challenges emerge due to the potential cyber kill chain cycles that combine adversarial model, prompt injection and conventional cyber attacks. Threat modeling methods enable the system designers to identify potential cyber threats and the relevant mitigations during the early stages of development. Although the cyber security community has extensive experience in applying these methods to software-based systems, the elicited threats are usually abstract and vague, limiting their effectiveness for conducting proper likelihood and impact assessments for risk prioritization, especially in complex systems with novel attacks surfaces, such as those involving LLMs. In this study, we propose a structured, goal driven risk assessment approach that contextualizes the threats with detailed attack vectors, preconditions, and attack paths through the use of attack trees. We demonstrate the proposed approach on a case study with an LLM agent-based healthcare system. ...