[2603.20976] Detection of adversarial intent in Human-AI teams using LLMs

Computer Science > Machine Learning arXiv:2603.20976 (cs) [Submitted on 21 Mar 2026] Title:Detection of adversarial intent in Human-AI teams using LLMs Authors:Abed K. Musaffar, Ambuj Singh, Francesco Bullo View a PDF of the paper titled Detection of adversarial intent in Human-AI teams using LLMs, by Abed K. Musaffar and 2 other authors View PDF Abstract:Large language models (LLMs) are increasingly deployed in human-AI teams as support agents for complex tasks such as information retrieval, programming, and decision-making assistance. While these agents' autonomy and contextual knowledge enables them to be useful, it also exposes them to a broad range of attacks, including data poisoning, prompt injection, and even prompt engineering. Through these attack vectors, malicious actors can manipulate an LLM agent to provide harmful information, potentially manipulating human agents to make harmful decisions. While prior work has focused on LLMs as attack targets or adversarial actors, this paper studies their potential role as defensive supervisors within mixed human-AI teams. Using a dataset consisting of multi-party conversations and decisions for a real human-AI team over a 25 round horizon, we formulate the problem of malicious behavior detection from interaction traces. We find that LLMs are capable of identifying malicious behavior in real-time, and without task-specific information, indicating the potential for task-agnostic defense. Moreover, we find that the maliciou...