![[2507.06134] OpenAgentSafety: A Comprehensive Framework for Evaluating Real-World AI Agent Safety](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2507.06134] OpenAgentSafety: A Comprehensive Framework for Evaluating Real-World AI Agent Safety
Summary
OpenAgentSafety introduces a modular framework for evaluating AI agent safety in real-world tasks, addressing critical vulnerabilities in existing benchmarks.
Why It Matters
As AI agents increasingly handle complex tasks in real-world settings, ensuring their safety is paramount. OpenAgentSafety provides a comprehensive evaluation framework that identifies unsafe behaviors, highlighting the need for improved safety measures before deployment. This research is crucial for developers and researchers aiming to enhance AI safety standards.
Key Takeaways
- OpenAgentSafety evaluates AI agents across eight critical risk categories.
- The framework supports real tools and over 350 multi-turn, multi-user tasks.
- Empirical analysis reveals significant safety vulnerabilities in popular LLMs.
- The framework allows for easy extensibility for researchers.
- Combines rule-based analysis with LLM-as-judge assessments for comprehensive safety evaluation.
Computer Science > Artificial Intelligence arXiv:2507.06134 (cs) [Submitted on 8 Jul 2025 (v1), last revised 16 Feb 2026 (this version, v2)] Title:OpenAgentSafety: A Comprehensive Framework for Evaluating Real-World AI Agent Safety Authors:Sanidhya Vijayvargiya, Aditya Bharat Soni, Xuhui Zhou, Zora Zhiruo Wang, Nouha Dziri, Graham Neubig, Maarten Sap View a PDF of the paper titled OpenAgentSafety: A Comprehensive Framework for Evaluating Real-World AI Agent Safety, by Sanidhya Vijayvargiya and 6 other authors View PDF HTML (experimental) Abstract:Recent advances in AI agents capable of solving complex, everyday tasks, from scheduling to customer service, have enabled deployment in real-world settings, but their possibilities for unsafe behavior demands rigorous evaluation. While prior benchmarks have attempted to assess agent safety, most fall short by relying on simulated environments, narrow task domains, or unrealistic tool abstractions. We introduce OpenAgentSafety, a comprehensive and modular framework for evaluating agent behavior across eight critical risk categories. Unlike prior work, our framework evaluates agents that interact with real tools, including web browsers, code execution environments, file systems, bash shells, and messaging platforms; and supports over 350 multi-turn, multi-user tasks spanning both benign and adversarial user intents. OpenAgentSafety is designed for extensibility, allowing researchers to add tools, tasks, websites, and adversarial st...
