![[2602.22953] General Agent Evaluation](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.22953] General Agent Evaluation
Summary
This paper introduces a framework for evaluating general-purpose agents, proposing a Unified Protocol and Exgentic framework, and benchmarks five agents across diverse environments, establishing a foundation for systematic research in this area.
Why It Matters
As general-purpose agents become more prevalent, a standardized evaluation method is crucial for understanding their capabilities and limitations. This research addresses the gap in systematic assessments, paving the way for advancements in AI that can operate effectively in unfamiliar environments.
Key Takeaways
- Current agent benchmarks are limited to domain-specific tasks.
- The paper proposes a Unified Protocol for fair evaluation of general agents.
- Five prominent agents were benchmarked, showing generalization across environments.
- The Exgentic framework facilitates practical evaluation of agent performance.
- A new Open General Agent Leaderboard was established to track advancements.
Computer Science > Artificial Intelligence arXiv:2602.22953 (cs) [Submitted on 26 Feb 2026] Title:General Agent Evaluation Authors:Elron Bandel, Asaf Yehudai, Lilach Eden, Yehoshua Sagron, Yotam Perlitz, Elad Venezian, Natalia Razinkov, Natan Ergas, Shlomit Shachor Ifergan, Segev Shlomov, Michal Jacovi, Leshem Choshen, Liat Ein-Dor, Yoav Katz, Michal Shmueli-Scheuer View a PDF of the paper titled General Agent Evaluation, by Elron Bandel and 14 other authors View PDF HTML (experimental) Abstract:The promise of general-purpose agents - systems that perform tasks in unfamiliar environments without domain-specific engineering - remains largely unrealized. Existing agents are predominantly specialized, and while emerging implementations like OpenAI SDK Agent and Claude Code hint at broader capabilities, no systematic evaluation of their general performance has been pursued. Current agentic benchmarks assume domain-specific integration, encoding task information in ways that preclude fair evaluation of general agents. This paper frames general-agent evaluation as a first-class research objective. We propose conceptual principles for such evaluation, a Unified Protocol enabling agent-benchmark integration, and Exgentic - a practical framework for general agent evaluation. We benchmark five prominent agent implementations across six environments as the first Open General Agent Leaderboard. Our experiments show that general agents generalize across diverse environments, achieving ...
