[2603.02601] AgentAssay: Token-Efficient Regression Testing for Non-Deterministic AI Agent Workflows

Computer Science > Artificial Intelligence arXiv:2603.02601 (cs) [Submitted on 3 Mar 2026] Title:AgentAssay: Token-Efficient Regression Testing for Non-Deterministic AI Agent Workflows Authors:Varun Pratap Bhardwaj View a PDF of the paper titled AgentAssay: Token-Efficient Regression Testing for Non-Deterministic AI Agent Workflows, by Varun Pratap Bhardwaj View PDF HTML (experimental) Abstract:Autonomous AI agents are deployed at unprecedented scale, yet no principled methodology exists for verifying that an agent has not regressed after changes to its prompts, tools, models, or orchestration logic. We present AgentAssay, the first token-efficient framework for regression testing non-deterministic AI agent workflows, achieving 78-100% cost reduction while maintaining rigorous statistical guarantees. Our contributions include: (1) stochastic three-valued verdicts (PASS/FAIL/INCONCLUSIVE) grounded in hypothesis testing; (2) five-dimensional agent coverage metrics; (3) agent-specific mutation testing operators; (4) metamorphic relations for agent workflows; (5) CI/CD deployment gates as statistical decision procedures; (6) behavioral fingerprinting that maps execution traces to compact vectors, enabling multivariate regression detection; (7) adaptive budget optimization calibrating trial counts to behavioral variance; and (8) trace-first offline analysis enabling zero-cost testing on production traces. Experiments across 5 models (GPT-5.2, Claude Sonnet 4.6, Mistral-Large-3,...