![[2602.05354] PATHWAYS: Evaluating Investigation and Context Discovery in AI Web Agents](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.05354] PATHWAYS: Evaluating Investigation and Context Discovery in AI Web Agents
Summary
The paper introduces PATHWAYS, a benchmark assessing AI web agents' ability to discover and utilize hidden contextual information in multi-step decision tasks, revealing significant performance limitations.
Why It Matters
Understanding the limitations of AI web agents in context discovery is crucial for improving their reliability and effectiveness in real-world applications. This research highlights the challenges faced by current architectures, which is vital for future developments in AI systems.
Key Takeaways
- PATHWAYS benchmark consists of 250 multi-step decision tasks for AI agents.
- Current web agents struggle to retrieve and integrate hidden contextual information.
- Performance declines sharply when tasks involve misleading signals.
- Explicit instructions can improve context discovery but may reduce overall accuracy.
- The findings indicate a need for better mechanisms in AI for adaptive investigation and evidence integration.
Computer Science > Artificial Intelligence arXiv:2602.05354 (cs) [Submitted on 5 Feb 2026 (v1), last revised 16 Feb 2026 (this version, v2)] Title:PATHWAYS: Evaluating Investigation and Context Discovery in AI Web Agents Authors:Shifat E. Arman, Syed Nazmus Sakib, Tapodhir Karmakar Taton, Nafiul Haque, Shahrear Bin Amin View a PDF of the paper titled PATHWAYS: Evaluating Investigation and Context Discovery in AI Web Agents, by Shifat E. Arman and 4 other authors View PDF HTML (experimental) Abstract:We introduce PATHWAYS, a benchmark of 250 multi-step decision tasks that test whether web-based agents can discover and correctly use hidden contextual information. Across both closed and open models, agents typically navigate to relevant pages but retrieve decisive hidden evidence in only a small fraction of cases. When tasks require overturning misleading surface-level signals, performance drops sharply to near chance accuracy. Agents frequently hallucinate investigative reasoning by claiming to rely on evidence they never accessed. Even when correct context is discovered, agents often fail to integrate it into their final decision. Providing more explicit instructions improves context discovery but often reduces overall accuracy, revealing a tradeoff between procedural compliance and effective judgement. Together, these results show that current web agent architectures lack reliable mechanisms for adaptive investigation, evidence integration, and judgement override. Comments...
