[2601.21961] How do Visual Attributes Influence Web Agents? A Comprehensive Evaluation of User Interface Design Factors

Computer Science > Artificial Intelligence arXiv:2601.21961 (cs) [Submitted on 29 Jan 2026 (v1), last revised 27 Feb 2026 (this version, v2)] Title:How do Visual Attributes Influence Web Agents? A Comprehensive Evaluation of User Interface Design Factors Authors:Kuai Yu, Naicheng Yu, Han Wang, Rui Yang, Huan Zhang View a PDF of the paper titled How do Visual Attributes Influence Web Agents? A Comprehensive Evaluation of User Interface Design Factors, by Kuai Yu and 4 other authors View PDF HTML (experimental) Abstract:Web agents have demonstrated strong performance on a wide range of web-based tasks. However, existing research on the effect of environmental variation has mostly focused on robustness to adversarial attacks, with less attention to agents' preferences in benign scenarios. Although early studies have examined how textual attributes influence agent behavior, a systematic understanding of how visual attributes shape agent decision-making remains limited. To address this, we introduce VAF, a controlled evaluation pipeline for quantifying how webpage Visual Attribute Factors influence web-agent decision-making. Specifically, VAF consists of three stages: (i) variant generation, which ensures the variants share identical semantics as the original item while only differ in visual attributes; (ii) browsing interaction, where agents navigate the page via scrolling and clicking the interested item, mirroring how human users browse online; (iii) validating through both ...