[2604.09155] CORA: Conformal Risk-Controlled Agents for Safeguarded Mobile GUI Automation

Computer Science > Machine Learning arXiv:2604.09155 (cs) [Submitted on 10 Apr 2026] Title:CORA: Conformal Risk-Controlled Agents for Safeguarded Mobile GUI Automation Authors:Yushi Feng, Junye Du, Qifan Wang, Zizhan Ma, Qian Niu, Yutaka Matsuo, Long Feng, Lequan Yu View a PDF of the paper titled CORA: Conformal Risk-Controlled Agents for Safeguarded Mobile GUI Automation, by Yushi Feng and 7 other authors View PDF HTML (experimental) Abstract:Graphical user interface (GUI) agents powered by vision language models (VLMs) are rapidly moving from passive assistance to autonomous operation. However, this unrestricted action space exposes users to severe and irreversible financial, privacy or social harm. Existing safeguards rely on prompt engineering, brittle heuristics and VLM-as-critic lack formal verification and user-tunable guarantees. We propose CORA (COnformal Risk-controlled GUI Agent), a post-policy, pre-action safeguarding framework that provides statistical guarantees on harmful executed actions. CORA reformulates safety as selective action execution: we train a Guardian model to estimate action-conditional risk for each proposed step. Rather than thresholding raw scores, we leverage Conformal Risk Control to calibrate an execute/abstain boundary that satisfies a user-specified risk budget and route rejected actions to a trainable Diagnostician model, which performs multimodal reasoning over rejected actions to recommend interventions (e.g., confirm, reflect, or ab...