[2603.24946] MobileDev-Bench: A Comprehensive Benchmark for Evaluating Language Models on Mobile Application Development

Computer Science > Software Engineering arXiv:2603.24946 (cs) [Submitted on 26 Mar 2026] Title:MobileDev-Bench: A Comprehensive Benchmark for Evaluating Language Models on Mobile Application Development Authors:Moshood A. Fakorede, Krishna Upadhyay, A.B. Siddique, Umar Farooq View a PDF of the paper titled MobileDev-Bench: A Comprehensive Benchmark for Evaluating Language Models on Mobile Application Development, by Moshood A. Fakorede and 3 other authors View PDF HTML (experimental) Abstract:Large language models (LLMs) have shown strong performance on automated software engineering tasks, yet existing benchmarks focus primarily on general-purpose libraries or web applications, leaving mobile application development largely unexplored despite its strict platform constraints, framework-driven lifecycles, and complex platform API interactions. We introduce MobileDev-Bench, a benchmark comprising 384 real-world issue-resolution tasks collected from 18 production mobile applications spanning Android Native (Java/Kotlin), React Native (TypeScript), and Flutter (Dart). Each task pairs an authentic developer-reported issue with executable test patches, enabling fully automated validation of model-generated fixes within mobile build environments. The benchmark exhibits substantial patch complexity: fixes modify 12.5 files and 324.9 lines on average, and 35.7% of instances require coordinated changes across multiple artifact types, such as source and manifest files. Evaluation of ...