[2602.24288] DARE-bench: Evaluating Modeling and Instruction Fidelity of LLMs in Data Science

Computer Science > Artificial Intelligence arXiv:2602.24288 (cs) [Submitted on 27 Feb 2026] Title:DARE-bench: Evaluating Modeling and Instruction Fidelity of LLMs in Data Science Authors:Fan Shu, Yite Wang, Ruofan Wu, Boyi Liu, Zhewei Yao, Yuxiong He, Feng Yan View a PDF of the paper titled DARE-bench: Evaluating Modeling and Instruction Fidelity of LLMs in Data Science, by Fan Shu and 6 other authors View PDF HTML (experimental) Abstract:The fast-growing demands in using Large Language Models (LLMs) to tackle complex multi-step data science tasks create an emergent need for accurate benchmarking. There are two major gaps in existing benchmarks: (i) the lack of standardized, process-aware evaluation that captures instruction adherence and process fidelity, and (ii) the scarcity of accurately labeled training data. To bridge these gaps, we introduce DARE-bench, a benchmark designed for machine learning modeling and data science instruction following. Unlike many existing benchmarks that rely on human- or model-based judges, all tasks in DARE-bench have verifiable ground truth, ensuring objective and reproducible evaluation. To cover a broad range of tasks and support agentic tools, DARE-bench consists of 6,300 Kaggle-derived tasks and provides both large-scale training data and evaluation sets. Extensive evaluations show that even highly capable models such as gpt-o4-mini struggle to achieve good performance, especially in machine learning modeling tasks. Using DARE-bench t...