[2603.02236] CUDABench: Benchmarking LLMs for Text-to-CUDA Generation

Computer Science > Machine Learning arXiv:2603.02236 (cs) [Submitted on 13 Feb 2026] Title:CUDABench: Benchmarking LLMs for Text-to-CUDA Generation Authors:Jiace Zhu, Wentao Chen, Qi Fan, Zhixing Ren, Junying Wu, Xing Zhe Chai, Chotiwit Rungrueangwutthinon, Yehan Ma, An Zou View a PDF of the paper titled CUDABench: Benchmarking LLMs for Text-to-CUDA Generation, by Jiace Zhu and 8 other authors View PDF HTML (experimental) Abstract:Recent studies have demonstrated the potential of Large Language Models (LLMs) in generating GPU Kernels. Current benchmarks focus on the translation of high-level languages into CUDA, overlooking the more general and challenging task of text-to-CUDA generation. Furthermore, given the hardware-specific and performance-critical features of GPU programming, accurately assessing the performance of LLM-generated GPU programs is nontrivial. In this work, we introduce CUDABench, a comprehensive benchmark designed to evaluate the text-to-CUDA capabilities of LLMs. First, we construct CUDABench-Set, which covers Breadth-Depth-Difficulty evaluation space in diverse application domains, including artificial intelligence, scientific computing, and data analytics, etc. Furthermore, we propose CUDABench-Score and Generative Verification Pipeline that assess (1) compilation correctness, (2) functional consistency through execution-based verification, and (3) a novel roofline-based metric, Performance-Score. Benchmarking state-of-the-art LLMs reveals insightful...