![[2602.22240] From Prompts to Performance: Evaluating LLMs for Task-based Parallel Code Generation](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.22240] From Prompts to Performance: Evaluating LLMs for Task-based Parallel Code Generation
Summary
This paper evaluates the performance of Large Language Models (LLMs) in generating task-based parallel code using various input prompts and programming frameworks, revealing strengths and weaknesses in their capabilities.
Why It Matters
As LLMs become increasingly integrated into software development, understanding their effectiveness in generating efficient parallel code is crucial for enhancing high-performance computing applications. This research provides insights into their capabilities and limitations, guiding future developments in LLM-assisted programming.
Key Takeaways
- LLMs show varying abilities in generating parallel code based on input type.
- Different programming frameworks impact the efficiency of LLM-generated solutions.
- The study highlights both strengths and weaknesses of LLMs in handling complex programming tasks.
- Findings can inform future LLM applications in high-performance and scientific computing.
- Understanding LLM performance can enhance development practices in parallel programming.
Computer Science > Programming Languages arXiv:2602.22240 (cs) [Submitted on 24 Feb 2026] Title:From Prompts to Performance: Evaluating LLMs for Task-based Parallel Code Generation Authors:Linus Bantel, Moritz Strack, Alexander Strack, Dirk Pflüger View a PDF of the paper titled From Prompts to Performance: Evaluating LLMs for Task-based Parallel Code Generation, by Linus Bantel and Moritz Strack and Alexander Strack and Dirk Pfl\"uger View PDF HTML (experimental) Abstract:Large Language Models (LLM) show strong abilities in code generation, but their skill in creating efficient parallel programs is less studied. This paper explores how LLMs generate task-based parallel code from three kinds of input prompts: natural language problem descriptions, sequential reference implementations, and parallel pseudo code. We focus on three programming frameworks: OpenMP Tasking, C++ standard parallelism, and the asynchronous many-task runtime HPX. Each framework offers different levels of abstraction and control for task execution. We evaluate LLM-generated solutions for correctness and scalability. Our results reveal both strengths and weaknesses of LLMs with regard to problem complexity and framework. Finally, we discuss what these findings mean for future LLM-assisted development in high-performance and scientific computing. Comments: Subjects: Programming Languages (cs.PL); Artificial Intelligence (cs.AI); Distributed, Parallel, and Cluster Computing (cs.DC) Cite as: arXiv:2602.22...
