![[2602.13611] From What to How: Bridging User Requirements with Software Development Using Large Language Models](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.13611] From What to How: Bridging User Requirements with Software Development Using Large Language Models
Summary
This paper explores the limitations of large language models (LLMs) in software design and code generation, proposing a new benchmark called DesBench to evaluate their capabilities.
Why It Matters
As LLMs become integral to software development, understanding their strengths and weaknesses in design tasks is crucial. This research highlights significant gaps in current LLM capabilities, emphasizing the need for improved methodologies in software design that leverage AI effectively.
Key Takeaways
- LLMs struggle with software design complexities, impacting code generation.
- The proposed DesBench benchmark evaluates LLMs on design-aware tasks.
- Acceptance test cases generated by LLMs show comparable quality to human-written tests.
Computer Science > Software Engineering arXiv:2602.13611 (cs) [Submitted on 14 Feb 2026] Title:From What to How: Bridging User Requirements with Software Development Using Large Language Models Authors:Xiao He, Ru Chen, Jialun Cao View a PDF of the paper titled From What to How: Bridging User Requirements with Software Development Using Large Language Models, by Xiao He and 2 other authors View PDF HTML (experimental) Abstract:Recently, large language models (LLMs) are extensively utilized to enhance development efficiency, leading to numerous benchmarks for evaluating their performance. However, these benchmarks predominantly focus on implementation, overlooking the equally critical aspect of software design. This gap raises two pivotal questions: (1) Can LLMs handle software design? (2) Can LLMs write code following the specific designs? To investigate these questions, this paper proposes DesBench, a design-aware benchmark for evaluating LLMs on three software design-related tasks: design-aware code generation, object-oriented modeling, and the design of acceptance test cases. DesBench comprises 30 manually crafted Java projects that include requirement documents, design models, implementations, and acceptance tests, amounting to a total of 30 design models, 194 Java classes, and 737 test cases. We evaluated seven state-of-the-art LLMs, including three DeepSeek R1, two Qwen2.5, and two GPT models, using DesBench. The results reveal that LLMs remain significantly challeng...
