[2603.13239] Benchmarking Zero-Shot Reasoning Approaches for Error Detection in Solidity Smart Contracts

Computer Science > Artificial Intelligence arXiv:2603.13239 (cs) [Submitted on 17 Feb 2026 (v1), last revised 20 Mar 2026 (this version, v2)] Title:Benchmarking Zero-Shot Reasoning Approaches for Error Detection in Solidity Smart Contracts Authors:Eduardo Sardenberg, Antonio José Grandson Busson, Daniel de Sousa Moraes, Julio Cesar Duarte, Sérgio Colcher View a PDF of the paper titled Benchmarking Zero-Shot Reasoning Approaches for Error Detection in Solidity Smart Contracts, by Eduardo Sardenberg and 4 other authors View PDF HTML (experimental) Abstract:Smart contracts play a central role in blockchain systems by encoding financial and operational logic. Still, their susceptibility to subtle security flaws poses significant risks of financial loss and erosion of trust. LLMs create new opportunities for automating vulnerability detection, yet the effectiveness of different prompting strategies and model choices in real-world contexts remains uncertain. This paper evaluates state-of-the-art LLMs on Solidity smart contract analysis using a balanced dataset of 400 contracts under two tasks: (i) Error Detection, where the model performs binary classification to decide whether a contract is vulnerable, and (ii) Error Classification, where the model must assign the predicted issue to a specific vulnerability category. Models are evaluated using zero-shot prompting strategies, including zero-shot, zero-shot Chain-of-Thought (CoT), and zero-shot Tree-of-Thought (ToT). In the Error...