![[2511.10855] ExPairT-LLM: Exact Learning for LLM Code Selection by Pairwise Queries](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2511.10855] ExPairT-LLM: Exact Learning for LLM Code Selection by Pairwise Queries
Summary
The paper presents ExPairT-LLM, an innovative algorithm for code selection in LLMs that improves accuracy by using pairwise queries, outperforming existing methods significantly.
Why It Matters
As code generation remains a challenging task for large language models (LLMs), ExPairT-LLM addresses critical shortcomings in current code selection algorithms. By introducing pairwise queries, it enhances the reliability of code selection, which is essential for developers and researchers working with AI-driven coding tools.
Key Takeaways
- ExPairT-LLM uses pairwise membership and equivalence queries for better code selection.
- The algorithm shows a pass@1 success rate improvement of up to 27.1% over state-of-the-art methods.
- It enhances the performance of LLMs in complex reasoning tasks by 24.0%.
- The approach is robust against some errors made by LLMs, increasing reliability.
- Evaluation was conducted on four popular code datasets, demonstrating practical applicability.
Computer Science > Machine Learning arXiv:2511.10855 (cs) [Submitted on 13 Nov 2025 (v1), last revised 20 Feb 2026 (this version, v3)] Title:ExPairT-LLM: Exact Learning for LLM Code Selection by Pairwise Queries Authors:Tom Yuviler, Dana Drachsler-Cohen View a PDF of the paper titled ExPairT-LLM: Exact Learning for LLM Code Selection by Pairwise Queries, by Tom Yuviler and Dana Drachsler-Cohen View PDF HTML (experimental) Abstract:Despite recent advances in LLMs, the task of code generation is still challenging. To cope, code selection algorithms select the best program from multiple programs generated by an LLM. However, existing algorithms can fail to identify the correct program, either because they can misidentify nonequivalent programs or because they rely on an LLM and assume it always correctly determines the output for every input. We present ExPairT-LLM, an exact learning algorithm for code selection that selects a program by posing to an LLM oracle two new types of queries: pairwise membership and pairwise equivalence. These queries are simpler for LLMs and enable ExPairT-LLM to identify the correct program through a tournament, which is robust to some LLM mistakes. We evaluate ExPairT-LLM on four popular code datasets. Its pass@1 (success rate) outperforms the state-of-the-art code selection algorithm on average by +13.0% and up to +27.1%. It also improves the pass@1 of LLMs performing complex reasoning by +24.0%. Subjects: Machine Learning (cs.LG) Cite as: arXi...
