[2603.04334] SpotIt+: Verification-based Text-to-SQL Evaluation with Database Constraints

Computer Science > Databases arXiv:2603.04334 (cs) [Submitted on 4 Mar 2026] Title:SpotIt+: Verification-based Text-to-SQL Evaluation with Database Constraints Authors:Rocky Klopfenstein, Yang He, Andrew Tremante, Yuepeng Wang, Nina Narodytska, Haoze Wu View a PDF of the paper titled SpotIt+: Verification-based Text-to-SQL Evaluation with Database Constraints, by Rocky Klopfenstein and 5 other authors View PDF HTML (experimental) Abstract:We present SpotIt+, an open-source tool for evaluating Text-to-SQL systems via bounded equivalence verification. Given a generated SQL query and the ground truth, SpotIt+ actively searches for database instances that differentiate the two queries. To ensure that the generated counterexamples reflect practically relevant discrepancies, we introduce a constraint-mining pipeline that combines rule-based specification mining over example databases with LLM-based validation. Experimental results on the BIRD dataset show that the mined constraints enable SpotIt+ to generate more realistic differentiating databases, while preserving its ability to efficiently uncover numerous discrepancies between generated and gold SQL queries that are missed by standard test-based evaluation. Subjects: Databases (cs.DB); Artificial Intelligence (cs.AI); Logic in Computer Science (cs.LO); Programming Languages (cs.PL) Cite as: arXiv:2603.04334 [cs.DB]   (or arXiv:2603.04334v1 [cs.DB] for this version)   https://doi.org/10.48550/arXiv.2603.04334 Focus to learn m...