[2604.03338] The Ideation Bottleneck: Decomposing the Quality Gap Between AI-Generated and Human Economics Research

Economics > General Economics arXiv:2604.03338 (econ) [Submitted on 3 Apr 2026] Title:The Ideation Bottleneck: Decomposing the Quality Gap Between AI-Generated and Human Economics Research Authors:Ning Li View a PDF of the paper titled The Ideation Bottleneck: Decomposing the Quality Gap Between AI-Generated and Human Economics Research, by Ning Li View PDF Abstract:Autonomous AI systems can now generate complete economics research papers, but they substantially underperform human-authored publications in head-to-head comparisons. This paper decomposes the quality gap into two independent components: research idea quality and execution quality. Using a two-model ensemble of fine-tuned language models trained on publication decisions (Gong, Li, and Zhou, 2026) to evaluate idea quality and a comprehensive six-dimension rubric assessed by Gemini 3.1 Flash Lite -- the same model family used as the APE tournament judge, ensuring methodological consistency -- to evaluate execution quality, we analyze 953 economics papers -- 912 AI-generated papers from the APE project and 41 human papers published in the American Economic Review and AEJ: Economic Policy. The idea quality gap is large (Cohen's d = 2.23, p < 0.001), with human papers achieving 47.1% mean ensemble exceptional probability versus 16.5% for AI. The execution quality gap is also significant but smaller (d = 0.90, p < 0.001), with human papers scoring 4.38/5.0 versus 3.84. Idea quality accounts for approximately 71% of ...