[2604.00222] Risk-Aware Batch Testing for Performance Regression Detection

Computer Science > Software Engineering arXiv:2604.00222 (cs) [Submitted on 31 Mar 2026] Title:Risk-Aware Batch Testing for Performance Regression Detection Authors:Ali Sayedsalehi, Peter C. Rigby, Gregory Mierzwinski View a PDF of the paper titled Risk-Aware Batch Testing for Performance Regression Detection, by Ali Sayedsalehi and 2 other authors View PDF HTML (experimental) Abstract:Performance regression testing is essential in large-scale continuous-integration (CI) systems, yet executing full performance suites for every commit is prohibitively expensive. Prior work on performance regression prediction and batch testing has shown independent benefits, but each faces practical limitations: predictive models are rarely integrated into CI decision-making, and conventional batching strategies ignore commit-level heterogeneity. We unify these strands by introducing a risk-aware framework that integrates machine-learned commit risk with adaptive batching. Using Mozilla Firefox as a case study, we construct a production-derived dataset of human-confirmed regressions aligned chronologically with Autoland, and fine-tune ModernBERT, CodeBERT, and LLaMA-3.1 variants to estimate commit-level performance regression risk, achieving up to 0.694 ROC-AUC with CodeBERT. The risk scores drive a family of risk-aware batching strategies, including Risk-Aged Priority Batching and Risk-Adaptive Stream Batching, evaluated through realistic CI simulations. Across thousands of historical Fire...