![[2602.13359] The Speed-up Factor: A Quantitative Multi-Iteration Active Learning Performance Metric](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.13359] The Speed-up Factor: A Quantitative Multi-Iteration Active Learning Performance Metric
Summary
This article introduces the Speed-up Factor, a new performance metric for evaluating multi-iteration active learning methods, demonstrating its effectiveness against traditional metrics.
Why It Matters
Active learning is crucial in machine learning for optimizing data annotation efficiency. This study provides a novel metric that enhances the evaluation of active learning strategies, potentially improving model performance and resource allocation in various applications.
Key Takeaways
- Introduces the Speed-up Factor as a new metric for active learning evaluation.
- Demonstrates the metric's accuracy and stability across multiple iterations.
- Compares the Speed-up Factor with existing state-of-the-art metrics.
- Utilizes diverse datasets to validate the metric's effectiveness.
- Highlights the importance of efficient annotation in machine learning.
Computer Science > Machine Learning arXiv:2602.13359 (cs) [Submitted on 13 Feb 2026] Title:The Speed-up Factor: A Quantitative Multi-Iteration Active Learning Performance Metric Authors:Hannes Kath, Thiago S. Gouvêa, Daniel Sonntag View a PDF of the paper titled The Speed-up Factor: A Quantitative Multi-Iteration Active Learning Performance Metric, by Hannes Kath and 2 other authors View PDF HTML (experimental) Abstract:Machine learning models excel with abundant annotated data, but annotation is often costly and time-intensive. Active learning (AL) aims to improve the performance-to-annotation ratio by using query methods (QMs) to iteratively select the most informative samples. While AL research focuses mainly on QM development, the evaluation of this iterative process lacks appropriate performance metrics. This work reviews eight years of AL evaluation literature and formally introduces the speed-up factor, a quantitative multi-iteration QM performance metric that indicates the fraction of samples needed to match random sampling performance. Using four datasets from diverse domains and seven QMs of various types, we empirically evaluate the speed-up factor and compare it with state-of-the-art AL performance metrics. The results confirm the assumptions underlying the speed-up factor, demonstrate its accuracy in capturing the described fraction, and reveal its superior stability across iterations. Subjects: Machine Learning (cs.LG) Cite as: arXiv:2602.13359 [cs.LG] (or ...
