[2603.22750] REALITrees: Rashomon Ensemble Active Learning for Interpretable Trees

Statistics > Machine Learning arXiv:2603.22750 (stat) [Submitted on 24 Mar 2026] Title:REALITrees: Rashomon Ensemble Active Learning for Interpretable Trees Authors:Simon D. Nguyen, Hayden McTavish, Kentaro Hoffman, Cynthia Rudin, Tyler H. McCormick View a PDF of the paper titled REALITrees: Rashomon Ensemble Active Learning for Interpretable Trees, by Simon D. Nguyen and 4 other authors View PDF HTML (experimental) Abstract:Active learning reduces labeling costs by selecting samples that maximize information gain. A dominant framework, Query-by-Committee (QBC), typically relies on perturbation-based diversity by inducing model disagreement through random feature subsetting or data blinding. While this approximates one notion of epistemic uncertainty, it sacrifices direct characterization of the plausible hypothesis space. We propose the complementary approach: Rashomon Ensembled Active Learning (REAL) which constructs a committee by exhaustively enumerating the Rashomon Set of all near-optimal models. To address functional redundancy within this set, we adopt a PAC-Bayesian framework using a Gibbs posterior to weight committee members by their empirical risk. Leveraging recent algorithmic advances, we exactly enumerate this set for the class of sparse decision trees. Across synthetic and established active learning baselines, REAL outperforms randomized ensembles, particularly in moderately noisy environments where it strategically leverages expanded model multiplicity to...