[2603.02174] De-paradox Tree: Breaking Down Simpson's Paradox via A Kernel-Based Partition Algorithm

Computer Science > Machine Learning arXiv:2603.02174 (cs) [Submitted on 2 Mar 2026] Title:De-paradox Tree: Breaking Down Simpson's Paradox via A Kernel-Based Partition Algorithm Authors:Xian Teng, Yu-Ru Lin View a PDF of the paper titled De-paradox Tree: Breaking Down Simpson's Paradox via A Kernel-Based Partition Algorithm, by Xian Teng and 1 other authors View PDF HTML (experimental) Abstract:Real-world observational datasets and machine learning have revolutionized data-driven decision-making, yet many models rely on empirical associations that may be misleading due to confounding and subgroup heterogeneity. Simpson's paradox exemplifies this challenge, where aggregated and subgroup-level associations contradict each other, leading to misleading conclusions. Existing methods provide limited support for detecting and interpreting such paradoxical associations, especially for practitioners without deep causal expertise. We introduce De-paradox Tree, an interpretable algorithm designed to uncover hidden subgroup patterns behind paradoxical associations under assumed causal structures involving confounders and effect heterogeneity. It employs novel split criteria and balancing-based procedures to adjust for confounders and homogenize heterogeneous effects through recursive partitioning. Compared to state-of-the-art methods, De-paradox Tree builds simpler, more interpretable trees, selects relevant covariates, and identifies nested opposite effects while ensuring robust esti...