![[2602.21036] Empirically Calibrated Conditional Independence Tests](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.21036] Empirically Calibrated Conditional Independence Tests
Summary
The paper presents Empirically Calibrated Conditional Independence Tests (ECCIT), a method designed to enhance the reliability of conditional independence tests in statistical analysis, particularly addressing issues of miscalibration in small and large samples.
Why It Matters
Conditional independence tests are crucial for causal discovery and feature selection in statistics and machine learning. The proposed ECCIT method aims to improve the accuracy of these tests, which is essential for researchers relying on statistical validity in their analyses. By correcting miscalibration, ECCIT offers a more robust framework for obtaining valid results, thereby enhancing the integrity of findings in various applications.
Key Takeaways
- ECCIT addresses common failure modes in conditional independence tests.
- The method provides valid false discovery rate control with higher power.
- ECCIT is test agnostic, making it applicable across various models.
- Empirical benchmarks demonstrate ECCIT's effectiveness over existing strategies.
- The approach is particularly beneficial for small sample sizes and misspecified models.
Statistics > Methodology arXiv:2602.21036 (stat) [Submitted on 24 Feb 2026] Title:Empirically Calibrated Conditional Independence Tests Authors:Milleno Pan, Antoine de Mathelin, Wesley Tansey View a PDF of the paper titled Empirically Calibrated Conditional Independence Tests, by Milleno Pan and 2 other authors View PDF HTML (experimental) Abstract:Conditional independence tests (CIT) are widely used for causal discovery and feature selection. Even with false discovery rate (FDR) control procedures, they often fail to provide frequentist guarantees in practice. We highlight two common failure modes: (i) in small samples, asymptotic guarantees for many CITs can be inaccurate and even correctly specified models fail to estimate the noise levels and control the error, and (ii) when sample sizes are large but models are misspecified, unaccounted dependencies skew the test's behavior and fail to return uniform p-values under the null. We propose Empirically Calibrated Conditional Independence Tests (ECCIT), a method that measures and corrects for miscalibration. For a chosen base CIT (e.g., GCM, HRT), ECCIT optimizes an adversary that selects features and response functions to maximize a miscalibration metric. ECCIT then fits a monotone calibration map that adjusts the base-test p-values in proportion to the observed miscalibration. Across empirical benchmarks on synthetic and real data, ECCIT achieves valid FDR with higher power than existing calibration strategies while remai...
