[2603.18413] Statistical Testing Framework for Clustering Pipelines by Selective Inference

Statistics > Machine Learning arXiv:2603.18413 (stat) [Submitted on 19 Mar 2026 (v1), last revised 23 Mar 2026 (this version, v2)] Title:Statistical Testing Framework for Clustering Pipelines by Selective Inference Authors:Yugo Miyata, Tomohiro Shiraishi, Shunichi Nishino, Ichiro Takeuchi View a PDF of the paper titled Statistical Testing Framework for Clustering Pipelines by Selective Inference, by Yugo Miyata and 3 other authors View PDF HTML (experimental) Abstract:A data analysis pipeline is a structured sequence of steps that transforms raw data into meaningful insights by integrating multiple analysis algorithms. In many practical applications, analytical findings are obtained only after data pass through several data-dependent procedures within such pipelines. In this study, we address the problem of quantifying the statistical reliability of results produced by data analysis pipelines. As a proof of concept, we focus on clustering pipelines that identify cluster structures from complex and heterogeneous data through procedures such as outlier detection, feature selection, and clustering. We propose a novel statistical testing framework to assess the significance of clustering results obtained through these pipelines. Our framework, based on selective inference, enables the systematic construction of valid statistical tests for clustering pipelines composed of predefined components. We prove that the proposed test controls the type I error rate at any nominal level ...