[2505.16017] GradPCA: Leveraging NTK Alignment for Reliable Out-of-Distribution Detection

Computer Science > Machine Learning arXiv:2505.16017 (cs) [Submitted on 21 May 2025 (v1), last revised 28 Feb 2026 (this version, v2)] Title:GradPCA: Leveraging NTK Alignment for Reliable Out-of-Distribution Detection Authors:Mariia Seleznova, Hung-Hsu Chou, Claudio Mayrink Verdun, Gitta Kutyniok View a PDF of the paper titled GradPCA: Leveraging NTK Alignment for Reliable Out-of-Distribution Detection, by Mariia Seleznova and 3 other authors View PDF HTML (experimental) Abstract:We introduce GradPCA, an Out-of-Distribution (OOD) detection method that exploits the low-rank structure of neural network gradients induced by Neural Tangent Kernel (NTK) alignment. GradPCA applies Principal Component Analysis (PCA) to gradient class-means, achieving more consistent performance than existing methods across standard image classification benchmarks. We provide a theoretical perspective on spectral OOD detection in neural networks to support GradPCA, highlighting feature-space properties that enable effective detection and naturally emerge from NTK alignment. Our analysis further reveals that feature quality -- particularly the use of pretrained versus non-pretrained representations -- plays a crucial role in determining which detectors will succeed. Extensive experiments validate the strong performance of GradPCA, and our theoretical framework offers guidance for designing more principled spectral OOD detectors. Subjects: Machine Learning (cs.LG); Computer Vision and Pattern Recogn...