[2411.05183] Why CNN Features Are not Gaussian: A Statistical Anatomy of Deep Representations

Computer Science > Computer Vision and Pattern Recognition arXiv:2411.05183 (cs) [Submitted on 7 Nov 2024 (v1), last revised 6 Apr 2026 (this version, v4)] Title:Why CNN Features Are not Gaussian: A Statistical Anatomy of Deep Representations Authors:David Chapman, Parniyan Farvardin View a PDF of the paper titled Why CNN Features Are not Gaussian: A Statistical Anatomy of Deep Representations, by David Chapman and 1 other authors View PDF HTML (experimental) Abstract:Deep convolutional neural networks (CNNs) are commonly analyzed through geometric and linear-algebraic perspectives, yet the statistical distribution of their internal feature activations remains poorly understood. In many applications, deep features are implicitly treated as Gaussian when modeling densities. In this work, we empirically examine this assumption and show that it does not accurately describe the distribution of CNN feature activations. Through a systematic study across multiple architectures and datasets, we find that the feature activations deviate substantially from Gaussian and are better characterized by Weibull and related long-tailed distributions. We further introduce a novel Discretized Characteristic Function Copula (DCF-Copula) method to model multivariate feature dependencies. We find that tail-length increases with network depth and that upper-tail dependence emerges between feature pairs. These statistical findings are not consistent with the Central Limit Theorem, and are instead ...