[2604.04717] The Infinite-Dimensional Nature of Spectroscopy and Why Models Succeed, Fail, and Mislead

Computer Science > Machine Learning arXiv:2604.04717 (cs) [Submitted on 6 Apr 2026] Title:The Infinite-Dimensional Nature of Spectroscopy and Why Models Succeed, Fail, and Mislead Authors:Umberto Michelucci, Francesca Venturini View a PDF of the paper titled The Infinite-Dimensional Nature of Spectroscopy and Why Models Succeed, Fail, and Mislead, by Umberto Michelucci and 1 other authors View PDF HTML (experimental) Abstract:Machine learning (ML) models have achieved strikingly high accuracies in spectroscopic classification tasks, often without a clear proof that those models used chemically meaningful features. Existing studies have linked these results to data preprocessing choices, noise sensitivity, and model complexity, but no unifying explanation is available so far. In this work, we show that these phenomena arise naturally from the intrinsic high dimensionality of spectral data. Using a theoretical analysis grounded in the Feldman-Hajek theorem and the concentration of measure, we show that even infinitesimal distributional differences, caused by noise, normalisation, or instrumental artefacts, may become perfectly separable in high-dimensional spaces. Through a series of specific experiments on synthetic and real fluorescence spectra, we illustrate how models can achieve near-perfect accuracy even when chemical distinctions are absent, and why feature-importance maps may highlight spectrally irrelevant regions. We provide a rigorous theoretical framework, confir...