[2504.21419] Kernel Density Machines

Statistics > Machine Learning arXiv:2504.21419 (stat) [Submitted on 30 Apr 2025 (v1), last revised 26 Mar 2026 (this version, v3)] Title:Kernel Density Machines Authors:Andrea Della Vecchia, Damir Filipovic, Paul Schneider View a PDF of the paper titled Kernel Density Machines, by Andrea Della Vecchia and Damir Filipovic and Paul Schneider View PDF HTML (experimental) Abstract:We introduce kernel density machines (KDM), an agnostic kernel-based framework for learning the Radon-Nikodym derivative (density) between probability measures under minimal assumptions. KDM applies to general measurable spaces and avoids the structural requirements common in classical nonparametric density estimators. We construct a sample estimator and prove its consistency and a functional central limit theorem. To enable scalability, we develop Nystrom-type low-rank approximations and derive optimal error rates, filling a gap in the literature where such guarantees for density learning have been missing. We demonstrate the versatility of KDM through applications to kernel-based two-sample testing and conditional distribution estimation, the latter enjoying dimension-free guarantees beyond those of locally smoothed methods. Experiments on simulated and real data show that KDM is accurate, scalable, and competitive across a range of tasks. Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Statistics Theory (math.ST) MSC classes: 62G07, 65D05, 65D15, 65C60, 62G10, 62G20 Cite as: arXiv:...