[2502.10600] Weighted quantization using MMD: From mean field to mean shift via gradient flows

Statistics > Machine Learning arXiv:2502.10600 (stat) [Submitted on 14 Feb 2025 (v1), last revised 1 Apr 2026 (this version, v3)] Title:Weighted quantization using MMD: From mean field to mean shift via gradient flows Authors:Ayoub Belhadji, Daniel Sharp, Youssef Marzouk View a PDF of the paper titled Weighted quantization using MMD: From mean field to mean shift via gradient flows, by Ayoub Belhadji and 2 other authors View PDF Abstract:Approximating a probability distribution using a set of particles is a fundamental problem in machine learning and statistics, with applications including clustering and quantization. Formally, we seek a weighted mixture of Dirac measures that best approximates the target distribution. While much existing work relies on the Wasserstein distance to quantify approximation errors, maximum mean discrepancy (MMD) has received comparatively less attention, especially when allowing for variable particle weights. We argue that a Wasserstein-Fisher-Rao gradient flow is well-suited for designing quantizations optimal under MMD. We show that a system of interacting particles satisfying a set of ODEs discretizes this flow. We further derive a new fixed-point algorithm called mean shift interacting particles (MSIP). We show that MSIP extends the classical mean shift algorithm, widely used for identifying modes in kernel density estimators. Moreover, we show that MSIP can be interpreted as preconditioned gradient descent and that it acts as a relaxation...