![[2602.16142] Ratio Covers of Convex Sets and Optimal Mixture Density Estimation](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.16142] Ratio Covers of Convex Sets and Optimal Mixture Density Estimation
Summary
This paper explores optimal mixture density estimation using Kullback-Leibler divergence, providing new insights into density estimation with unbounded ratios and convex sets.
Why It Matters
Understanding optimal mixture density estimation is crucial for improving statistical models in machine learning. This research presents innovative covering results that enhance the theoretical framework for density estimation, which can lead to better performance in various applications, including multi-objective optimization.
Key Takeaways
- Introduces optimal high-probability guarantees for density estimation.
- Establishes new covering results for convex sets that have broader implications.
- Demonstrates improved rates for mixture density estimation compared to existing methods.
Mathematics > Statistics Theory arXiv:2602.16142 (math) [Submitted on 18 Feb 2026] Title:Ratio Covers of Convex Sets and Optimal Mixture Density Estimation Authors:Spencer Compton, Gábor Lugosi, Jaouad Mourtada, Jian Qian, Nikita Zhivotovskiy View a PDF of the paper titled Ratio Covers of Convex Sets and Optimal Mixture Density Estimation, by Spencer Compton and 4 other authors View PDF HTML (experimental) Abstract:We study density estimation in Kullback-Leibler divergence: given an i.i.d. sample from an unknown density $p$, the goal is to construct an estimator $\widehat p$ such that $\mathrm{KL}(p,\widehat p)$ is small with high probability. We consider two settings involving a finite dictionary of $M$ densities: (i) model aggregation, where $p$ belongs to the dictionary, and (ii) convex aggregation (mixture density estimation), where $p$ is a mixture of densities from the dictionary. Crucially, we make no assumption on the base densities: their ratios may be unbounded and their supports may differ. For both problems, we identify the best possible high-probability guarantees in terms of the dictionary size, sample size, and confidence level. These optimal rates are higher than those achievable when density ratios are bounded by absolute constants; for mixture density estimation, they match existing lower bounds in the special case of discrete distributions. Our analysis of the mixture case hinges on two new covering results. First, we provide a sharp, distribution-free u...
