![[2602.14682] Exposing Diversity Bias in Deep Generative Models: Statistical Origins and Correction of Diversity Error](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.14682] Exposing Diversity Bias in Deep Generative Models: Statistical Origins and Correction of Diversity Error
Summary
This paper investigates the diversity bias in deep generative models, revealing that these models often underestimate the diversity of the underlying data distribution and proposing methods to correct this bias.
Why It Matters
Understanding and correcting diversity bias in generative models is crucial for improving their performance and reliability in various applications. This research highlights the limitations of current models and offers strategies for enhancing their ability to capture data diversity, which is essential for fair and effective AI systems.
Key Takeaways
- Deep generative models often exhibit a systematic downward diversity bias.
- Diversity scores from generated samples are consistently lower than those from actual data.
- Finite sample sizes can lead to underestimating true data diversity.
- Optimizing for empirical data distributions may reduce diversity.
- Diversity-aware regularization strategies can mitigate bias effectively.
Computer Science > Machine Learning arXiv:2602.14682 (cs) [Submitted on 16 Feb 2026] Title:Exposing Diversity Bias in Deep Generative Models: Statistical Origins and Correction of Diversity Error Authors:Farzan Farnia, Mohammad Jalali, Azim Ospanov View a PDF of the paper titled Exposing Diversity Bias in Deep Generative Models: Statistical Origins and Correction of Diversity Error, by Farzan Farnia and 2 other authors View PDF HTML (experimental) Abstract:Deep generative models have achieved great success in producing high-quality samples, making them a central tool across machine learning applications. Beyond sample quality, an important yet less systematically studied question is whether trained generative models faithfully capture the diversity of the underlying data distribution. In this work, we address this question by directly comparing the diversity of samples generated by state-of-the-art models with that of test samples drawn from the target data distribution, using recently proposed reference-free entropy-based diversity scores, Vendi and RKE. Across multiple benchmark datasets, we find that test data consistently attains substantially higher Vendi and RKE diversity scores than the generated samples, suggesting a systematic downward diversity bias in modern generative models. To understand the origin of this bias, we analyze the finite-sample behavior of entropy-based diversity scores and show that their expected values increase with sample size, implying that ...
