[2507.06867] Conformal Prediction for Long-Tailed Classification

Statistics > Machine Learning arXiv:2507.06867 (stat) [Submitted on 9 Jul 2025 (v1), last revised 26 Feb 2026 (this version, v3)] Title:Conformal Prediction for Long-Tailed Classification Authors:Tiffany Ding, Jean-Baptiste Fermanian, Joseph Salmon View a PDF of the paper titled Conformal Prediction for Long-Tailed Classification, by Tiffany Ding and 2 other authors View PDF HTML (experimental) Abstract:Many real-world classification problems, such as plant identification, have extremely long-tailed class distributions. In order for prediction sets to be useful in such settings, they should (i) provide good class-conditional coverage, ensuring that rare classes are not systematically omitted from the prediction sets, and (ii) be a reasonable size, allowing users to easily verify candidate labels. Unfortunately, existing conformal prediction methods, when applied to the long-tailed setting, force practitioners to make a binary choice between small sets with poor class-conditional coverage or sets that have very good class-conditional coverage but are extremely large. We propose methods with marginal coverage guarantees that smoothly trade off set size and class-conditional coverage. First, we introduce a new conformal score function called prevalence-adjusted softmax that optimizes for macro-coverage, defined as the average class-conditional coverage across classes. Second, we propose a new procedure that interpolates between marginal and class-conditional conformal predict...