[2603.01698] Towards Principled Dataset Distillation: A Spectral Distribution Perspective

Computer Science > Computer Vision and Pattern Recognition arXiv:2603.01698 (cs) [Submitted on 2 Mar 2026] Title:Towards Principled Dataset Distillation: A Spectral Distribution Perspective Authors:Ruixi Wu, Shaobo Wang, Jiahuan Chen, Zhiyuan Liu, Yicun Yang, Zhaorun Chen, Zekai Li, Kaixin Li, Xinming Wang, Hongzhu Yi, Kai Wang, Linfeng Zhang View a PDF of the paper titled Towards Principled Dataset Distillation: A Spectral Distribution Perspective, by Ruixi Wu and Shaobo Wang and Jiahuan Chen and Zhiyuan Liu and Yicun Yang and Zhaorun Chen and Zekai Li and Kaixin Li and Xinming Wang and Hongzhu Yi and Kai Wang and Linfeng Zhang View PDF HTML (experimental) Abstract:Dataset distillation (DD) aims to compress large-scale datasets into compact synthetic counterparts for efficient model training. However, existing DD methods exhibit substantial performance degradation on long-tailed datasets. We identify two fundamental challenges: heuristic design choices for distribution discrepancy measure and uniform treatment of imbalanced classes. To address these limitations, we propose Class-Aware Spectral Distribution Matching (CSDM), which reformulates distribution alignment via the spectrum of a well-behaved kernel function. This technique maps the original samples into frequency space, resulting in the Spectral Distribution Distance (SDD). To mitigate class imbalance, we exploit the unified form of SDD to perform amplitude-phase decomposition, which adaptively prioritizes the real...