[2511.07970] Continual Unlearning for Text-to-Image Diffusion Models: A Regularization Perspective

Computer Science > Machine Learning arXiv:2511.07970 (cs) [Submitted on 11 Nov 2025 (v1), last revised 2 Mar 2026 (this version, v2)] Title:Continual Unlearning for Text-to-Image Diffusion Models: A Regularization Perspective Authors:Justin Lee, Zheda Mai, Jinsu Yoo, Chongyu Fan, Cheng Zhang, Wei-Lun Chao View a PDF of the paper titled Continual Unlearning for Text-to-Image Diffusion Models: A Regularization Perspective, by Justin Lee and 5 other authors View PDF HTML (experimental) Abstract:Machine unlearning--the ability to remove designated concepts from a pre-trained model--has advanced rapidly, particularly for text-to-image diffusion models. However, existing methods typically assume that unlearning requests arrive all at once, whereas in practice they often arrive sequentially. We present the first systematic study of continual unlearning in text-to-image diffusion models and show that popular unlearning methods suffer from rapid utility collapse: after only a few requests, models forget retained knowledge and generate degraded images. We trace this failure to cumulative parameter drift from the pre-training weights and argue that regularization is crucial to addressing it. To this end, we study a suite of add-on regularizers that (1) mitigate drift and (2) remain compatible with existing unlearning methods. Beyond generic regularizers, we show that semantic awareness is essential for preserving concepts close to the unlearning target, and propose a gradient-project...