![[2602.20114] Benchmarking Unlearning for Vision Transformers](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.20114] Benchmarking Unlearning for Vision Transformers
Summary
This article presents a benchmarking study on unlearning algorithms for Vision Transformers (VTs), highlighting their performance compared to CNNs and establishing a reference baseline for future research.
Why It Matters
As machine unlearning becomes essential for developing safe AI, this research fills a gap by evaluating unlearning techniques specifically for Vision Transformers. It provides a foundational benchmark that can guide future studies and applications in AI safety and fairness.
Key Takeaways
- The study benchmarks unlearning algorithms across different Vision Transformer architectures.
- It assesses the impact of dataset scale and complexity on unlearning performance.
- Unified evaluation metrics are introduced to measure forget quality and accuracy.
- The research reveals how Vision Transformers memorize training data compared to CNNs.
- Establishes a reference performance baseline for future unlearning algorithm comparisons.
Computer Science > Computer Vision and Pattern Recognition arXiv:2602.20114 (cs) [Submitted on 23 Feb 2026] Title:Benchmarking Unlearning for Vision Transformers Authors:Kairan Zhao, Iurie Luca, Peter Triantafillou View a PDF of the paper titled Benchmarking Unlearning for Vision Transformers, by Kairan Zhao and 2 other authors View PDF HTML (experimental) Abstract:Research in machine unlearning (MU) has gained strong momentum: MU is now widely regarded as a critical capability for building safe and fair AI. In parallel, research into transformer architectures for computer vision tasks has been highly successful: Increasingly, Vision Transformers (VTs) emerge as strong alternatives to CNNs. Yet, MU research for vision tasks has largely centered on CNNs, not VTs. While benchmarking MU efforts have addressed LLMs, diffusion models, and CNNs, none exist for VTs. This work is the first to attempt this, benchmarking MU algorithm performance in different VT families (ViT and Swin-T) and at different capacities. The work employs (i) different datasets, selected to assess the impacts of dataset scale and complexity; (ii) different MU algorithms, selected to represent fundamentally different approaches for MU; and (iii) both single-shot and continual unlearning protocols. Additionally, it focuses on benchmarking MU algorithms that leverage training data memorization, since leveraging memorization has been recently discovered to significantly improve the performance of previously SO...
