![[2602.19219] Controlled Face Manipulation and Synthesis for Data Augmentation](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.19219] Controlled Face Manipulation and Synthesis for Data Augmentation
Summary
The paper presents a novel method for controlled face manipulation to augment data for facial expression analysis, addressing label scarcity and class imbalance in deep learning models.
Why It Matters
This research is significant as it tackles the challenge of data scarcity in facial expression analysis, a critical area in computer vision. By improving data augmentation techniques, the study enhances model training efficiency and accuracy, which can lead to better applications in areas like emotion recognition and human-computer interaction.
Key Takeaways
- Introduces a method for manipulating facial expressions in a controlled manner.
- Addresses issues of label scarcity and class imbalance in training datasets.
- Utilizes dependency-aware conditioning to reduce feature entanglement.
- Demonstrates improved accuracy in AU detection with generated data.
- Outperforms existing data-efficient training strategies with fewer artifacts.
Computer Science > Computer Vision and Pattern Recognition arXiv:2602.19219 (cs) [Submitted on 22 Feb 2026] Title:Controlled Face Manipulation and Synthesis for Data Augmentation Authors:Joris Kirchner, Amogh Gudi, Marian Bittner, Chirag Raman View a PDF of the paper titled Controlled Face Manipulation and Synthesis for Data Augmentation, by Joris Kirchner and 3 other authors View PDF HTML (experimental) Abstract:Deep learning vision models excel with abundant supervision, but many applications face label scarcity and class imbalance. Controllable image editing can augment scarce labeled data, yet edits often introduce artifacts and entangle non-target attributes. We study this in facial expression analysis, targeting Action Unit (AU) manipulation where annotation is costly and AU co-activation drives entanglement. We present a facial manipulation method that operates in the semantic latent space of a pre-trained face generator (Diffusion Autoencoder). Using lightweight linear models, we reduce entanglement of semantic features via (i) dependency-aware conditioning that accounts for AU co-activation, and (ii) orthogonal projection that removes nuisance attribute directions (e.g., glasses), together with an expression neutralization step to enable absolute AU edit. We use these edits to balance AU occurrence by editing labeled faces and to diversify identities/demographics via controlled synthesis. Augmenting AU detector training with the generated data improves accuracy an...
