[2511.04401] Spurious Correlation-Aware Embedding Regularization for Worst-Group Robustness

Computer Science > Machine Learning arXiv:2511.04401 (cs) [Submitted on 6 Nov 2025 (v1), last revised 2 Mar 2026 (this version, v2)] Title:Spurious Correlation-Aware Embedding Regularization for Worst-Group Robustness Authors:Subeen Park, Joowang Kim, Hakyung Lee, Sunjae Yoo, Kyungwoo Song View a PDF of the paper titled Spurious Correlation-Aware Embedding Regularization for Worst-Group Robustness, by Subeen Park and 4 other authors View PDF HTML (experimental) Abstract:Deep learning models achieve strong performance across various domains but often rely on spurious correlations, making them vulnerable to distribution shifts. This issue is particularly severe in subpopulation shift scenarios, where models struggle in underrepresented groups. While existing methods have made progress in mitigating this issue, their performance gains are still constrained. They lack a rigorous theoretical framework connecting the embedding space representations with worst-group error. To address this limitation, we propose Spurious Correlation-Aware Embedding Regularization for Worst-Group Robustness (SCER), a novel approach that directly regularizes feature representations to suppress spurious cues. We show theoretically that worst-group error is influenced by how strongly the classifier relies on spurious versus core directions, identified from differences in group-wise mean embeddings across domains and classes. By imposing theoretical constraints at the embedding level, SCER encourages m...