[2603.02675] From Shallow to Deep: Pinning Semantic Intent via Causal GRPO

Computer Science > Machine Learning arXiv:2603.02675 (cs) [Submitted on 3 Mar 2026] Title:From Shallow to Deep: Pinning Semantic Intent via Causal GRPO Authors:Shuyi Zhou, Zeen Song, Wenwen Qiang, Jiyan Sun, Yao Zhou, Yinlong Liu, Wei Ma View a PDF of the paper titled From Shallow to Deep: Pinning Semantic Intent via Causal GRPO, by Shuyi Zhou and 6 other authors View PDF Abstract:Large Language Models remain vulnerable to adversarial prefix attacks (e.g., ``Sure, here is'') despite robust standard safety. We diagnose this vulnerability as Shallow Safety Alignment, stemming from a pathology we term semantic representation decay: as the model generates compliant prefixes, its internal malicious intent signal fades. To address this, we propose Two-Stage Causal-GRPO (TSC-GRPO), a framework designed to achieve intent pinning. First, grounded in causal identifiability theory, we train a causal intent probe to disentangle invariant intent from stylistic perturbations. Second, we internalize this causal awareness into the policy via Group Relative Policy Optimization. By employing a cumulative causal penalty within ``fork-in-the-road'' training scenarios, we force the model to learn that accumulating harmful tokens monotonically decreases reward, enabling robust late-stage refusals. Experiments show that TSC-GRPO significantly outperforms baselines in defending against jailbreak attacks while preserving general utility. Subjects: Machine Learning (cs.LG) Cite as: arXiv:2603.02675...