[2603.01792] ALTER: Asymmetric LoRA for Token-Entropy-Guided Unlearning of LLMs

Computer Science > Computation and Language arXiv:2603.01792 (cs) [Submitted on 2 Mar 2026] Title:ALTER: Asymmetric LoRA for Token-Entropy-Guided Unlearning of LLMs Authors:Xunlei Chen, Jinyu Guo, Yuang Li, Zhaokun Wang, Yi Gong, Jie Zou, Jiwei Wei, Wenhong Tian View a PDF of the paper titled ALTER: Asymmetric LoRA for Token-Entropy-Guided Unlearning of LLMs, by Xunlei Chen and 7 other authors View PDF HTML (experimental) Abstract:Large language models (LLMs) have advanced to encompass extensive knowledge across diverse domains. Yet controlling what a LLMs should not know is important for ensuring alignment and thus safe use. However, effective unlearning in LLMs is difficult due to the fuzzy boundary between knowledge retention and forgetting. This challenge is exacerbated by entangled parameter spaces from continuous multi-domain training, often resulting in collateral damage, especially under aggressive unlearning strategies. Furthermore, the computational overhead required to optimize State-of-the-Art (SOTA) models with billions of parameters poses an additional barrier. In this work, we present ALTER, a lightweight unlearning framework for LLMs to address both the challenges of knowledge entanglement and unlearning efficiency. ALTER operates through two phases: (I) high entropy tokens are captured and learned via the shared A matrix in LoRA, followed by (II) an asymmetric LoRA architecture that achieves a specified forgetting objective by parameter isolation and unlea...