![[2602.15823] CrispEdit: Low-Curvature Projections for Scalable Non-Destructive LLM Editing](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.15823] CrispEdit: Low-Curvature Projections for Scalable Non-Destructive LLM Editing
Summary
CrispEdit introduces a novel algorithm for editing large language models (LLMs) that preserves their capabilities while allowing for targeted behavior changes, addressing a key challenge in LLM editing.
Why It Matters
This research is significant as it tackles the critical issue of capability preservation in LLMs, which is essential for maintaining model integrity during edits. The proposed method could enhance the reliability of LLMs in various applications, making it a valuable contribution to the field of machine learning and AI.
Key Takeaways
- CrispEdit is a second-order editing algorithm focused on capability preservation.
- It formulates editing as constrained optimization, improving upon existing methods.
- The algorithm utilizes low-curvature projections to minimize capability loss.
- CrispEdit achieves an average capability degradation of less than 1%.
- The method demonstrates significant improvements over prior LLM editing techniques.
Computer Science > Machine Learning arXiv:2602.15823 (cs) [Submitted on 17 Feb 2026] Title:CrispEdit: Low-Curvature Projections for Scalable Non-Destructive LLM Editing Authors:Zarif Ikram, Arad Firouzkouhi, Stephen Tu, Mahdi Soltanolkotabi, Paria Rashidinejad View a PDF of the paper titled CrispEdit: Low-Curvature Projections for Scalable Non-Destructive LLM Editing, by Zarif Ikram and 4 other authors View PDF Abstract:A central challenge in large language model (LLM) editing is capability preservation: methods that successfully change targeted behavior can quietly game the editing proxy and corrupt general capabilities, producing degenerate behaviors reminiscent of proxy/reward hacking. We present CrispEdit, a scalable and principled second-order editing algorithm that treats capability preservation as an explicit constraint, unifying and generalizing several existing editing approaches. CrispEdit formulates editing as constrained optimization and enforces the constraint by projecting edit updates onto the low-curvature subspace of the capability-loss landscape. At the crux of CrispEdit is expressing capability constraint via Bregman divergence, whose quadratic form yields the Gauss-Newton Hessian exactly and even when the base model is not trained to convergence. We make this second-order procedure efficient at the LLM scale using Kronecker-factored approximate curvature (K-FAC) and a novel matrix-free projector that exploits Kronecker structure to avoid constructing ma...
