![[2602.15143] Protecting Language Models Against Unauthorized Distillation through Trace Rewriting](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.15143] Protecting Language Models Against Unauthorized Distillation through Trace Rewriting
Summary
This paper explores methods to protect language models from unauthorized knowledge distillation by modifying reasoning traces, focusing on anti-distillation and API watermarking techniques.
Why It Matters
As language models become increasingly valuable, unauthorized distillation poses a significant threat to intellectual property and model integrity. This research provides innovative solutions to safeguard these models, ensuring that developers can protect their investments and maintain competitive advantages in AI.
Key Takeaways
- Unauthorized distillation exploits the efforts of model developers.
- The paper introduces anti-distillation methods to degrade the usefulness of unauthorized queries.
- API watermarking techniques embed verifiable signatures in student models.
- Dynamic rewriting of reasoning outputs preserves correctness while deterring unauthorized use.
- Experiments show effective watermark detection with minimal false alarms.
Computer Science > Artificial Intelligence arXiv:2602.15143 (cs) [Submitted on 16 Feb 2026] Title:Protecting Language Models Against Unauthorized Distillation through Trace Rewriting Authors:Xinhang Ma, William Yeoh, Ning Zhang, Yevgeniy Vorobeychik View a PDF of the paper titled Protecting Language Models Against Unauthorized Distillation through Trace Rewriting, by Xinhang Ma and 3 other authors View PDF HTML (experimental) Abstract:Knowledge distillation is a widely adopted technique for transferring capabilities from LLMs to smaller, more efficient student models. However, unauthorized use of knowledge distillation takes unfair advantage of the considerable effort and cost put into developing frontier models. We investigate methods for modifying teacher-generated reasoning traces to achieve two objectives that deter unauthorized distillation: (1) \emph{anti-distillation}, or degrading the training usefulness of query responses, and (2) \emph{API watermarking}, which embeds verifiable signatures in student models. We introduce several approaches for dynamically rewriting a teacher's reasoning outputs while preserving answer correctness and semantic coherence. Two of these leverage the rewriting capabilities of LLMs, while others use gradient-based techniques. Our experiments show that a simple instruction-based rewriting approach achieves a strong anti-distillation effect while maintaining or even improving teacher performance. Furthermore, we show that our rewriting ap...
