![[2602.17385] Dataless Weight Disentanglement in Task Arithmetic via Kronecker-Factored Approximate Curvature](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.17385] Dataless Weight Disentanglement in Task Arithmetic via Kronecker-Factored Approximate Curvature
Summary
This paper presents a novel dataless approach to disentangling task vectors in task arithmetic using Kronecker-Factored Approximate Curvature, enhancing modularity and performance without requiring external data.
Why It Matters
The research addresses significant challenges in adapting foundation models for multiple tasks without data dependency, which is crucial for applications with privacy constraints. By proposing a method that maintains performance while eliminating the need for held-out tuning, it opens new avenues for robust AI model development.
Key Takeaways
- Introduces a dataless method for disentangling task vectors.
- Utilizes Kronecker-Factored Approximate Curvature for regularization.
- Achieves state-of-the-art results in task addition and negation.
- Promotes robustness against task vector rescaling.
- Maintains constant complexity regardless of the number of tasks.
Computer Science > Artificial Intelligence arXiv:2602.17385 (cs) [Submitted on 19 Feb 2026] Title:Dataless Weight Disentanglement in Task Arithmetic via Kronecker-Factored Approximate Curvature Authors:Angelo Porrello, Pietro Buzzega, Felix Dangel, Thomas Sommariva, Riccardo Salami, Lorenzo Bonicelli, Simone Calderara View a PDF of the paper titled Dataless Weight Disentanglement in Task Arithmetic via Kronecker-Factored Approximate Curvature, by Angelo Porrello and 6 other authors View PDF HTML (experimental) Abstract:Task Arithmetic yields a modular, scalable way to adapt foundation models. Combining multiple task vectors, however, can lead to cross-task interference, causing representation drift and degraded performance. Representation drift regularization provides a natural remedy to disentangle task vectors; however, existing approaches typically require external task data, conflicting with modularity and data availability constraints (e.g., privacy requirements). We propose a dataless approach by framing regularization against representation drift as a curvature matrix approximation problem. This allows us to leverage well-established techniques; in particular, we adopt Kronecker-Factored Approximate Curvature and obtain a practical regularizer that achieves state-of-the-art results in task addition and negation. Our method has constant complexity in the number of tasks and promotes robustness to task vector rescaling, eliminating the need for held-out tuning. Commen...
