[2504.21023] Param$Δ$ for Direct Weight Mixing: Post-Train Large Language Model at Zero Cost

Computer Science > Computation and Language arXiv:2504.21023 (cs) [Submitted on 23 Apr 2025] Title:Param$Δ$ for Direct Weight Mixing: Post-Train Large Language Model at Zero Cost Authors:Sheng Cao, Mingrui Wu, Karthik Prasad, Yuandong Tian, Zechun Liu View a PDF of the paper titled Param$\Delta$ for Direct Weight Mixing: Post-Train Large Language Model at Zero Cost, by Sheng Cao and 4 other authors View PDF Abstract:The post-training phase of large language models is essential for enhancing capabilities such as instruction-following, reasoning, and alignment with human preferences. However, it demands extensive high-quality data and poses risks like overfitting, alongside significant computational costs due to repeated post-training and evaluation after each base model update. This paper introduces $Param\Delta$, a novel method that streamlines post-training by transferring knowledge from an existing post-trained model to a newly updated base model with ZERO additional training. By computing the difference between post-trained model weights ($\Theta_\text{post}$) and base model weights ($\Theta_\text{base}$), and adding this to the updated base model ($\Theta'_\text{base}$), we define $Param\Delta$ Model as: $\Theta_{\text{Param}\Delta} = \Theta_\text{post} - \Theta_\text{base} + \Theta'_\text{base}$. This approach surprisingly equips the new base model with post-trained capabilities, achieving performance comparable to direct post-training. We did analysis on LLama3, Llam...