🤗 PEFT welcomes new merging methods

Back to Articles 🤗 PEFT welcomes new merging methods Published February 19, 2024 Update on GitHub Upvote 29 +23 Sourab Mangrulkar smangrul Follow Sayak Paul sayakpaul Follow Model merging has quickly become the de-facto standard of pushing the performance limits of large language models. On the Open LLM Leaderboard, we continue to notice merged models topping up the charts. Our very own Omar Sanseviero, made a little sprint on model merging and discovered interesting findings. The typical way of model merging, so far, has been to take a set of models and merge them. This post gives a nice primer on this topic. Generally, for merging multiple models, we first download their checkpoints and then perform merging. Depending on the merge algorithm and the sizes of the underlying model, this process can be quite memory-intensive. The mergekit library provides optimized ways for handling this, making the process manageable on limited memory. But what if we wanted to merge different “adapters” obtained from the same model? You might have four different LoRA checkpoints obtained from the same base model, and you want to experiment with different merging techniques. Eventually, you want to settle with the best merge, giving you the best results for your task. A couple of things become evident when approaching such a developer experience: When dealing with adapters such as LoRA, it’s common for users to swap in and out different adapters or even combine them. Adapters can be activate...