Welcome aMUSEd: Efficient Text-to-Image Generation

Back to Articles Welcome aMUSEd: Efficient Text-to-Image Generation Published January 4, 2024 Update on GitHub Upvote 13 +7 Isamu Isozaki Isamu136 Follow guest Suraj Patil valhalla Follow Will Berman williamberman Follow Sayak Paul sayakpaul Follow We’re excited to present an efficient non-diffusion text-to-image model named aMUSEd. It’s called so because it’s a open reproduction of Google's MUSE. aMUSEd’s generation quality is not the best and we’re releasing a research preview with a permissive license. In contrast to the commonly used latent diffusion approach (Rombach et al. (2022)), aMUSEd employs a Masked Image Model (MIM) methodology. This not only requires fewer inference steps, as noted by Chang et al. (2023), but also enhances the model's interpretability. Just as MUSE, aMUSEd demonstrates an exceptional ability for style transfer using a single image, a feature explored in depth by Sohn et al. (2023). This aspect could potentially open new avenues in personalized and style-specific image generation. In this blog post, we will give you some internals of aMUSEd, show how you can use it for different tasks, including text-to-image, and show how to fine-tune it. Along the way, we will provide all the important resources related to aMUSEd, including its training code. Let’s get started 🚀 Table of contents How does it work? Using in diffusers Fine-tuning aMUSEd Limitations Resources We have built a demo for readers to play with aMUSEd. You can try it out in this Space...