Tricks from OpenAI gpt-oss YOU 🫵 can use with transformers

Back to Articles Tricks from OpenAI gpt-oss YOU 🫵 can use with transformers Published September 11, 2025 Update on GitHub Upvote 179 +173 Aritra Roy Gosthipaty ariG23498 Follow Sergio Paniego sergiopaniego Follow Vaibhav Srivastav reach-vb Follow Pedro Cuenca pcuenq Follow Arthur Zucker ArthurZ Follow Nathan Habib SaylorTwift Follow Cyril Vallez cyrilvallez Follow OpenAI recently released their GPT-OSS series of models. The models feature some novel techniques like MXFP4 quantization, efficient kernels, a brand new chat format, and more. To enable the release of gpt-oss through transformers, we have upgraded the library considerably. The updates make it very efficient to load, run, and fine-tune the models. In this blog post, we talk about all the upgrades in-depth, and how they become part of the transformers toolkit so other models (current and future) can benefit from them. Providing clean implementations of new methods in transformers also allows the community to quickly understand and adopt them. Frameworks such as MLX, llama.cpp or vLLM can use the transformers code as a reference to build their own implementations. For this release, we worked on: Zero-build Kernels, downloadable from the Hub MXFP4 Quantization Tensor Parallelism Expert Parallelism Dynamic Sliding Window Layer & Cache Continuous Batching & Paged Attention Load larger models faster Best part: Most of these features should work across all major models within transformers! Zero-build Kernels, downloadab...