Introducing Storage Buckets on the Hugging Face Hub

Back to Articles Introducing Storage Buckets on the Hugging Face Hub Published March 10, 2026 Update on GitHub Upvote 183 +177 Lucain Pouget Wauplin Follow Eliott Coyac coyotte508 Follow Adrien Carreira XciD Follow Victor Mustar victor Follow Julien Chaumond julien-c Follow Quentin Lhoest lhoestq Follow Pierric Cistac pierric Follow Sylvestre Bcht Sylvestre Follow Hugo Larcher hlarcher Follow Rajat Arya rajatarya Follow Di Xiao seanses Follow Assaf Vayner assafvayner Follow Hugging Face Models and Datasets repos are great for publishing final artifacts. But production ML generates a constant stream of intermediate files (checkpoints, optimizer states, processed shards, logs, traces, etc.) that change often, arrive from many jobs at once, and rarely need version control. Storage Buckets are built exactly for this: mutable, S3-like object storage you can browse on the Hub, script from Python, or manage with the hf CLI. And because they are backed by Xet, they are especially efficient for ML artifacts that share content across files. Why we built Buckets Git starts to feel like the wrong abstraction pretty quickly when you're dealing with: Training clusters writing checkpoints and optimizer states throughout a run Data pipelines processing raw datasets iteratively Agents storing traces, memory, and shared knowledge graphs The storage need in all these cases is the same: write fast, overwrite when needed, sync directories, remove stale files, and keep things moving. A Bucket i...