![[2507.08333] Token-Based Audio Inpainting via Discrete Diffusion](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2507.08333] Token-Based Audio Inpainting via Discrete Diffusion
Summary
This article presents a novel method for audio inpainting using discrete diffusion techniques to restore missing segments in audio recordings, particularly in music.
Why It Matters
Audio inpainting is crucial for enhancing audio quality in various applications, including music restoration and sound design. This research introduces a new approach that significantly improves the restoration of larger gaps in audio, which has implications for both academic research and practical applications in audio processing.
Key Takeaways
- Introduces a token-based approach for audio inpainting using discrete diffusion.
- Outperforms existing methods in restoring long gaps in audio recordings.
- Incorporates innovative training techniques for improved audio restoration.
Computer Science > Sound arXiv:2507.08333 (cs) [Submitted on 11 Jul 2025 (v1), last revised 17 Feb 2026 (this version, v4)] Title:Token-Based Audio Inpainting via Discrete Diffusion Authors:Tali Dror, Iftach Shoham, Moshe Buchris, Oren Gal, Haim Permuter, Gilad Katz, Eliya Nachmani View a PDF of the paper titled Token-Based Audio Inpainting via Discrete Diffusion, by Tali Dror and 6 other authors View PDF HTML (experimental) Abstract:Audio inpainting seeks to restore missing segments in degraded recordings. Previous diffusion-based methods exhibit impaired performance when the missing region is large. We introduce the first approach that applies discrete diffusion over tokenized music representations from a pre-trained audio tokenizer, enabling stable and semantically coherent restoration of long gaps. Our method further incorporates two training approaches: a derivative-based regularization loss that enforces smooth temporal dynamics, and a span-based absorbing transition that provides structured corruption during diffusion. Experiments on the MusicNet and MAESTRO datasets with gaps up to 750 ms show that our approach consistently outperforms strong baselines across range of gap lengths, for gaps of 150 ms and above. This work advances musical audio restoration and introduces new directions for discrete diffusion model training. Visit our project page for examples and code. Subjects: Sound (cs.SD); Artificial Intelligence (cs.AI); Information Theory (cs.IT); Machine Learn...
