![[2602.16305] BAT: Better Audio Transformer Guided by Convex Gated Probing](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.16305] BAT: Better Audio Transformer Guided by Convex Gated Probing
Summary
The paper introduces the Better Audio Transformer (BAT), which utilizes a novel Convex Gated Probing method to enhance audio self-supervised learning, achieving state-of-the-art results on audio benchmarks.
Why It Matters
This research addresses the limitations of current audio self-supervised learning models that rely heavily on fine-tuning. By proposing a more efficient probing method, it aims to improve the reliability and reproducibility of audio models, which is crucial for advancing audio processing technologies.
Key Takeaways
- Introduces Convex Gated Probing (CGP) to improve audio SSL models.
- BAT achieves state-of-the-art performance on audio benchmarks.
- CGP allows for efficient use of frozen layers in audio models.
- Refines data preprocessing and model architecture for better results.
- Addresses the shortcomings of fine-tuning in audio SSL.
Computer Science > Sound arXiv:2602.16305 (cs) [Submitted on 18 Feb 2026] Title:BAT: Better Audio Transformer Guided by Convex Gated Probing Authors:Houtan Ghaffari, Lukas Rauch, Christoph Scholz, Paul Devos View a PDF of the paper titled BAT: Better Audio Transformer Guided by Convex Gated Probing, by Houtan Ghaffari and 3 other authors View PDF HTML (experimental) Abstract:Probing is widely adopted in computer vision to faithfully evaluate self-supervised learning (SSL) embeddings, as fine-tuning may misrepresent their inherent quality. In contrast, audio SSL models still rely on fine-tuning because simple probing fails to unlock their full potential and alters their rankings when competing for SOTA on AudioSet. Hence, a robust and efficient probing mechanism is required to guide the trajectory of audio SSL towards reliable and reproducible methods. We introduce Convex Gated Probing (CGP), a prototype-based method that drastically closes the gap between fine-tuning and probing in audio. CGP efficiently utilizes all frozen layers via a gating mechanism and exposes the location of latent task-relevant information. Guided by CGP, we rework the entire SSL pipeline of current SOTA audio models that use legacy implementations of prior SSL methods. By refining data preprocessing, model architecture, and pre-training recipe, we introduce Better Audio Transformer (BAT), and establish new SOTA on audio benchmarks. Subjects: Sound (cs.SD); Machine Learning (cs.LG) Cite as: arXiv:26...
