![[2602.16008] MAEB: Massive Audio Embedding Benchmark](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.16008] MAEB: Massive Audio Embedding Benchmark
Summary
The MAEB paper introduces a comprehensive benchmark for evaluating audio models across 30 tasks in over 100 languages, highlighting performance disparities among model types.
Why It Matters
MAEB addresses the need for standardized evaluation in audio processing, providing insights into model performance across diverse tasks. This benchmark can guide future research and development in audio AI, enhancing understanding of model capabilities and limitations.
Key Takeaways
- MAEB covers 30 diverse audio tasks, including speech and music.
- No single model excels across all tasks, indicating specialization is necessary.
- Performance on acoustic tasks does not correlate with linguistic task success.
- MAEB integrates into the MTEB ecosystem for unified evaluation.
- The benchmark is accompanied by a leaderboard and code for further research.
Computer Science > Sound arXiv:2602.16008 (cs) [Submitted on 17 Feb 2026] Title:MAEB: Massive Audio Embedding Benchmark Authors:Adnan El Assadi, Isaac Chung, Chenghao Xiao, Roman Solomatin, Animesh Jha, Rahul Chand, Silky Singh, Kaitlyn Wang, Ali Sartaz Khan, Marc Moussa Nasser, Sufen Fong, Pengfei He, Alan Xiao, Ayush Sunil Munot, Aditya Shrivastava, Artem Gazizov, Niklas Muennighoff, Kenneth Enevoldsen View a PDF of the paper titled MAEB: Massive Audio Embedding Benchmark, by Adnan El Assadi and 17 other authors View PDF Abstract:We introduce the Massive Audio Embedding Benchmark (MAEB), a large-scale benchmark covering 30 tasks across speech, music, environmental sounds, and cross-modal audio-text reasoning in 100+ languages. We evaluate 50+ models and find that no single model dominates across all tasks: contrastive audio-text models excel at environmental sound classification (e.g., ESC50) but score near random on multilingual speech tasks (e.g., SIB-FLEURS), while speech-pretrained models show the opposite pattern. Clustering remains challenging for all models, with even the best-performing model achieving only modest results. We observe that models excelling on acoustic understanding often perform poorly on linguistic tasks, and vice versa. We also show that the performance of audio encoders on MAEB correlates highly with their performance when used in audio large language models. MAEB is derived from MAEB+, a collection of 98 tasks. MAEB is designed to maintain tas...