![[2508.17742] EEG-FM-Bench: A Comprehensive Benchmark for the Systematic Evaluation of EEG Foundation Models](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2508.17742] EEG-FM-Bench: A Comprehensive Benchmark for the Systematic Evaluation of EEG Foundation Models
Summary
The paper presents EEG-FM-Bench, a standardized benchmark for evaluating EEG foundation models, addressing inconsistencies in current evaluation methods and enabling fair comparisons across models.
Why It Matters
Standardized evaluation benchmarks are crucial for advancing research in EEG models, as they facilitate reliable comparisons and improve understanding of model performance. This benchmark aims to unify fragmented results in the field, promoting reproducibility and scientific progress.
Key Takeaways
- EEG-FM-Bench integrates 14 datasets and 10 paradigms for comprehensive evaluation.
- Multi-task learning helps mitigate overfitting in data-scarce EEG scenarios.
- Pre-training efficiency is hindered by conflicts between reconstruction and downstream tasks.
- Compact architectures with domain-specific biases outperform larger models.
- The benchmark promotes reproducibility and interpretable advances in EEG research.
Electrical Engineering and Systems Science > Signal Processing arXiv:2508.17742 (eess) [Submitted on 25 Aug 2025 (v1), last revised 13 Feb 2026 (this version, v2)] Title:EEG-FM-Bench: A Comprehensive Benchmark for the Systematic Evaluation of EEG Foundation Models Authors:Wei Xiong, Jiangtong Li, Jie Li, Kun Zhu, Changjun Jiang View a PDF of the paper titled EEG-FM-Bench: A Comprehensive Benchmark for the Systematic Evaluation of EEG Foundation Models, by Wei Xiong and Jiangtong Li and Jie Li and Kun Zhu and Changjun Jiang View PDF HTML (experimental) Abstract:Electroencephalography foundation models (EEG-FMs) have advanced brain signal analysis, but the lack of standardized evaluation benchmarks impedes model comparison and scientific progress. Current evaluations rely on inconsistent protocols that render cross-model comparisons unreliable, while a lack of diagnostic analyses obscures the internal mechanisms driving transfer efficiency and scaling behaviors. To address this, we introduce \textbf{EEG-FM-Bench}, a unified system for the standardized evaluation of EEG-FMs. The benchmark integrates 14 datasets across 10 paradigms and incorporates diverse experimental settings, including multiple fine-tuning strategies, task organizations, and classifier configurations, supported by tools for gradient and representation analysis. Our experiments and analysis reveal several critical insights: (1) multi-task learning acts as a critical regularizer to mitigate overfitting in dat...
