[2509.24789] Fidel-TS: A High-Fidelity Multimodal Benchmark for Time Series Forecasting

Computer Science > Machine Learning arXiv:2509.24789 (cs) [Submitted on 29 Sep 2025 (v1), last revised 7 May 2026 (this version, v4)] Title:Fidel-TS: A High-Fidelity Multimodal Benchmark for Time Series Forecasting Authors:Zhijian Xu, Wanxu Cai, Xilin Dai, Zhaorong Deng, Qiang Xu View a PDF of the paper titled Fidel-TS: A High-Fidelity Multimodal Benchmark for Time Series Forecasting, by Zhijian Xu and 4 other authors View PDF HTML (experimental) Abstract:The evaluation of time series forecasting models is hindered by a lack of high-quality benchmarks, leading to overestimated assessments of progress. Existing datasets suffer from issues ranging from small-scale, low-frequency, pre-training data contamination in unimodal designs to the temporal and description leakage prevalent in early multimodal designs. To address this, we formalize the core principles of high-fidelity benchmarking, focusing on data sourcing integrity, leak-free design, and structural clarity. We introduce Fidel-TS, a new large-scale benchmark built from these principles. Our experiments reveal the limitations of prior benchmarks and the potential discrepancies in model evaluation, providing new insights into multiple existing unimodal and multimodal forecasting models and LLMs across various evaluation tasks. Comments: Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML) Cite as: arXiv:2509.24789 [cs.LG]   (or arXiv:2509.24789v4 [cs.LG] for this version)   https://doi.org/10.48550/arXiv.2509....