[2604.00064] Forecast collapse of transformer-based models under squared loss in financial time series

Statistics > Machine Learning arXiv:2604.00064 (stat) [Submitted on 31 Mar 2026] Title:Forecast collapse of transformer-based models under squared loss in financial time series Authors:Pierre Andreoletti (IDP) View a PDF of the paper titled Forecast collapse of transformer-based models under squared loss in financial time series, by Pierre Andreoletti (IDP) View PDF Abstract:We study trajectory forecasting under squared loss for time series with weak conditional structure, using highly expressive prediction models. Building on the classical characterization of squared-loss risk minimization, we emphasize regimes in which the conditional expectation of future trajectories is effectively degenerate, leading to trivial Bayes-optimal predictors (flat for prices and zero for returns in standard financial settings). In this regime, increased model expressivity does not improve predictive accuracy but instead introduces spurious trajectory fluctuations around the optimal predictor. These fluctuations arise from the reuse of noise and result in increased prediction variance without any reduction in bias. This provides a process-level explanation for the degradation of Transformerbased forecasts on financial time series. We complement these theoretical results with numerical experiments on high-frequency EUR/USD exchange rate data, analyzing the distribution of trajectory-level forecasting errors. The results show that Transformer-based models yield larger errors than a simple line...