![[2602.13550] Out-of-Support Generalisation via Weight Space Sequence Modelling](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.13550] Out-of-Support Generalisation via Weight Space Sequence Modelling
Summary
This paper presents a novel approach to out-of-support generalization in machine learning, introducing the WeightCaster framework for improved predictions beyond training data limits.
Why It Matters
As AI applications expand into safety-critical areas, reliable predictions outside training data ranges are essential. This research addresses the challenge of out-of-support generalization, enhancing AI's applicability and trustworthiness in real-world scenarios.
Key Takeaways
- Introduces the WeightCaster framework for out-of-support generalization.
- Reformulates the generalization problem as a sequence modeling task in weight space.
- Demonstrates competitive performance on synthetic and real-world datasets.
- Enhances reliability of AI predictions in safety-critical applications.
- Avoids explicit inductive biases while maintaining computational efficiency.
Computer Science > Machine Learning arXiv:2602.13550 (cs) [Submitted on 14 Feb 2026] Title:Out-of-Support Generalisation via Weight Space Sequence Modelling Authors:Roussel Desmond Nzoyem View a PDF of the paper titled Out-of-Support Generalisation via Weight Space Sequence Modelling, by Roussel Desmond Nzoyem View PDF HTML (experimental) Abstract:As breakthroughs in deep learning transform key industries, models are increasingly required to extrapolate on datapoints found outside the range of the training set, a challenge we coin as out-of-support (OoS) generalisation. However, neural networks frequently exhibit catastrophic failure on OoS samples, yielding unrealistic but overconfident predictions. We address this challenge by reformulating the OoS generalisation problem as a sequence modelling task in the weight space, wherein the training set is partitioned into concentric shells corresponding to discrete sequential steps. Our WeightCaster framework yields plausible, interpretable, and uncertainty-aware predictions without necessitating explicit inductive biases, all the while maintaining high computational efficiency. Emprical validation on a synthetic cosine dataset and real-world air quality sensor readings demonstrates performance competitive or superior to the state-of-the-art. By enhancing reliability beyond in-distribution scenarios, these results hold significant implications for the wider adoption of artificial intelligence in safety-critical applications. Com...