![[2602.22479] Efficient Continual Learning in Language Models via Thalamically Routed Cortical Columns](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.22479] Efficient Continual Learning in Language Models via Thalamically Routed Cortical Columns
Summary
This paper presents TRC², a novel architecture for continual learning in language models that mitigates catastrophic forgetting while maintaining efficiency in training and inference.
Why It Matters
As language models are increasingly deployed in dynamic environments, the ability to learn continuously without forgetting prior knowledge is crucial. TRC² addresses the limitations of traditional training methods, offering a solution that balances stability and adaptability, which is essential for real-world applications.
Key Takeaways
- TRC² architecture enhances continual learning in language models.
- It combines sparse thalamic routing with memory and feedback mechanisms.
- The model achieves a better stability-plasticity tradeoff during training.
- TRC² allows for rapid adaptation to new data while preserving previous learning.
- A reproducible evaluation framework is provided to measure performance.
Computer Science > Machine Learning arXiv:2602.22479 (cs) [Submitted on 25 Feb 2026] Title:Efficient Continual Learning in Language Models via Thalamically Routed Cortical Columns Authors:Afshin Khadangi View a PDF of the paper titled Efficient Continual Learning in Language Models via Thalamically Routed Cortical Columns, by Afshin Khadangi View PDF HTML (experimental) Abstract:Continual learning is a core requirement for deployed language models, yet standard training and fine-tuning pipelines remain brittle under non-stationary data. Online updates often induce catastrophic forgetting, while methods that improve stability frequently increase latency, memory footprint, or dense computation in ways that do not scale well to long contexts. We introduce TRC$^{2}$ (Thalamically Routed Cortical Columns), a decoder-only backbone that addresses continual learning at the architectural level. TRC$^{2}$ combines sparse thalamic routing over cortical columns with mechanisms for modulation, prediction, memory, and feedback, together with a fast corrective pathway that supports rapid adaptation without destabilizing slower parameters. The resulting block is sparse and chunk-parallel, enabling efficient training and inference while preserving clean ablations of each subsystem. We instantiate a reproducible training and evaluation stack and a continual-learning harness that measures proxy forgetting under streaming domain shifts. Across language modeling and continual learning benchmar...