![[2602.17993] Turbo Connection: Reasoning as Information Flow from Higher to Lower Layers](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.17993] Turbo Connection: Reasoning as Information Flow from Higher to Lower Layers
Summary
The paper introduces Turbo Connection, a novel architecture that enhances reasoning in Transformers by allowing multiple residual connections from higher to lower layers, significantly improving performance on various benchmarks without retraining the entire model.
Why It Matters
This research addresses a critical limitation in Transformer architectures, specifically their fixed-depth computation paths. By introducing Turbo Connection, the authors demonstrate a method to enhance reasoning capabilities in large language models (LLMs), which could lead to more effective AI applications in complex problem-solving tasks.
Key Takeaways
- Turbo Connection allows multiple residual connections, enhancing reasoning in Transformers.
- The architecture improves accuracy on benchmarks by up to 10%.
- Dense backward connections are more effective than sparse alternatives.
- TurboConn can be integrated into existing LLMs without full retraining.
- The depth of computational paths is crucial for reasoning ability.
Computer Science > Machine Learning arXiv:2602.17993 (cs) [Submitted on 20 Feb 2026] Title:Turbo Connection: Reasoning as Information Flow from Higher to Lower Layers Authors:Mohan Tang, Sidi Lu View a PDF of the paper titled Turbo Connection: Reasoning as Information Flow from Higher to Lower Layers, by Mohan Tang and 1 other authors View PDF HTML (experimental) Abstract:Complex problems, whether in math, logic, or planning, are solved by humans through a sequence of steps where the result of one step informs the next. In this work, we adopt the perspective that the reasoning power of Transformers is fundamentally limited by a fixed maximum number of steps along any latent path of computation. To address this, we introduce Turbo Connection (TurboConn), a novel architecture that overcomes the fixed-depth constraint by routing multiple residual connections from the higher-layer hidden states of each token $t$ to the lower layers of token $t+1$. Fine-tuning pre-trained LLMs with our method not only yields accuracy gains of 0.9% to over 10% on benchmarks like GSM8K, Parity, and multi-step arithmetic, but also demonstrates that the density of these backward connections is critical; our dense interaction significantly outperforms "sparse" alternatives that only pass a single hidden state or vector. Notably, TurboConn can be integrated into pre-trained LLMs to overcome task-specific plateaus: while a fine-tuned Qwen-3-1.7B achieves only 53.78% on Parity, adding our architectural...
