![[2602.16490] From Growing to Looping: A Unified View of Iterative Computation in LLMs](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.16490] From Growing to Looping: A Unified View of Iterative Computation in LLMs
Summary
This paper explores the relationship between looping and depth growing in large language models (LLMs), proposing a unified view that highlights their shared mechanisms for enhancing reasoning capabilities.
Why It Matters
Understanding the interplay between looping and depth growth in LLMs is crucial for advancing AI reasoning. This research offers insights into how these techniques can be combined to improve model performance, which is significant for developers and researchers in machine learning and AI.
Key Takeaways
- Looping and depth growing are complementary techniques that enhance reasoning in LLMs.
- The study reveals that both methods share similar depth-wise signatures, indicating a common iterative computation mechanism.
- Applying inference-time looping to depth-grown models can significantly boost accuracy on reasoning tasks.
- Higher-quality training data improves the effectiveness of depth-grown models, especially in reasoning tasks.
- These findings suggest practical applications for improving AI models through iterative computation strategies.
Computer Science > Computation and Language arXiv:2602.16490 (cs) [Submitted on 18 Feb 2026] Title:From Growing to Looping: A Unified View of Iterative Computation in LLMs Authors:Ferdinand Kapl, Emmanouil Angelis, Kaitlin Maile, Johannes von Oswald, Stefan Bauer View a PDF of the paper titled From Growing to Looping: A Unified View of Iterative Computation in LLMs, by Ferdinand Kapl and 4 other authors View PDF HTML (experimental) Abstract:Looping, reusing a block of layers across depth, and depth growing, training shallow-to-deep models by duplicating middle layers, have both been linked to stronger reasoning, but their relationship remains unclear. We provide a mechanistic unification: looped and depth-grown models exhibit convergent depth-wise signatures, including increased reliance on late layers and recurring patterns aligned with the looped or grown block. These shared signatures support the view that their gains stem from a common form of iterative computation. Building on this connection, we show that the two techniques are adaptable and composable: applying inference-time looping to the middle blocks of a depth-grown model improves accuracy on some reasoning primitives by up to $2\times$, despite the model never being trained to loop. Both approaches also adapt better than the baseline when given more in-context examples or additional supervised fine-tuning data. Additionally, depth-grown models achieve the largest reasoning gains when using higher-quality, math...
