Depth-first pruning seems to transfer from GPT-2 to Llama (unexpectedly well)
Reddit - Artificial Intelligence
1 min read
About this article
TL;DR: Removing the right transformer layers (instead of shrinking all layers) gives smaller, faster models with minimal quality loss — and this seems to transfer from GPT-2 to Llama. been experimenting with a simple idea: instead of shrinking model width, just remove entire layers based on sensitivity and then recover with distillation. Originally tested it on GPT-2 (124M) and it worked pretty well. Decided to try the exact same approach on TinyLlama 1.1B to see if it was just a fluke. but i...
You've been blocked by network security.To continue, log in to your Reddit account or use your developer tokenIf you think you've been blocked by mistake, file a ticket below and we'll look into it.Log in File a ticket
Originally published on March 31, 2026.
Curated by AI News.
![[2603.23966] Policy-Guided Threat Hunting: An LLM enabled Framework with Splunk SOC Triage](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}