![[2506.21220] Complexity-aware fine-tuning](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2506.21220] Complexity-aware fine-tuning
Summary
The paper presents a novel method for fine-tuning large language models (LLMs) by categorizing training data based on complexity, resulting in improved accuracy and reduced data usage.
Why It Matters
This research addresses the inefficiencies in traditional fine-tuning methods for LLMs, proposing a complexity-aware approach that enhances performance while significantly reducing data requirements. This has implications for resource management in AI development and deployment.
Key Takeaways
- Introduces a complexity-aware fine-tuning method for LLMs.
- Achieves higher accuracy (0.58) compared to standard fine-tuning (0.45).
- Utilizes 81% less data while maintaining performance.
- Categorizes training data based on entropy to optimize learning.
- Demonstrates the effectiveness of distillation in complex scenarios.
Computer Science > Machine Learning arXiv:2506.21220 (cs) [Submitted on 26 Jun 2025 (v1), last revised 24 Feb 2026 (this version, v4)] Title:Complexity-aware fine-tuning Authors:Andrey Goncharov, Daniil Vyazhev, Petr Sychev, Edvard Khalafyan, Alexey Zaytsev View a PDF of the paper titled Complexity-aware fine-tuning, by Andrey Goncharov and 4 other authors View PDF HTML (experimental) Abstract:General-purpose Large Language Models (LLMs) are frequently fine-tuned through supervised fine-tuning (SFT) to enhance performance in specific domains. Better results can be achieved by distilling the chain-of-thought of a larger model at the cost of numerous expensive calls and a much greater amount of data. We propose a novel blueprint for efficient fine-tuning that uses reasoning only for complex data identified by entropy. Specifically, across three small open models ($\approx 3B$) we split the training data into complexity categories by a single token answer entropy (ROC AUC $0.73$), fine-tune large language models (LLMs) via SFT and distillation, and show that our pipeline significantly outperforms the standard SFT approach ($0.58$ vs $0.45$ average accuracy) and outperforms the distillation approach ($0.58$ vs $0.56$ average accuracy) while using $81\%$ less data. Subjects: Machine Learning (cs.LG); Computation and Language (cs.CL) Cite as: arXiv:2506.21220 [cs.LG] (or arXiv:2506.21220v4 [cs.LG] for this version) https://doi.org/10.48550/arXiv.2506.21220 Focus to learn mor...