[2603.00031] GRIP: Geometric Refinement and Adaptive Information Potential for Data Efficiency

Computer Science > Computation and Language arXiv:2603.00031 (cs) [Submitted on 4 Feb 2026] Title:GRIP: Geometric Refinement and Adaptive Information Potential for Data Efficiency Authors:Changhao Wang, Jiaolong Yang, Xinhao Yao, Yunfei Yu, Peng Jiao, Lu Yu, Junpeng Fang, Riccardo Cantoro, Qing Cui, Jun Zhou View a PDF of the paper titled GRIP: Geometric Refinement and Adaptive Information Potential for Data Efficiency, by Changhao Wang and 9 other authors View PDF HTML (experimental) Abstract:The performance of Large Language Models (LLMs) is increasingly governed by data efficiency rather than raw scaling volume. However, existing selection methods often decouple global distribution balancing from local instance selection, compromising the hierarchical integrity of the training set. We introduce \textbf{GRIP} (Geometric Refinement and Adaptive Information Potential), a framework that unifies these dimensions by modeling the corpus as an information-dense geometric space. GRIP employs a \textbf{Rapid Adaptation Probe (RAP)} to quantify the information potential of semantic clusters, dynamically re-allocating the sampling budget to regions with the highest representation deficits. Subsequently, we perform Intra-Cluster Selection using a \textbf{length-rectified geometric prior} to counteract embedding density artifacts and preserve long-tail logical sequences. Extensive evaluations on Mixture-of-Experts (MoE) models up to 300B tokens demonstrate that GRIP consistently outp...