[2601.21343] Self-Improving Pretraining: using post-trained models to pretrain better models

Computer Science > Computation and Language arXiv:2601.21343 (cs) [Submitted on 29 Jan 2026 (v1), last revised 5 Apr 2026 (this version, v3)] Title:Self-Improving Pretraining: using post-trained models to pretrain better models Authors:Ellen Xiaoqing Tan, Jack Lanchantin, Shehzaad Dhuliawala, Danwei Li, Thao Nguyen, Jing Xu, Ping Yu, Ilia Kulikov, Sainbayar Sukhbaatar, Jason Weston, Xian Li, Olga Golovneva View a PDF of the paper titled Self-Improving Pretraining: using post-trained models to pretrain better models, by Ellen Xiaoqing Tan and 11 other authors View PDF HTML (experimental) Abstract:Large language models are classically trained in stages: pretraining on raw text followed by post-training for instruction following and reasoning. However, this separation creates a fundamental limitation: many desirable behaviors such as safety, factuality, overall generation quality, and reasoning ability are only added at a late stage, even though the patterns learned earlier strongly shape a model's capabilities. To tackle this issue, we introduce a new way to pretrain and mid-train models that incorporates these behaviors earlier. We utilize an existing strong, post-trained model to both rewrite pretraining data and to judge policy model rollouts, thus using reinforcement earlier in training. In our experiments, we show this can give strong gains in quality, safety, factuality and reasoning. Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine ...