[2510.06084] Spectrum Tuning: Post-Training for Distributional Coverage and In-Context Steerability

Computer Science > Computation and Language arXiv:2510.06084 (cs) [Submitted on 7 Oct 2025 (v1), last revised 3 Mar 2026 (this version, v2)] Title:Spectrum Tuning: Post-Training for Distributional Coverage and In-Context Steerability Authors:Taylor Sorensen, Benjamin Newman, Jared Moore, Chan Park, Jillian Fisher, Niloofar Mireshghallah, Liwei Jiang, Yejin Choi View a PDF of the paper titled Spectrum Tuning: Post-Training for Distributional Coverage and In-Context Steerability, by Taylor Sorensen and 7 other authors View PDF HTML (experimental) Abstract:Language model post-training has enhanced instruction-following and performance on many downstream tasks, but also comes with an often-overlooked cost on tasks with many possible valid answers. On many tasks such as creative writing, synthetic data generation, or steering to diverse preferences, models must cover an entire distribution of outputs, rather than a single correct answer. We characterize three desiderata for conditional distributional modeling: in-context steerability, valid output space coverage, and distributional alignment, and document across three model families how current post-training can reduce these properties. In particular, we disambiguate between two kinds of in-context learning: ICL for eliciting existing underlying knowledge or capabilities, and in-context steerability, where a model must use in-context information to override its priors and steer to a novel data generating distribution. To better...