[2603.20217] Expected Reward Prediction, with Applications to Model Routing

Computer Science > Computation and Language arXiv:2603.20217 (cs) [Submitted on 3 Mar 2026] Title:Expected Reward Prediction, with Applications to Model Routing Authors:Kenan Hasanaliyev, Silas Alberti, Jenny Hamer, Dheeraj Rajagopal, Kevin Robinson, Jasper Snoek, Victor Veitch, Alexander Nicholas D'Amour View a PDF of the paper titled Expected Reward Prediction, with Applications to Model Routing, by Kenan Hasanaliyev and 7 other authors View PDF HTML (experimental) Abstract:Reward models are a standard tool to score responses from LLMs. Reward models are built to rank responses to a fixed prompt sampled from a single model, for example to choose the best of n sampled responses. In this paper, we study whether scores from response-level reward models lifted to score a model's suitability for a prompt, prior to seeing responses from that model. Specifically, we show that it is straightforward to predict the expected reward that an LLM would earn from the reward model under repeated sampling. Further, we show that these expected reward predictions are precise and discriminative enough to support an application to a model routing protocol that routes prompts to models at inference time to maximize reward while controlling computational cost. We demonstrate the performance of this routing procedure on the open-perfectblend dataset, using a model pool composed of Llama3.1-Instruct 8B/70B, Gemma2-IT 9B/27B, and Gemma1-IT 7B models. Our simple expected reward prediction--based r...