![[2601.22323] Models Under SCOPE: Scalable and Controllable Routing via Pre-hoc Reasoning](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2601.22323] Models Under SCOPE: Scalable and Controllable Routing via Pre-hoc Reasoning
Summary
The paper presents SCOPE, a novel routing framework for language models that dynamically predicts cost and performance, enhancing efficiency and accuracy in model selection.
Why It Matters
As AI models proliferate, efficient routing mechanisms become crucial for optimizing resource use. SCOPE allows for flexible adaptation to user needs, balancing accuracy and cost, which is vital for practical applications in machine learning and AI.
Key Takeaways
- SCOPE predicts model performance and cost dynamically, enhancing routing decisions.
- It adapts to new and unseen models, overcoming limitations of fixed model selection.
- Users can prioritize accuracy or cost savings based on their specific needs.
- The framework demonstrates significant improvements in accuracy and cost reduction.
- Reinforcement learning is utilized for training, making SCOPE a robust solution.
Computer Science > Machine Learning arXiv:2601.22323 (cs) [Submitted on 29 Jan 2026 (v1), last revised 13 Feb 2026 (this version, v2)] Title:Models Under SCOPE: Scalable and Controllable Routing via Pre-hoc Reasoning Authors:Qi Cao, Shuhao Zhang, Ruizhe Zhou, Ruiyi Zhang, Peijia Qin, Pengtao Xie View a PDF of the paper titled Models Under SCOPE: Scalable and Controllable Routing via Pre-hoc Reasoning, by Qi Cao and 5 other authors View PDF HTML (experimental) Abstract:Model routing chooses which language model to use for each query. By sending easy queries to cheaper models and hard queries to stronger ones, it can significantly reduce inference cost while maintaining high accuracy. However, most existing routers treat this as a fixed choice among a small set of models, which makes them hard to adapt to new models or changing budget constraints. In this paper, we propose SCOPE (Scalable and Controllable Outcome Performance Estimator), a routing framework that goes beyond model selection by predicting their cost and performance. Trained with reinforcement learning, SCOPE makes reasoning-based predictions by retrieving how models behave on similar problems, rather than relying on fixed model names, enabling it to work with new, unseen models. Moreover, by explicitly predicting how accurate and how expensive a model will be, it turns routing into a dynamic decision problem, allowing users to easily control the trade-off between accuracy and cost. Experiments show that SCOPE i...
