[2603.04444] vLLM Semantic Router: Signal Driven Decision Routing for Mixture-of-Modality Models

Computer Science > Networking and Internet Architecture arXiv:2603.04444 (cs) [Submitted on 23 Feb 2026] Title:vLLM Semantic Router: Signal Driven Decision Routing for Mixture-of-Modality Models Authors:Xunzhuo Liu, Huamin Chen, Samzong Lu, Yossi Ovadia, Guohong Wen, Zhengda Tan, Jintao Zhang, Senan Zedan, Yehudit Kerido, Liav Weiss, Bishen Yu, Asaad Balum, Noa Limoy, Abdallah Samara, Brent Salisbury, Hao Wu, Ryan Cook, Zhijie Wang, Qiping Pan, Rehan Khan, Avishek Goswami, Houston H. Zhang, Shuyi Wang, Ziang Tang, Fang Han, Zohaib Hassan, Jianqiao Zheng, Avinash Changrani View a PDF of the paper titled vLLM Semantic Router: Signal Driven Decision Routing for Mixture-of-Modality Models, by Xunzhuo Liu and 27 other authors View PDF HTML (experimental) Abstract:As large language models (LLMs) diversify across modalities, capabilities, and cost profiles, the problem of intelligent request routing -- selecting the right model for each query at inference time -- has become a critical systems challenge. We present vLLM Semantic Router, a signal-driven decision routing framework for Mixture-of-Modality (MoM) model deployments. The central innovation is composable signal orchestration: the system extracts heterogeneous signal types from each request -- from sub-millisecond heuristic features (keyword patterns, language detection, context length, role-based authorization) to neural classifiers (domain, embedding similarity, factual grounding, modality) -- and composes them through c...