[2603.04716] SLO-Aware Compute Resource Allocation for Prefill-Decode Disaggregated LLM Inference

Computer Science > Distributed, Parallel, and Cluster Computing arXiv:2603.04716 (cs) [Submitted on 5 Mar 2026] Title:SLO-Aware Compute Resource Allocation for Prefill-Decode Disaggregated LLM Inference Authors:Luchang Li, Dongfang Li, Bozhao Gong, Yu Zhang View a PDF of the paper titled SLO-Aware Compute Resource Allocation for Prefill-Decode Disaggregated LLM Inference, by Luchang Li and 3 other authors View PDF Abstract:Prefill-Decode (P/D) disaggregation has emerged as a widely adopted optimization strategy for Large Language Model (LLM) inference. However, there currently exists no well-established methodology for determining the optimal number of P/D hardware resources, subject to constraints on total throughput, service level objectives (SLOs), and request characteristics - specifically input and output lengths. To address this gap, we propose a hybrid approach that combines theoretical modeling with empirical benchmarking. First, we present a theoretical model for calculating P/D resource counts, which is based on total throughput requirements, request input and output lengths, as well as prefill and decode throughput. Then, to obtain the actual prefill and decode throughput under SLO constraints, we model the prefill process using M/M/1 queuing theory, deriving the achieved prefill throughput from the benchmarked maximum prefill throughput and Time-To-First-Token (TTFT). For the decode phase, we determine the decode batch sizes that meet Time-Per-Output-Token (TPO...