[2605.00005] Cloud Is Closer Than It Appears: Revisiting the Tradeoffs of Distributed Real-Time Inference

Computer Science > Machine Learning arXiv:2605.00005 (cs) [Submitted on 17 Feb 2026] Title:Cloud Is Closer Than It Appears: Revisiting the Tradeoffs of Distributed Real-Time Inference Authors:Pragya Sharma, Hang Qiu, Mani Srivastava View a PDF of the paper titled Cloud Is Closer Than It Appears: Revisiting the Tradeoffs of Distributed Real-Time Inference, by Pragya Sharma and 2 other authors View PDF HTML (experimental) Abstract:The increasing deployment of deep neural networks (DNNs) in cyber-physical systems (CPS) enhances perception fidelity, but imposes substantial computational demands on execution platforms, posing challenges to real-time control deadlines. Traditional distributed CPS architectures typically favor on-device inference to avoid network variability and contention-induced delays on remote platforms. However, this design choice places significant energy and computational demands on the local hardware. In this work, we revisit the assumption that cloud-based inference is intrinsically unsuitable for latency-sensitive control tasks. We demonstrate that, when provisioned with high-throughput compute resources, cloud platforms can effectively amortize network and queueing delays, enabling them to match or surpass on-device performance for real-time decision-making. Specifically, we develop a formal analytical model that characterizes distributed inference latency as a function of the sensing frequency, platform throughput, network delay, and task-specific saf...