[2603.20821] Compass: Optimizing Compound AI Workflows for Dynamic Adaptation

Computer Science > Distributed, Parallel, and Cluster Computing arXiv:2603.20821 (cs) [Submitted on 21 Mar 2026] Title:Compass: Optimizing Compound AI Workflows for Dynamic Adaptation Authors:Milos Gravara, Juan Luis Herrera, Stefan Nastic View a PDF of the paper titled Compass: Optimizing Compound AI Workflows for Dynamic Adaptation, by Milos Gravara and 2 other authors View PDF HTML (experimental) Abstract:Compound AI is a distributed intelligence approach that represents a unified system orchestrating specialized AI/ML models with engineered software components into AI workflows. Compound AI production deployments must satisfy accuracy, latency, and cost objectives under varying loads. However, many deployments operate on fixed infrastructure where horizontal scaling is not viable. Existing approaches optimize solely for accuracy and do not consider changes in workload conditions. We observe that compound AI systems can switch between configurations to fit infrastructure capacity, trading accuracy for latency based on current load. This requires discovering multiple Pareto-optimal configurations from a combinatorial search space and determining when to switch between them at runtime. We present Compass, a novel framework that enables dynamic configuration switching through offline optimization and online adaptation. Compass consists of three components: COMPASS-V algorithm for configuration discovery, Planner for switching policy derivation, and Elastico Controller for ...