![[2602.13334] Ask the Expert: Collaborative Inference for Vision Transformers with Near-Edge Accelerators](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.13334] Ask the Expert: Collaborative Inference for Vision Transformers with Near-Edge Accelerators
Summary
This article presents a collaborative inference framework for deploying Vision Transformers on edge devices, addressing computational challenges and latency issues through a novel routing mechanism and specialist training strategy.
Why It Matters
As the demand for efficient AI processing on edge devices grows, this research offers a significant advancement in deploying Vision Transformers, balancing performance and resource constraints. The findings can influence future designs in edge computing and AI applications, making them more practical for real-world use.
Key Takeaways
- Introduces a collaborative inference framework for Vision Transformers.
- Achieves a 4.12% improvement in expert specialization accuracy.
- Reduces latency by up to 45% and energy consumption by 46%.
- Utilizes a novel routing mechanism for selecting relevant experts.
- Demonstrates effectiveness through extensive experiments on CIFAR-100.
Computer Science > Computer Vision and Pattern Recognition arXiv:2602.13334 (cs) [Submitted on 11 Feb 2026] Title:Ask the Expert: Collaborative Inference for Vision Transformers with Near-Edge Accelerators Authors:Hao Liu, Suhaib A. Fahmy View a PDF of the paper titled Ask the Expert: Collaborative Inference for Vision Transformers with Near-Edge Accelerators, by Hao Liu and Suhaib A. Fahmy View PDF HTML (experimental) Abstract:Deploying Vision Transformers on edge devices is challenging due to their high computational complexity, while full offloading to cloud resources presents significant latency overheads. We propose a novel collaborative inference framework, which orchestrates a lightweight generalist ViT on an edge device and multiple medium-sized expert ViTs on a near-edge accelerator. A novel routing mechanism uses the edge model's Top-$\mathit{k}$ predictions to dynamically select the most relevant expert for samples with low confidence. We further design a progressive specialist training strategy to enhance expert accuracy on dataset subsets. Extensive experiments on the CIFAR-100 dataset using a real-world edge and near-edge testbed demonstrate the superiority of our framework. Specifically, the proposed training strategy improves expert specialization accuracy by 4.12% on target subsets and enhances overall accuracy by 2.76% over static experts. Moreover, our method reduces latency by up to 45% compared to edge execution, and energy consumption by up to 46% com...
