[2603.26551] Beyond MACs: Hardware Efficient Architecture Design for Vision Backbones

Computer Science > Computer Vision and Pattern Recognition arXiv:2603.26551 (cs) [Submitted on 27 Mar 2026] Title:Beyond MACs: Hardware Efficient Architecture Design for Vision Backbones Authors:Moritz Nottebaum, Matteo Dunnhofer, Christian Micheloni View a PDF of the paper titled Beyond MACs: Hardware Efficient Architecture Design for Vision Backbones, by Moritz Nottebaum and 2 other authors View PDF HTML (experimental) Abstract:Vision backbone networks play a central role in modern computer vision. Enhancing their efficiency directly benefits a wide range of downstream applications. To measure efficiency, many publications rely on MACs (Multiply Accumulate operations) as a predictor of execution time. In this paper, we experimentally demonstrate the shortcomings of such a metric, especially in the context of edge devices. By contrasting the MAC count and execution time of common architectural design elements, we identify key factors for efficient execution and provide insights to optimize backbone design. Based on these insights, we present LowFormer, a novel vision backbone family. LowFormer features a streamlined macro and micro design that includes Lowtention, a lightweight alternative to Multi-Head Self-Attention. Lowtention not only proves more efficient, but also enables superior results on ImageNet. Additionally, we present an edge GPU version of LowFormer, that can further improve upon its baseline's speed on edge GPU and desktop GPU. We demonstrate LowFormer's w...