[2410.20791] From Cool Demos to Production-Ready FMware: Core Challenges and a Technology Roadmap

Computer Science > Software Engineering arXiv:2410.20791 (cs) [Submitted on 28 Oct 2024 (v1), last revised 6 Apr 2026 (this version, v3)] Title:From Cool Demos to Production-Ready FMware: Core Challenges and a Technology Roadmap Authors:Gopi Krishnan Rajbahadur, Gustavo A. Oliva, Dayi Lin, Jiho Shin, Ahmed E. Hassan View a PDF of the paper titled From Cool Demos to Production-Ready FMware: Core Challenges and a Technology Roadmap, by Gopi Krishnan Rajbahadur and 4 other authors View PDF HTML (experimental) Abstract:The rapid expansion of foundation models (FMs), such as large language models (LLMs), has given rise to FMware, software systems that integrate FM(s) as core components. While building demonstration-level FMware is relatively straightforward, transitioning to production-ready systems presents numerous challenges, including reliability, high implementation costs, scalability, and compliance with privacy regulations. Our paper conducts a semi-structured thematic synthesis to identify key challenges in productionizing FMware across diverse data sources, including our industry experience developing FMArts, a FMware lifecycle engineering platform, and its integration into Huawei Cloud; grey literature; academic publications; hands-on involvement in the Open Platform for Enterprise AI (OPEA); organizing the AIware conference and bootcamp; and co-leading the ISO SPDX SBOM working group on AI and datasets. We identify critical issues in FM(s) selection, data and model a...