![[2602.19843] MAS-FIRE: Fault Injection and Reliability Evaluation for LLM-Based Multi-Agent Systems](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.19843] MAS-FIRE: Fault Injection and Reliability Evaluation for LLM-Based Multi-Agent Systems
Summary
The paper presents MAS-FIRE, a framework for evaluating the reliability of LLM-based Multi-Agent Systems through fault injection, addressing semantic failures that traditional methods overlook.
Why It Matters
As LLM-based Multi-Agent Systems become more prevalent in complex tasks, understanding their reliability is crucial. MAS-FIRE offers a systematic approach to identify and mitigate failures, enhancing the robustness of these systems, which is vital for their effective deployment in real-world applications.
Key Takeaways
- MAS-FIRE introduces a framework for fault injection in Multi-Agent Systems.
- A taxonomy of 15 fault types helps diagnose cognitive and coordination errors.
- Stronger foundation models do not always lead to improved robustness.
- Architectural design significantly impacts fault tolerance.
- The framework provides actionable insights for improving system reliability.
Computer Science > Software Engineering arXiv:2602.19843 (cs) [Submitted on 23 Feb 2026] Title:MAS-FIRE: Fault Injection and Reliability Evaluation for LLM-Based Multi-Agent Systems Authors:Jin Jia, Zhiling Deng, Zhuangbin Chen, Yingqi Wang, Zibin Zheng View a PDF of the paper titled MAS-FIRE: Fault Injection and Reliability Evaluation for LLM-Based Multi-Agent Systems, by Jin Jia and 4 other authors View PDF HTML (experimental) Abstract:As LLM-based Multi-Agent Systems (MAS) are increasingly deployed for complex tasks, ensuring their reliability has become a pressing challenge. Since MAS coordinate through unstructured natural language rather than rigid protocols, they are prone to semantic failures (e.g., hallucinations, misinterpreted instructions, and reasoning drift) that propagate silently without raising runtime exceptions. Prevailing evaluation approaches, which measure only end-to-end task success, offer limited insight into how these failures arise or how effectively agents recover from them. To bridge this gap, we propose MAS-FIRE, a systematic framework for fault injection and reliability evaluation of MAS. We define a taxonomy of 15 fault types covering intra-agent cognitive errors and inter-agent coordination failures, and inject them via three non-invasive mechanisms: prompt modification, response rewriting, and message routing manipulation. Applying MAS-FIRE to three representative MAS architectures, we uncover a rich set of fault-tolerant behaviors that we...
