[2603.23271] A Multimodal Framework for Human-Multi-Agent Interaction

Computer Science > Robotics arXiv:2603.23271 (cs) [Submitted on 24 Mar 2026] Title:A Multimodal Framework for Human-Multi-Agent Interaction Authors:Shaid Hasan, Breenice Lee, Sujan Sarker, Tariq Iqbal View a PDF of the paper titled A Multimodal Framework for Human-Multi-Agent Interaction, by Shaid Hasan and 3 other authors View PDF HTML (experimental) Abstract:Human-robot interaction is increasingly moving toward multi-robot, socially grounded environments. Existing systems struggle to integrate multimodal perception, embodied expression, and coordinated decision-making in a unified framework. This limits natural and scalable interaction in shared physical spaces. We address this gap by introducing a multimodal framework for human-multi-agent interaction in which each robot operates as an autonomous cognitive agent with integrated multimodal perception and Large Language Model (LLM)-driven planning grounded in embodiment. At the team level, a centralized coordination mechanism regulates turn-taking and agent participation to prevent overlapping speech and conflicting actions. Implemented on two humanoid robots, our framework enables coherent multi-agent interaction through interaction policies that combine speech, gesture, gaze, and locomotion. Representative interaction runs demonstrate coordinated multimodal reasoning across agents and grounded embodied responses. Future work will focus on larger-scale user studies and deeper exploration of socially grounded multi-agent ...