![[2602.15294] EAA: Automating materials characterization with vision language model agents](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.15294] EAA: Automating materials characterization with vision language model agents
Summary
The paper introduces Experiment Automation Agents (EAA), a system leveraging vision-language models to automate complex microscopy workflows, enhancing efficiency and accessibility in materials characterization.
Why It Matters
This research is significant as it addresses the challenges of automating experimental procedures in microscopy, which can be complex and require specialized knowledge. By integrating multimodal reasoning and user-guided interactions, EAA aims to democratize access to advanced microscopy techniques, potentially transforming research methodologies in materials science.
Key Takeaways
- EAA automates microscopy workflows, reducing the need for expert intervention.
- The system integrates multimodal reasoning for enhanced task execution.
- EAA supports both fully autonomous and user-guided experimental procedures.
- The architecture allows for flexible task management and tool compatibility.
- Demonstrated applications show improved efficiency in beamline operations.
Computer Science > Artificial Intelligence arXiv:2602.15294 (cs) [Submitted on 17 Feb 2026] Title:EAA: Automating materials characterization with vision language model agents Authors:Ming Du, Yanqi Luo, Srutarshi Banerjee, Michael Wojcik, Jelena Popovic, Mathew J. Cherukara View a PDF of the paper titled EAA: Automating materials characterization with vision language model agents, by Ming Du and 5 other authors View PDF HTML (experimental) Abstract:We present Experiment Automation Agents (EAA), a vision-language-model-driven agentic system designed to automate complex experimental microscopy workflows. EAA integrates multimodal reasoning, tool-augmented action, and optional long-term memory to support both autonomous procedures and interactive user-guided measurements. Built on a flexible task-manager architecture, the system enables workflows ranging from fully agent-driven automation to logic-defined routines that embed localized LLM queries. EAA further provides a modern tool ecosystem with two-way compatibility for Model Context Protocol (MCP), allowing instrument-control tools to be consumed or served across applications. We demonstrate EAA at an imaging beamline at the Advanced Photon Source, including automated zone plate focusing, natural language-described feature search, and interactive data acquisition. These results illustrate how vision-capable agents can enhance beamline efficiency, reduce operational burden, and lower the expertise barrier for users. Subject...