![[2602.16140] Human-AI Collaboration in Large Language Model-Integrated Building Energy Management Systems: The Role of User Domain Knowledge and AI Literacy](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.16140] Human-AI Collaboration in Large Language Model-Integrated Building Energy Management Systems: The Role of User Domain Knowledge and AI Literacy
Summary
This study explores how user domain knowledge and AI literacy influence the effectiveness of human-AI interactions in building energy management systems integrated with large language models.
Why It Matters
Understanding the dynamics of human-AI collaboration is crucial as it can enhance the efficiency of energy management systems. This research highlights the importance of user knowledge and AI literacy, which can inform the design of more effective AI tools for energy conservation.
Key Takeaways
- User domain knowledge and AI literacy significantly impact interactions with AI in energy management.
- Participants primarily used concise prompts, indicating a reliance on AI's analytical capabilities.
- Only one out of twenty metrics showed significant differences based on AI literacy, suggesting LLMs can level the playing field.
- The study employed a systematic role-playing experiment to gather data on human-AI interactions.
- Insights from this research can guide the development of user-centric AI systems in energy management.
Computer Science > Human-Computer Interaction arXiv:2602.16140 (cs) [Submitted on 18 Feb 2026] Title:Human-AI Collaboration in Large Language Model-Integrated Building Energy Management Systems: The Role of User Domain Knowledge and AI Literacy Authors:Wooyoung Jung, Kahyun Jeon, Prosper Babon-Ayeng View a PDF of the paper titled Human-AI Collaboration in Large Language Model-Integrated Building Energy Management Systems: The Role of User Domain Knowledge and AI Literacy, by Wooyoung Jung and 2 other authors View PDF Abstract:This study aimed to comprehend how user domain knowledge and artificial intelligence (AI) literacy impact the effective use of human-AI interactive building energy management system (BEMS). While prior studies have investigated the potential of integrating large language models (LLMs) into BEMS or building energy modeling, very few studies have examined how user interact with such systems. We conducted a systematic role-playing experiment, where 85 human subjects interacted with an advanced generative pre-trained transformer (OpenAI GPT-4o). Participants were tasked with identifying the top five behavioral changes that could reduce home energy use with the GPT model that functioned as an LLM-integrated BEMS. Then, the collected prompt-response data and participant conclusions were analyzed using an analytical framework that hierarchically assessed and scored human-AI interactions and their home energy analysis approaches. Also, participants were class...
