[2503.07928] The StudyChat Dataset: Analyzing Student Dialogues With ChatGPT in an Artificial Intelligence Course

Computer Science > Artificial Intelligence arXiv:2503.07928 (cs) [Submitted on 11 Mar 2025 (v1), last revised 4 Mar 2026 (this version, v4)] Title:The StudyChat Dataset: Analyzing Student Dialogues With ChatGPT in an Artificial Intelligence Course Authors:Hunter McNichols, Fareya Ikram, Andrew Lan View a PDF of the paper titled The StudyChat Dataset: Analyzing Student Dialogues With ChatGPT in an Artificial Intelligence Course, by Hunter McNichols and 2 other authors View PDF HTML (experimental) Abstract:The widespread availability of large language models (LLMs), such as ChatGPT, has significantly impacted education, raising both opportunities and challenges. Students can frequently interact with LLM-powered, interactive learning tools, but their usage patterns need to be observed and understood. We introduce StudyChat, a publicly available dataset capturing real-world student interactions with an LLM-powered tutoring chatbot in a semester-long, university-level artificial intelligence (AI) course. We deploy a web application that replicates ChatGPT's core functionalities, and use it to log student interactions with the LLM while working on programming assignments. We collect 16,851 interactions, which we annotate using a dialogue act labeling schema inspired by observed interaction patterns and prior research. We analyze these interactions, highlight usage trends, and analyze how specific student behavior correlates with their course outcome. We find that students who pr...