![[2510.12915] Toward LLM-Supported Automated Assessment of Critical Thinking Subskills](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2510.12915] Toward LLM-Supported Automated Assessment of Critical Thinking Subskills
Summary
This paper explores the feasibility of using large language models (LLMs) to automate the assessment of critical thinking subskills in educational contexts, focusing on argumentative essay writing.
Why It Matters
As AI-generated content proliferates, the ability to critically evaluate information becomes essential. This research addresses a gap in educational data mining by proposing methods for assessing critical thinking skills, which are vital for students' success in a digital age. The findings could influence future educational practices and assessment methodologies.
Key Takeaways
- The study investigates automated scoring methods for critical thinking subskills.
- Fine-tuning Llama 3.1 8B model yielded the best results in assessing nuanced proficiency levels.
- The research highlights the importance of combining automated assessments with human validation.
- Critical thinking skills are increasingly necessary in an era of misinformation.
- Future studies should focus on scalable assessment methods in authentic educational settings.
Computer Science > Computers and Society arXiv:2510.12915 (cs) [Submitted on 14 Oct 2025 (v1), last revised 18 Feb 2026 (this version, v2)] Title:Toward LLM-Supported Automated Assessment of Critical Thinking Subskills Authors:Marisa C. Peczuh, Nischal Ashok Kumar, Ryan Baker, Blair Lehman, Danielle Eisenberg, Caitlin Mills, Payu Wittawatolarn, Kushaan Naskar, Keerthi Chebrolu, Sudhip Nashi, Cadence Young, Brayden Liu, Sherry Lachman, Andrew Lan View a PDF of the paper titled Toward LLM-Supported Automated Assessment of Critical Thinking Subskills, by Marisa C. Peczuh and 13 other authors View PDF Abstract:As the world becomes increasingly saturated with AI-generated content, disinformation, and algorithmic persuasion, critical thinking - the capacity to evaluate evidence, detect unreliable claims, and exercise independent judgment - is becoming a defining human skill. Developing critical thinking skills through timely assessment and feedback is crucial; however, there has not been extensive work in educational data mining on defining, measuring, and supporting critical thinking. In this paper, we investigate the feasibility of measuring "subskills" that underlie critical thinking. We ground our work in an authentic task where students operationalize critical thinking by writing argumentative essays. We developed a coding rubric based on an established skills progression and completed human coding for a corpus of student essays. We then evaluated three distinct approaches ...
