[2603.02830] Faster, Cheaper, More Accurate: Specialised Knowledge Tracing Models Outperform LLMs

Computer Science > Computation and Language arXiv:2603.02830 (cs) [Submitted on 3 Mar 2026] Title:Faster, Cheaper, More Accurate: Specialised Knowledge Tracing Models Outperform LLMs Authors:Prarthana Bhattacharyya, Joshua Mitton, Ralph Abboud, Simon Woodhead View a PDF of the paper titled Faster, Cheaper, More Accurate: Specialised Knowledge Tracing Models Outperform LLMs, by Prarthana Bhattacharyya and 2 other authors View PDF HTML (experimental) Abstract:Predicting future student responses to questions is particularly valuable for educational learning platforms where it enables effective interventions. One of the key approaches to do this has been through the use of knowledge tracing (KT) models. These are small, domain-specific, temporal models trained on student question-response data. KT models are optimised for high accuracy on specific educational domains and have fast inference and scalable deployments. The rise of Large Language Models (LLMs) motivates us to ask the following questions: (1) How well can LLMs perform at predicting students' future responses to questions? (2) Are LLMs scalable for this domain? (3) How do LLMs compare to KT models on this domain-specific task? In this paper, we compare multiple LLMs and KT models across predictive performance, deployment cost, and inference speed to answer the above questions. We show that KT models outperform LLMs with respect to accuracy and F1 scores on this domain-specific task. Further, we demonstrate that LLMs...