[2603.18873] Evaluating LLM-Generated Lessons from the Language Learning Students' Perspective: A Short Case Study on Duolingo

Computer Science > Computation and Language arXiv:2603.18873 (cs) [Submitted on 19 Mar 2026 (v1), last revised 22 Mar 2026 (this version, v2)] Title:Evaluating LLM-Generated Lessons from the Language Learning Students' Perspective: A Short Case Study on Duolingo Authors:Carlos Rafael Catalan, Patricia Nicole Monderin, Lheane Marie Dizon, Gap Estrella, Raymund John Sarmimento, Marie Antoinette Patalagsa View a PDF of the paper titled Evaluating LLM-Generated Lessons from the Language Learning Students' Perspective: A Short Case Study on Duolingo, by Carlos Rafael Catalan and 5 other authors View PDF HTML (experimental) Abstract:Popular language learning applications such as Duolingo use large language models (LLMs) to generate lessons for its users. Most lessons focus on general real-world scenarios such as greetings, ordering food, or asking directions, with limited support for profession-specific contexts. This gap can hinder learners from achieving professional-level fluency, which we define as the ability to communicate comfortably various work-related and domain-specific information in the target language. We surveyed five employees from a multinational company in the Philippines on their experiences with Duolingo. Results show that respondents encountered general scenarios more frequently than work-related ones, and that the former are relatable and effective in building foundational grammar, vocabulary, and cultural knowledge. The latter helps bridge the gap toward p...