🇵🇭 FilBench - Can LLMs Understand and Generate Filipino?

Back to Articles 🇵🇭 FilBench - Can LLMs Understand and Generate Filipino? Published August 12, 2025 Update on GitHub Upvote 23 +17 Lj V. Miranda ljvmiranda921 Follow guest Elyanah Aco acocodes Follow guest Conner Manuel connermanuel Follow guest Jan Christian Blaise Cruz jcblaise Follow SEACrowd Jan Christian Blaise Cruz jcblaise Follow SEACrowd Joseph Imperial josephimperial Follow SEACrowd Daniel van Strien davanstrien Follow Nathan Habib SaylorTwift Follow Clémentine Fourrier clefourrier Follow As large language models (LLMs) become increasingly integrated into our lives, it becomes crucial to assess whether they reflect the nuances and capabilities of specific language communities. For example, Filipinos are among the most active ChatGPT users globally, ranking fourth in ChatGPT traffic (behind the United States, India, and Brazil [1] [2]), but despite this strong usage, we lack a clear understanding of how LLMs perform for their languages, such as Tagalog and Cebuano. Most of the existing evidence is anecdotal, such as screenshots of ChatGPT responding in Filipino as proof that it is fluent. What we need instead is a systematic evaluation of LLM capabilities in Philippine languages. That’s why we developed FilBench: a comprehensive evaluation suite to assess the capabilities of LLMs for Tagalog, Filipino (the standardized form of Tagalog), and Cebuano, on fluency, linguistic and translation abilities, as well as specific cultural knowledge. We used it to evaluate 20+ ...