[2504.05995] NativQA Framework: Enabling LLMs and VLMs with Native, Local, and Everyday Knowledge

Computer Science > Computation and Language arXiv:2504.05995 (cs) [Submitted on 8 Apr 2025 (v1), last revised 7 Apr 2026 (this version, v3)] Title:NativQA Framework: Enabling LLMs and VLMs with Native, Local, and Everyday Knowledge Authors:Firoj Alam, Md Arid Hasan, Sahinur Rahman Laskar, Mucahid Kutlu, Kareem Darwish, Shammur Absar Chowdhury View a PDF of the paper titled NativQA Framework: Enabling LLMs and VLMs with Native, Local, and Everyday Knowledge, by Firoj Alam and 5 other authors View PDF Abstract:The rapid progress of large language models (LLMs) raises concerns about cultural bias, fairness, and performance in diverse languages and underrepresented regions. Addressing these gaps requires large-scale resources grounded in multilingual, local, and cultural contexts. We systematize and extend the earlier NativQA framework to multimodality by adding image, audio, and video support, enabling scalable construction of culturally and regionally aligned QA datasets in native languages. Given user-defined seed queries, the framework uses search engines to collect location-specific everyday information. We evaluate it across 39 locations in 24 countries and 7 languages, spanning extremely low-resource to high-resource settings, and collect over $\sim$300K text QA pairs, $\sim$312K images, and $\sim$29K videos with associated audio. The developed resources can be used for LLMs benchmarking and further fine-tuning. The framework has been made publicly available for the com...