University of Tartu thesis: transfer learning boosts Estonian AI models
NewsERR{{1775739600000 | amCalendar}}{{contentCtrl.likes}} share FacebookX (Twitter)e-mail The University of Tartu's main building. Source: Simo Sepp/Minupilt.err.ee NewsA doctoral thesis at the University of Tartu reveals that effective Estonian-language artificial intelligence models can be developed despite limited data by utilizing cross-lingual transfer learning. Modern language models require vast amounts of text, but Estonian, like many small languages, lacks sufficient digital data. This creates a key challenge regarding how to build capable models with scarce training data.According to the recently defended thesis author, Hele-Andra Kuulmets, the solution lies not just in collecting more data, but in combining existing resources more intelligently.Most language model methods have been developed for English and cannot be directly applied to smaller languages. This is where transfer learning comes in — reusing knowledge learned from one language to improve models in another. When models are trained on multiple languages, their internal representations begin to align, allowing what is learned in one language to support understanding in others.Keelemudelid Autor/allikas: AIrika Harrik/ERRKuulmets' results show this works well in practice. The best-performing models used multilingual data in two stages: large-scale pretraining followed by fine-tuning for a specific language. These clearly outperformed models trained only on Estonian data.The study found that large mode...
