The South African-built tool helping the world’s under-resourced languages enter the AI age

A software tool developed by South African researchers is helping under-resourced languages enter the AI age. TextAugment, created by Professor Vukosi Marivate and researcher Tshephisho Sefara, addresses the lack of digital data needed to train AI systems for languages with limited digital resources. The tool has been downloaded over 286,000 times and is used in projects involving Swahili, Arabic, Uzbek, and other languages.

On July 16, Marivate and the TextAugment team were awarded the inaugural NSTF-SADiLaR Research Software Award for Human Language Technologies for their work developing open-source software that helps create synthetic training data. This recognition comes after Marivate received the Order of Mapungubwe in Silver for his contributions to data science, artificial intelligence, and natural language processing.

African languages are classified as "low-resource" languages in AI research due to the scarcity of digital data and tools that enable machines to process them effectively. Marivate explains that one of the biggest challenges in training robust machine learning models for these languages is data scarcity. TextAugment helps by synthetically expanding existing datasets, replacing words with synonyms while preserving their original meaning.

This technology supports various language-processing applications, including translation, sentiment analysis, and summarization. However, Marivate acknowledges that some augmentation methods have limitations, as replacing a word with a synonym does not always account for the context in which the word is used. His research group is also examining how AI models understand idioms and proverbs in African languages, including the challenges of representing African languages in AI systems and ensuring that technology engages with the cultural context in which those languages are spoken.