How well do AI models read Tanglish? I tested 13 of them
This is a submission for the Kaggle Benchmarking Challenge What I Benchmarked I'm a Tamil speaker. When Tamil people text, we mostly type Tamil words in English letters and throw in English words wherever it is easier. That mix is called Tanglish. "Naalaikku kaalaila ezhara manikku bus" is a normal message. In Tamil script the same thing is நாளைக்கு காலைல ஏழரை மணிக்கு பஸ், and very few people…
A researcher tested 13 AI models on their ability to comprehend Tamil text written in Tanglish, a mix of English and Tamil letters. The study evaluated how the script change affected the models' comprehension. Out of 60 everyday comprehension questions, all models scored perfectly on English versions, but Tanglish versions varied.
Google's Gemini 3.5 Flash and Gemini 3.7 Flash models achieved perfect scores of 100% in both English and Tanglish. Claude Sonnet 5, Gemma 4 26B, and Gemini 3.1 Flash-Lite models scored 98% in both scripts, missing at most one question each. The small Gemma models performed best, while Claude Haiku 4.5 and gpt-oss-20b scored significantly lower, with 93% and 70% respectively.
The research highlights that Google's models show minimal impact from the script change, while smaller models struggle more with colloquial Tamil and Tanglish versions.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
