{
  "id": 5998794,
  "title": "I Built a Tiny GPT That Speaks Sanskrit in a Weekend — Here’s What Broke",
  "url": "https://urgent.news/2026/09/06/i-built-a-tiny-gpt-that-speaks-sanskrit-in-a-weekend-heres-what-broke",
  "topic": "ai",
  "section": "AI",
  "published": "2026-09-06T16:00:00.000Z",
  "source": {
    "name": "HackerNoon",
    "slug": "hackernoon",
    "url": "https://hackernoon.com/i-built-a-tiny-gpt-that-speaks-sanskrit-in-a-weekend-heres-what-broke?source=rss"
  },
  "original_language": "en",
  "account": "A weekend project resulted in a tiny GPT capable of predicting the next Sanskrit syllable. The author, familiar with transformers, wanted to test their understanding by building a model from scratch. The goal was to predict the next syllable, with the model trained on Sanskrit text. However, the complexity lay in dealing with Sanskrit's unique character structure. A \"character\" in Sanskrit is not always a single Unicode code point, as several code points can represent a single syllable. This led to a bug in the initial attempt, where the model struggled to learn the correct spelling patterns and generated nonsensical output. The solution was to use grapheme clusters, which represent what humans perceive as a single character. This change improved the model's performance significantly. The author emphasizes that tokenization choice is critical in building such a model, as it directly impacts the quality of the generated text. Despite the model being relatively simple, the data preparation process was the most challenging part. Converting PDFs to clean Sanskrit text was arduous and required handling various data issues like scanned images, non-Unicode fonts, and reordered vowel signs. The author concludes that data preparation should be prioritized over model building when embarking on a similar project.",
  "summary": "The off-the-shelf models are bad at Sanskrit largely because of tokenization and data scarcity",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}