{
  "id": 9533639,
  "title": "The Model Was Never the Bottleneck: What Shipping a Text Classifier Into a Government Office Taught",
  "url": "https://urgent.news/2026/09/24/the-model-was-never-the-bottleneck-what-shipping-a-text-classifier",
  "topic": "ai",
  "section": "AI",
  "published": "2026-09-24T06:17:41.000Z",
  "source": {
    "name": "HackerNoon",
    "slug": "hackernoon",
    "url": "https://hackernoon.com/the-model-was-never-the-bottleneck-what-shipping-a-text-classifier-into-a-government-office-taught?source=rss"
  },
  "original_language": "en",
  "account": "The government office receives written appeals from citizens, and someone must read each one to determine its type, assign it to the correct department, and initiate the response process. Existing operators varied in accuracy, labeling appeals 67% correctly on average, with a Fleiss kappa of 0.81 when compared to expert judgments. The goal was not to replace operators but to reduce their workload by automating the boring 80% of the task.\n\nVarious machine learning approaches were tested, including human operators (baseline), Bag-of-Words + SVM, TF-IDF + SVM, fastText, Word2Vec + LSTM, and Multilingual BERT. The smallest model with the fastest training time - fastText - outperformed the other techniques. The fastText model required only three minutes of training and did not necessitate lemmatization, unlike the standard advice for morphologically rich languages. Despite BERT's superior accuracy, the decision was made to deploy fastText due to hardware constraints, training cadence, and the marginal impact of a few percentage points of accuracy on the workflow.\n\nLoad testing demonstrated that the ML component contributed minimally to the overall processing time, with inference taking under two seconds per appeal. The majority of the time was spent on human review. This realization led the authors to focus on reducing human effort rather than improving model accuracy. The most common errors were due to dual intent, bureaucratic register, and mixed framing in the appeals. Recognizing these patterns highlighted the importance of taxonomy in categorizing appeals, as a single label could not accurately represent the diverse intents present in the text.",
  "summary": "We shipped a Word2Vec+LSTM over a more accurate BERT, then load testing showed the model was 0.3% of the wall-clock time. The queue was five human reviewers.",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}