{
  "id": 12612292,
  "title": "Building Named Entity Recognition for Code-Mixed Text with XLM-RoBERTa",
  "url": "https://urgent.news/2026/10/07/building-named-entity-recognition-for-code-mixed-text-with-xlm-roberta",
  "topic": "ai",
  "section": "AI",
  "published": "2026-10-07T06:00:02.000Z",
  "source": {
    "name": "HackerNoon",
    "slug": "hackernoon",
    "url": "https://hackernoon.com/building-named-entity-recognition-for-code-mixed-text-with-xlm-roberta?source=rss"
  },
  "original_language": "en",
  "account": "Named Entity Recognition (NER) is a technique used to identify different types of entities, such as people, places, and organizations, within a given text. One of the most significant challenges with this project was the limited availability of datasets containing rich Roman Urdu–English code-mixed text. Code-mixed text refers to the practice of using multiple languages, in this case, English and Roman Urdu, within the same sentence. This poses difficulties for NER because Roman Urdu can be written in various ways, causing inconsistencies in the data. Moreover, people may spell Roman Urdu words differently based on their personal writing style, further complicating the learning process for a model. The project aimed to develop a system capable of recognizing three entity types: Person (PER), Location (LOC), and Organization (ORG), while handling the numerous spelling variations and language shifts present in the data. The main challenge in this project was the myriad of spelling variations in Roman Urdu, as different individuals may use their unique writing styles for the same word. This makes it difficult for a model to identify consistent patterns. Additionally, Roman Urdu and English can coexist within a single sentence, necessitating the model to learn contextual patterns rather than relying on memorization of entity names. To address these issues, the project employed the BIO format for dataset creation, which assigns labels to each token in a sentence. The B label indicates the beginning of an entity, I denotes a token inside or continuing an entity, and O signifies a token outside an entity. For example, the name \"Mehwish Faryad\" can be represented as Mehwish — B-PER and Faryad — I-PER. Tokens not belonging to an entity receive the O label. The dataset comprised 4,348 sentences, 138,879 tokens, and 7,936 named entities. The project focused on three entity types: Person (PER), Location (LOC), and Organization (ORG). XLM-RoBERTa was selected for the project due to its ability to process multilingual data, making it suitable for code-mixed text containing both Roman Urdu and English. XLM-RoBERTa is a multilingual transformer model pre-trained on a vast dataset of languages, allowing it to learn patterns across different languages. This model was fine-tuned to predict BIO labels for each token in the input text, enabling it to recognize named entities like Person, Location, and Organization. The system architecture begins with the user's Roman Urdu–English code-mixed text. The text undergoes preprocessing, normalizing the input and eliminating unnecessary characters. The preprocessed text is then segmented into smaller components and passed through the XLM-RoBERTa tokenizer. The tokenized input is processed by the fine-tuned XLM-RoBERTa model, which predicts BIO labels for each token. These predictions are subsequently reconstructed into recognized entities such as Person, Location, and Organization. The Flask API manages the model inference and delivers the results to the web interface, where the recognized entities can be displayed to the user. The model was trained on Google Colab and developed and tested on a local laptop. The training process utilized Hugging Face Transformers and tokenizers for model training and text processing. Initially, the dataset was cleaned, and irrelevant sentences with only O tags were removed. The BIO-formatted dataset was then created using Python, and sentences containing only O labels were retained to ensure the model could differentiate between entities and non-entities. During training, overfitting was observed, and K-fold cross-validation was employed to improve model generalization. GPU availability and stability during training posed challenges, leading to the decision to utilize Google Colab for training, while the local laptop primarily handled development and testing.",
  "summary": "Build a Named Entity Recognition system for Roman Urdu-English code-mixed text using XLM-RoBERTa, BIO tagging, and Flask.",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}