{
  "id": 19640444,
  "title": "Building a Private Classifier with Apple's On-Device Model and NLContextualEmbedding",
  "url": "https://urgent.news/2026/11/12/building-a-private-classifier-with-apples-on-device-model-and",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-11-12T11:00:25.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/backlog/building-a-private-classifier-with-apples-on-device-model-and-nlcontextualembedding-4ago"
  },
  "original_language": "en",
  "account": "Apple's on-device model is a small language model that runs offline on macOS and iOS devices, costing nothing per call and never sending user data anywhere. This article discusses its capabilities, limitations, and how to build a classification engine using two components: NLContextualEmbedding and the Foundation Models framework.\n\nThe model has approximately 3 billion parameters and a context window of about 4,096 tokens at launch, covering the instructions, prompt, and response together. However, the context size may change between OS versions, so it's essential to check it for your target version. Long documents won't fit, so plan to chunk them.\n\nThe model runs on the Neural Engine and GPU, making it fast enough for interactive use on Apple silicon. However, throughput varies depending on the hardware and output length, so it's crucial to measure on your own device before relying on specific numbers.\n\nThe model is well-suited for classifying and tagging text, extracting structured fields from short text, summarizing or rewriting short passages, and any use case that requires privacy or offline processing. It's not ideal for broad world knowledge, heavy reasoning or math, or long documents without chunking.\n\nThe key feature of this model is structured output. With the @Generable protocol, it can return typed Swift values instead of free text. For classification purposes, it means the model can only respond with predefined category names and confidence levels.\n\nTo build a classification engine, import the FoundationModels framework and define a struct that conforms to @Generable, specifying the allowed category names and confidence range. Create an instance of LanguageModelSession with instructions for the expected categories. Generate a Verdict struct to receive the classification results.\n\nBefore running the full language model for every item, consider using NLContextualEmbedding, a BERT-style model built into the NaturalLanguage framework. It converts text into vectors, allowing you to score items by similarity to each category without any generation. Embed each category once and cache their vectors. Then, embed the input text and compute cosine similarity against every category.\n\nIf the top score clearly outperforms the second, accept it as the classification. However, if the top two scores are close, use the LanguageModelSession to ask the model to choose between just those two candidates, using the @Generable struct for a more reliable answer. This approach is fast, uses no generation, and minimizes the risk of misclassification.",
  "summary": "Apple's on-device model is a small language model that ships with macOS and iOS. It runs offline, costs nothing per call, and never sends your data anywhere. This article covers what it is good at, where it falls short, and how to build a classification engine from two pieces: NLContextualEmbedding and the Foundation Models framework. How good is it? Size and context. The model has roughly 3…",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}