{
  "id": 5070528,
  "title": "On-Device AI in Kotlin",
  "url": "https://urgent.news/2026/09/02/on-device-ai-in-kotlin",
  "topic": "ai",
  "section": "AI",
  "published": "2026-09-02T11:03:27.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/pielounw/on-device-ai-in-kotlin-189o"
  },
  "original_language": "en",
  "account": "This Kotlin tutorial teaches readers how to run a large language model (LLM) directly on a user's device, eliminating the need for servers or API keys. It begins with a simple chat exchange and gradually introduces advanced features such as multimodal input, speech-to-text, text-to-speech, voice activity detection, tool calling, and RAG. Before diving into the code, each concept is thoroughly explained, making it accessible for beginners in on-device AI. Running AI on-device offers several advantages: it works offline, ensures privacy by keeping user data on the device, provides low latency, and eliminates cloud costs. However, on-device models are smaller and less powerful than frontier cloud models. NobodyWho library is used throughout the tutorial, wrapping llama.cpp in Rust and offering bindings for Kotlin, Python, Expo/React Native, Swift, Flutter & Godot. It supports running models locally in .gguf format on Android and desktop JVM (Linux, macOS, Windows). The tutorial includes instructions on how to add the library to your build file, load a model, and communicate with it. For Android, lifecycleScope or viewModelScope should be used instead of runBlocking. In a real chat interface, streaming tokens as they arrive is essential to avoid a blank screen until generation finishes. Multimodal models can ingest images and audio by using a multimodal LLM and its projection model, which converts images and/or audio into tokens the LLM can consume. Speech to text can be achieved by integrating Whisper models in ONNX format through SpeechToText.",
  "summary": "In this Kotlin tutorial, you'll learn how to run a large language model (LLM) directly on a user's device: no server, no API key needed. We'll start from scratch with a simple chat exchange, and progressively introduce more advanced features: multimodal input, speech-to-text, text-to-speech, voice activity detection, tool calling and RAG. Each concept is explained before the code, so you can…",
  "key_points": [
    "Kotlin tutorial teaches running LLM on user device",
    "Advantages: offline, privacy, low latency, no cloud costs",
    "NobodyWho library wraps llama.cpp for Kotlin, Python, React Native"
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}