{
  "id": 18010740,
  "title": "Google Launches LiteRT Tensor API for Building, Customizing, and Optimizing ML Models for On-Device",
  "url": "https://urgent.news/2026/11/03/google-launches-litert-tensor-api-for-building-customizing-and",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-11-03T19:00:00.000Z",
  "source": {
    "name": "InfoQ",
    "slug": "infoq",
    "url": "https://www.infoq.com/news/2026/11/litert-tensor-api/"
  },
  "original_language": "en",
  "account": "Google has unveiled the LiteRT Tensor API, a lightweight C++ API designed for constructing, tailoring, and optimizing machine learning models to run on devices locally. This new API empowers developers to craft, adapt, and enhance machine learning models natively for on-device processing. Highlighting its flexibility and cross-platform compatibility, Google asserts that the LiteRT Tensor API merges the freedom of native model creation with universal applicability. In contrast to the conventional method where developers manually encode computations in C++ for precise control, leading to backend-specific implementations (e.g., Metal, OpenCL, or Vulkan) that must be rewritten for each hardware, the LiteRT Tensor API simplifies this process. By constructing computation graphs operationally in C++, developers gain superior control over model architecture, optimization, and accelerator execution. Notably, the API supports direct weight ingestion, enabling the loading of weights from checkpoints, including Hugging Face .safetensors files, and offers zero-copy buffer interoperability to prevent data duplication across preprocessing, inference, and postprocessing stages. Moreover, LiteRT Tensor API is compatible with AI coding agents capable of constructing, modifying, validating, and benchmarking model graphs by generating C++ code. Google demonstrated the API's capabilities by creating a real-world application for multi-object segmentation and tracking using a coding agent. This application was subsequently used to benchmark performance across three platforms: a macOS laptop (Metal), an iOS phone (Metal), and an Android phone (OpenCL). The tests revealed significant speed enhancements, with the MacBook Pro runtime running approximately 1.5 times faster than Apple MLX in fp16 mode, involving both image encoding and mask decoding. Laurence Moroney, Arm's director of AI, underscored the significance of starting with model conversion and utilizing the C++ API only when additional graph-level control or specific pre and post-processing on the accelerator is required. He also noted the transformative impact of defining model graphs in plain C++: transitioning from under 10 frames per second to over 30 frames per second at 512x512 resolution on WebGPU, with the coding agent significantly accelerating the optimization process. Google is also actively developing further enhancements, such as direct ML Drift backend support, additional NPU-friendly operations, automatic full-graph quantization, and expanded dynamic-shape support for varying image sizes, batch sizes, and sequence lengths.",
  "summary": "Google has introduced the LiteRT Tensor API, a lightweight C++ API that allows developers to build, customize, and optimize machine learning models directly for on-device execution. According to Google, it combines the flexibility of native model authoring with cross-platform portability. By Sergio De Simone",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}