{
  "id": 19492387,
  "title": "Google gooses Python AI performance by... removing Python",
  "url": "https://urgent.news/2026/11/11/google-gooses-python-ai-performance-by-removing-python-19492387",
  "topic": "ai",
  "section": "AI",
  "published": "2026-11-11T11:30:00.000Z",
  "source": {
    "name": "The Register",
    "slug": "the-register",
    "url": "https://www.theregister.com/devops/2026/11/11/google-gooses-python-ai-performance-by-removing-python/5308995"
  },
  "original_language": "en",
  "account": "Google's approach to AI performance is to train models in Python and serve them using C++. While Python is user-friendly for data scientists, incorporating its interpreter can introduce overhead and deployment challenges. To address this, the OpenXLA Project offers tools for compiling machine learning computations into native code for various processors. Google has now open sourced XSE, or the XLA Serving Engine, which is particularly well-suited for smaller, task-oriented models.\n\nLLM serving tends to garner more attention, but many production inferences involve ranking, recommendation, embedding, and vision models. These models deal with short requests and have tight latency budgets. Google's XSE has already managed more than six million queries per second across services such as YouTube Shorts, Google Ads, and Search. In Google's tests, XSE delivered two to five times the throughput of the best optimized Python server while using a quarter of the memory.\n\nIn this setup, ML engineers write their model code in Python using Google's JAX for computation. They also plan to integrate TorchTPU, allowing PyTorch users to export models for C++ serving on TPUs. An export tool packages the model's computations in StableHLO, a portable intermediate representation, alongside trained weights in an XLA SavedModel. The C++ serving runtime then uses XLA to compile these computations into native code for the target silicon, bypassing the need for a Python interpreter. Both GPUs and CPUs can run XLA SavedModels, and Google has partnered with Arm and Intel to develop the XLA ecosystem. However, Google prefers running models on its own TPU accelerators.",
  "summary": "Newly open sourced XSE handles six million queries a second across services including Search, Ads, and YouTube Shorts",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}