{
  "id": 5330196,
  "title": "Running Whisper + LLMs on an AMD NPU under Linux",
  "url": "https://urgent.news/2026/09/03/running-whisper-llms-on-an-amd-npu-under-linux",
  "topic": "ai",
  "section": "AI",
  "published": "2026-09-03T12:40:54.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/jac-76/running-whisper-llms-on-an-amd-npu-under-linux-2o1h"
  },
  "original_language": "en",
  "account": "On a MSI Stealth A16 AI+ laptop with Ryzen AI 9 365 APU and XDNA2 NPU, transcribing a 30-second clip using OpenAI's whisper-large-v3-turbo achieved a real-time factor (RTF) of approximately 0.18, which is roughly one-tenth of the energy cost on a CPU, while also allowing an LLM to answer through an OpenAI-compatible API. This end-to-end setup runs locally and offline on Linux. The key components include the kernel driver amdxdna (in-tree since Linux 6.14), XRT (Xilinx/AMD Runtime) and the xrt-plugin-amdxdna shim, both installed via pacman on Arch Linux. The crucial gotcha is setting an unlimited memlock rlimit in the limits.conf file, which is typically too low for NPU tasks. FastFlowLM simplifies the process by providing prebuilt xclbins and libwhisper_npu.so, allowing for a standalone Whisper NPU server that can be served via HTTP API or a CLI. Benchmarks show transcription completion in 5.2 seconds (5.17–5.6 s range) with a real-time factor of around 0.17–0.19, significantly faster than real time without impacting other system components.",
  "summary": "TL;DR — On a MSI Stealth A16 AI+ (Ryzen AI 9 365, XDNA2 NPU) running Arch, I got OpenAI's whisper-large-v3-turbo transcribing on the NPU — not the CPU, not the GPU — at RTF ≈ 0.18 (a 30 s clip in ~5.2 s) for roughly a tenth of the energy the same job costs on the CPU, plus an LLM answering on the same NPU through an OpenAI-compatible API. The whole path is local and offline. This is the write-up…",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}