{
  "id": 13849618,
  "title": "Is Your Phone's LLM 37x Too Slow? Check This One Compiler Flag",
  "url": "https://urgent.news/2026/10/12/is-your-phones-llm-37x-too-slow-check-this-one-compiler-flag",
  "topic": "ai",
  "section": "AI",
  "published": "2026-10-12T02:14:50.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/pingredsai/is-your-phones-llm-37x-too-slow-check-this-one-compiler-flag-3mcc"
  },
  "original_language": "en",
  "account": "Is Your Phone's LLM 37 Times Slower? This One Compiler Flag Can Speed It Up Part 3 of a series. The first step was discovering that llama.cpp for Android, when built with a debug Gradle variant, uses the -O0 optimization level. This results in a speed of 37 times slower than the release version. The metrics show generation at 0.5 tokens per second on debug compared to 18.5 on release, a 37x improvement. The first token took 1863 ms on debug versus 52 ms on release, a 36x difference. Prefill with 43 tokens took 42 seconds on debug versus 0.556 seconds on release, a 75x improvement. Model load took 24 seconds on debug versus 3.9 seconds on release, a 6x improvement. The issue was easily overlooked because the disassembled binary showed the sdot instruction, suggesting all was well. The sdot instruction only indicates the instruction set, not the optimization level. A reader on DEV pointed out that the build type is the culprit, as Gradle debug variants default to CMAKE_BUILD_TYPE=Debug. Changing the CMakeLists.txt to set the build type to Release, and telling Gradle to use Release, resolved the issue. Verification showed 82 instances of -O3 in the build.ninja file and -DNDEBUG present, while the release version had -O3 and -DNDEBUG. The binary size dropped significantly when optimized, indicating actual optimization occurred. The optimization affected inlining, loop unrolling, auto-vectorization, register allocation, and dead code elimination, all of which are crucial for LLM inference. Without these optimizations, the code runs very slowly, even with a full CPU clock and threads on big cores. To avoid this issue when building llama.cpp for Android, check the optimization flags and binary size. An optimized build should show -O3 flags and a noticeably smaller binary size. Optimization level should be checked first, followed by architecture flags, thread count and affinity, memory and IO, and finally, the algorithm itself. A basic oversight like this can significantly impact performance, highlighting the importance of checking fundamental aspects before moving on to more complex factors.",
  "summary": "Is Your Phone's LLM 37x Too Slow? Check This One Compiler Flag Part 3 of a series. Part 1: found the missing -march (2.5x faster) . Part 2: ruled out scheduling and frequency scaling (negative results). Part 3: found the real culprit — -O0 . 1. The result first If you build llama.cpp for Android with a Gradle debug variant, you get an -O0 binary. It is 37x slower than Release. Metric Debug (-O0)…",
  "key_points": [
    "Debug Gradle variant uses -O0 optimization level, causing 37x slower performance",
    "Generation speed drops from 18.5 tokens/second on release to 0.5 on debug",
    "Changing build type to Release resolves significant performance improvement"
  ],
  "editors_take": "Defaulting to debug variants during the build process creates a massive performance bottleneck that can mislead developers into searching for complex algorithmic solutions instead of addressing basic compiler settings.",
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}