{
  "id": 724492,
  "title": "The Browsing LLM's Frustrating Limitations",
  "url": "https://urgent.news/2026/08/13/llm",
  "topic": "ai",
  "section": "AI",
  "published": "2026-08-13T03:07:17.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/jh5_pulse/liu-lan-qi-pao-llm-de-gu-gan-xian-zhi-140d"
  },
  "original_language": "zh",
  "account": "Running LLMs directly in the browser is perfect, as it prioritizes privacy and eliminates the need for complex GPU settings. From WebLLM to Transformers.js, the front-end community has sparked a trend of \"edge LLMs.\" However, when deploying models in users' browsers, the first challenge is whether WebGPU is truly faster than WASM.\n\nRecent tests have yielded surprisingly dramatic results. For miniature models under 500M, WASM is actually 12% faster. However, for large models over 3B, WASM becomes a disaster, causing Chrome to crash.\n\nTesting Environment:\n- Machine: MacBook Air M2, 8GB RAM\n- GPU: Apple M2, 8 cores\n- Disk available space: 9.7GB\n- Browser: Chrome 136 (Playwright headless)\n- Framework: WebLLM 0.2.84, @xenova/transformers 2.17.2\n\nInitially, the test encountered a problem with the MacBook Air M2 having only 9.7GB of disk space left, which triggered the browser's Cache API physical quota limit (about 10% of available space, approximately 970MB). This limit determines the size of the model that can be loaded on the front end.\n\nPhi-3.5 Mini @ MBA\nThe plan was to test the popular lightweight model Phi-3.5-mini-instruct (3.8B parameters). However, on an 8GB MacBook Air, the model couldn't even be loaded.\n\nPhi-3.5-mini-instruct-q4f16_1-MLC requires about 2,520 MB of VRAM. When the weight file was downloaded to the browser, a QuotaExceededError occurred. Even with various caching strategies, it still didn't work.\n\nPhi-3.5 Mini @ MStudio\nTo confirm if it was a storage quota issue, the test was run on an M2 Max with the same WebLLM code and Phi-3.5-mini-instruct-q4f16_1-MLC model.\n\nResults:\n- tok/s: 57.4\n- Accuracy: 90.0% (9 out of 10 questions)\n- Total tokens: 956\n- Total time: 16.66s\n\nThe model loaded successfully, and the output was smooth, with 57.4 tok/s.\n\nThe first hurdle for running front-end LLMs on lightweight clients is often the browser's Storage Allocator, not the GPU itself.\n\nWhat model can the MacBook Air run?\n1. TinyLlama-1.1B\nWhen running TinyLlama-1.1B-Chat with @xenova/transformers (v2.17.2), although the weights could be downloaded successfully, an error occurred during inference: RangeError: offset is out of bounds at Uint8Array.set.\n\n2. SmolLM2-360M\nTesting SmolLM2-360M-Instruct (q4f16) yielded:\n- Speed: 38.4 tok/s\nThe model ran stably but had limited reasoning capabilities.\n\nWebGPU vs WASM Performance Comparison\nA fair comparison of WebGPU and WASM performance was conducted using various models on a Mac Studio.\n\nGPT-2 (124M) was tested twice.\n\nResults:\n- Backend tok/s accuracy total time\n- WASM 23.2 30% 16.39s\n- WebGPU 20.4 30% 18.68s\n\nSurprisingly, WASM outperformed WebGPU by about 12% for the 124M model.\n\nWhy did WASM win for miniature models?\n- Data transfer overhead: Tokens need to be copied frequently between CPU and GPU memory, and the transfer time directly affects performance when the computation is not large enough.\n- Kernel startup cost: Each GPU kernel launch has a fixed overhead that cannot be evenly distributed by small-scale parallel computing.\n- WASM evolution: The mature instruction set and JIT optimization allow the CPU to efficiently handle matrix operations of the 124M scale.\n\n3.8B Model Comparison\nUsing the Phi-3.5-mini-instruct-q4f16_1 model on an M2 Max, WebGPU and WASM were compared.\n\nResults:\n- Backend tok/s accuracy total tokens total time\n- WebGPU 59.7 70% (7 out of 10) 490 8.20s\n- WASM 0.5 60% (6 out of 10) 413 827s (13.8 minutes)\n\nWASM was 119 times slower than WebGPU.\n\nWhy such a large difference?\nA reasonable guess is that WASM runs on the CPU, and the CPU's memory bandwidth and computing power are fixed.\n\nWebGPU, on the other hand, offloads computations to the GPU's parallel cores, and its advantage becomes more apparent as the model size increases.\n\nFor larger models like GPT-2 (124M), the kernel startup and data transfer overhead of GPU outweigh the benefits of parallelization.\n\nA larger model, Qwen2.5-7B-Instruct (7B, q4f16), was tested on WebGPU.\n\nResults:\n- tok/s: 35.5\n- Accuracy: 80% (8 out of 10)\n- Total tokens: 643\n- Total time: 18.13s\n\nThe 7B model ran at 35.5 tok/s on WebGPU.\n\nTo improve the Phi-3.5 Mini model on WebGPU, max_tokens was increased from 50 to 400, and accuracy improved from 70% to 100%, with tok/s stable between 59.7-65.2.\n\nConclusion:\n- Small models do not require WebGPU: For tasks that only need miniature models (<500M), using WASM directly can avoid compatibility issues with WebGPU on different browsers and eliminate the need for complex fallback logic.\n- Storage space is a hidden killer: When deploying models over 1B, prioritize checking the storage quota, as insufficient user disk space leading to cache failure often causes user experience collapse.",
  "summary": "直接在瀏覽器跑 LLM，既兼顧隱私又不用複雜的GPU設定，太完美了吧？ 從 WebLLM 到 Transformers.js，前端社群有一群反骨仔吹起一股「邊緣 LLM」的熱潮，可是，當真正將模型落地到使用者的瀏覽器時，第一個面對的考驗就是，WebGPU 真的有比 WASM 快嗎？ 這陣子實測的結論比想像中更加戲劇化， 500M 以下的微型模型，WASM 反而快了 12%，但是對於 3B 以上的大模型，WASM 直接變成悲劇，直接讓Chrome撞的頭破血流（Chrome 沒有頭 ＝ Headless Chrome）。 測試環境 項目 規格 機器 MacBook Air M2, 8GB RAM GPU Apple M2, 8 核心 磁碟可用空間 9.7GB 瀏覽器 Chrome 136 (Playwright headless) 框架 WebLLM 0.2.84,…",
  "key_points": [
    "Browser-based LLMs face performance limits",
    "WASM outperforms WebGL/WebGPU for models <500 million parameters",
    "Chrome crashes with models >3 billion parameters due to lack of physical head"
  ],
  "editors_take": "Running large language models in browsers is limited by storage capacity and performance, with small models handling adequately but larger ones facing significant challenges.",
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}