A 4 GB Laptop GPU Beats a 12-Core CPU by 4.3x on Gemma 4
This article compares two ways of serving the same small language model on the same laptop: CPU-only, and on the 4 GB GTX 1650 Ti sitting in the same chassis. The payload is byte-identical on both arms and the command lines differ by a single flag. The card takes decode by 4.3x . The repository is at https://github.com/xbill9/gemma4-dev What Is Being Compared? One machine, a 13th Gen Intel Core…
A laptop equipped with a 12-core Intel Core i7-1360P CPU and a 4 GB NVIDIA GTX 1650 Ti Max-Q GPU was used to compare two methods of serving the same small language model. The model, a 3.35 GB quantization-aware GGUF, was run on both the CPU and GPU arms of the laptop, with the only difference being a single flag in the command line.
The same endpoint was used for both runs, and the results showed that the GPU arm was significantly faster, decoding the model at a rate 4.3 times faster than the CPU arm. The study also found that the GPU arm's performance improved as the context grew, while the CPU arm's performance decreased. Despite the limited size of the GPU card, it was found to be sufficient for serving the model, holding only 1598 MiB of the 3.334 GB of tensor bytes in memory.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
