De combien de mémoire un LLM local a-t-il besoin ?
Avant de télécharger un modèle pour le faire tourner en local, il faut savoir s'il tient en mémoire. L'estimation est la somme de trois termes, comparée à ce que la machine peut réellement donner au modèle : need = weights + KV cache + overhead fits if need <= 0.8 × usable memory Les chiffres ci-dessous suivent l'estimation utilisée par le navigateur de modèles locaux de l'application de bureau…
Determining the memory requirements for a local LLM involves calculating three main factors: weights, KV cache, and overhead. The total memory need is approximately 80% of the usable memory. For a 14B model in Q4_K_M quantization, the weight memory is 7.84 GB, while the KV cache ranges from 768 MB to 6 GB depending on the context length.
Overhead adds an estimated 0.9 GB. The total memory usage for this model falls between 8 and 9 GB, with 8.4 GB being the reported value. Available memory on a system depends on the type of GPU and the total RAM; a 16 GB Mac can provide around 12 GB. The need for memory must be less than 80% of the usable memory, with additional margin for the system and other applications. Higher quantization levels result in lower memory requirements but also slower generation speeds.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
