More RAM Changed What Matters in My Local AI Setup
Six months ago, I started running local LLMs seriously on a Mac with 16 GB of unified memory. Most of my decisions were dictated by memory: which models fit, which quantization to use, and how much context I could afford. Moving to 48 GB changed that. Models such as Qwen3.8-27B became practical for everyday use, and my attention shifted toward a different problem: how well does the model work…
Six months ago, a Mac user with 16 GB of unified memory began running local LLMs seriously. This decision was heavily influenced by the available memory: which models could fit, which quantization techniques were viable, and how much contextual information could be accommodated. Switching to 48 GB of unified memory altered this landscape entirely.
Models like Qwen3.8-27B became feasible for regular use, prompting a shift in focus towards assessing the model's performance during actual coding-agent sessions. Memory was merely the initial constraint. With the enhanced memory capacity, other factors such as context length emerged as significant considerations. Longer sessions tend to accumulate tool outputs and code, which can interrupt the flow before anticipated.
While thinking can enhance certain tasks, it simultaneously introduces latency. A model that appears impressive in a brief interaction can present a markedly different experience when it must manage a codebase, engage with multiple files, utilize tools, implement changes, and retain sufficient context to continue its reasoning process.
Consequently, the evaluation criteria for local models shifted. Now, users are less concerned about whether a model technically fits their machine and more interested in determining how long they can effectively work with it before the workflow begins to impede their productivity. The actual setup that was adopted after extensive experimentation consists of a Mac with 48 GB of unified memory connected to LM Studio, Qwen3.8/Splash, and OpenCode.
The ongoing optimization efforts now revolve around this stack, focusing on selecting an appropriate context size, deciding when to enable thinking, understanding the reasons behind compaction, and balancing model capabilities against interactive speed. Despite the potential for further experimentation, the clear visibility of trade-offs offered by increased memory has made a substantial difference.
A comprehensive account of the current configuration, the changes from the initial 16 GB setup, and the insights gained from using it for real development tasks is available in the full article: https://ammar-najjar.com/blog/local-ai-setup-six-months-later/. For those who utilize local LLMs with coding agents, the article invites comparisons of experiences.
Once model size and memory constraints no longer represent the primary challenges, what other bottlenecks tend to surface?
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
