The article tested on a Ryzen AI Max 390 laptop running Arch Linux and ROCm, with its integrated Radeon graphics memory set to an 8GB BIOS allocation. That is a capable platform, but it uses AMD’s unified-memory design: CPU and GPU draw from the same physical RAM pool. AMD’s own documentation describes Variable Graphics Memory and UMA as ways to reallocate system RAM to the iGPU, rather than providing the fixed, dedicated GDDR6 pool found on a discrete graphics card. A BIOS value of 8GB is therefore not a clean stand-in for an 8GB GeForce or Radeon card, particularly when the test machine can be configured with as much as 24GB of graphics allocation.
That caveat is more than lab methodology. It changes the advice a reader should act on: these are viable 8GB-class models when loaded in a suitable quantized format and run with conservative context settings. They are not proof that an 8GB GPU can comfortably absorb full-precision weights, long conversations, a large prompt, and a desktop display workload at once.
The benchmark leaves out the numbers readers need
MakeUseOf says its Auto graphics-memory setting performed 5% to 10% better than an 8GB cap, but it does not publish the per-model token-per-second figures, prompt lengths, output lengths, model files, quantization formats, context limits, or RAM configuration needed to reproduce the result. It calls the exercise a benchmark, yet readers do not get a benchmark table.
Those omissions are especially significant because local-model performance is not determined by parameter count alone. A 7B or 8B model can be offered in FP16, Q8, Q6, Q5, Q4, or more aggressive quantizations, and each consumes dramatically different memory. The inference engine matters too: Ollama, llama.cpp, LM Studio, KoboldCpp, vLLM, ONNX Runtime, and ROCm builds can reserve and offload memory differently.
The source also reports using
ollama-rocmand forcing the Ryzen AI Max 390 iGPU to operate as an unsupported GPU after ROCm initially did not detect it. That is a useful workaround for an enthusiast experiment, but it makes the results less transferable to an out-of-the-box Windows system. It also means the article is measuring a particular combination of Arch Linux, an evolving ROCm software stack, a workaround, and a unified-memory laptop—not the experience of plugging an 8GB desktop GPU into Windows 11 and clicking Run.
For readers who want a practical conclusion, the published test supports a narrower one: quantized small models can be useful on 8GB-class hardware, but the claimed comfort level depends on settings the article does not disclose.
Phi-3.5 Mini exposes the gap between “fits” and “runs”
Microsoft’s Phi-3.5 Mini is the most revealing entry in the list. At 3.8 billion parameters, it is small enough to be attractive for local summarization, extraction, lightweight chat, and some coding assistance. Microsoft describes Phi-3.5 Mini as a 3.8B-parameter model with a 128K-token capability, and it is a sensible option for constrained machines.
But MakeUseOf says Phi-3.5 Mini can run in FP16 while using relatively little video memory. On a true 8GB GPU, that claim needs a major qualification. FP16 weights alone require roughly 7.6GB before memory used by the runtime, token buffers, the KV cache that stores conversational context, and graphics-driver overhead. Microsoft’s current Foundry Local compatibility material lists Phi-3.5 Mini at 8.428GB of required GPU memory for one supported configuration.
That does not mean Phi-3.5 Mini cannot run on an 8GB card. It means the practical route is quantization, reduced context, or partial CPU offload—not assuming the full FP16 model will load reliably. For a Windows PC with an 8GB discrete GPU, a Q4 or Q5 build is the safer starting point, leaving memory for actual prompts and output rather than gambling that the loader can squeeze into the last few hundred megabytes.
The same principle applies to all eight picks. “The weights fit” is only the first checkpoint. Usable local AI requires room for the model, the active context, the output being generated, the inference backend, and the operating system.
Llama 3.1, Qwen2.5, and Mistral 7B are viable—but context is the limiter
Meta’s Llama 3.1 8B remains an understandable general-purpose recommendation. Its official model card specifies a 128K context window, multilingual support, and grouped-query attention, while 4-bit versions can commonly load within the memory budget of an 8GB GPU. It is a credible all-rounder for chat, rewriting, basic local document work, and light coding.
Qwen2.5 7B is similarly strong on paper. Alibaba’s Qwen documentation advertises 128K context support, structured-output improvements, and support for more than 29 languages. The MakeUseOf description that Qwen produces more formal responses is a subjective behavioral assessment rather than a reproducible benchmark result, but its recommendation for technical tasks is reasonable if a user values JSON, tables, instruction following, or multilingual work.
Neither model becomes a 128K-document assistant merely because the architecture advertises 128K support. Long context consumes memory through the KV cache, and 8GB will force a much lower usable context in many local configurations. Running a Q4 7B model with a 4K or 8K context can be straightforward. Attempting to feed it a massive codebase, a book-length PDF, or an extended chat history can turn a successful load into an out-of-memory error or a crawl.
Mistral 7B Instruct v0.3 is another model that can fit in a 4-bit form, and Mistral lists it with a 32K context window and approximately 5GB GPU RAM at FP4. Yet MakeUseOf’s August 2026 recommendation omits an important status detail: Mistral’s documentation lists the model’s retirement date as March 30, 2025, with Ministral 3 8B as its replacement. Open weights do not stop working after retirement, so existing Mistral 7B installs remain usable. But it is difficult to call a retired model the best current default recommendation when newer, maintained alternatives are available.
Gemma 2 9B and DeepSeek-R1 trade responsiveness for capability
Google’s Gemma 2 9B is the largest model in the roundup, and it is the entry most likely to make an 8GB machine feel constrained. Google positioned Gemma 2 9B as a high-performing small open model, and the model family was trained with an 8,192-token context length. A 4-bit build can be made to load on 8GB hardware, but “load” should not be confused with breathing room.
A user who selects Gemma 2 9B should lower the context setting, keep other GPU-heavy applications closed, and expect less headroom for a long session than with Phi-3.5 Mini, Mistral 7B, Llama 3.1 8B, or Qwen2.5 7B. It is the model in this group most likely to be worth the wait for a difficult prompt—and least likely to be the pleasant choice for rapid back-and-forth desktop chat.
DeepSeek-R1-Distill-Qwen-7B has a different performance cost. DeepSeek’s model card confirms that it is a Qwen2.5-Math-7B-derived distilled model trained using reasoning data from DeepSeek-R1. Its visible reasoning-style output can make it more methodical on math, logic, and some programming tasks, but it often generates many more tokens before reaching a final answer. That increases latency and expands the active context faster than a direct-response chat model.
For an 8GB machine, DeepSeek-R1-Distill-Qwen-7B is best treated as a specialist. Use it for a problem where deliberate intermediate reasoning is worth a slower answer; keep Llama or Qwen around for ordinary chat and document work.
The two smallest specialists need more precise labels
DeepSeek-Coder 6.7B is a legitimate coding-focused family, trained on a code-heavy corpus with a 16K context window and fill-in-the-middle capabilities for code completion and insertion. That can make it useful for local editor integration, boilerplate generation, and work inside a contained project.
However, MakeUseOf does not identify whether its test used the base checkpoint, the instruction-tuned checkpoint, or a particular GGUF quantization. That is not a trivial missing detail. The base model is designed around completion and infilling; the instruction model is the one most readers would expect to use as a chat-style debugging assistant. Before installing it, Windows users should verify the exact model tag rather than treating “DeepSeek-Coder 6.7B” as one interchangeable download.
Gemma 2 2B is the least demanding option and the most honest fit for low-memory systems. Its small size makes it suitable for short summaries, classification, extraction, quick rewriting, and local translation. MakeUseOf is right to warn that it has less depth than the 7B-to-9B entries. The practical advantage is that it leaves enough memory to run with longer prompts or alongside normal desktop applications without constantly operating at the edge of VRAM exhaustion.
The useful 8GB local-AI setup is therefore a pair, not a single “best” model: one compact general model such as Phi-3.5 Mini, Llama 3.1 8B, or Qwen2.5 7B in Q4 or Q5; and one specialist such as DeepSeek-R1-Distill-Qwen-7B for reasoning or DeepSeek-Coder 6.7B Instruct for code. Gemma 2 9B can be the quality-first third option, provided the user accepts tighter context limits and slower responses.
An 8GB card remains enough to learn, experiment, and build useful private local workflows. It is no longer enough to ignore memory settings. The decisive control is not the model name on the download page; it is the quantization and context value selected before the first prompt.