That is a useful distinction. This is not a claim that a budget local setup replaces every cloud assistant. It is an account of matching narrowly defined tasks to hardware already on the desk—and resisting the idea that every model release requires a new graphics card.
Three models, three different jobs
Jonker’s lineup consists of Gemma 4 E4B, DeepSeek-R1-Distill-Llama-8B, and the community-developed FluentlyQwen3-Coder-4B. All three are reportedly used in Q4_K_M quantized form, but they serve different purposes.
The most transferable lesson is the division of labor, not a universal recommendation to download the same files.
Gemma 4 E4B: explaining documents and screenshots
Jonker uses Gemma 4 E4B in LM Studio to explain information supplied in the conversation, including screenshots of complicated documents. Google’s official model card confirms that E4B accepts text, images, and audio and produces text output. It also supports structured function calling. Those are model capabilities, however—not a guarantee that every application and model package exposes every input type.
Google also confirms the architectural distinction behind the “E” label: it means effective parameters. Per-layer embedding tables contribute to the total parameter count but operate as lookups, helping explain why the model’s computational profile differs from its headline size. That should not be read as permission to ignore the memory occupied by the selected package.
There is a measurable capability trade-off. Google reports an MMLU Pro score of 69.4% for E4B, compared with 85.2% for Gemma 4 31B. Those results concern the evaluated instruction-tuned models, not an independent test of Jonker’s Q4_K_M setup. They support treating E4B as a compact assistant rather than a pocket-sized equivalent of the larger model.
For document work, a sensible approach is to request a plain-English explanation alongside the exact passage supporting it. With screenshots, verify names, dates, units, and figures against the original before relying on the interpretation. The objective is easier reading—not delegating consequential decisions to a chatbot.
DeepSeek-R1-Distill-Llama-8B: bounded reasoning tasks
Jonker assigns budget comparisons, numerical questions, and untidy notes to DeepSeek-R1-Distill-Llama-8B, reporting a Q4_K_M download of approximately 4.9GB.
DeepSeek’s official documentation confirms that this checkpoint derives from Llama 3.1 8B, rather than being the full R1 model squeezed into a smaller file. It belongs to a family of distilled models based on Qwen and Llama architectures.
The original benchmark comparison also supports Jonker’s distinction between the Llama 8B and Qwen 7B variants. DeepSeek reports higher AIME 2024 and MATH-500 scores for Qwen 7B, while Llama 8B edges ahead on LiveCodeBench and Codeforces. These are developer-reported results under specified evaluation conditions, not guarantees about a particular quantized download.
For everyday use, the practical recommendation is narrower than “trust it with the numbers.” Ask the model to identify assumptions and express the calculation clearly, then check the arithmetic independently. A detailed generated explanation is something to examine, not a certificate of correctness.
FluentlyQwen3-Coder-4B: small tools, limited scope
The third model is the specialist. Jonker uses FluentlyQwen3-Coder-4B for single-file HTML tools, YAML frontmatter cleanup in Obsidian, and regular expressions for extracting dates from notes.
Project Fluently’s model card identifies it as a community coding fine-tune in the Qwen3-4B lineage and lists NVIDIA’s OpenCodeReasoning dataset. It should therefore be distinguished from an official Qwen coding-model release. The card makes broad improvement claims, but those claims are not a substitute for measured task results.
Jonker reports a 2.38GB Q4_K_M file. A corresponding GGUF repository lists that size and quantization, corroborating the download-size claim for that package—not its peak VRAM consumption or coding reliability.
The useful boundary is Jonker’s decision to keep assignments small enough to inspect. For readers following that approach, use dummy information first, preserve a working copy, and test both ordinary inputs and obvious failure cases before introducing personal records. “It opens in a browser” is a starting point, not a completed acceptance test.
The important catch: download size is not VRAM usage
The reporting leaves several hardware questions unanswered: the GPU model, context settings, generation speed, and peak memory usage are not supplied. Consequently, these experiences cannot establish that every 8GB graphics card will deliver the same result.
Treat the quoted gigabytes as file sizes, not complete memory budgets. A sensible evaluation should include the model’s working context and runtime requirements rather than merely subtracting the download size from eight.
For a Windows PC owner, the buying decision should start with a representative task:
- Does the model load with the settings you intend to use?
- Does it remain usable with a realistically sized document or conversation?
- Is its response time acceptable?
- How much correction does its output require?
That last question matters as much as raw speed. A model that produces an answer quickly but needs extensive repair has simply moved the waiting time onto the user.
Local inference helps privacy—but tools change the boundary
There is a concrete privacy benefit behind this workflow. LM Studio’s documentation says downloaded models can operate offline and that its document-chat processing remains on the machine. It separately identifies model searches, downloads, runtime downloads, and update checks as operations that require network access. Offline inference and an application that never contacts the internet are different claims.
Jonker also reports connecting Gemma to web and filesystem tools through MCP. That introduces another boundary: keeping model inference local does not mean every connected tool operates locally or has harmless permissions.
For sensitive sessions, the cautious approach is to disable unnecessary integrations, restrict file access, and download the required components beforehand. If the task does not need connectivity, LM Studio documents that ordinary local chat and document processing can continue without it.
Upgrade for a demonstrated limit, not a release announcement
Jonker’s experience makes a credible case for postponing an upgrade—not for declaring 8GB sufficient for everyone. The workflow succeeds on its own terms because it retains cloud services for demanding work and gives smaller local models constrained assignments.
The broader lesson is refreshingly practical: define the job, test the existing machine, and judge the result. If capacity, speed, or output quality repeatedly blocks useful work, an upgrade has a concrete justification. Until then, the graphics card does not need replacing merely because a larger model exists.
References
- These small local models keep saving my 8GB GPU from an upgrade XDA · 2026-11-17T11:00:21+00:00
- Gemma 4 model card - Gemma Documentation gemma.ai
- Gemma 4 model card - Gemma Documentation ai.google.dev