A laptop runs a local AI summarizing a sales report, while Task Manager shows memory usage at 99%.
A week with a small local AI model on a Windows laptop produced an odd result. The author, Tony Phillips of How-To Geek, found the answers were good enough. What sent him back to Claude was the work around the answers: the waiting, the steering, and the fact that his hardware kept demanding attention. This is a single person's trial, not a benchmark, so treat it as a field report.

A laptop runs a local AI summarizing a sales report, while Task Manager shows memory usage at 99%. The setup​

Phillips ran Qwen3-4B-Instruct-2507 (Q4_K_M, 2.5GB) locally through LM Studio Bionic on an Intel Core i9-13900H laptop with 16GB of RAM. He compared it with Claude Sonnet 5.5 Free at its default Medium effort setting. He skipped formal benchmarks and used the kinds of questions he might put to an everyday assistant.

Some details are not given, so don't read them into the result:

  • The laptop's make and GPU
  • The Windows build
  • The runtime version
  • Sampling settings
  • How many prompts were tested
  • Whether the model ran on the CPU or the GPU

The reported results come from one machine and one set of prompts.

What the model is​

The Hugging Face model card for Qwen3-4B-Instruct-2507 describes a 4.0-billion-parameter model. It is the updated non-thinking version of Qwen3-4B, and LM Studio's listing says it improves on instruction following, logical reasoning, text comprehension, mathematics, science, coding and tool usage. The card also lists a native context length of 262,144 tokens. It notes that if users hit out-of-memory problems, they should consider reducing the context length to something like 32,768. These are vendor claims. They are not results from the author's week.

The 2.5GB file size is also not the model's full memory cost. A third-party requirements page puts Qwen3 4B Instruct 2507 at about 2.9 GiB at Q4_K_M for a 4K context, which covers the GGUF file plus one FP16 KV cache before runtime overhead. Longer inputs and runtime overhead add to that, which helps explain why a long paste can push a 16GB machine hard.

Where the local model did well​

The first surprise was competence. Two tests stood out:

  • An Excel LET question. Phillips judged the explanation accurate, with correct syntax and a sensible example. He said it read more like documentation than expert advice, but he would have been satisfied with it while working in Excel.
  • A long Windows announcement. He pasted in the text and asked for a summary. Qwen picked out the major changes and key updates, and he called the result solid.

These are his judgments of specific outputs. No prompts, transcripts or external fact-checks were supplied. Still, they fit a reasonable pattern: lookup-style and summarization tasks are where small local models tend to hold up best. That pattern comes from general industry experience, not from this test.

Where it struggled​

The trouble began when Phillips moved from asking for information to asking for judgment. He asked for headlines, article ideas and story angles.

  • An invented discovery. One suggested angle was that USB ports damage phone batteries over time. The model then supplied a testing method, a finding and a payoff, as if the investigation had already been done. Phillips's complaint is that the ideas sounded plausible while the reasoning behind them was not grounded. That is not evidence the USB claim is true.
  • Themes nobody asked for. Asked whether a week of local AI testing could make an article, the model brought in digital sovereignty, cloud independence and digital freedom. None of those were in his prompt.
  • Recovery after pushback. When he challenged its assumptions, the model acknowledged the problem and gave a more grounded answer. But he says he often spent as much time evaluating its reasoning as using its ideas.

He gave Claude the same prompt. In his account, Claude pushed back on his framing and suggested the more interesting story was which tasks really need a cloud model. He disagreed with parts of that, but felt it stayed closer to the evidence. His summary was that Claude felt like a conversation, while the local model felt more like a batch process.

This is one subjective comparison with no scoring. It does not show that Claude always reasons better. It does show a useful failure mode: a fluent small model can present speculation with confidence, and you may have to catch it yourself.

The hardware cost​

The most concrete Windows detail came from the summary test.

  • Phillips waited roughly five minutes for the response to finish.
  • Task Manager showed overall system memory use of about 97%.
  • The cooling fans were loud enough that he compared them to a 747 preparing for takeoff.
  • Claude finished the same task quickly. No exact time was given.

He says this changed how he worked. He thought harder about long prompts, watched RAM, and weighed whether a question was worth the wait. In his words, running AI locally made the infrastructure visible.

Intel's spec page shows the i9-13900H is a Q1 2023 mobile part with 14 cores (six performance, eight efficient) and 20 threads. It has a 45W base power and a 115W maximum turbo power. That tells you what class of chip was tested. It can't explain this laptop's fan noise or speed, because cooling design, power settings, runtime choices and workload all matter.

Several things remain unknown. The article doesn't say what else was running during the test, whether a different quantization or a smaller context window would have helped, or whether the model was on the CPU or the GPU. Don't take five minutes as typical for Qwen or LM Studio. It was one long-input run on one 16GB laptop.

What local AI does better​

Phillips credits local AI with real advantages:

  • No usage anxiety. As a Claude Free user, he never knows exactly how much usage he has left. The local model never made him wonder whether he could ask another question.
  • Control. He chooses the model, the quantization and when to update.
  • Offline use and local processing. He counts both as legitimate advantages, including for privacy.
  • No subscription. He calls the model effectively free after setup. Hardware and electricity still cost money, but subscriptions and cloud usage limits disappear. He gives no power readings or cost comparison, so there is no break-even figure here.

A caution on privacy​

Be careful with the privacy claim. LM Studio describes Bionic as an agent that can use local models or switch to open-source models in the cloud for heavier tasks. It lists flexible execution options: running locally, connecting through LM Link, or using LM Studio Secure Cloud. LM Studio also says cloud requests are not retained after processing.

So running Bionic does not by itself mean your data stays on the machine. Privacy depends on using a downloaded local model and not switching on cloud features. Phillips did not audit Bionic's network behavior or telemetry, and neither do we. If privacy is your reason for going local, check which mode each project uses.

What to check before trying this yourself​

Phillips's account supports a few practical checks. It doesn't set a minimum spec.

  1. Memory headroom. Watch Task Manager during your first long prompt. A model file that is small on disk can still push a 16GB system near its limit.
  2. Context length. Use shorter inputs, or reduce the context window, if you run into memory problems. The Qwen model card suggests this.
  3. Task fit. Test it first on bounded jobs such as explanations and summaries. Be more skeptical of brainstorming, where plausible invention is the risk.
  4. Verification habits. Expect to challenge the output. Ask it to separate what it knows from what it is assuming.
  5. Your tolerance for waiting. If a question isn't worth several minutes, a cloud model may suit it better.

Bottom line​

The experiment doesn't show that small local models are bad. It shows that answer quality is only one part of whether an assistant is practical. In this test the local model was competent on a documentation-style Excel answer and a summary. It was weaker on open-ended editorial judgment, and its cost showed up as time, memory and fan noise. Phillips says he would keep a local model for experimentation, quick questions, and cases where privacy or offline access matters more than speed. For his daily work, Claude's conversational flow and editorial judgment won out.

Your results will depend on your RAM, cooling, model, quantization and tolerance for supervising a confident but sometimes overeager assistant.

 

References

  1. I tried replacing Claude with a free local AI model for a week. The responses weren't the problem How-To Geek 2026-11-20T18:31:15+00:00
  2. Intel® Core™ i9-13900H Processor (24M Cache, up to 5.40 GHz) - Product Specifications | Intel intel.com
  3. Qwen/Qwen3-4B-Instruct-2507 · Hugging Face huggingface.co