A desktop PC with an RTX 3070 sits beside a monitor displaying a split-screen AI interface and coding workspace.
A 9-billion-parameter model that borrows its reasoning style from Anthropic's Claude is running on a six-year-old RTX 3070 with 8GB of VRAM. Its user says it has shrunk their reliance on a paid Claude subscription, though not eliminated it. XDA Developers writer Nolen Jonker tested Qwythos 9B v2 and drew that conclusion. It is a useful reality check for anyone with a modest gaming GPU who is wondering how much cloud AI they still need.

The result is narrower than the headline suggests, and the caveats matter. Here is what was tested, what the model publisher says, and where the evidence stops.

A desktop PC with an RTX 3070 sits beside a monitor displaying a split-screen AI interface and coding workspace. What Qwythos 9B v2 is​

Qwythos is a free reasoning model from Empero AI. Empero describes itself as an independent AI research lab in Germany that builds small, efficient language models people can run themselves. A reasoning model thinks through a problem in a separate block before it answers. Most local runners let you toggle that behavior on or off.

Several details of its lineage are worth separating:

  • Base architecture: It is built on the Qwen3.5-9B architecture, a hybrid design that mixes Gated-DeltaNet linear attention and full attention at a 3:1 ratio.
  • Licensing: The Hugging Face card lists Apache-2.0.
  • What v2 changed: Empero says the gains are in eliminating looping, restoring the multi-token-prediction head and cleaner behavior, "not higher raw scores." Looping, which affected 6.7% of greedy generations before, is reported at 0% in v2.
  • Intended use: Empero aims it at math, science, medical explanations and cybersecurity. It is deliberately uncensored, which is worth knowing before you deploy it anywhere shared.

The "trained on Claude" claim​

The Claude connection needs careful reading. Empero's own site says the model was distilled from over 500M tokens of Claude Mythos 5 and Claude Fable 5 traces. The XDA review adds two qualifications:

  • The step-by-step reasoning in the training data was written by Empero's own tool, called rethink. The author reads the Claude link as more about style than copied thinking.
  • Empero hasn't said how it obtained the Mythos data. The author notes that Anthropic has been vocal about cracking down on labs that harvest Claude's outputs. The reviewer admits there is no clean answer on that yet.

The Hugging Face model tree traces the model back through Qwen3.5-9B and an earlier Empero model, Qwythos-9B-Claude-Mythos-5-1M. That shows the claimed lineage. It does not independently show where the underlying data came from or whether its use was authorized. Treat the "Claude" label as the publisher's claim, not a verified fact.

Running it on an 8GB card​

The author used an RTX 3070 with 8GB of VRAM and the Q4_K_M quantization, loaded through LM Studio. These are firsthand results from one machine:

  • File size: The author reports just under 7GB.
  • Context: They had room for roughly 10,000 to 30,000 tokens, depending on how many layers were offloaded. They ran comfortably at 14,000.
  • Speed: Generation ran at about 20 to 22 tokens per second, and most answers began in under a second.

Empero's GGUF card lists the standard Q4_K_M file at 5.34 GiB (5.74 GB). It lists a separate MTP-enabled Q4_K_M at 5.50 GiB (5.90 GB). The larger Q5_K_M is 6.08 GiB, Q6_K is 6.95 GiB, and Q8_0 is 8.87 GiB. Those are file sizes, not total VRAM needs, because the context cache and runtime overhead come on top. That is likely why the author's "just under 7GB" differs from the card's listing. The model card itself doesn't explain the gap.

Q8_0 is clearly too big for an 8GB card once you add context. Q4_K_M or Q5_K_M are the realistic choices.

Why the context fits​

The author attributes the modest memory footprint to Qwen 3.5's design. Most models keep a growing conversation memory in every block, which eats VRAM as a chat gets longer. In Qwen 3.5 only a quarter of the blocks do that, and the rest hold a fixed-size memory. That matches the 3:1 hybrid ratio on the model card.

The model advertises a 1,048,576-token context. Empero's own guidance says a single H100/H200-class GPU comfortably handles 256k to 512k, and the full 1M typically needs multi-GPU or aggressive KV-cache offload. The headline number is a server-class figure, not something an 8GB card delivers.

Getting it running​

Empero's GGUF card says the files work with llama.cpp, Ollama, LM Studio, Jan and KoboldCpp. For LM Studio, the card says to drop the .gguf into the runtime's model directory. It adds that the standard Qwen3.5 chat template loads automatically from the file.

Empero's recommended sampling settings are:

  • temperature 0.6
  • top_p 0.95
  • top_k 20
  • repeat penalty 1.05, which is now optional
  • up to 16,384 new tokens, so the model has room to reason

Because looping is trained out, the card says greedy decoding at temperature 0 stays coherent. If your runtime doesn't support MTP draft speculation, use the standard files rather than the MTP ones.

How it did against Claude-style tasks​

The tests were informal and personal, not a benchmark. The author mostly uses Claude for explain-this and sort-this-out tasks, so those were the focus.

TaskResult
Travel-time puzzle (thinking off)Correct answer of leaving at 9:12 in under a second, but it ignored a different departure time stated at the start of the question
Why coffee makes you crash (thinking on)About 28 seconds of thinking, then an accurate explanation
Beginner explainer on local AI (thinking off)Looser, went over the word limit, and used an inconsistent GPU analogy
Latest LM Studio versionSaid it had no internet access instead of inventing an answer
Same question with Brave Search MCP enabledAnswered correctly
Screenshot design feedbackGeneric suggestions, some reading like guesses

Three takeaways stand out:

  1. A correct answer can still miss a constraint. The travel puzzle is a good reminder to check that a model honored every condition in your prompt.
  2. Thinking mode earns its keep on explanations. The cost is waiting while it reasons.
  3. Offline models need tools for current facts. The honest refusal was the right behavior. Live information requires you to wire up web search through MCP or a similar tool.

The vision result is unsurprising. Empero's GGUF card says all Qwythos training was text-only, the vision tower was never fine-tuned, and image reasoning hasn't been independently evaluated for this release.

What the benchmarks do and don't say​

Empero's card reports these v2 results from its own internal harness:

  • 83.8% on MMLU with chain-of-thought, and 69.6% on the 5-shot log-likelihood setup
  • 96.4% on ARC-Challenge
  • 49.0% on GPQA-diamond
  • 93.6% on GSM8K
  • 77.4% on HumanEval pass@1

The sample sizes are modest: 500 questions for MMLU, ARC and GSM8K, 198 for GPQA-diamond and 164 for HumanEval. Empero itself says to use official harnesses for strict comparisons. Independent coverage also notes these numbers come from Empero's internal framework rather than standard leaderboard evaluations, which makes direct comparison with other scores harder.

On coding, the card is candid. HumanEval is 77.4 for v2 against 81.7 for the raw Qwen3.5-9B base. That matches the author's experience that coding is a weak spot. The MMLU note is also worth remembering. The model scores far higher when allowed to reason than on the log-likelihood setup, so a benchmark comparison depends on which one you use.

Can it replace a Claude subscription?​

The author's verdict is measured. Qwythos can take over a decent chunk of small tasks like logic problems and explainers. It would also save on Claude rate limits. Coding and design stay in the cloud.

The author also pays for Claude's workspace features, including Projects, Artifacts and Docs. They suggest local front ends like AnythingLLM could replace some of that. That is a personal assessment, and feature parity wasn't tested here.

In practical terms, this is a routing strategy, not a cancellation notice:

  • Good local candidates: explainers, logic puzzles, quick summaries and anything where privacy or cost matters.
  • Keep in the cloud: heavier coding, design critique, and anything needing live data unless you configure a search tool.
  • Verify regardless: Empero itself says the model can confidently get facts wrong without web search.

The bottom line​

The strongest takeaway is hardware accessibility. A 9B reasoning model in a roughly 5.3 to 5.5 GiB quantized file can run at usable speed on an older mid-range 8GB GPU. That is a real shift for Windows users with gaming PCs, and the free price tag helps.

The caveats are equally real. The speed and context figures come from one RTX 3070 setup, the benchmarks are the publisher's own, and the "Claude" training data has no verified provenance. For those reasons it is a sensible addition to a local AI toolbox, not a proven replacement for a frontier cloud model.

 

References

  1. I ran a 9B model trained on Claude's reasoning on my 8GB GPU, and it made me rethink paying for Claude XDA 2026-10-24T21:30:20+00:00
  2. empero-ai/Qwythos-9B-v2-GGUF · Hugging Face huggingface.co
  3. empero-ai/Qwythos-9B-v2 · Hugging Face huggingface.co