The setup: Qwythos 9B v2 on 8GB of VRAM
Jonker ran Qwythos 9B v2 locally. Qwythos-9B-v2 is a community fine-tune from Empero AI, built on Qwen3.5-9B, with the looping behavior fixed. The quantized files are distributed in GGUF format for runtimes including llama.cpp, Ollama, LM Studio, jan and KoboldCpp.
The model's file sizes explain why it fits on a card like the 3070:
| File | Size | Role |
|---|---|---|
| Qwythos-9B-v2-Q4_K_M | 5.34 GiB / 5.74 GB | Recommended default text weights |
| Qwythos-9B-v2-Q5_K_M | 6.08 GiB / 6.52 GB | Balanced quality and size |
| Qwythos-9B-v2-Q8_0 | 8.87 GiB / 9.53 GB | Near-lossless, too big for 8GB |
| mmproj BF16 | 0.86 GiB / 0.92 GB | Vision projector, required for images |
Those figures come from the model listing. The listing says the projector is required for image input and pairs with any of the text quantizations.
File size is not the same as VRAM use. The listing makes no promise about how much GPU memory a given setup needs. Context length, image handling and how much is offloaded to the GPU all change the total. Jonker says his setup fits and runs smoothly at around 20 tokens per second. That is his experience on one card, not a guarantee.
Where the "eyes" come from
Empero did not train this model to see better. Its own listing says the vision tower is inherited unchanged from Qwen3.5-9B and was frozen through both fine-tuning stages, so image behavior matches the base model. The GGUF page adds a candid caveat: all Qwythos training was text-only, and image-grounded reasoning has not been independently evaluated for this release.
So when Jonker's screenshots came out well, the credit goes to Alibaba's Qwen3.5 vision stack, not to Empero's tuning.
Runtime support also matters. Empero's main model card says that for vLLM serving, the multimodal stack is "text-only in practice." That is the Safetensors card, not the GGUF one, so it should not be read as a limit on the GGUF build. A related Qwythos GGUF listing says LM Studio detects the matching mmproj file in the same folder and enables the image-attach button automatically. That listing is for the earlier Qwythos release, so check that the projector actually loads for the v2 files you download.
What the benchmarks do and don't say
Jonker cites two Alibaba figures for Qwen3.5-9B: 89.2 on OCRBench, which tests reading text inside images, and 65.2 on ScreenSpot Pro, which tests locating interface elements on software screens. Both are Alibaba's own numbers for the base model. They are not results from his RTX 3070 run, and they don't show Qwythos beating Claude or Gemini.
They do fit the pattern he reports: strong at reading text, shakier at judging what to do on a busy interface. Reading a label and deciding whether it matters are different skills.
What the hands-on comparison found
Jonker compared Qwythos with Claude Opus and Gemini Flash. He says he used the same images and prompts he normally sends to cloud models. The comparison has clear limits. There were no repeated trials, no scoring rubric, and the prompts and settings were not published. The model labels are the author's own. Treat the findings as one experienced user's impressions, not a benchmark.
Where Qwythos did well:
- Text-heavy screenshots. Asked to pull one chatbot reply out of a busy desktop screenshot, it copied the text line for line. Jonker says that matched Opus almost word for word, while Gemini pulled the wrong part of the image.
- File and folder views. It handled file browsers and settings panels, which are mostly labels and numbers. It also noticed a file whose listed type didn't match its extension.
Where it was weaker:
- Decisions in complex interfaces. In a video-editor popup about changing a project's frame rate, Qwythos read everything correctly but explained each button without recommending one.
- Spotting inconsistencies. Opus picked the issue and flagged that the filename in the popup didn't match the clip on the timeline. Qwythos read both names correctly but didn't treat the mismatch as worth checking.
- Verbosity. Its answers run long. Jonker says local settings can rein that in, which hosted chat apps don't expose.
The practical takeaway is that extraction and judgment are separate jobs. A small local model looks competent at the first and less dependable at the second.
The privacy angle
The strongest argument is about where the image goes. Jonker's examples are a photo of medication, or a screenshot showing personal filenames, logins, statements or contracts. For those, running inference locally means the image is not uploaded to a third-party model service. He also says it spares him from reading retention policies or opting out of training.
That is a reasonable point, with some limits:
- His report doesn't audit LM Studio, the model, or the PC for network activity. "Local" depends on your actual setup.
- Not having to read a cloud provider's policy is a convenience, not proof that cloud use is unsafe or that local use is risk-free.
- Reading text is not the same as interpreting it. Don't rely on a 9B model alone for medical interpretation, credentials, contracts or changes to active projects.
Should you cancel a subscription?
Jonker's own conclusion is personal. Claude handles his heavy work, Qwythos handles his private work, and that left little for his Gemini plan to do. The evidence supports that for him, on this kind of task. It does not show that a 9B model replaces Claude or Gemini in general, or that its quality, speed or privacy matches theirs.
How to try it on your own PC
- Check your hardware. An 8GB card can be enough for the Q4_K_M build plus the projector in Jonker's setup. Results vary with context length and settings.
- Get both files. Download a text quantization and the matching mmproj file. Per Jonker, LM Studio's built-in downloader fetches the projector for you.
- Confirm image input works. Make sure your runtime accepts image attachments before you rely on it.
- Test with non-sensitive samples first. Use screenshots you could safely share.
- Compare against your current chatbot. Try pure transcription, then a task that needs judgment, as in the frame-rate popup example.
- Verify the output. Check any transcription against the original image, especially for numbers and names.
- Keep an eye on length. If answers run long, adjust your local settings or prompt.
Bottom line
For screenshot transcription and reading file or settings panels, a free 9B model on an 8GB GPU looks like a credible local option. It is a better fit than a cloud service for images you'd rather not upload. For tasks where the model must decide what matters on screen, the author's own example showed a frontier model still ahead. That makes this a case for splitting the work, not for cancelling everything.
References
- I ran a local vision model on my 8GB GPU against Claude and Gemini, and it made me rethink paying for either XDA · 2026-11-19T15:30:22+00:00
- Qwen/Qwen3.5-9B · Hugging Face huggingface.co
- empero-ai/Qwythos-9B-v2-GGUF · Hugging Face cdn-avatars.qwak.ai