The results make a useful distinction: being able to load a local AI model is not the same as running it comfortably. They also leave the headline claim behind the experiment—45.6 tokens per second on a 16 GB RTX 5060 Ti—unverified.
What is different about this Qwen file?
Kingy tested Qwen3.8-27B-GSQ-RCO-IQ3_S-mtp.gguf, reporting a 12.12 GB download with an additional multi-token prediction, or MTP, head. Its underlying IQ3_S model is approximately 11.77 GB; the prediction head adds roughly 0.35 GB. Kingy says its downloads matched the publisher’s SHA-256 fingerprints.
According to ISTA-DASLab’s model card, the compression combines Gumbel-Softmax Quantization, or GSQ, with Riemannian Constrained Optimization, or RCO. Instead of assigning identical precision throughout the model, the process allocates different quantization types to individual tensors within a size budget. The publisher says the resulting files use standard GGUF formats compatible with llama.cpp, Ollama and LM Studio. That general compatibility should not be mistaken for confirmation that every application exposes the same MTP controls.
The lab describes IQ3_S as retaining the base model’s performance on selected benchmarks. That is a publisher-reported result, not a guarantee that compression leaves every answer unchanged. Its model card reports matching AIME25 and LiveCodeBench v6 scores, with a small deficit on GPQA-Diamond.
Three machines, three different experiences
Kingy’s tests took place on November 8, 2026. The configurations matter:
| Test system | Runtime and backend | Important constraint |
|---|---|---|
| Ryzen AI Max+ 395, Radeon 8060S, 128 GB shared memory | Ubuntu 26.04.1; llama.cpp 79e2e74eb; Vulkan | Model reportedly fit fully in GPU-accessible memory |
| MacBook Pro M4 Pro, 24 GB | Homebrew llama.cpp 0.6.0; Metal | Model fit, but the machine had substantial swap usage |
| ASUS ROG Strix G16, RTX 5070 Laptop 8 GB, 32 GB RAM | Windows 11; llama.cpp b12199; CUDA | Model required a CPU/GPU split |
The Windows laptop was plugged in and running in Turbo mode. Kingy reports that its fit tool placed 27 of 64 layers on the GPU for the benchmark, while estimating 24 GPU layers for the longer-context chat-server configuration. The remaining layers ran on the CPU. These were therefore comparisons of complete systems and software configurations—not a controlled contest between operating systems.
Reported generation speeds
| Machine | MTP disabled | MTP enabled | Reported wait for an 8,217-token prompt |
|---|---|---|---|
| Ryzen AI Max+ 395 | 16.1 tok/s | 28.8 tok/s | 25–26 seconds |
| MacBook Pro M4 Pro | 11.4 tok/s | 10.9 tok/s | 82–87 seconds |
| RTX 5070 Laptop, 8 GB | 7.0 tok/s | 7.8 tok/s | 19–20 seconds |
Kingy averaged four prompts, with two-token MTP drafts for the enabled results. Crucially, these speed runs consumed their entire 1,024-token output allowance on reasoning: they measure token generation, not delivery of completed, useful answers. Quality was assessed separately.
For Windows laptop owners, the distinction between prompt processing and generation is particularly interesting. The RTX machine had the shortest reported document-processing wait but the slowest sustained output. In this experiment, “reads quickly” and “answers quickly” were different properties.
Kingy’s separate llama-bench figures followed the same generation-speed order: 16.7 tok/s on AMD, 12.3 on the Mac and 7.4 on the Windows laptop. With an 8K context already present, those figures fell to 16.0, 10.9 and 6.0 respectively. The AMD system’s older UD-Q3_K_XL file had previously reached 15.8 tok/s.
MTP helped AMD, not every machine
The llama.cpp documentation explains speculative decoding as drafting several tokens and then checking them together with the target model. It explicitly lists draft-mtp as using prediction heads from the main model. The potential advantage comes from batched verification; correct guesses still have to save more time than drafting and checking consume.
Kingy enabled it with these server options:
--spec-type draft-mtp --spec-draft-n-max 2
On AMD, generation rose from 16.1 to 28.8 tok/s—approximately 1.8 times the baseline. Longer three- and four-token drafts were slightly slower, at 27.8 and 27.4 tok/s.
The Mac accepted roughly the same proportion of draft guesses as AMD, around 70%, but gained no speed. Kingy suggests verification overhead may explain that outcome, while explicitly acknowledging that the explanation was not proven. The Windows laptop gained only modestly.
The practical lesson is to benchmark MTP rather than assume it helps. An acceptance percentage alone is not an end-to-end performance result. The llama.cpp project also documents a SPEED-Bench client for comparing baseline and speculative-decoding throughput, latency and acceptance across prompts—a more useful evaluation approach than watching the acceptance counter in isolation.
Kingy additionally observed different wording with MTP enabled, even at temperature zero. It did not establish a corresponding quality loss, but those observations should discourage treating the two configurations as byte-for-byte interchangeable.
A perfect score—with important boundaries
On Kingy’s 20-task automated quality test, the compressed file scored 20/20 in 26 minutes on the Mac. Its comparison table gives ChatGPT, labeled gpt-6.1-sol by Kingy, the same score in five minutes. Older Qwen3.8 quantizations scored 19/20, while Qwen3-8B scored 18/20.
That supports a narrow conclusion: the compressed file performed well on this particular suite. It does not establish general equivalence to ChatGPT.
Two qualifications carry considerable weight:
- Grading changed: Kingy says three rules had been loosened after its earlier tests. Under the original strict rules, this file scored 19/20 and the ChatGPT comparison scored 18/20.
- Coding was inconsistent: A parser passed all 14 hidden tests in two of three attempts. The unsuccessful attempt passed nine tests.
Two successes in three attempts demonstrate capability, not a dependable two-thirds success probability across programming work. With only 20 tasks and generally one run per task, Kingy itself advises treating a one-point difference as a tie.
What remains unproven about “16 GB is enough”
The experiment was prompted by Manolo Remiddi’s September 12, 2026 video. As described by Kingy, the video claimed 45.6 tok/s on a 16 GB RTX 5060 Ti during a browser-agent task and 136 tok/s on an RTX 5090. Kingy tested neither configuration.
Those figures should remain separate from the three-machine results. Different tasks and untested hardware prevent either confirmation or refutation.
Likewise, a 12.12 GB file being smaller than 16 GB does not establish a universal working-context guarantee. The report’s own laptop configurations illustrate why: changing the workload and retained context changed the estimated number of layers that would fit on its GPU. File size is a starting point, not a complete memory budget.
The practical takeaway
These results are encouraging for compact local models, but they are not a hardware shopping verdict.
The AMD result was obtained on Linux, so it should not be presented as a measured Windows result for Ryzen AI Max+ systems. The Mac comparison was affected by 7–16 GB of swap usage and some battery operation; Kingy also discarded an earlier run that slowed dramatically. Different llama.cpp builds and backends further limit clean platform comparisons.
For an owner of the tested class of 8 GB Windows laptop, the evidence is straightforward: this file can run through partial GPU offloading, but generation was slow and MTP offered little relief. For the tested high-memory AMD system, MTP was substantially more useful.
The broader finding is not that one checkbox makes local AI fast. It is that model size, memory placement, runtime version and workload must be evaluated together. Compression opened the door; the hardware and software still determined how quickly anything walked through it.