The setup: a fanless relic meets modern AI
Gibson's test machine was a Zotac ZBOX C1325 nano with a Celeron N3160, 8 GB of RAM and a cheap SSD. Zotac's own product page for the closely named ZBOX CI325 nano gives the context. It lists an Intel Celeron N3160 (quad-core, 1.6GHz, up to 2.24GHz), two DDR3L-1600 SO-DIMM slots for up to 8GB, and integrated Intel HD Graphics. It's a passively cooled box sold for web browsing, email and light productivity. Its supported OS list tops out at Windows 10. Nobody designed it with LLMs in mind.
Gibson installed Ubuntu Server and compiled llama.cpp, the open-source inference engine. He configured it for CPU-only inference and built three tools with two parallel jobs:
cmake --build build --target llama-server llama-cli llama-bench -j 2
The -j flag is standard CMake behavior. llama.cpp's build documentation says it runs multiple jobs in parallel, for example -j 8 runs 8 jobs. On a four-core Celeron, a modest value like 2 is a sensible way to avoid overwhelming the machine during compilation. The write-up doesn't give the exact CMake configuration flags or the llama.cpp revision, so anyone reproducing this should treat the build as a starting point.
Models and access
He chose three small models, all in Q4_K_M quantization:
- SmolLM2 (the 360M variant appears in the task tests)
- Qwen3.5 0.8B
- Qwen3.5 2B
Quantization stores model weights at lower precision to cut memory and compute demands, at some cost in quality. He ran llama-server and opened zotac-ai.home.arpa:8080 from a browser, which gave him a ChatGPT-style interface while the Zotac did the inference.
A note on that address: it reflects his own network. By default, llama-server binds to localhost. The server documentation lists the default listen address as 127.0.0.1. Reaching it from another machine means deliberately changing the host setting. The default port also depends on the documentation and version you read. Some guides say 8080. The newer llama.app documentation says the server listens on 127.0.0.1:9931 by default. Check the docs for the build you actually compile. The write-up doesn't describe authentication or firewall setup, so don't copy a LAN-exposed server without adding them.
How the models performed
These were hand-picked tasks, not a formal benchmark. They still show a clear pattern.
| Task | SmolLM2 360M | Qwen3.5 0.8B | Qwen3.5 2B |
|---|---|---|---|
| Polite email rewrite | Dropped the 3 pm deadline and the presentation, and added details never supplied | Kept the deadline and presentation context | Produced a polite, suitable rewrite |
| Four-field extraction (12 keyboards, Chicago, Thursday) | Put "12" in the Item field | All four fields correct | Correct |
| Two-sentence Wi-Fi summary | Reversed the facts: said the Chicago work was done and Boston was still working on it | Left out the reason for the replacement | Covered the replacement, the disconnections behind it and what staff must do |
The 2B model also solved 2x² − 7x + 3 = 0. It factored it as (2x − 1)(x − 3) = 0, found x = ½ and x = 3, and verified both by substitution. Gibson is appropriately cautious about it. One correct answer doesn't make a math tutor.
The SmolLM2 summary failure is the most instructive. A confident, fluent answer that inverts reality is harder to catch than a garbled one. Small models can produce text that reads fine and is wrong. That's a general limitation of the technology, not just this machine.
The real bottleneck was waiting, not memory
Gibson reports that all four CPU cores were maxed out during inference. Memory use ranged from about 390 MiB with SmolLM2 to about 1.3 GiB with Qwen3.5 2B, and swap was never touched. So 8 GB was plenty for these models. The CPU was the limit.
The most useful numbers come from the first email task. Both Qwen models processed a 396-token prompt:
| Model | Total server time | Generation speed |
|---|---|---|
| Qwen3.5 0.8B | 90 seconds | 2.63 tokens/second |
| Qwen3.5 2B | 228 seconds | 1.23 tokens/second |
For the 2B model, roughly 202 seconds went to reading the prompt and another 26 seconds to writing the short rewrite. That adds up to about the 228-second total.
Output at about one token per second is tolerable for a short answer. Waiting more than three minutes for it to start is the real problem. Prompt processing, not generation, dominated the wait. The prompt included the formatting and instructions wrapped around his request, not just the text itself.
This matches what llama.cpp's build documentation says about BLAS. It says BLAS support may improve prompt processing at batch sizes above 32, but doesn't affect generation performance. That doesn't mean BLAS would rescue a Celeron N3160. Gibson didn't test it, and I wouldn't promise gains. It does show that prompt processing and generation are separate performance problems.
Practical takeaways
- Keep prompts short and focused. A short answer doesn't guarantee a quick reply if the input and instructions are long.
- Start with the smallest model that meets your needs. In Gibson's tests, 0.8B was the most promising compromise, though it still missed an instruction. The 2B model was more accurate but took much longer.
- Verify everything. Every model below 2B made at least one factual or instruction-following error in these tests.
- Watch CPU, not RAM. On this class of hardware, memory had headroom while the processor was saturated. Longer context windows would change the memory picture, and the write-up doesn't test them.
- Don't expose the server casually. The default localhost binding exists for a reason. If you open it to your LAN, add proper access controls.
What this does and doesn't prove
It would be a mistake to read this as "you don't need a GPU for local AI." The narrower finding is that small quantized models can run on a very old fanless CPU, slowly and with limited accuracy. Gibson himself concludes that he'd be better off writing the email or summary himself, and he's retiring the Zotac from AI duty. The experiment also says nothing about power consumption, other PCs or current model rankings. The timings and quality observations are one author's first-person results, not independently reproduced benchmarks.
Where it succeeds is as a cheap, low-risk way to find out whether you enjoy tinkering with local models before buying hardware. For Windows and homelab users with a dusty mini PC in a drawer, that's a reasonable weekend project. Just bring patience and something to watch while the prompt processes.