The three-expert idea
Raj says no single local model leads at deep research, coding and image generation. So he rotates three:
- BigBang-v1 for web research
- FLUX 3 Image Klein for image generation and editing
- Qwen3.8-27B for coding
Raj says he has tested dozens of local LLMs through 2026. This is his current working set, and he expects to swap models within weeks. That caveat fits how fast open models are moving.
BigBang-v1: a research model that still needs a search engine
BigBang-v1 is a fine-tune of Qwen3.6-35B-A3B under the Apache 2.0 license. It is a mixture-of-experts (MoE) model with 35B total parameters and about 3B active per token.
Raj highlights the model card's BrowseComp score of 76.5. BrowseComp measures how well a model finds hard-to-locate web information. The project's own README lists that score and a 54.2 on SWE-Bench Pro. Those are the developers' figures. The README notes its evaluation used its own tool policy, with search, page-visit and code-execution tools, and that other harnesses may produce different scores.
"Local" here means the model weights run locally. The search does not. Raj connected llama.cpp's web UI to Exa's search engine over MCP with a free API key. Every query still goes to an outside service. Exa's MCP page calls its server free and open-source. That doesn't tell you what API limits apply to heavier use.
Fitting 19.3GB onto a 16GB card
Raj's IQ4_XS quantization from bartowski is 19.3GB, which exceeds his VRAM. He uses llama.cpp's --n-cpu-moe flag to move about 7GB of expert weights into system RAM. He reports a 128K context window and 68 to 78 tokens per second.
The flag does one narrow thing. --n-cpu-moe N keeps the MoE feed-forward expert weights for the first N layers in system RAM instead of VRAM. The tradeoff is simple: higher N means less VRAM used and more CPU work per token, while lower N keeps more experts on the GPU and generates faster. Attention and the KV cache stay on the GPU. The expert tensors of the first N layers sit in system RAM and run on the CPU, while attention, shared weights and the KV cache stay on the GPU.
Don't copy Raj's offload amount. One 2026 guide advises against copying a value and says to sweep it for your own card. The right setting depends on your quantization, context length, llama.cpp version and what else is using VRAM. A practical routine is to start with the experts on the GPU. Then raise the offload until the model loads, and lower it step by step while watching memory.
The Windows 11 26H2 test, and its caveat
Raj asked the model about known issues in Windows 11 26H2, a topic past the model's training cutoff. He says five searches turned up every issue Microsoft had confirmed at the time, with correct dates and KB numbers. The model also added a KB number and a fix for one issue that Raj couldn't trace to Microsoft.
That is the most useful detail in the piece for IT readers. Raj gives no query text, no result log and no list of the issues, so the test can't be replicated from what he published. Treat it as an anecdote. If a local model with web search tells you about a patch or a workaround, confirm the KB number and the fix on Microsoft's own pages before you act. A plausible but unsupported fix is the kind of error that costs an admin an afternoon.
FLUX 3 Image Klein: generation and editing
Black Forest Labs describes FLUX 3 Image Klein as a 5.6B-parameter open-weights model. One model handles text-to-image, editing with up to 10 reference images, and layout control with bounding boxes. The official repository lists a distilled version at 16 steps and a base version at 50 steps. It says the distilled version runs about six times faster. The base version is meant for fine-tuning and research.
Raj runs the distilled version in ComfyUI. He reports an average of about 16 seconds for a 1024x1024 image across five runs. Roughly 4 seconds of that is sampling. Most of the rest goes to a prompt upsampler that expands his prompt.
It's three models, not one
The upsampler and text encoder are part of the pipeline. Black Forest Labs says the upsampler is a 9B vision-language model fine-tuned from Qwen3.5-9B. The README says not to use the model without it, because the model is trained on the captions it writes. Raj's setup also uses a 4B Qwen3-VL text encoder. He runs INT8 builds of all three, and ComfyUI loads each into VRAM as its stage runs.
Black Forest Labs' repository says the INT8 files in ComfyUI run on 8GB GPUs such as the RTX 3070. A 16GB card therefore has headroom. Raj also says ComfyUI can be a pain for first-timers. That is an observation, not a measured usability comparison.
What "free" covers
ComfyUI's launch page says running the model locally costs nothing per image beyond your own hardware, and that no API key is needed. Its separate FLUX 3 Image Partner Node calls Black Forest Labs' hosted API and runs a larger model. That is a different product.
The license has limits. Per ComfyUI, the FLUX Kommunity License allows free non-commercial use. Commercial use of generated images is allowed if you and your affiliates earn under US$5M a year, and you must filter or review outputs and disclose AI generation where the law requires it. Other commercial uses, including commercial fine-tunes, need a separate license from Black Forest Labs. "No usage limits" is accurate for local running. "No strings" is not.
Qwen3.8-27B: a slower, denser coding model
Raj prefers Qwen3.8-27B for coding, even though BigBang-v1 can also write code. He points to Qwen's 61.7 on SWE-bench Pro against BigBang-v1's 54.2. Both scores are self-reported, and the two vendors used different evaluation setups. Qwen's model card describes the Claude Code harness, a 256K context window and corrected tasks. So the comparison isn't controlled or run on the same hardware. Raj's preference rests on his own experience writing Pygame projects for fun, and he says it's the only trade-off he can confidently report.
Dense versus MoE on 16GB
Qwen3.8-27B is dense, so all 27B parameters work on every token. The --n-cpu-moe trick doesn't apply because there are no experts to offload. Raj uses Unsloth's UD-IQ4_XS quantization at 14.3GB, which fits in VRAM on its own.
The context window is where it gets tight. At 32K context, he says the cache needs about 2GB on top of the weights. llama.cpp then moves 6 of the model's 64 layers into system RAM. Generation drops to roughly 15 to 17 tokens per second, against 68 to 78 for BigBang-v1.
| Model | Role | Architecture | Reported context | Reported speed |
|---|---|---|---|---|
| BigBang-v1 (IQ4_XS, 19.3GB) | Research | MoE, 3B active of 35B | 128K | 68–78 tokens/s |
| Qwen3.8-27B (UD-IQ4_XS, 14.3GB) | Coding | Dense, 27B | 32K | 15–17 tokens/s |
| FLUX 3 Image Klein (distilled) | Images | 5.6B, plus 9B upsampler and 4B encoder | n/a | ~16 s per 1024x1024 image |
All figures in the table are Raj's, from one machine.
What this means for PC owners
Several things hold up despite the limits of a single-author test:
- A 16GB gaming GPU is enough for a capable multi-model workflow. The catch is that you swap models rather than run them all at once.
- MoE models make 16GB workable. CPU expert offload lets a 19GB-plus model run with a large context. Plan for the system RAM it uses and the speed cost.
- Dense models are slower here. They avoid the offload trick but cost you generation speed and context.
- Local doesn't mean offline. The research workflow relies on an external search service, and every search leaves your machine.
- Free weights aren't free of costs. You still pay for the hardware and the electricity, and you may face search API limits and license terms.
- Verify what comes back. The 26H2 example shows how a model with search can still invent a detail.
The bottom line
Raj's account doesn't show that a $0 stack replaces cloud AI for everyone. It shows that one GPU can host a research model, an image model and a coding model, if you accept slower dense-model speeds and a setup with some friction. Every performance and benchmark figure comes from the author or the model developers, so test on your own hardware. Treat anything the research model says about patches or known issues as a lead until Microsoft's documentation confirms it.
References
- My self-hosted AI models research, code, and generate images, and all they cost is a GPU I already own XDA · 2026-10-25T10:00:20+00:00
- README.md github.com
- README.md github.com