PrismML Bonsai 27B: 1-Bit AI Model Memory Requirements and 4GB GPU Limits

What Bonsai 27B actually is​

Bonsai 27B is not a new architecture. It moves Qwen3.6-27B into binary or ternary weights rather than being a new pretrain. The parameter count stays at 27 billion. What changes is how each weight is stored.

PrismML ships two variants:

• 1-bit Bonsai 27B: binary {−1, +1} weights at 1.125 bits per weight, for 3.9GB.
• Ternary Bonsai 27B: weights of −1, 0 or +1, which PrismML puts at 1.71 effective bits per weight and about 5.9GB. It retains 94.6% of FP16 quality, against 89.5% for the 1-bit build.

The 1.125-bit figure comes from the way the format works. In the Q1_0_g128 format, each weight is a single sign bit. Every group of 128 weights shares one FP16 scale factor. That adds up to roughly 14 times smaller than FP16.

The "54GB" baseline is the rough arithmetic of 27 billion parameters at 16 bits each. PrismML's own memory table lists a BF16 GGUF reference at 47.73 GiB. The two numbers use different conventions, so treat them as the same ballpark. Either way, the original is far beyond any consumer GPU.

One more caveat: PrismML hasn't explained how it chooses the sign for each weight. The MakeUseOf report says an independent study confirmed the released weights work well. The same study found that naive 1-bit quantization in the public format collapses. In other words, the format alone isn't the secret. The training method is, and it isn't published.

The 4GB number is the weights, not the bill​

LM Studio's model page is blunt on this point. Actual runtime memory also includes the KV cache, activations, runtime buffers and the optional vision tower, and grows with context length. Its listing says you need at least 4GB of RAM for the smallest Bonsai 27B. That is a platform minimum for loading the model. It does not promise that a 4GB graphics card can run it entirely on the GPU.

PrismML's repository gives memory estimates for the earlier Bonsai 27B family. They cover weights, activations, an FP16 KV cache and about 1.2 GiB of overhead, and they are text-only.

Adding the vision projector costs roughly another 0.9 GiB. PrismML says an FP16 KV cache costs about 64 KiB per token, or around 6.3 GiB at 100K tokens. Its optional 4-bit mean-centered KV cache cuts that to roughly 1.8 GiB at 100K.

Published figures differ slightly by source and by unit. The Hugging Face model card lists 3.79 for weights and 5.2 for a 4K-context peak. It lists 11.6 for 100K, with an FP16 cache. The GitHub table shows lower values in GiB. MakeUseOf's reporting repeats the model card's numbers. If you're sizing a purchase or a download, use the model card as a conservative guide.

The model also supports a 262K-token context window. That is a ceiling. It is not a promise that the full window fits in a small card.

A 4GB laptop GPU test​

MakeUseOf's Yadullah Abidi tried the model on a laptop with a 4GB GTX 1650 Ti. This is one person's test, not a lab result.

• LM Studio flagged the model for partial GPU offload straight away. It defaulted to only four GPU layers.
• Forcing full offload at a 4,096-token context failed with a "failed to allocate buffer for kv cache" error.
• The model loaded once offload dropped to 50 percent, using about 3.6GB of VRAM. The CPU handled the rest.
• Speed was about 4 tokens per second with half the model on the GPU and the rest on a Core i5-9300H. With no layers on the GPU it fell to 2.4.
• A prompt of about 1,600 tokens took nearly a minute to produce its first token.
• A simple question was answered correctly, but only after more than three minutes of thinking.

The takeaway is that a 4GB card can hold the weights but not much else. The author's own conclusion is that 6GB or 8GB is the realistic floor for comfortable use. That is an inference from the memory tables and this one test, not a formal PrismML requirement.

The download is also bigger than the headline. MakeUseOf saw LM Studio pull 4.73GB because it also fetched a 0.93GB vision component. LM Studio's own page lists 3.80GB for the main file. Your total depends on whether the vision tower comes along.

How much quality survives​

All the benchmark numbers below are PrismML's own, from 15 benchmarks run in thinking mode. They are not independent results.

Math and coding hold up best. Instruction following, tool calling and vision take the biggest hits. PrismML itself says long, multi-file agentic coding isn't a strength of this release.

Independent data is thinner and more mixed:

• A GitHub test harness ran 60 programmatically scored probes on a V100 32GB. It scored the 1-bit build 47 out of 60, against 50 for base Qwen at Q4_K_M and Q6_K. The ternary build scored 49. The 1-bit build decoded fastest, at 55.1 tokens per second against 33.1 for Q4_K_M. The harness authors caution that quality gaps on thinking probes were within noise.
• MakeUseOf cites a DataCamp tester who saw about 92 tokens per second on an RTX 5070 Ti. That tester still found a Q4 build of Qwen3.6-27B more reliable for coding-agent work.
• On a Ryzen 5 5600, a tester called Big Iron measured 6.7 tokens per second for generation. Bonsai scored 32 of 40 on homelab questions, while the much smaller Qwen 3-4B scored 35.

So the compression is genuine, but the quality cost is real. A smaller, higher-precision model can sometimes match it on narrow tasks.

Running it on a Windows PC​

Runtime support has moved on since the July launch. PrismML's current guide says Q1_0 is supported in upstream llama.cpp on CPU, Metal, CUDA and Vulkan. That means a stock build can load the 1-bit file. PrismML's fork is no longer strictly required, though early guides told users to build it. The MakeUseOf report adds that the fork is still faster for CPU prompt processing, at 13.19 versus 7.91 tokens per second.

Apple users should note that 1-bit MLX support is still pending upstream. Until it merges, PrismML points to its own MLX fork.

For Windows users, the repository documents a PowerShell route. You set the family and size, then run the setup and server scripts:

1. Set $env:BONSAI_FAMILY = "bonsai" for the 1-bit family.
2. Set $env:BONSAI_MODEL = "27B".
3. Allow scripts for the session with Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass.
4. Run .\setup.ps1, then .\scripts\start_llama_server.ps1.

Set the same family and size in every new terminal. The guide warns that leaving the family unset makes the scripts pick Bonsai 2 instead. The easier path is LM Studio, which lists the model directly. MakeUseOf says it applies PrismML's suggested sampling settings (temperature 0.7, top-p 0.95, top-k 20) automatically. I couldn't confirm the top-p and top-k values in the primary pages I opened, so treat them as the author's report.

Don't confuse it with Bonsai 2​

PrismML's repository now treats Bonsai 2 27B as the default, and the 1-bit and ternary models above are described as the earlier family. Bonsai 2 has different requirements. One third-party guide says it needs PrismML's llama.cpp fork and used just over 12GB of VRAM with default llama.cpp server settings. The same guide says that was because llama.cpp reserves KV cache for four parallel request slots by default. The memory and quality figures in this article apply to the 1-bit Bonsai 27B only.

Who should bother​

• 4GB GPU owners: you can experiment through partial offload, but expect slow, patient use.
• 8GB and 12GB GPU owners: this is the sweet spot. A conventional 27B model won't fit at all, and Bonsai will.
• Apple silicon laptop owners: MakeUseOf's author puts these in the same sweet spot.
• Anyone with spare memory: the ternary build is worth a look. It costs more memory but retains more of the original quality.
• Coding-agent users: a Q4 build of the full model may still serve you better, per the DataCamp tester.

Bonsai 27B is a real step forward in making large models fit on ordinary hardware. The honest summary is that 4GB gets you the weights, and everything else costs extra.