For Windows users, the sensible starting point is therefore not “Which AI PC should I buy?” It is “What model do I want to run, and where will it fit?”
Memory capacity comes first
There is no universal RAM requirement for “local AI.” Even one model can have substantially different requirements depending on its numerical format and inference software. OpenAI’s Transformers guide, for example, lists approximately 16GB of VRAM for gpt-oss-20b using MXFP4 quantization, but about 48GB of memory when using bfloat16 instead. That is a major difference without changing the model’s parameter count.
Quantization reduces the storage needed for model weights by representing them at lower precision. It is one reason sizeable language models can run on consumer hardware. OpenAI’s model card explains that quantizing gpt-oss’s mixture-of-experts weights to MXFP4 allows its smaller model to run on systems with as little as 16GB of memory.
However, a model’s weights are not its entire working memory requirement. NVIDIA identifies three important consumers of inference memory:
- Model weights: the learned parameters.
- Intermediate data: information used during computation.
- The key-value cache: stored attention information that grows with sequence length and batch size.
The practical consequence is straightforward: do not treat a download size as a complete hardware specification. Leave room for inference overhead and the context you intend to use, rather than planning a configuration that only just accommodates the weights. This is a sizing recommendation based on those documented memory requirements.
System RAM and GPU VRAM are not interchangeable
Ollama can place a model entirely in system memory, entirely on a GPU, or across both. Its documentation explicitly distinguishes these arrangements, so having enough total memory does not necessarily mean the model is running wholly on the graphics card.
Why does that matter? During text generation, the system repeatedly moves weights and other inference data from memory to the processor. NVIDIA explains that these transfers can dominate decoding latency. Consequently, a graphics card’s arithmetic capability alone is not a reliable predictor of conversational speed. Memory capacity, bandwidth and model placement deserve attention too.
There is also an important exception to simplistic parameter-count calculations. OpenAI lists gpt-oss-20b as having approximately 21 billion parameters, with 3.6 billion active for each token. That mixture-of-experts arrangement reduces the parameters used in each computation; it does not turn the complete model into a 3.6-billion-parameter download.
Bottom line: distinguish the memory needed to hold a model from the processing work needed to generate its next token.
Context length can upset an otherwise sensible configuration
A short question and a lengthy document are different workloads—even when you use the same model.
Ollama’s dedicated context-length documentation states that increasing context raises memory requirements. This means a configuration that handles ordinary conversation may need different settings for document analysis or coding sessions with extensive input.
Before concluding that your PC needs an upgrade, make a controlled comparison:
- Keep the model and runtime unchanged.
- Test a short prompt.
- Repeat with the longer input you actually need.
- Check whether the model’s CPU/GPU placement changes.
This is a diagnostic recommendation, not a benchmark claim: it combines Ollama’s documented context-memory relationship with its model-placement reporting.
Windows users can verify where the model runs
Here is a useful check before spending money.
With Ollama installed and a local model loaded, open PowerShell or Command Prompt and run:
ollama ps
Inspect the PROCESSOR column:
| Reported placement | Meaning |
|---|---|
100% GPU | The model is loaded entirely on the GPU |
100% CPU | The model is loaded entirely in system memory |
| A CPU/GPU percentage split | The model is distributed across both |
These are Ollama’s documented interpretations—not percentages of instantaneous processor utilization. A split placement is not automatically an error, but it tells you that the model is not wholly GPU-resident.
If an expected GPU is not being used, check hardware and driver compatibility before assuming insufficient VRAM is the only explanation. Ollama maintains separate NVIDIA, AMD and Vulkan support information, with platform-specific boundaries.
An NPU helps only through compatible software
Engadget’s Zenbook experiment highlights runtime choice: shifting Mistral inference from Ollama to OpenVINO reduced CPU load and improved responsiveness.
The purchasing lesson is to verify the software path, not merely the accelerator badge. Ollama’s Windows documentation specifies particular GPU and driver requirements; accelerator support depends on the runtime you choose.
Storage also deserves a place on the checklist. Ollama documents at least 4GB for its Windows installation, plus additional space for models that can occupy tens or hundreds of gigabytes. Its native Windows application makes the command line available in PowerShell and Command Prompt.
Buy for the workload, not the label
The most useful hardware specification is a small worksheet: model, quantization, context length, supported runtime, available memory and actual placement. Those factors explain far more than a generic “AI-ready” label.
Start by testing the machine you already own. Then upgrade the constraint you can demonstrate—capacity, GPU compatibility or responsiveness—rather than buying impressive specifications and hoping the model agrees.