Edge0-145B on Windows: Local AI Hardware Requirements, RTX 5090 Results and Deployment Limits

The most useful finding is tucked inside Edge0’s model documentation: its constrained-memory performance results came from an RTX 5090 D 24 GB operating under restricted memory budgets, not from a separate 16 GB graphics card or a representative Windows laptop. That distinction should shape both purchasing decisions and performance expectations.

What “145 billion parameters” actually means​

Edge0-145B uses a latent mixture-of-experts architecture. According to the project’s model card, it has 145 billion total parameters but activates 10.12 billion per token. Edge0-120B similarly activates 8.64 billion of its 120 billion parameters. Both derive from Qwen3.8-Flash-Next.

In practical terms, the model selects portions of its expert network rather than using every parameter for every generated token. Edge0 says its runtime coordinates GPU memory, system memory and SSD access, keeping frequently needed experts available while overlapping data movement with computation. The engineering is about managing a large workload efficiently—not making its underlying data disappear.

Windows support comes with specific boundaries​

The project’s platform matrix lists NVIDIA GPUs and CUDA 13 for the 145B and 120B models on Windows and Linux. Apple Silicon support uses Metal. Windows CPU and Vulkan support listed elsewhere in the repository applies to the older 35B model, not automatically to these new releases.

Edge0’s 145B runtime guide lists at least 16 GiB of GPU memory and 20 GiB of system memory for Windows and Linux. Its model card qualifies the constrained-memory measurements as test budgets rather than general minimum requirements. Read together, these documents support an evaluation starting point—not a guarantee that any machine meeting those capacities will deliver the advertised experience.

The command-line route also requires both edge0-infer and edge0-strata to be built. The guide documents a Windows-specific CUDA dependency: if the required runtime DLL cannot be found, startup can fail with exit code 0xC0000135. This remains a technical deployment, even if the broader project also offers a chat application.

Bottom line: verify the model-specific backend and dependencies before treating an existing PC as compatible.

The performance headline needs its footnotes​

Edge0’s announcement reports 29 GB peak memory, generation above 80 tokens per second, prompt processing above 1,100 tokens per second and typical power near 120 W. These are company-reported results, not independently established performance guarantees.

Its detailed model card illustrates the variation. For Edge0-145B Q3_K, it reports chat generation of 99.7 tokens per second on an RTX 5090 D 24 GB, 69.9 on an RTX 4090 24 GB, and 36.8 in the constrained-memory test. The same table separates chat from agent decoding, so those figures should not be mixed into one universal speed claim.

Context limits need similar care. Although the announcement promotes 128K–256K context, the runtime guide defaults to 131,072 tokens on Windows/Linux and 32,768 on macOS. A model’s advertised capability and its shipping configuration are not necessarily identical.

Why businesses should evaluate—not immediately migrate​

Edge0 says local inference avoids corresponding cloud-model token fees and keeps core capabilities available offline. It also acknowledges that hardware and electricity still cost money, and that functions requiring current online information remain connected workloads. Those are useful boundaries around the cost and independence argument.

For developers, the repository provides command-line, local-server and chat-app interfaces. But the 145B guide documents operational limitations: concurrent requests queue, some status fields lack memory statistics, and identical requests at temperature zero can produce different answers. These details deserve attention before a single-user demonstration becomes a shared business service.

A sensible pilot should therefore:

• Use non-sensitive, representative tasks.
• Measure completion quality and waiting time on the intended hardware.
• Record installation, support and operating costs—not just avoided API charges.
• Review model-license terms separately from the inference repository’s Apache-2.0 label.
• Assess the complete application’s data handling before approving confidential workloads.

The recommendation is straightforward: treat Edge0 as a candidate for controlled evaluation, not as evidence that cloud AI has become obsolete. Its documented Windows support makes the release relevant; its hardware-dependent results make careful testing essential. The interesting question is no longer merely whether a large model can start on a personal computer, but whether it can finish your actual work reliably enough to justify running it there.