A laptop displays a glowing schematic linking AI software workflows to its internal processors and hardware.
Ryzen AI's NPU has spent two years as a spec-sheet number, with Windows Studio Effects as its main job. AMD and Unsloth have now given it a more demanding one. AMD and Unsloth announced that local NPU inference is available in Unsloth Desktop, starting with AMD Ryzen AI NPUs of XDNA 2 and later. Unsloth Desktop is the local AI app, and the NPU is the neural engine inside Ryzen AI chips. This piece covers what the announcement changes, what it takes to run it, and which numbers are vendor claims.

A laptop displays a glowing schematic linking AI software workflows to its internal processors and hardware. What actually changed​

This is a packaging and integration story. It is not a new inference engine. The NPU support uses AMD's open-source Lemonade and FastFlowLM code for model serving and inference. AMD's FastFlow SDK documentation shows FastFlowLM already exists as an NPU runtime that runs Q4NX 4-bit models. The Q4NX detail comes from Unsloth's own guide.

The new part is that Unsloth Desktop can now offer that path in its own interface. A compatible application sends a model request to the Unsloth Desktop API. Lemonade and FastFlowLM then handle model serving and inference on the XDNA 2 NPU. Hardware Busters describes the endpoint as OpenAI-compatible. It lists Hermes Agent, OpenClaw, Open WebUI, Cline, LangChain and GitHub Copilot as compatible tools.

AMD says more than 100 agentic AI apps and workflows can reach the NPU this way. That count is AMD's. Nobody has independently verified it, and it doesn't mean every one of those apps has been tested.

One independent write-up adds a useful caveat. Agent orchestration and other application work stay on the CPU and GPU. This adds a local execution option and does not move an entire agent onto the NPU. Only the model inference goes to the NPU.

Supported hardware and operating systems​

Unsloth's documentation lists the supported processors as Ryzen AI 300 (Strix Point, Krackan Point), Ryzen AI Max (Strix Halo) and Ryzen AI 400 (Gorgon Point), each with an XDNA 2 NPU. It says you need a supported AMD XDNA 2 NPU on Windows or Linux x64. Phoronix reports that AMD confirmed in advance that the support works with the Linux builds and is not limited to Windows.

Hardware Busters gives Linux prerequisites. These are kernel 7.0 or newer with the in-tree amdxdna driver, or a DKMS package on older kernels, plus current NPU firmware and an unlimited memlock limit. I could not confirm those specifics in the pages I reviewed, so treat them as the report's account of Unsloth's guide and check the live documentation before you rely on them. AMD's FastFlow SDK documentation adds that validated driver versions are listed per release. It also advises running fastflow self-test after changing a driver.

Setup on Windows​

AMD's walkthrough starts by installing Unsloth Desktop on a device with an XDNA 2 NPU. Unsloth's guide says on a supported Ryzen AI PC, the model picker automatically shows an NPU section. The first setup is a one-time job: Unsloth downloads FastFlowLM and checks your NPU driver version.

The steps, as AMD describes them:

  1. Install Unsloth Desktop.
  2. Open the model hub and filter for NPU models.
  3. Select Enable NPU.
  4. Download a listed model. AMD warns that progress can look stuck.
  5. Go to New Chat, pick a model in the NPU section, and load it with the context length you want.

Unsloth also says it manages its own copies of Lemonade and FastFlowLM, with separate runtime, configuration and model folders. An existing Lemonade or FastFlowLM install therefore shouldn't collide with it.

Memory is the real spec​

The headline demo is Qwen3.6-35B-A3B, a 35-billion-parameter mixture-of-experts model, running locally through Unsloth Desktop and FastFlowLM. The catch is memory. The demonstrated 35B model needs around 28 GB of available NPU memory. AMD's guide says that under Windows' default 50% shared allocation this works out to 64 GB of system RAM. Hardware Busters repeats the 64 GB recommendation.

Three figures get mixed up here, so keep them separate:

  • Model file size: 20.6 GB for the 35B model, per Unsloth's figures as reported.
  • Memory available to the NPU: about 28 GB with full context.
  • Recommended system RAM: 64 GB.

Hardware Busters relays these vendor figures from Unsloth's lineup:

ModelFile sizeRecommended RAMTokens/s
Qwen3.6-35B-A3B20.6 GB64 GB17.2 (3.13 s to first token)
Qwen3.8-27B18.1 GB48 GB2.0
Gemma4-12B-IT8.7 GB32 GB6.8
Qwen3.5-9B7.6 GB32 GB9.5
LFM3-1.7B1.5 GB16 GB40.7

These are not independent benchmarks. The tables don't say which chip or power mode produced them. Treat them as best-case or at least unverified. Unsloth's guide confirms that FastFlowLM's default power mode is Performance, so battery use may give lower numbers.

Why the 35B model beats the 27B​

The 35B figure sits well above the 27B one. Hardware Busters explains this with the model design. The 35B model is a mixture-of-experts that activates only about 3B parameters per token. The 27B model is dense. The outlet concludes that memory bandwidth, not the NPU's TOPS rating, is the limit. That reading is plausible, but it is the outlet's interpretation and nobody has measured it here. The practical takeaway still stands. If you are buying a Ryzen AI machine for local agents, total parameter count and TOPS tell you less than the model architecture and how much RAM you can fit.

Limits and caveats​

  • No arbitrary GGUF files. Models must be in Q4NX format. A pull request from the Unsloth contributor who built the integration says Unsloth GGUFs and fine-tunes cannot run on the NPU, because FastFlowLM needs its own model format. Hardware Busters says conversion is available through Unsloth's Export page, which runs on the CPU, or through AMD's open-source converter.
  • Energy claims are unverified. Unsloth says NPUs use two to four times less energy per token than a GPU. No test conditions were available to check that.
  • Licensing question. The same pull request flagged that FastFlowLM's runtime license is MIT, but its repository still has terms restricting commercial use of the NPU binaries for companies above USD 10 million in annual revenue. The contributor suggested confirming with AMD which terms apply. That was raised during development, so I couldn't confirm whether it has been settled. IT teams in larger organizations should check before any deployment.
  • Early-access plumbing. AMD's FastFlow SDK calls itself an early-access release in its own README. That refers to the SDK, not to Unsloth Desktop.

Security and administration​

Hardware Busters reports that Unsloth pins and SHA-256-checks its runtime downloads. It also says the managed Lemonade server accepts only local connections and uses a fresh auth key on every start. Setting UNSLOTH_STUDIO_ENABLE_NPU=0 before launch disables the feature. I couldn't confirm these in the AMD announcement, so treat them as the report's account of Unsloth's documentation.

Local-only binding limits network exposure. It does not make the agents and tools that connect to the endpoint safe. Admins who want to keep the NPU path off managed machines have the environment variable to work with.

Who should care​

For developers and enthusiasts with a Strix Halo or other XDNA 2 system and plenty of RAM, this is a low-friction way to run a capable MoE model without tying up the GPU. For everyone else, the NPU does not change the basic rule of local AI: memory capacity decides what you can run. Wait for independent benchmarks that state the chip, power mode and context length before treating 17 tokens per second as typical.

 

References

  1. Unsloth Desktop NPU Support Brings 35B Local Models to AMD Ryzen AI Laptops on Windows and Linux - Hardware Busters Hardware Busters 2026-11-19T19:10:22+00:00
  2. Studio: add chat on the AMD Ryzen AI NPU through Lemonade and FastFlowLM by oobabooga · Pull Request #11743 · unslothai/unsloth · GitHub github.com
  3. Unsloth Desktop Now Supports Using The AMD Ryzen AI NPU - Including On Linux - Phoronix phoronix.com