Run Qwen3.8-27B Locally with GitHub Copilot and Foundry Local on Windows

What Microsoft announced​

The post comes from Microsoft's Foundry team and is dated November 5, 2026. It makes three claims:

• The model: Qwen3.8-27B is Microsoft's main optimization target for local coding. A related pull request in the Azure model-asset repository adds Qwen3.8-27B base and CUDA GPU Foundry Local assets. They are registered with a 262,144-token context and a minimum Foundry Local version of 2.2.1.
• The hardware: Microsoft says 4-bit quantization shrinks the model to roughly 18 GB. That makes it workable on consumer GPUs with 24 GB of VRAM or more.
• The integration: Foundry Local exposes a local endpoint. The GitHub Copilot app uses its Bring Your Own Key (BYOK) mechanism to talk to that endpoint.

Microsoft also lists future plans: more devices (Macs and AMD GPUs), more coding-optimized local models, and deeper Copilot integration. These are plans, not current support.

The benchmark numbers, and how to read them​

Microsoft reproduces four scores for Qwen3.8-27B:

These come from Qwen's published model card, not from Microsoft testing the Copilot workflow. They are benchmark scores. They don't predict how often the model will finish your tasks.

The model card describes the test conditions. SWE-bench Pro and DeepSWE 1.1 ran through a Claude Code harness at temperature 1.0, top-p 0.95 and a 256K context window. NL2Repo-Bench also used the Claude Code harness, with some repository-access commands such as pip download and git clone disabled to prevent reward hacking. Copilot's agent loop, prompts and tool definitions differ from that harness, so results in Copilot may differ too.

The model card also lists Apache-2.0 licensing, a 27B dense architecture with native vision-language support, and a native context of 262,144 tokens, extensible to 1,000,000. Thinking mode is on by default. Don't assume every one of those features is exposed through this Copilot integration. Microsoft's post demonstrates coding only.

Under the hood: the optimization stack​

Microsoft says Foundry Local runs the model on ONNX Runtime and tunes the whole pipeline on NVIDIA GPUs. That covers prefill, decoding, memory management, long-context handling and concurrent requests. The named techniques fall into three groups.

Speed

• DFlash2 speculative decoding. A small drafter model proposes several tokens at once, and the main model verifies them. This aims to get past the memory-bandwidth limit on decoding.
• CUDA graph fusions and kernel tuning for NVIDIA's SM121 architecture.

Memory

• INT4 weight handling, KV-cache quantization, embedding memory sharing and CPU offloading.
• Configurable, memory-aware inference to trade memory against performance.

Long context and concurrency

• Prefix caching, so repeated prompt context isn't recomputed.
• Continuous batching for parallel requests.

Prefix caching matters for agentic coding. Agents resend large, mostly identical context on every turn, so reusing it saves a lot of work.

The performance claim versus llama.cpp​

Microsoft compared Foundry Local with llama.cpp on a 4-bit Qwen3.8-27B. The test systems were an RTX 5090 with 32 GB of VRAM and an NVIDIA DGX Spark with 128 GB of unified memory. The dataset was real GitHub issues. Microsoft says Foundry Local generated faster at every tested prompt length, and that the lead widened beyond 131K tokens.

Treat that as a vendor claim. The results are shown as charts in the post, not as text figures, so I'm not quoting speedups. The post also doesn't give software versions, the number of prompts or the run methodology. If you already run a llama.cpp setup that you're happy with, benchmark your own prompts before switching.

Setup guide for Windows​

Prerequisites​

• An NVIDIA GPU with at least 24 GB of VRAM. This is the requirement for Microsoft's Copilot walkthrough.
• Foundry Local CLI 0.13.0 or later.
• CUDA and cuDNN runtimes installed separately. Foundry Local does not download them.
• The GitHub Copilot app.

The 0.13.0 release is labelled a public preview and was published on November 2, 2026. Its notes call it an early build that may have rough edges and change between releases. They also say it updates the SDK and runtime packages to 2.3.0, ONNX Runtime to 1.31.0 and ONNX Runtime GenAI to 0.18.0. The improvements listed include better support for bring-your-own models and support for coding models.

The release notes give these runtime requirements:

• Windows and Linux x64 need CUDA 12 and a compatible cuDNN 9.
• Windows and Linux ARM64 need CUDA 13 and a compatible cuDNN 9.

The release lists Windows ARM64, Apple Silicon and Linux assets. That does not mean the Qwen3.8-27B Copilot setup works on all of them. Microsoft's walkthrough requires an NVIDIA GPU with at least 24 GB.

Steps​

1. Install the CLI and check the version.
   
   Code:

   winget install Microsoft.FoundryLocal
   foundry --version

   The version should be 0.13.0 or higher.
2. Configure the CUDA and cuDNN paths. On Windows, add the runtime DLL directories to PATH. Open a new terminal afterwards. If Foundry Local was already running, run foundry server restart from that new terminal.
3. Load the model and get the endpoint.
   
   Code:

   foundry model load qwen3.8-27b
   foundry server status

   The load command downloads the model if it isn't already present. That is a large download, so check your disk space and connection first. Use the endpoint that foundry server status reports on your machine. Don't copy the port from Microsoft's screenshot.
4. Add the provider in the GitHub Copilot app. Open Settings, choose Model providers, then Add provider, and select Foundry Local. Paste the endpoint into the Base URL field.
5. Add the model. Search for Qwen3.8-27B in Browse Catalog and add it.
6. Select it. Switch the provider to Foundry Local and pick Qwen3.8-27B in the model picker.

Troubleshooting​

• The version is below 0.13.0. Update through WinGet, then open a new terminal and check again.
• The GPU backend fails to load. Check the CUDA and cuDNN versions against the release notes. Check that the DLL directories are on PATH. Then restart the server from a fresh terminal.
• The endpoint doesn't work. Re-run foundry server status and use the address it prints now.
• The Foundry Local option is missing. GitHub's documentation says BYOK availability depends on the client and configuration method. On Copilot Business or Enterprise plans, an organization policy can disable users' ability to configure their own provider. Ask your admin if the option is missing.

What "private" does and doesn't mean​

This is the point to read carefully. GitHub's BYOK documentation says user-configured BYOK sends model requests from the client straight to your chosen provider, not through GitHub's Copilot API. A model hosted on your own machine therefore handles those requests locally. GitHub also notes that BYOK allows model use without a Copilot subscription. A cloud-hosted provider still needs internet access. For an air-gapped environment, the endpoint just has to be reachable from the client.

Microsoft's claim that "your code and your context never leave the machine" is reasonable for the inference step. It is not a blanket guarantee for everything else. The Copilot app, extensions, repository services and tool calls can still use the network. Sign-in and account features are examples. I'd read the claim as being about where the model runs, not as a verified end-to-end air gap. If you have strict data-residency or offline requirements, test the whole setup with the network monitored.

Other people have wired Foundry Local into Copilot before. A Sense of [AI] write-up from June describes pointing Copilot CLI at a Foundry Local endpoint using BYOK environment variables. It also notes that a model without tool calling produces an error instead of silently falling back to a GitHub-hosted model. That is useful behavior to know about. Tool calling is central to agent workflows, and a model that lacks it won't work in them.

Practical take​

Reasons to try it

• You have a 24 GB or larger NVIDIA GPU and want zero per-token inference charges.
• You work with code that shouldn't go to a third-party model provider.
• You work offline or on flaky connections.

Reasons for caution

• Microsoft admits smaller local models can reason less well than advanced cloud models. Its recommendation is a hybrid workflow. Use local for everyday coding and targeted subtasks, and cloud models for complex, high-reasoning work.
• The CLI is a preview build, and the setup depends on separately installed CUDA and cuDNN runtimes.
• The headline speed comparison is Microsoft's own, with limited methodology detail.
• "Predictable cost" isn't "free". Power and hardware costs shift to you, and a 24 GB-plus GPU is a significant purchase.

One more observation, from general industry knowledge. Copilot's BYOK path and Foundry Local's OpenAI-style endpoint make the model a swappable part. If Microsoft's roadmap delivers Mac and AMD support and more coding models, the model matters less than the wiring. That is the more durable part of this announcement.

Bottom line​

For Windows developers with high-VRAM NVIDIA cards, this is a documented, fairly short route to a local coding agent inside Copilot. Three conditions apply: you need Foundry Local CLI 0.13.0 or later, you need the CUDA and cuDNN runtimes installed yourself, and you should keep cloud models available for hard problems. Check the speed and quality claims against your own repositories before relying on them. The privacy benefit is real for where inference runs, but verify the rest of the data flow yourself.