Azure Databricks Private Network Gateway: VNet Setup, Routing Priorities and Preview Limits

For Azure architects, cloud administrators, and Windows enterprise data teams, the pitch is straightforward: stop building a brand-new bridge every time a serverless job needs to talk to an internal database. However, a close look at Microsoft Learn’s Azure Databricks documentation reveals that while PNG significantly streamlines virtual network (VNet) routing, it comes with strict routing precedence rules, regional constraints, and upcoming December 2026 billing milestones that IT teams need to plan around before flipping the switch.

The Serverless Networking Bottleneck: Why Classic Compute Lingered​

To appreciate why a subnet gateway matters, consider the fundamental architectural split between classic and serverless compute in Databricks. Classic compute clusters run inside a customer's own virtual network, meaning they can reach internal data sources using standard cloud primitives like VNet peering, VPNs, or Azure ExpressRoute.

Serverless compute, by contrast, runs inside a Databricks-managed network. While that frees data engineering teams from waiting on cluster spin-ups and managing infrastructure versions, it creates a strict security boundary. Giving that Databricks-managed serverless environment a private path back into customer-hosted databases, Kafka clusters, or internal APIs has historically been a high-friction exercise.

According to Databricks' announcement blog, more than 75% of users in the company's customer interviews cited their existing network configuration—and the sheer complexity of pushing through network changes—as the main reason specific workloads remained stuck on classic compute instead of migrating to serverless. It is worth noting from an analytical standpoint that this 75% figure comes from internal vendor interviews without a published sample size or independent survey methodology, so enterprise architects should view it as directional customer feedback rather than a hard industry benchmark. Even so, the underlying operational headache will ring painfully true to anyone who has ever sat in a change-advisory board meeting.

Previously, while Serverless Private Link worked well for cloud-managed Platform-as-a-Service (PaaS) offerings like Azure Data Lake Storage or AWS S3, connecting to self-managed or on-premises resources required standing up a separate private endpoint, load balancer, and proxy VM for each individual resource. Connecting to three databases meant three sets of infrastructure; connecting across dozens of internal APIs and on-premises systems meant weeks of three-way ticket tennis between Databricks administrators, Azure cloud engineers, and corporate network security teams.

Section Summary:

• The Problem: Serverless compute runs in a Databricks-managed network, requiring explicit private paths back to customer-hosted data.
• The Old Way: Per-resource private endpoints, load balancers, and proxy VMs created administrative bottlenecks across cloud and security teams.
• The Impact: Databricks reports that over three-quarters of interviewed customers kept workloads on classic compute primarily to avoid this networking complexity.

How Private Network Gateway Works on Microsoft Azure​

Private Network Gateway shifts the unit of configuration from the individual endpoint to the network level. Instead of provisioning proxy VMs, an Azure administrator delegates a single subnet in their Azure VNet to Microsoft.Databricks/workspaces. When PNG is configured, Azure Databricks injects a managed, encrypted gateway directly into that delegated subnet.

Once established, serverless workloads can route traffic through the gateway into the VNet and onward to any downstream destination that the VNet can reach—including peered VNets and transitively connected on-premises environments linked over Azure ExpressRoute or site-to-site VPNs.

Databricks positions five primary enterprise scenarios for PNG:

• Private Database Ingestion: Using Lakeflow Connect on serverless compute to pull data from private databases across VNets, VPCs, and connected on-premises data centers.
• AI Agent Connectivity: Allowing Databricks-hosted AI agents to securely call privately hosted Model Context Protocol (MCP) servers and internal APIs.
• Private Package Repositories: Installing Python libraries directly from internal PyPI, Artifactory, or Nexus repositories without exposing those artifact servers to the public internet.
• Centralized Firewall Inspection: Routing serverless outbound traffic through existing customer firewalls (such as Palo Alto Networks appliances or Azure Firewall) for packet inspection and native logging.
• Consistent Source IPs for SaaS Allowlisting: Ensuring external SaaS platforms, such as Salesforce, see outbound traffic originating from customer-owned IP addresses rather than shared cloud egress ranges.

Early enterprise adopters cited by Databricks and SDxCentral highlight the reduction in cross-team friction. Primož Gale, lead technical architect at Slovenian banking group NLB, stated that PNG simplified private connectivity setup, cut operational overhead, and helped jobs run faster. Michel Schwandner, a platform engineer at furniture hardware manufacturer Julius Blum GmbH, noted that the gateway allowed his team to access internal services directly and manage outbound traffic centrally through existing governance controls. Similarly, Fabiano Seki, data engineering tech lead at Banco Bradesco, pointed out in Databricks' announcement that eliminating per-connection load balancers and Private Link services simplified firewall approvals across Azure and on-premises sources. As with any vendor-published testimonials, IT leaders should treat claims of faster job execution as reflections of streamlined workflow and reduced setup latency rather than audited throughput benchmarks.

Section Summary:

• Architecture: Customers delegate one subnet to Microsoft.Databricks/workspaces, and Databricks manages the gateway lifecycle inside it.
• Reach: Traffic flows from serverless compute into the VNet and across transitive links like ExpressRoute or VPN.
• Key Scenarios: Private database ingestion, internal MCP servers for AI agents, private Python repositories, firewall inspection, and stable source IP allowlisting.

Traffic Modes, Routing Precedence, and Security Boundaries​

handing a third-party platform a delegated subnet inside a corporate VNet naturally raises a critical question for security architects: who actually controls the perimeter?

According to Databricks and Microsoft Learn documentation, each gateway is dedicated to a single customer rather than operating as a shared multi-tenant appliance, with isolation enforced through traffic encapsulation and network security rules. Workspace-level access controls are evaluated before traffic ever leaves the serverless environment. Once traffic lands in the delegated Azure subnet, the customer's own Azure route tables, Network Security Groups (NSGs), DNS resolvers, and firewall appliances govern where packets are allowed to travel.

Administrators control what enters the gateway by choosing between two distinct traffic_mode settings when creating the resource:

1. SPECIFIC_DESTINATIONS Mode: Routes only the explicitly listed Fully Qualified Domain Names (FQDNs) through the gateway, while all other serverless traffic follows its normal path. This mode supports domain suffix matching—for instance, listing mydb.contoso.com automatically covers sub.mydb.contoso.com. Microsoft recommends this mode for the majority of deployments because it provides surgical control without altering unrelated traffic flows. Furthermore, Azure Databricks automatically allowlists these specified destinations in serverless egress control policies.
2. ALL_TRAFFIC Mode: Routes all outbound serverless traffic into the customer's network through the gateway, unless a higher-priority route applies. This mode is designed for organizations that mandate deep packet inspection through a corporate firewall or require all internet-bound traffic to exit via customer-owned IP addresses. Crucially, unlike SPECIFIC_DESTINATIONS mode, using ALL_TRAFFIC requires administrators to explicitly add every intended destination to their serverless egress network policy.

The Azure Databricks Routing Hierarchy​

One of the most important technical nuances documented in Microsoft Learn is that PNG is complementary to—not a replacement for—Azure Private Link. Even if you enable ALL_TRAFFIC mode, PNG does not hijack every packet. Azure Databricks evaluates outbound serverless connections in a strict priority order:

Why does Priority 2 matter so much on Azure? Because Azure Blob Storage and Azure Data Lake Storage traffic routed via Azure service endpoints will always follow the native Azure backbone path. You cannot force Azure Blob Storage traffic through a Private Network Gateway; for direct private access to cloud object storage, Private Link remains the required tool.

Section Summary:

• Two Modes: SPECIFIC_DESTINATIONS routes only listed FQDNs (with automatic egress policy allowlisting), while ALL_TRAFFIC routes broad egress for firewall inspection (requiring manual egress policy allowlisting).
• Precedence: Private Link rules (Priority 1) and Azure Service Endpoints (Priority 2) always override PNG (Priority 3).

Prerequisites, Preview Limits, and Regional Availability on Azure​

Before Azure administrators rush into the portal, Microsoft Learn outlines specific technical prerequisites and preview boundaries that govern how PNG can be deployed today:

• Licensing & Billing: PNG requires an Azure Databricks account on the Premium plan. While the feature itself is free to use during the initial preview window, Microsoft Learn explicitly notes that billing for Private Network Gateway begins on or after December 1, 2026.
• Subnet Sizing & Isolation: Databricks recommends creating a brand-new, dedicated subnet with a /28 CIDR block or larger. The subnet cannot be used by any other Azure resource.
• Tenant & Regional Alignment: The VNet subscription and the Azure Databricks workspace subscription must reside in the same Microsoft Entra ID tenant. Furthermore, the gateway, the delegated subnet, the Network Connectivity Configuration (NCC), and the target workspaces must all be in the exact same Azure region.
• Supported Azure Regions: During Public Preview, PNG is available in 25 Azure regions: australiaeast, brazilsouth, canadacentral, centralindia, centralus, eastasia, eastus, eastus2, francecentral, germanywestcentral, japaneast, koreacentral, northcentralus, northeurope, southcentralus, southeastasia, swedencentral, switzerlandnorth, uaenorth, uksouth, westcentralus, westeurope, westus, westus2, and westus3.
• Capacity & Object Quotas: Azure gateways offer provisioned bandwidth tiers of 10, 30, 50, and 100 Gbps, defaulting to 30 Gbps in the account console. Each NCC supports a maximum of two gateways, and each individual gateway supports at most two custom DNS resolvers and 100 destination rules.
• Supported Workloads: The preview supports serverless Azure Databricks Runtime products and AI services.

Step-by-Step Guide: Configuring Private Network Gateway in Azure Databricks​

PNG reuses Databricks' existing Network Connectivity Configuration (NCC)—an account-level management object—rather than inventing a separate governance hierarchy. Administrators can deploy PNG via the Databricks Account Console UI, the Networking v1 REST API (/api/networking/v1/), or Terraform using the databricks_private_network_gateway resource.

Here is the standard deployment workflow for Azure environments:

Step 1: Delegate the Azure VNet Subnet​

1. Sign in to the Azure Portal and navigate to your target Virtual Network (VNet).
2. Select Subnets from the left navigation pane and create or choose a dedicated, unused subnet (at least /28 in the same region as your NCC and workspaces).
3. Under Subnet delegation, select Microsoft.Databricks/workspaces and click Save.

Step 2: Configure DNS Resolution​

1. By default, Databricks recommends leaving custom DNS resolvers empty ([] in the API) so the gateway uses Azure-provided DNS for public hostnames and Azure Private DNS zones linked to your VNet.
2. If your target resources reside in an Azure Private DNS zone, ensure that zone is explicitly linked to the VNet containing your delegated gateway subnet.
3. If you use on-premises DNS or have overlapping private DNS zone names, prepare the IP addresses of up to two custom DNS resolvers that are directly reachable from the delegated subnet.

Step 3: Create the Gateway Inside an NCC​

1. Log in to the Azure Databricks Account Console with Account Admin privileges.
2. Navigate to Security > Network connectivity configurations and select an existing NCC in your target region (or create a new one).
3. Open the Private network gateway tab and click Add private network gateway.
4. Provide a display name, specify the full Azure resource ID of your delegated subnet, select your bandwidth tier (10, 30, 50, or 100 Gbps), and choose SPECIFIC_DESTINATIONS (along with your target FQDNs) or ALL_TRAFFIC.
5. Wait for the gateway state to transition from CREATING to ESTABLISHED.

Step 4: Attach the NCC to Your Workspaces​

1. In the Account Console, open Workspaces, select the target workspace in the same region, and click Update Workspace.
2. Under Network connectivity configuration, select your NCC and save. Repeat this for every workspace in that region that needs access to the gateway.

Admin Pro-Tip for Lifecycle Management: While you can update destinations, DNS resolvers, traffic modes, and bandwidth tiers in place via PATCH requests (note that bandwidth scaling takes effect asynchronously without a completion callback), you cannot change a gateway's subnet in place. Moving to a different subnet requires deleting the gateway—which immediately severs active workload connections—and provisioning a fresh one.

Troubleshooting Common Azure Deployment Failures​

Even with a simplified gateway model, enterprise networking rarely goes off without a hitch on day one. Microsoft Learn documents several common failure modes and their exact fixes:

• Gateway Stuck in CREATING for Over 10 Minutes: Usually caused by missing subnet delegation or insufficient IP addresses. Verify that the subnet is at least a /28 block, has no competing Azure resources inside it, and is properly delegated to Microsoft.Databricks/workspaces.
• DNS Fails with SERVFAIL or NXDOMAIN: Indicates that your custom DNS resolver IP cannot be reached from the gateway subnet, or your Azure Private DNS zone is not linked to the VNet hosting the gateway subnet.
• DNS Resolves, But Connections Time Out: A classic firewall or NSG block. Inspect the inbound and outbound NSG rules on both the delegated gateway subnet and the target resource's subnet, and verify your VNet route tables.
• JDBC or Database Handshake Fails After DNS Resolution: Check the destination database's own host firewall or authentication allowlist; ensure it permits incoming traffic from the IP range of the delegated gateway subnet.
• Traffic to Azure SQL or Key Vault Ignores the Gateway: If the target FQDN matches a public-facing Azure service endpoint suffix, Priority 2 routing forces it onto the Azure service endpoint backbone instead of PNG. For private-only resources, configure a more specific internal DNS_NAME on the gateway.
• Configured FQDN Traffic Is Dropped Immediately: Check if a Serverless Egress Gateway (SEG) policy is active on overlapping workspaces; SEG policies are evaluated prior to PNG and will drop traffic if the destination is denied.

What This Means for Enterprise Cloud Architects​

Databricks’ Private Network Gateway represents a pragmatic maturity step for serverless data and AI architectures on Microsoft Azure. For years, cloud vendors have pitched serverless as an effortless panacea, while enterprise infrastructure teams quietly absorbed the complexity of stitching managed compute clouds back into hybrid, ExpressRoute-connected corporate networks. By collapsing per-resource proxy sprawl into a single /28 delegated VNet subnet, PNG removes one of the strongest architectural excuses for keeping legacy clusters running.

Still, Windows and Azure administrators should approach the preview with eyes wide open: audit your Private Link versus service-endpoint routing paths, keep the 100-destination and single-region NCC boundaries in mind, and factor the upcoming December 1, 2026 billing start date into your cloud budget forecasts.

If you are planning to test Private Network Gateway in your Azure environment or integrating hybrid workloads over ExpressRoute, head over to our Cloud Computing & Azure Architecture and Enterprise Networking & Security sections here on WindowsForum.com to share your deployment topologies and compare notes with fellow sysadmins. Have you been holding back serverless migrations because of VNet proxy sprawl, or are you sticking with classic compute for good? Let us know in the discussion thread below!