Microsoft Foundry Agent Service Makes Voice Its Own Agent Type
In the September 24 Azure blog post, Tina Schuchman, Microsoft's corporate vice president for Foundry, says many voice experiences today are built by putting speech on top of text agents. That approach needs separate components for speech, orchestration, monitoring and deployment. Microsoft's answer is a native agent type that goes through the same build, deploy, observe, evaluate and optimize cycle as every other Foundry agent. Voice agents, now in public preview, are a new agent type supporting both prompt and hosted agents.
The difference between prompt agents and hosted agents matters here. Microsoft's Foundry Agent Service overview describes prompt agents as configuration only: you supply instructions, a model and tools, and Foundry runs the agent without any application code. Hosted agents are your own code, built with Microsoft Agent Framework, LangGraph, the OpenAI Agents SDK or another framework. They ship as a container or zip file, and Foundry runs them with a managed endpoint, scaling and an Entra identity. So a "voice agent" can be a no-code configuration or a custom codebase. How much integration work you do depends on which one you pick.
For models, Microsoft says developers can choose from industry-leading voice models— GPT Realtime, Azure Realtime, MAI and more—to balance latency, reasoning depth, and cost. Customers can also bring their own model. The announcement does not compare the models or list prices. Microsoft says the agents handle natural turn-taking, recover smoothly when a user interrupts, and support more than 80 languages and 140 locales. Those counts are Microsoft's figures. The documentation notes that supported settings and voice families depend on the model you choose, so the numbers should not be read as equal coverage for every combination.
Customization covers three areas. Teams can fine-tune speech recognition for domain vocabulary, create a custom voice, or add a photo or video avatar. Agents can also reuse the tools, enterprise knowledge and governance policies already set up for a team's other Foundry agents. For organizations that already have Foundry agents in production, this is the most useful part of the announcement.
Six Months of Voice Live Previews Led to Foundry's New Voice Agent Type
Voice in Foundry did not start this week. It has been developing through previews all year. When Foundry Agent Service reached general availability in March 2026, Microsoft's Foundry blog listed Voice Live (preview) + Foundry Agents: Real-time speech-to-speech, fully managed, wired natively to your agent's prompt, tools, and tracing. The Tech Community post for that launch described the problem this way: building a production voice agent meant assembling speech recognition (STT), language reasoning and agent logic, tool orchestration, speech synthesis (TTS) and real-time streaming infrastructure, and developers often had to manage latency, synchronization, and conversational state across separate services.
Build 2026 brought the code-first path. Microsoft said then that for teams that need full control over their agent runtime and orchestration framework, hosted agents with Voice Live is available in public preview. The hosted agents update also added a one-click option: Voice Live integration with hosted text agent is now in public preview, enabling real-time voice experiences through both the Responses and Invocations protocols. For native speech-to-speech hosted agents, hosted agents now support WebSocket and WebRTC for real-time voice scenarios. Region coverage was clearly still growing at that point. The same post listed plans to expand Voice Live / WebSocket coverage beyond North Central US to additional regions.
The September release brings these pieces together as one named agent type with its own observability and deployment paths. It also comes shortly after an older route was retired. Microsoft's quickstart for the earlier integration warned that this classic integration uses Agent Service (classic) and will be deprecated 8/31/2026. Teams that built voice features on the classic Agent Service should treat the new voice agent type as the migration target, not an optional extra.
Speech-to-Speech or Cascaded: The Architecture Choice Inside Foundry Voice Agents
Microsoft's current Foundry Agent Service documentation explains how the managed version, the voice-based prompt agent, works. You configure a model, instructions, audio settings, an optional greeting and tools. Foundry manages the agent's lifecycle, and Voice Live handles speech recognition, turn-taking, model interaction and speech synthesis. Clients connect over a real-time WebSocket instead of sending separate text requests. With a fully managed Voice Live model, you don't need to deploy a separate audio model.
The documentation describes two architectures, and choosing between them is the main design decision:
| Architecture | Pipeline | What Microsoft says it offers | Trade-off |
|---|---|---|---|
| Native speech-to-speech | Caller audio → real-time speech model → spoken response | Natural, low-latency conversation; the model handles audio in and out directly | Supported settings and voice families depend on the chosen model |
| Cascaded text model | Caller audio → speech recognition → text model → speech synthesis → spoken response | Wider choice of text models and Azure voices, plus explicit control over transcription, phrase lists, turn detection and interim responses | Each stage adds to end-to-end latency |
The cascaded pipeline is the likely choice when a deployment needs tight control over vocabulary, such as product names, medical terms or account formats, or when the team wants a particular text model for reasoning. Native speech-to-speech suits cases where conversational feel and response time matter most. The documentation lists the resources a typical portal-built voice agent uses: a Foundry resource and project, a versioned voice-based prompt agent, a managed Voice Live model or an eligible customer-deployed model, Azure Speech capabilities where the architecture requires them, Application Insights for traces and evaluation, and Azure Communication Services with Azure Event Grid for phone channels.
This overview page was last updated September 24, the same day as the announcement. It describes the platform as it stands now, and it may not reflect a list of features that all launched this week.
Teams Phone, Twilio and the Fine Print on Foundry Voice Agent Channels
The announcement lists four deployment targets: web channels, Microsoft Teams, Teams Phone, and Twilio inbound and outbound telephony. Microsoft's documentation of the current portal gives a more detailed picture, and anyone planning a rollout should read it:
- Browser testing works from the agent playground, and a preview web app is available when the agent endpoint supports the Responses protocol.
- Phone-number channels currently mean Microsoft Teams Phone Extensibility and Twilio.
- Direct SIP is not currently offered as a supported customer flow, so organizations that want to connect their own SIP trunk cannot yet do so this way.
- The standard Teams and Microsoft 365 Copilot app publishing card for text agents isn't currently shown in the voice agent Channels layout, and Microsoft warns against treating that path as the phone integration.
- Avatars appear only in the browser, over WebRTC. Telephone callers receive audio only.
The avatar limit affects anyone comparing this with Google's announcement. A photo or video avatar is a browser feature. On a phone call, the customer hears an audio agent only.
Twilio's role goes beyond Foundry's own channel list. At Build, Microsoft quoted Twilio saying that hosted agents' framework-agnostic design let developers deploy Twilio Agent Connect directly inside its serverless runtime. Contact-center teams already on Twilio therefore have both a managed channel and a code-level option.
Observability, AZD AI and the Foundry Toolkit for VS Code That Isn't Here Yet
Voice agents add their own problems to observability. Conversations are streams, not single requests, and latency is part of the user experience. Microsoft says voice agent observability covers tracing, monitoring, transcripts and evaluation designed for spoken interactions. The documentation lists what that means today:
- Voice-specific summary cards, token breakdowns and connection-latency charts require Application Insights.
- Voice traces use a single trace-focused view, because voice traces don't have a Responses API response_id.
- Full-conversation evaluation of existing conversations is supported. Single-turn and simulated-conversation evaluation are not yet supported for voice agents.
- The action that creates a dataset from the trace page isn't currently available for voice agents.
That last point sits awkwardly beside the rest of the September announcement. Microsoft's broader update builds a "hill-climbing" loop in which Insights in Foundry, the Rubric evaluator, trace-to-dataset generation and the Agent optimizer take production evidence and use it to improve agents. The blog says the Rubric evaluator, trace-to-dataset generation and Agent optimizer are generally available later this month. Voice agents can't yet use every part of that loop, particularly dataset creation from traces. Teams expecting to optimize voice agents the same way they optimize text agents should plan for some manual steps for now.
On developer tooling, the blog names AZD AI as a repeatable, source-controlled workflow for configuring and deploying voice agents. It says the Foundry Toolkit for Visual Studio Code is "coming soon" for voice agents, and will let developers create, run, speak with and debug voice agents inside the editor. The same blog says the toolkit already supports hosted agents and Microsoft Agent Framework for long-running work. What is still coming is the voice-specific workflow in the toolkit.
Gemini 3.8 Live Avatar Arrives a Day Earlier With a Different Pitch
Neowin presented Microsoft's news as a response to Google's Gemini Live. The dates are close. On September 24, Google published "Introducing Gemini 3.8 Live with Live Avatar", building on its Gemini 3.8 Live model launch the week before. Google says the feature pairs live dialogue with low-latency streaming video, producing an agent that listens, sees and speaks with an animated visual persona. It includes lip-sync, natural expressions and turn-taking, and is available in Gemini Enterprise from launch day.
Google stresses two capabilities. The first is asynchronous tool calling: the avatar can start tool calls and fetch data in the background while the conversation continues. The second is multilingual support: Google says Live Avatar can switch among 97 languages mid-conversation, adapting lip-sync and expressions as it goes. Organizations can use preset avatars or generate a custom one from a reference image, though Google says custom creation is currently available only through enterprise allowlisting. Google also says all output is watermarked with SynthID.
The two products have different focus. Google's announcement is about a visual, avatar-led experience on its own model. Microsoft's is about development and operations: choice of voice models including OpenAI and Microsoft MAI options, reuse of existing Foundry tools and governance, and deployment into Teams and phone systems. In Microsoft's version, avatars are optional and browser-only. Neither company has published a benchmark against the other, and nothing in Microsoft's post shows it was a direct reply to Google. The two launches landed at nearly the same time.
What this means for Foundry teams weighing voice agents
Teams that already run Foundry agents, especially any still on the classic Agent Service voice integration, should start testing the new voice agent type now. Teams planning production phone deployments should wait until the channel, evaluation and SLA gaps are resolved. Microsoft's documentation says preview capabilities have no service-level agreement and are not recommended for production until customers have reviewed support, security, compliance and availability requirements. Availability depends on the Foundry resource region, the model and voice selected, the subscription, and which preview features are enabled.
Cost also needs attention. Microsoft's documentation lists possible charges for model input and output, speech recognition and synthesis, custom voice training and hosting, avatar use, Application Insights ingestion and retention, Azure Communication Services calling and phone numbers, Teams and telephony services, and any connected tools and data. Budget for the whole setup, not a single voice feature.
- Voice agents in Foundry Agent Service are in public preview as a new agent type that covers both prompt agents and hosted agents.
- The Agent Service (classic) voice quickstart was marked for deprecation on August 31, 2026, so classic-based voice projects should move to the new model.
- Choose native speech-to-speech for the lowest latency, or the cascaded pipeline for control over transcription, phrase lists and turn detection.
- Supported phone channels are Teams Phone Extensibility and Twilio. Direct SIP is not a supported customer flow, and avatars do not appear to phone callers.
- Enable Application Insights from the start, because voice monitoring summaries and latency charts depend on it.
- The Foundry Toolkit for VS Code workflow for voice agents is still "coming soon". Use AZD AI for source-controlled deployment in the meantime.
For Foundry customers, voice is now part of the same agent platform instead of a separate project: the same tools, governance and deployment pipeline, with a voice model chosen per workload. The main thing to watch is when voice agents gain full evaluation support, trace-to-dataset support, the VS Code toolkit and a production SLA. Until then, the preview is best used to test voice features on existing Foundry agents and to model the full cost before any customer-facing phone line depends on it.