TomTom and Microsoft: In‑Car Conversational AI Powered by Azure OpenAI

TomTom and Microsoft have quietly rewired a familiar part of the car — the voice assistant — into something closer to a conversational co‑pilot, pairing TomTom’s navigation and in‑vehicle software with Microsoft’s Azure OpenAI stack to deliver an in‑car AI assistant that can hold multi‑turn conversations, handle complex routing and POI requests, and control vehicle functions through natural speech. This ties TomTom’s Digital Cockpit to Azure OpenAI (with supporting Azure services) and positions automakers to ship richer voice experiences without building the large‑language‑model plumbing themselves.

Background / Overview​

TomTom announced the joint effort with Microsoft in December 2023 and showcased the work publicly during the CES cycle, positioning the feature as part of its Digital Cockpit product — an OEM‑ready, modular infotainment SDK that can be embedded or rebranded inside vehicle systems. The project leverages Microsoft’s Azure OpenAI Service (the managed way to call OpenAI‑style models on Azure) together with other Azure components such as Azure Kubernetes Service and Azure Cosmos DB to provide back‑end scale and state management. This is not TomTom’s first step into conversational AI: the company previously published a ChatGPT plugin that linked its maps to ChatGPT to let users plan trips through chat, and it has an ongoing commercial relationship with Microsoft that stretches back to Azure Maps and mapping services. That prior collaboration gave TomTom a practical starting point for integrating Microsoft cloud services at scale inside vehicles.

---

What TomTom built — the product and the promises​

TomTom presents the assistant as an OEM‑grade, conversational agent embedded in or alongside the Digital Cockpit. The company emphasizes three practical capabilities:

• Natural, multi‑turn conversation for routing and discovery: ask for a destination, then ask follow‑ups (e.g., “Find an EV charger along that route with a café nearby”), and the assistant will reason across steps.
• Deep integration with navigation and vehicle controls: route planning, EV range and charger planning, and direct vehicle commands (temperature, media, windows) can be issued with a single voice flow.
• OEM control over branding and UX: TomTom’s SDK model lets carmakers keep the look, voice, and gating policies for the in‑vehicle experience while outsourcing the heavy language work to Azure.

TomTom’s internal write‑ups and interviews with its engineers say the prototype went from concept to working demo in months (TomTom spoke of a focused nine‑month engineering effort in post‑demo materials), and the company highlighted how map‑centric context and telematics data strengthen the assistant’s responses compared with purely general‑purpose chatbots.

How TomTom frames the business case​

TomTom pitches this as a way for automakers to regain control over the in‑car experience that many drivers already prefer to outsource to their phones. The company cites studies showing users often prefer phone apps to built‑in car assistants; the new assistant targets that gap by combining natural conversation with onboard sensor and map awareness. OEMs get a shorter time‑to‑market route for richer voice features and retain the option to monetize or control updates themselves via TomTom’s SDK and cloud hooks.

---

The technology stack and architecture (what’s under the hood)​

TomTom’s public materials and independent reporting make the architecture clear: a hybrid model where in‑cab wake‑word detection and safety gating typically run locally, while heavy conversation and reasoning go to cloud endpoints (Azure OpenAI) that are augmented with TomTom’s navigation services and back‑end data.
Key cloud components TomTom and partners report using:

• Azure OpenAI Service for LLM inference and conversational reasoning.
• Azure Kubernetes Service (AKS) to run scalable microservices and orchestration.
• Azure Cosmos DB for contextual session state and telematics indexing.
• Azure Cognitive Services (speech, transcription, TTS) for robust voice I/O.

This hybrid pattern is now a template in the automotive world: keep latency‑sensitive and safety‑critical elements as close to the vehicle as possible, and farm out open‑ended reasoning to the cloud where large models and retrieval systems can be combined with up‑to‑date knowledge. Mercedes’ integration of ChatGPT into its MBUX assistant — which ran as a beta in mid‑2023 and used Azure OpenAI as the cloud back end — is a high‑profile precedent for the hybrid approach.

Retrieval and safety layers​

TomTom’s approach uses its own map and POI data to ground LLM responses (retrieval augmentation), a necessary guard against the hallucination risks of pure LLM output. In simple terms, when a driver asks for a location, the system pulls authoritative map data and availability (EV charger status, live traffic) before composing a human‑facing answer — often an architecture called Retrieval‑Augmented Generation (RAG). TomTom and Microsoft documents indicate that retrieval plus on‑model safety filters are central to product design.

---

The user experience: what works and what doesn’t yet​

TomTom’s demos and early press coverage show a conversational flow that feels more fluid than traditional command lists. Examples the company highlights include:

• Ask for a scenic route to a named restaurant, then ask to add a stop at a charger that’s rated at least 4 stars. The assistant chains those constraints and builds a route.
• Ask follow‑ups without repeating context: “How long will the stop add to my trip?” after asking for a detour. The system retains context for multi‑turn dialogue.

These are practical, real‑world improvements over rigid voice menus. But the demo environment is controlled; the real test is the messy variety of accents, noisy cabins, partial connectivity, and edge cases (obscure POIs, conflicting requests) that occur at scale.

Limitations and UX risks​

• Latency: cloud inference introduces round‑trip delays. TomTom reports engineering optimisations, but in poor cellular coverage the system must gracefully degrade to local commands or cached maps.
• Distraction: richer conversations can prolong interactions and increase cognitive load. Overly conversational assistants risk pulling attention from the road despite being hands‑free. Mercedes and other OEMs explicitly frame these systems to limit complex interactions while driving, but behavioral risk remains.
• Expectation mismatch: drivers may assume the assistant has perfect situational awareness (full sensor access, current traffic, vehicle health). Integration gaps and inconsistent data can erode trust quickly.

---

Safety, privacy, and governance — the hard tradeoffs​

Embedding LLMs in cars amplifies well‑known AI governance problems through the lens of mobility and safety.

• Data flows: voice snippets, vehicle telemetry and location history traverse cloud services. TomTom and Microsoft state that OEMs retain branding and integration control, but the telemetry pathways and retention policies are implementation decisions that determine privacy exposure. TomTom’s and Microsoft’s marketing materials promise enterprise controls and opt‑ins but don’t enumerate all retention timelines in public release notes; those are matters OEMs and regulators will have to scrutinize.
• Regulatory scrutiny: voice assistants that can act (e.g., book a service or join a corporate meeting while driving) overlap with consumer protection, telematics regulation, and workplace privacy rules. Fleet and corporate use cases will need stricter governance than private car scenarios. Microsoft’s guidance for Azure OpenAI emphasizes responsible use and tenant governance; automakers will need to map those controls into their firmware and telematics policies.
• Safety gating: the canonical hybrid architecture keeps wake‑word detection, simple command execution (e.g., HVAC control) and safety interlocks local on the vehicle. Anything that could materially alter vehicle behavior must be explicitly safety‑gated, auditable, and revertible. The industry has converged on this, but verification and auditing of model outputs remain a live engineering challenge.

---

How TomTom’s move fits into the broader auto‑AI landscape​

TomTom’s product is part of a broader pattern: hyperscalers (Microsoft, Google, Amazon) and specialist suppliers (Cerence, TomTom, BlackBerry QNX) are packaging cloud AI for automakers so OEMs can offer richer cabin experiences without inventing large‑language models themselves.

• Microsoft has been active with partners; beyond TomTom, collaborations with Cerence and Mercedes (the Mercedes MBUX ChatGPT beta) show Microsoft’s Azure OpenAI story is a platform play across multiple OEM partners.
• OEMs differ in strategy: some (e.g., certain premium brands) will push cloud‑first assistants as premium differentiators, while volumes and fleet customers may prefer lightweight on‑device options with limited cloud fallbacks.

TomTom’s advantage is its mapping and POI dataset combined with an SDK that targets OEM integration — a natural complement to the cloud reasoning layer Microsoft supplies. That makes TomTom attractive to mid‑tier OEMs that want a tested, map‑aware assistant without building LLM integrations in‑house.

---

Commercial and aftermarket considerations​

• Time‑to‑market and OTA updates: TomTom’s SDK model and Azure’s managed endpoints enable over‑the‑air (OTA) updates of assistant features and model endpoints, sidestepping costly dealer visits. That mirrors the way Mercedes and other OEMs have pushed conversational upgrades in recent years.
• Monetization: OEMs could bundle advanced conversational features as subscription services (navigation + concierge + workplace integrations), while TomTom can offer the stack as a licensed feature with customization services. Expect tiers: basic onboard voice control (local), advanced generative conversation (cloud), and enterprise/fleet variants with audit and logging. TomTom’s Digital Cockpit and partnerships with suppliers such as Marelli and CARIAD show the route to embedding the SDK across vehicle lines.
• Integration partnerships: TomTom announced deals to pre‑integrate its cockpit SDK with suppliers (for example, Marelli) and has co‑developed navigation solutions with group software units (e.g., CARIAD for Volkswagen Group), demonstrating the commercial channels automakers typically use to bring such technologies into production vehicles.

---

Verifiable claims, vendor metrics, and what’s still opaque​

TomTom and Microsoft have supplied several quantitative claims in vendor pages and case studies. Two are worth calling out and verifying independently:

• TomTom’s engineering timeline: TomTom engineers described a roughly nine‑month focused effort to build a working assistant prototype that combines maps with Azure LLMs. This comes directly from TomTom’s newsroom interviews and is a verifiable company claim.
• Reported accuracy and latency improvements: Microsoft’s partner materials (regionally published case pages) report internal metrics such as reducing response times from 12 seconds to 2.5 seconds and achieving “95% correct responses” across 300 evaluated scenarios in one evaluation. These are promising vendor results but should be treated as vendor‑supplied benchmarks; independent third‑party tests under real‑world conditions have not been published and would be needed to validate those percentages and latency numbers universally. Until independent audits or peer reviews are available, these performance figures must be considered provisional vendor claims.

When vendors publish internal benchmarks, they’re useful indicators of readiness, but the real test is deployment at scale across languages, noise profiles, and connectivity regimes.

---

Risk matrix — what to watch for as this technology scales​

• Safety & distraction (high risk): Richer, longer conversations can increase driver cognitive load. Design must favor short, actionable replies while deferring long interactions until parked.
• Privacy & data governance (high risk): Continuous audio capture, location history, and inferred driver profiles create privacy exposure. Clear retention, opt‑in controls, and on‑device preprocessing are essential mitigations.
• Hallucinations & mistrust (medium risk): If the assistant fabricates POI facts or charger availability, user trust collapses. Grounding answers in authoritative maps and live feeds (and surfacing provenance) reduces risk.
• Vendor lock‑in & platform dependencies (medium risk): OEMs must weigh reliance on a single cloud provider versus hybrid or multi‑cloud strategies to avoid future commercial risk. TomTom’s SDK model offers a partial escape hatch by packaging the UX and retrieval layer under OEM control.
• Regulatory pressure (emerging risk): Voice assistants that act on behalf of drivers or that integrate with workplace systems will attract privacy, safety and consumer regulation. OEMs need compliance roadmaps.

---

Practical guidance for OEMs, fleets, and IT leaders​

• Start with constrained features: prioritize navigation, charging guidance, and basic vehicle control as the first wave. Keep business‑critical or safety‑critical actions local or strictly gated.
• Instrument telemetry, but with privacy: collect interaction telemetry for improvement while offering explicit opt‑ins, retention windows, and user‑visible controls. Use on‑device preprocessing to limit raw audio uploads.
• Validate with real drivers: vendor demos and lab tests are useful; large live pilots (diverse accents, languages, noise conditions, connectivity states) are mandatory before broad rollout. Mercedes’ three‑month beta approach is an instructive model for iterative learning.
• Design for graceful degradation: ensure the assistant degrades back to local voice commands and cached navigation during connectivity outages.
• Demand explainability and provenance: surface the data source in responses (e.g., “I found that charger on TomTom’s live network; availability updated 3 minutes ago”) to keep user trust high.

---

Conclusion — incremental revolution, not instant perfection​

TomTom’s collaboration with Microsoft marks an important, pragmatic moment in automotive AI: the industry is moving from simple command lists to conversational, map‑aware assistants that can synthesize route logic, live telematics and natural language reasoning. That shift unlocks convenience (better EV routing, hands‑free discovery, contextual assistance) but also raises the stakes on privacy, safety and trust.
The technical pattern is clear: combine the OEM’s domain data (maps, telematics, vehicle state) with cloud LLMs for reasoning, keep safety gates local, and iterate via OTA updates and pilot programs. TomTom’s Digital Cockpit and Microsoft’s Azure OpenAI Service offer a repeatable blueprint for OEMs that want to accelerate their in‑car AI roadmaps without reinventing large‑model infrastructure. That said, vendor claims about accuracy and latency should be validated independently. The first wave of these assistants will likely be a mixed bag: impressive when networks and contexts align, less reliable in fringe conditions. Success will hinge not on LLM novelty alone but on disciplined systems engineering — conservative gating, rigorous privacy controls, clear provenance, and real‑world testing at scale.
TomTom and Microsoft have made the car more conversational; the next phase is making those conversations reliably useful, safe, and trustworthy for millions of drivers.

---

Source: Mashable TomTom makes cars more chatty with the help of Microsoft AI