What Cohere announced
Cohere announced the beta on October 22, 2026. It describes Model Vault as a dedicated, fully managed model inference platform. The confidential option extends that service to financial information, clinical records, privileged documents, proprietary source code and intellectual property.
The roles are split three ways. Microsoft Azure supplies the underlying confidential computing infrastructure. NVIDIA Confidential Computing carries that protection onto the H100 GPUs. Cohere supplies the managed serving, the encrypted request path, remote attestation and customer-side verification.
Cohere says it is also working toward NVIDIA Blackwell support, including B200 GPUs, on the way to general availability. That is a roadmap item, not a feature of the current beta.
How the protection works
Encryption normally covers data at rest and data in transit. Cohere's pitch is that this design also covers data in use, while the model is running inference. Cohere describes the pieces this way:
- Confidential VM. The operating system, drivers, inference server and their memory run inside a hardware-isolated confidential virtual machine. Cohere says operators have no interactive path into the workload.
- GPU protection. NVIDIA Confidential Computing protects GPU memory and encrypts CPU-to-GPU traffic. This matters because much of the model computation happens on the accelerator. A protected CPU alone would leave a gap.
- Encrypted payloads. Requests use Oblivious HTTP (OHTTP). The application payload is encrypted to a key tied to the attested environment. It can cross intermediary infrastructure as ciphertext, and only the confidential environment can decrypt it. Responses are protected on the way back.
- Attestation before release. Customer-side software checks signed evidence about the hardware, firmware and software configuration against an approved policy. If the check fails, the client refuses to send the request.
Cohere's documentation adds detail on what attestation covers:
- It can show the hardware is a genuine trusted execution environment.
- It can show the environment is securely configured, with debug disabled and GPU confidential-computing mode on.
- It can show the expected measured components are running, such as firmware, kernel, inference workload, container policy and GPU firmware.
- Any change to the vault image, model or configuration changes the measurements. An unexpected build should therefore fail verification instead of quietly serving traffic.
- Each inference response includes an attestation certificate that applications can keep as a record.
For Azure admins, the FAQ says the confidential VM is AMD SEV-SNP on Azure, or Intel TDX. It is paired with a confidential-computing-capable NVIDIA GPU such as the H100. Cohere says the exact options depend on model, capacity and region, so confirm availability with the vendor.
The practical bit: integration
Cohere says the API stays the same. Traffic goes through the Cohere OHTTP proxy, either standalone or as a Python wrapper. The proxy verifies attestation, encrypts requests and decrypts responses. Cohere says application code can stay unchanged. Because the proxy runs on the customer side, it is the piece that refuses to send data to an environment that fails verification.
The fine print on "public beta"
Cohere's launch post says customers can start deploying encrypted workloads today. Its documentation is more cautious. It says the encrypted option is in beta and that interested customers should contact Cohere to join. Its pricing page says the option is priced separately from Standard Vault. It also says the beta is free for design partners, with general-availability pricing to come.
Coverage from earlier in the cycle points the same way. One report describes beta access as limited and gated through Cohere sales. It also says encrypted deployments support only a subset of the models in standard Model Vault. Cohere's own FAQ says the supported-model list is expanding. Treat "public beta" as an invitation to talk to Cohere, not a self-service signup, until you've confirmed otherwise.
Earlier reporting also suggests this didn't start on October 22. VentureBeat reported the capability went live on September 16, as an extension of the existing single-tenant Model Vault. The October 22 announcement widens that earlier limited beta.
What it doesn't do
Cohere is fairly candid about the limits. This matters more than the marketing line "the operator technically can't read it."
- Plaintext still exists. The model processes plaintext inside the protected environment. This is not homomorphic encryption.
- Side channels and physical attacks. Timing, cache and microarchitectural leaks, and attacks needing deep physical access, are listed as residual risks.
- Metadata is visible. Request timing, sizes, traffic volume and routing information stay outside the encrypted payload. Cohere says the API credential and model identifier also stay outside it.
- Availability. An operator who can't read a workload can still stop, delay or restart it.
- Application security. Prompt injection, insecure tool use and authorization bugs are not fixed by running inside a trusted execution environment.
- Compliance. Cohere says this is not a data-residency mechanism. It does not replace contracts, certifications or security reviews.
- Trusted base. The CPU and GPU hardware, firmware, reference measurements and attestation infrastructure all remain trusted. Cohere's own summary is that trust gets "smaller, explicit, measurable, and verifiable" rather than eliminated.
One caution on the claims. The statements that Cohere, the cloud provider and hypervisor admins sit outside the trust boundary are Cohere's descriptions of its design. The sources I reviewed don't include an independent audit. Attestation also only helps if you validate it against a sensible policy.
Why Azure and security teams should care
Single-tenant hosting answers "who shares my hardware?" Confidential computing answers a different question: can the operator or cloud staff see plaintext while it's processed? Cohere itself frames the encrypted option as adding protection in use to a Standard Vault that already protects data at rest and in transit.
Cohere names legal due diligence, fraud investigations, patent material and proprietary code search as the target use cases. Microsoft's Azure confidential computing product lead is quoted as saying enterprises need protection "by more than contractual promises." That is a vendor's view, but it matches where regulated buyers are heading.
A checklist before you pilot
- Confirm beta eligibility and whether you'd be a design partner.
- Ask which models, regions and CPU technology (SEV-SNP or TDX) your deployment would use.
- Define your attestation policy. Know who maintains the approved measurements.
- Decide what routing and timing metadata your threat model can tolerate.
- Ask when the confidential stack will be open-sourced. Cohere says it intends to open-source it and builds on Kata Containers and Confidential Containers. That is a stated plan, not a shipped fact.
- Keep your own application security and compliance reviews. The hardware doesn't replace them.
Confidential inference on Azure is a meaningful step for teams that have kept their most sensitive data away from managed AI. It's still a beta with gated access, and the strongest honest claim is a narrower trust boundary you can check yourself, not a guarantee that no one can ever see your data.
References
- Cohere opens public beta of Confidential Computing in Model Vault Foro3D · 2026-10-24T13:30:22+00:00
- Inside Cohere Model Vault: Building a verifiable confidential AI inference cohere.com
- Confidential Computing on Cohere Model Vault cohere.com