A glowing blue diagram shows an AI-powered data workflow connecting cloud storage, documents, search, and a chatbot.
Azure Cosmos DB's Semantic Reranker is now generally available. It's a second-stage ranking step for search, RAG and agent workloads. The Azure Updates entry marks it "Launched", which Microsoft defines as fully released and production-ready for all Azure customers. The Azure Cosmos DB team announced it at Microsoft Ignite 2026. Semantic Reranker uses an AI model to score and reorder the results of any query based on how well each document matches the user's intent.

The Azure Updates page itself shows no body text when fetched. The details below come from Microsoft's Ignite blog post and the Learn documentation. One of them is a significant change from the preview-era guidance.

A glowing blue diagram shows an AI-powered data workflow connecting cloud storage, documents, search, and a chatbot. What the feature does, and what it doesn't​

Semantic Reranker does not fetch anything new. You retrieve candidates from a container first. That can be vector, full-text, hybrid or any other query. The reranker then scores those candidates against a query or context and puts them in a new order. According to the Learn documentation, it doesn't retrieve more documents from your container.

Microsoft's preview post explains why this matters. Vector and hybrid search are good at finding candidates, but the top slot isn't always the best answer. The example it gave was "best surf spots in Hawaii". A curated surf guide should beat a general Hawaii travel article, even when the two score alike on raw similarity.

Microsoft says the service uses the Microsoft AI Semantic Ranker model. That model also powers Azure AI Search's Semantic Ranker.

What comes back​

Each result can include:

  • Document index: the zero-based position in your submitted candidate list. Use it to map a score back to your original record.
  • Relevance score: a value from 0 to 1, where higher means more relevant.
  • Original document: returned only if you ask for it.
  • Sentence scores: optional, covering up to the two highest-scoring sentences per returned document, not every sentence.
  • Latency and model details: optional timing information and the model name and version reported by the service.

The index matters in practice. The documentation's examples request only a subset of results, so the response may not contain every candidate you sent. Keep your own mapping from index to original record instead of assuming the order is unchanged.

The setup has changed since preview​

Microsoft's June 2026 preview post described reranking as a call built into the Cosmos DB SDKs, made on the container you already use. The current Learn documentation describes a different pattern:

  1. Use the Azure Cosmos DB SDK to retrieve candidates.
  2. Send those candidates and your query to a separate inference endpoint. You do this with the Azure Data AI SDK's InferenceClient.

The Learn page states plainly that reranking isn't a method on the Cosmos DB container in its examples. If you're working from preview-era code or blog posts, treat their setup steps as historical. The current documentation is the reference.

Setup in practice​

According to the Learn documentation:

  • Portal route: open your Cosmos DB account and find Semantic Reranker in the resource menu. Turning it on creates a companion inference account with the same name as the Cosmos DB account. If the inference resource provider isn't registered, an administrator needs to register it first.
  • Command-line route: this suits automation and explicit permission scope. The inference account is a separate Microsoft.InferenceService/inferenceAccounts resource. Its endpoint is not your Cosmos DB document endpoint.
  • Regions: the article's example uses uksouth. It lists eastus2, westus3 and uksouth as the available inference regions.
  • Authentication: the examples use DefaultAzureCredential with Microsoft Entra ID, not account keys.
  • Sample prerequisites: these include .NET 8 or Python 3.10 or later, and PowerShell 7.4 or later for the setup commands. The sample also uses a Movies container with full-text search configured on /text. Those are requirements of the walkthrough, not of the product.

Two separate permission checks​

Runtime access involves two permission domains, and neither one covers the other:

  • Cosmos DB data-plane permissions, so the identity can query source items.
  • The Semantic Reranker User role on the inference account, so it can call the reranker.

Microsoft says inference access doesn't grant permission to query Cosmos DB. It also says Cosmos DB read access doesn't grant inference access. Role assignments can take a few minutes to propagate. For deployed apps, grant these to the app's managed identity rather than a person.

If a call fails with an authorization error, check the endpoint, the identity and both role assignments. Microsoft advises against wrapping the call in more application-level retries.

The portal also offers a bulk "Grant permissions" action. Use it only if every identity it lists should be allowed to rerank. For a single user or managed identity, use the scoped commands instead.

Limits, cost and trade-offs​

  • Batch size: the preview post gave a limit of 50 documents per rerank call. It also gave 2,048 tokens as the maximum for a context-document pair. The latest Learn material the article draws on repeats both figures.
  • JSON candidates: if you send JSON, set documentType to json and supply target paths. You can specify at most 100 target paths per request. For plain text, use documentType="text" and omit paths.
  • Price: $1 USD per 1,000 rerank calls. Microsoft notes that regional and cloud pricing might vary, so check the Cosmos DB pricing page for your deployment.
  • Latency: reranking adds a network call plus model inference time. Microsoft's advice is to measure relevance before and after reranking. Then decide whether the gain is worth the extra latency for your workload.
  • Retries: the inference client has its own retry policy, separate from the Cosmos DB client. The examples configure two retries after the first attempt for eligible failures such as HTTP 429.
  • Support: for support cases, capture the X-Correlation-ID response header when it's present.

One caveat on diagnostics: the service's phase timings aren't client end-to-end elapsed time. Reranker token usage is also not a Cosmos DB request-unit charge.

How it got here​

The feature was in private preview around Ignite 2025. That preview used Azure AI Search's reranking model, with .NET (preview) and Python SDKs. Public preview followed at Build in June 2026, and general availability came at Ignite 2026.

The same Ignite 2026 announcement also made full-text BM25 score projection generally available. That feature is separate from the reranker. It lets applications see the relevance scores that full-text search produces.

Practical takeaways​

  • It isn't a free quality upgrade. Microsoft itself says it won't help every workload equally. Build a small evaluation set of real queries and compare results with and without reranking.
  • Plan for the extra hop. A separate inference resource means extra provisioning, extra RBAC and one more dependency in the request path.
  • Keep candidate sets sensible. With a 50-document ceiling per call, retrieve a focused set rather than a huge one.
  • Check the SDK matrix before committing. Preview material named .NET, Python and Java. The current standalone walkthrough centers on .NET and Python. I'd confirm language support for your stack before planning around it.

For teams already running RAG or agent workloads on Cosmos DB for NoSQL, the GA label makes reranking a production option. Treat it as a measured addition, not a default. Check the current Learn page for setup, because the preview-era instructions no longer match.

 

References

  1. (Launched) Generally Available: Semantic Reranker in Azure Cosmos DB Azure Updates 2026-11-17T23:07:42Z
  2. Announcing the Public Preview of Semantic Reranker in Azure Cosmos DB for NoSQL - Azure Cosmos DB Blog devblogs.microsoft.com
  3. Semantic reranker - Azure Cosmos DB | Microsoft Learn learn.microsoft.com