What Microsoft actually built
The system is the MAI Diagnostic Orchestrator, or MAI-DxO. It is a model-agnostic orchestrator that simulates a panel of physicians, proposes likely differential diagnoses and strategically selects high-value, cost-effective tests. The Reuters report carried by Deccan Herald says most medical AI models take an all-at-once approach. They look at a patient's symptoms, lab results and imaging together and produce a diagnosis.
The orchestrator works differently:
- It first decides what to ask about the patient's history and what matters in the physical exam.
- It then picks tests based on what it has learned, reviews the results, and orders more tests if needed.
- Throughout, it keeps updating its working hypotheses and plans, the research leaders said in an interview.
The orchestrator is not a standalone Microsoft model. In the Science paper it is a model-agnostic system that simulates a panel of clinical experts with complementary roles. It applies cost-conscious test stewardship at each step and aggregates diagnoses across independent runs. Medical Economics reports that the main results ran on GPT-5. Strictly, the headline figures belong to an orchestration layer plus a commercial language model. Medical Economics refers to the system as DxO.
The 2025 preprint makes a related design argument. Because the system is model-agnostic, it reduces reliance on any single model and avoids the need to "version chase" each new model release. The preprint also reported that the techniques boosted the accuracy of off-the-shelf models from a variety of providers by an average of 11 percentage points.
Why the benchmark matters
Dr. Dominic King, corporate vice president of the health unit at Microsoft AI, made the case against the usual tests. AI models can reach a "100% pass rate" on multiple-choice assessments, he said, but those are not real-world. His line: "No patient ever came to see me with a nice summary."
The Sequential Diagnosis Benchmark (SDBench) is built to avoid that problem. It converts 304 New England Journal of Medicine clinicopathological conference cases and 325 emergency department-to-admission encounters from the MIMIC-IV-ED dataset into interactive, stepwise diagnostic encounters. Medical Economics adds how the information flow works. Each case opens with a brief clinical vignette. The diagnostician, whether physician or AI, can then ask history or examination questions, order tests, or commit to a final diagnosis. A separate AI gatekeeper releases case details only when they are requested. Each test is assigned an estimated cost based on hospital chargemaster pricing.
The benchmark measures more than the final answer. It also shows whether a system asks sensible questions, avoids pointless tests and knows when to stop.
The numbers, with their conditions
Medical Economics reports these figures:
| Dataset | DxO accuracy | Physician accuracy |
|---|---|---|
| NEJM clinicopathological cases | 83.2% | 36.1% (with search and reference tools) |
| Emergency department admissions (MIMIC-IV-ED) | 80.3% | 46.8% |
Physicians without reference tools averaged 22.3% on the NEJM cases. The best individual physician scored 51.8% on those cases and 62.5% on the emergency department cases.
"More than twice as accurate" holds for the NEJM cases. On the emergency department set, the gap is large but under 2x. The headline is also shorthand for a comparison with individual generalists.
The study also found the AI worked differently from the physicians:
- Physicians asked on average 5.6 questions before ordering their first test.
- DxO asked 25.5 questions first.
- When DxO was wrong, it usually named a closely related diagnosis at the wrong level of specificity. Physician errors tended to land further from the answer.
The cost claim needs care
Reuters says diagnostic processes cost less when the orchestrator designs the work-ups than when human doctors do. Medical Economics qualifies that. Without budget limits, DxO's average testing cost per case was higher than the physicians'. The figures were $5,431 against $4,725 on the NEJM cases, and $4,979 against $3,400 on the emergency department cases.
The savings come when budget limits are imposed. On the NEJM cases, DxO reached 80.9% accuracy at $2,241 per case and 79.6% at $1,303. On the emergency department cases, budget-limited versions scored 73.2% at $2,634 and 69.5% at $2,037. That is still above the physician average, but it gives up some accuracy. The lower costs are also estimates built from standardized U.S. pricing, not savings measured in hospitals.
What the study does not show
The authors list several limits, as reported by Medical Economics:
- Comparison group: The comparison was with individual generalist physicians, not specialists or multidisciplinary teams. The authors expect the gap to narrow against specialists matched to each case.
- Case selection: The NEJM cases are chosen for teaching value, and every case had a pathological diagnosis. Healthy and benign visits were excluded, so the case mix does not reflect routine practice.
- Text only: The benchmark did not include imaging, waveforms or pathology slides. It also left out the uncertainty of real encounters, such as patients who forget or omit information.
- Narrow scope: Only the diagnostic phase was measured, not treatment decisions, patient safety or outcomes.
- Cost model: Cost estimates did not account for invasiveness, patient discomfort or time to results.
The preprint says much the same. These results do not yet establish the clinical efficacy of MAI-DxO in real-world decision support.
There is also a disclosure to weigh. All authors are Microsoft employees and hold equity in the company. Medical Economics adds that eight authors are named inventors on a related Microsoft patent application, and the study received no outside funding. That doesn't invalidate the work, which went through Science. But the benchmark designers, the system builders and the beneficiaries are the same group, so independent replication is the obvious next test.
What comes next
Reuters reports that, in collaboration with Beth Israel Deaconess Medical Center in Boston, the researchers will next test the tool's ability to triage patients in emergency department waiting rooms. That is a different task from working up a complex case, so these results won't carry over automatically. Medical Economics says the authors plan workflows where clinicians and AI work together. They also see uses in medical education and continuing professional development.
What IT and enterprise readers should take from it
- The architecture is the transferable idea. A role-based, cost-aware orchestration layer that sits on top of interchangeable models is a pattern enterprise AI teams will recognize. The architecture, not the benchmark score, is the part to carry over.
- Evaluation design shapes the result. Static tests flatter models. Interactive tests that track cost and stopping decisions are harder, and more informative.
- Don't treat this as a product. Nothing in the reporting describes a shipping clinical product, a regulatory clearance or an integration into Microsoft's health offerings. Health IT teams should read it as research.
The result is a strong sign that structured, step-by-step AI reasoning can beat one-shot answers on a well-built simulation. Whether it helps real patients is a question only prospective testing can answer.
References
- Microsoft develops AI tool that can diagnose 'sequentially' like a doctor - Deccan Herald Deccan Herald · 2026-10-23T01:23:44+00:00
- Microsoft Diagnostic AI Outperforms Generalist Physicians in Science Study unite.ai
- (Science Through Covers) Microsoft AI diagnostic tool outperforms doctors in accuracy - DongA Science DongA Science · 2026-10-24T23:00:00+00:00