Microsoft’s AI diagnosed 83% of hard medical cases correctly versus 36% for generalist doctors

Microsoft says its diagnostic AI, MAI-DxO, named the correct diagnosis in 83% of difficult simulated medical cases, while generalist physicians working on their own got 36% right. The company published the result in the journal Science on Oct. 22, 2026, and the comparison is Microsoft’s own: its researchers designed the benchmark, ran the system and scored the physicians.

Make AI work for you: The Neuron delivers the latest AI tools, tutorials and news straight to your inbox. Join 700,000+ readers and catch up in 3 minutes a day.

Get The Neuron →

We may earn a commission if you sign up.

Mustafa Suleyman, the Microsoft AI chief who has been pushing what he calls medical superintelligence, presented the figures as a step toward that goal, and described himself and his team as “true believers” in the approach. He said Microsoft expects healthcare models to get “an order of magnitude better in capability within the next 24 to 36 months,” a timeline that depends on far more than one benchmark.

MAI-DxO and its panel of virtual physicians

MAI-DxO is not a single chatbot. According to Microsoft’s announcement, the Microsoft AI Diagnostic Orchestrator works like a panel of virtual physicians: several AI agents propose diagnoses, choose tests, and challenge one another over multiple rounds, weighing the value of each test against its cost. TechCrunch’s report on the Science study adds that the system was tested with models from OpenAI, Google, Anthropic, Meta and DeepSeek, and did best with OpenAI’s GPT-5 and GPT-5.5.

The test bed is the Sequential Diagnosis Benchmark, known as SDBench. Microsoft’s post says the main comparison used 304 complex cases drawn from New England Journal of Medicine case proceedings, converted into step-by-step encounters in which the diagnostician starts with a short summary and must ask for each additional finding or test, paying a simulated cost every time. The score covers both accuracy and spending, and the cases are teaching exercises, not a random sample of patients walking into a clinic. A system that orders every test on the menu would score well on accuracy and badly on cost, which is why Microsoft reports the two together.

What the generalist doctors were allowed to use

The 36% figure comes with conditions attached. TechCrunch describes the comparison group as primary care doctors and general internists working individually with reference materials. Microsoft’s announcement gives the split: physicians without external resources reached 22% accuracy on the NEJM cases, and physicians allowed textbooks and the internet reached 36%. Against those baselines, the 83% for MAI-DxO is about 3.7 times the first number and 2.3 times the second.

The comparison also left out the people who would normally take such cases. Specialists and physician teams were not in the 36% group, and TechCrunch notes that cases this complex usually go to them. The number of physicians in the 2026 comparison was not stated in the coverage reviewed, so the figure cannot be read as a measure of the average doctor in the field.

An earlier Microsoft preprint, Sequential Diagnosis with Language Models, posted in June 2025 with Suleyman and Dominic King among its authors, used the same 304-case design. The Decoder’s coverage of that preprint reported 21 experienced physicians from the United States and the United Kingdom who scored 19.9% without outside resources, against 79.9% for MAI-DxO paired with OpenAI’s o3.

The gap between 83% and 36% is large, but it measures two different ways of working. The software is built to ask for tests in sequence and to be argued with by its own agents. The doctors, in the benchmark, were asked to behave like solo clinicians, which is a narrower job than hospital medicine, where colleagues, consults and follow-up visits are routine.

Where the gains come from, and where they shrink

The Science paper tests the orchestrator against several underlying models, and the results show how much depends on the model underneath. Per TechCrunch, GPT-5 alone scored 78.9% on complex cases at about $8,400 per case; adding MAI-DxO cut the cost by about $3,000 but raised accuracy by only about four points. With GPT-5.5, released in April 2026, accuracy on complex cases climbed from 72% to nearly 87% at slightly lower cost. For Gemini 2.5 Flash the lift was larger, from 46.4% to nearly 69%.

Emergency cases told a quieter story. On typical emergency department cases, Microsoft reports MAI-DxO at 80% against 47% for physicians, yet TechCrunch says the system produced little improvement in accuracy or cost for the models tested there. Microsoft’s own researchers say the study’s data point to the underlying AI models as the source of most of the improvement, with the orchestrator adding gains on top that shrink as models approach the theoretical maximum.

Beth Israel Deaconess and the step to real patients

Every number above comes from a simulation, not from patients. Dominic King, Microsoft AI’s vice president of health, framed the stakes around supply: “The biggest challenge in healthcare delivery at the moment is the growing mismatch between demand and supply of medical expertise.” He called the results “something very substantial and exciting.”

The real-world test is still ahead. Microsoft and Beth Israel Deaconess Medical Center in Boston plan to compare MAI-DxO’s recommendations with human doctors’ decisions in emergency and urgent care settings. A Mayo Clinic healthcare foundation model is planned for release in 2027, and Suleyman has said Microsoft wants clinical decision support products that rank number one for state-of-the-art performance within the next few years.

This article was produced with the assistance of AI and reviewed by Morning Overview editors prior to publication.

Make AI work for you: Get The Neuron →

We may earn a commission if you sign up.