Study finds AI beats human peer review
Archived — this story has rotated out of today’s deck. It is kept here in full.
The gist
Microsoft's MAI-DxO reached the correct diagnosis in 83.2% of 304 challenging cases, versus 36.1% for generalist doctors.
The Science-published result tests AI on historical text cases, not live patients.
Background
MAI-DxO (Microsoft AI Diagnostic Orchestrator) is Microsoft's diagnostic AI system, which coordinates multiple AI agents acting like a panel of doctors — questioning a patient, ordering tests and debating a diagnosis. The work had circulated for a year as a non-peer-reviewed preprint on arXiv; publication in Science gives the results independent scrutiny. Microsoft AI CEO Mustafa Suleyman framed the result as a step toward "medical superintelligence," and the company joins Google and OpenAI in claiming its AI can match or outperform doctors.
How it unfolded
- Jun 2025An earlier preprint of the work was published, reporting improvements using models from Google, Anthropic, Meta and other companies.
- Apr 2026GPT-5.5, a newer model, was released.
- Oct 22, 2026Microsoft researchers reported that MAI-DxO identified difficult conditions more than twice as often as generalist doctors, in a study published in Science titled "Sequential Diagnosis with Language Models."
- Oct 23, 2026Coverage reports the system reached the correct diagnosis in 83.2% of 304 challenging New England Journal of Medicine cases versus 36.1% for generalist doctors; on 325 emergency department admissions it scored 80.3% versus 46.8%. Microsoft released the SDBench code, with clinical datasets requiring separate authorization.
- recentlyTesting with Beth Israel Deaconess Medical Center in Boston is expected within months; Microsoft has not set a commercialization timeline or announced a start date.
Who’s saying what
- Official
- Harsha Nori, head of AI at Microsoft AI Health, said the team is measuring not just accuracy but how "judiciously and carefully" the system goes through a diagnostic work-up.
- Official
- Microsoft AI CEO Mustafa Suleyman called the result a step toward "medical superintelligence," and vice president Dominic King said the biggest challenge in healthcare is the growing mismatch between demand and supply of medical expertise.
- Caution
- The study authors acknowledged the human baseline was generalist physicians and said they "would expect the performance gap to narrow" against expert specialists.
- Caution
- Coverage notes the tests used historical text-based cases without live patients, physical examinations, images or pathology specimens, and the system has no regulatory approval and is not intended for clinical use.
Still unverified
The study does not prove MAI-DxO is better than doctors in real clinical settings, since it used historical text-based cases and compared the AI with individual general practitioners rather than teams of specialists; cost savings were estimates based on the cases rather than actual medical bills.