On October 22, Microsoft announced that its AI medical diagnosis system, Diagnostic Orchestrator, outperformed human doctors in tests involving complex cases, with the findings published in the international academic journal Science. Mustafa Suleyman, CEO of Microsoft AI, stated that the company is progressing toward the goal of superintelligence in healthcare, expecting that within a few years, AI systems will perform at a level indistinguishable from human experts in the medical field.
Microsoft compared its proprietary AI diagnostic system with human doctors using a series of medical cases and emergency department data, establishing an evaluation benchmark known as SDBench. Unlike traditional tests that require answering a single medical multiple-choice question, this benchmark requires participants to analyze the condition step-by-step, ask questions, select examination items, and ultimately provide a diagnosis, thereby more realistically simulating the diagnostic process of clinical doctors. The core idea of Diagnostic Orchestrator is to enable multiple AI agents to work collaboratively, simulating a consultation team composed of doctors from different specialties.
The system determines the next questions to ask and examination items based on the patient's symptoms and medical history, continuously adjusting its judgments based on newly acquired information until a final diagnosis is reached.
Microsoft's AI Diagnosis System Performs Exceptionally in Tests
In tests based on hundreds of case reports published in the New England Journal of Medicine, the system achieved a diagnostic accuracy of approximately 83%. In comparison, general practitioners with access to reference materials had an accuracy rate of about 36%, while those without access had an accuracy rate of around 22%. In another set of cases from the emergency patient database of Beth Israel Deaconess Medical Center in Boston, the AI system's accuracy reached 80%, while the average accuracy of participating human doctors was about 47%.
In addition to higher diagnostic accuracy, Microsoft also noted that the system often arrives at diagnostic results at a lower estimated cost during tests, partly because the system can select examination items based on existing information rather than indiscriminately conducting numerous tests. However, these cost results are based on a unified estimation method in the study and do not imply that the system has proven capable of directly reducing actual hospital operating expenses in real medical environments.
Microsoft Plans to Apply AI in Clinical Care
The system is currently powered by OpenAI's GPT-5, but Microsoft emphasizes that it is not just the foundational model that plays a crucial role; the orchestration mechanism responsible for organizing the diagnostic process, coordinating multiple agents, and validating reasoning results is equally important. The research team stated that even as the foundational model continues to upgrade, multi-agent collaboration may still yield additional benefits. The system was initially developed during the advanced model phase of GPT-4.1, and subsequent tests have shown that improvements in model capabilities do not necessarily diminish the value of the collaborative diagnostic framework.
Microsoft also acknowledged that the tests primarily focused on past cases and selected complex diagnostic scenarios, and it cannot yet be proven that the system would achieve the same performance in real clinical environments. Further research is needed to evaluate its reliability when faced with common symptoms and its performance under real-world conditions such as incomplete information and changing medical conditions. For the system to be truly implemented in clinical use, it must undergo rigorous clinical validation.
Dominic King, who oversees Microsoft's AI healthcare business, stated that diagnosis is just one part of the company's healthcare strategy. Microsoft also hopes that AI can assist in coordinating clinical care, developing treatment plans, and addressing the complex issues involved in patient communication and interpersonal interactions in healthcare services. Microsoft plans to collaborate with Beth Israel Deaconess Medical Center to conduct tests in a hospital environment, with related work potentially starting in the coming months, although the company has not announced a specific timeline, nor are there concrete dates or product release plans for commercialization.
Suleyman emphasized that AI does not mean that doctors will disappear from the healthcare system. The work of doctors involves not only diagnosis but also handling ambiguous information, building trust with patients and their families, and making comprehensive judgments based on specific situations. Microsoft hopes that AI can reduce the time doctors spend on repetitive and tedious tasks, allowing them to focus more on patient care.