MS develops AI that diagnoses like a doctor, step by step
Microsoft researchers report that an AI tool, which diagnoses patients in a step-by-step manner akin to that of physicians, demonstrated an accuracy rate exceeding that of general practitioners by more than twofold when addressing complex medical cases, as detailed in a study released on Thursday in Science. The model identifies and coordinates the questions that should be posed to a patient, along with the examinations and tests that ought to be ordered, utilising a “orchestrator” that structures how AI will address a problem by leveraging a simulated panel of clinical experts working together to resolve diagnostic cases, as stated in the report.
Current medical AI models generally adopt a comprehensive approach, analysing a patient’s complete set of symptoms, laboratory results, and imaging studies to formulate a diagnosis. With the orchestrator coordinating the steps to reaching a diagnosis, the AI model initially determines which aspects of the patient’s history to enquire about and identifies what is pertinent in the physical examination. Based on its learning, it determines which tests to order, evaluates the results, and requests additional testing as necessary. Throughout this process, the research leaders indicated in an interview that they are continuously refining their working hypotheses and plans.
Tasked with diagnosing over 300 highly unusual and perplexing medical dilemmas, the model utilising the diagnostic orchestrator demonstrated more than double the accuracy in “working up” patients and determining diagnoses compared to general physicians relying on textbooks and the internet. The tool also surpassed human physicians when faced with scenarios derived from a significant database of emergency room admissions. Assessments that juxtapose these older systems with human physicians typically employ multiple-choice questions, according to the researchers. “AI has become very good at getting a 100% pass rate on those,” said Dr. Dominic King, the corporate vice president of the health unit at Microsoft AI, whose team is developing the new technology.
“But of course, that’s not real-world. No patient ever came to see me with a nice summary.” The researchers also found that diagnostic processes incur lower costs when the orchestrator designs the work-ups compared to when human doctors are involved. “We’re not just measuring the accuracy of the systems. We’re also measuring how judiciously and carefully it goes through a diagnostic work-up,” said Harsha Nori. In partnership with Beth Israel Deaconess Medical Center in Boston, the researchers will proceed to evaluate the tool’s efficacy in triaging patients within emergency department waiting rooms.
