Microsoft says AI outperforms doctors at complex medical diagnoses
Tech giants are racing to develop superintelligence, which refers to an AI system that exceeds human intellectual abilities in every way – and they're promising to use it to upend healthcare systems around the world.
For the latest experiment, Microsoft tested an AI diagnostic system against 21 experienced physicians, using real-world case studies from 304 patients that were published in the New England Journal of Medicine, a leading medical journal.
The AI tool correctly diagnosed up to 85.5 per cent of cases – roughly four times more than the group of doctors from the United Kingdom and the United States, who had between five and 20 years of experience.
The model was also cheaper than human doctors, ordering fewer scans and tests to reach the correct diagnosis, the analysis found.
Microsoft said the findings indicate that AI models can reason through complex diagnostic problems that stump physicians, who specialise in their fields but are not experts in every aspect of medicine.
However, AI 'can blend both breadth and depth of expertise, demonstrating clinical reasoning capabilities that, across many aspects of clinical reasoning, exceed those of any individual physician,' Microsoft executives said in a press release.
'This kind of reasoning has the potential to reshape healthcare'.
Microsoft does not see AI replacing doctors anytime soon, saying the tools will instead help physicians automate some routine tasks, personalise patients' treatment, and speed up diagnoses.
How the model works
Microsoft's AI system made diagnoses by mimicking a doctor's process of collecting a patient's details, ordering tests, and eventually narrowing down a medical diagnosis.
A 'gatekeeper agent' had information from the patient case studies. It interacted with a 'diagnostic orchestrator' that asked questions and ordered tests, receiving results from the real-world workups.
The company tested the system with leading AI models, including GPT, Llama, Claude, Gemini, Grok, and DeepSeek.
OpenAI's o3 model, which is integrated into ChatGPT, correctly solved 85.5 per cent of the patient cases, compared to an average of 20 per cent among the group of 21 experienced doctors.
Limitations and next steps
The researchers published their findings online as a preprint article, meaning it has not yet been peer-reviewed.
Microsoft also acknowledged some key limitations, notably that the AI tool has only been tested for complicated health problems, not more common, everyday issues.
The panel of doctors also worked without access to their colleagues, textbooks, or other tools that they might typically use when making diagnoses.
'This was done to enable a fair comparison to raw human performance,' Microsoft said.
The company called for more real-world evidence on AI's potential in health clinics, and said it will 'rigorously test and validate these approaches' before making them more widely available.
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Euronews
6 hours ago
- Euronews
Microsoft says AI outperforms doctors at complex medical diagnoses
Microsoft said it is one step closer to 'medical superintelligence' after a new artificial intelligence (AI) tool beat doctors at diagnosing complex medical problems. Tech giants are racing to develop superintelligence, which refers to an AI system that exceeds human intellectual abilities in every way – and they're promising to use it to upend healthcare systems around the world. For the latest experiment, Microsoft tested an AI diagnostic system against 21 experienced physicians, using real-world case studies from 304 patients that were published in the New England Journal of Medicine, a leading medical journal. The AI tool correctly diagnosed up to 85.5 per cent of cases – roughly four times more than the group of doctors from the United Kingdom and the United States, who had between five and 20 years of experience. The model was also cheaper than human doctors, ordering fewer scans and tests to reach the correct diagnosis, the analysis found. Microsoft said the findings indicate that AI models can reason through complex diagnostic problems that stump physicians, who specialise in their fields but are not experts in every aspect of medicine. However, AI 'can blend both breadth and depth of expertise, demonstrating clinical reasoning capabilities that, across many aspects of clinical reasoning, exceed those of any individual physician,' Microsoft executives said in a press release. 'This kind of reasoning has the potential to reshape healthcare'. Microsoft does not see AI replacing doctors anytime soon, saying the tools will instead help physicians automate some routine tasks, personalise patients' treatment, and speed up diagnoses. How the model works Microsoft's AI system made diagnoses by mimicking a doctor's process of collecting a patient's details, ordering tests, and eventually narrowing down a medical diagnosis. A 'gatekeeper agent' had information from the patient case studies. It interacted with a 'diagnostic orchestrator' that asked questions and ordered tests, receiving results from the real-world workups. The company tested the system with leading AI models, including GPT, Llama, Claude, Gemini, Grok, and DeepSeek. OpenAI's o3 model, which is integrated into ChatGPT, correctly solved 85.5 per cent of the patient cases, compared to an average of 20 per cent among the group of 21 experienced doctors. Limitations and next steps The researchers published their findings online as a preprint article, meaning it has not yet been peer-reviewed. Microsoft also acknowledged some key limitations, notably that the AI tool has only been tested for complicated health problems, not more common, everyday issues. The panel of doctors also worked without access to their colleagues, textbooks, or other tools that they might typically use when making diagnoses. 'This was done to enable a fair comparison to raw human performance,' Microsoft said. The company called for more real-world evidence on AI's potential in health clinics, and said it will 'rigorously test and validate these approaches' before making them more widely available.
Euronews
7 hours ago
- Euronews
Meta announces new ‘superintelligence' unit to work on AI
Meta announced a new division to create an artificial intelligence (AI) system that is as intelligent as humans. Mark Zuckerberg, the CEO of Meta, wrote in an internal memo on Monday that Meta Superintelligence Labs, MSL, will be a dedicated part of the company focused on artificial general intelligence (AGI). The memo said that MSL would house several research teams that are working on foundational AI models such as Llama software. It will be lead by Alexandr Wang, the former CEO of Scale AI, an American data annotation company that builds training data sets for AI companies. Meta bought a 49 percent share in Scale AI last month for $14.3 billion (€12.4 billion), with the company saying at the time that it would employ Wang and a few other members of his team. Meta's hiring spree Zuckerberg's memo included the names of several more people joining Meta from rivals OpenAI, Google and Anthropic. Former OpenAI leaders Hongyu Ren, Jiahui Yu, Shengjia Zhao and researchers Trapit Bansal, Shuchao Bi, Huiwen Chang and Ji Lin are now being brought into Meta, Zuckerberg's internal memo added. Euronews Next contacted OpenAI to get their reaction to the news but did not receive an immediate reply. OpenAI CEO Sam Altman recently accused Meta of offering his employees $100 million (€87 million) signing bonuses if they joined his company. Altman told his brother Jack in his podcast at the time that he respects Meta's 'aggression' in competing with the company but that promising 'a ton of upfront guarnateed comp(ensation) … [won't] set up a great culture'. 'I think people look at the two paths [OpenAI vs Meta] and they say OpenAI's got a really good shot, a much better shot on actually delivering on super intelligence and may eventually be the more valuable company,' he said.
France 24
2 days ago
- France 24
Meta spending big on AI talent but will it pay off?
OpenAI boss Sam Altman recently lamented that Meta has offered $100 million bonuses to engineers who jump to Zuckerberg's ship, where hefty salaries await. A few OpenAI employees have reportedly taken Meta up on the offer, joining Scale AI founder and former chief executive Alexandr Wang at the Menlo Park-based tech titan. Meta paid more than $14 billion for a 49 percent stake in Scale AI in mid-June, bringing Wang on board as part of the deal. Scale AI labels data to better train AI models for businesses, governments and labs. "Meta has finalized our strategic partnership and investment in Scale AI," a Meta spokesperson told AFP. "As part of this, we will deepen the work we do together producing data for AI models and Alexandr Wang will join Meta to work on our superintelligence efforts." US media outlets have reported that Meta's recruitment effort has also targeted OpenAI co-founder Ilya Sutskever; Google rival Perplexity AI, and hot AI video startup Runway. Meta chief Zuckerberg is reported to have sounded the charge himself due to worries Meta is lagging rivals in the generative AI race. The latest version of Meta AI model Llama finished behind its heavyweight rivals in code writing rankings at an LM Arena platform that lets users evaluate the technology. Meta is integrating recruits into a new team dedicated to developing "superintelligence," or AI that outperforms people when it comes to thinking and understanding. 'Mercenary' Tech blogger Zvi Moshowitz felt Zuckerberg had to do something about the situation, expecting Meta to succeed in attracting hot talent but questioning how well it will pay off. "There are some extreme downsides to going pure mercenary... and being a company with products no one wants to work on," Moshowitz told AFP. "I don't expect it to work, but I suppose Llama will suck less." While Meta's share price is nearing a new high with the overall value of the company approaching $2 trillion, some investors have started to worry. Institutional investors are concerned about how well Meta is managing its cash flow and reserves, according to Baird strategist Ted Mortonson. "Right now, there are no checks and balances" with Zuckerberg free to do as he wishes running Meta, Mortonson noted. The potential for Meta to cash in by using AI to rev its lucrative online advertising machine has strong appeal but "people have a real big concern about spending," said Mortonson. Meta executives have laid out a vision of using AI to streamline the ad process from easy creation to smarter targeting, bypassing creative agencies and providing a turnkey solution to brands. AI talent hires are a long-term investment unlikely to impact Meta's profitability in the immediate future, according to CFRA analyst Angelo Zino. "But still, you need those people on board now and to invest aggressively to be ready for that phase" of generative AI, Zino said. According to The New York Times, Zuckerberg is considering shifting away from Meta's Llama, perhaps even using competing AI models instead. Penn State University professor Mehmet Canayaz sees potential for Meta to succeed with AI agents tailored to specific tasks at its platform, not requiring the best large language model. "Even firms without the most advanced LLMs, like Meta, can succeed as long as their models perform well within their specific market segment," Canayaz said.
