A recent study has revealed that ChatGPT outperformed human doctors in diagnosing various diseases and medical conditions. Published last month, the research suggests that artificial intelligence (AI) chatbots may excel in analyzing patient histories and conditions, thereby delivering more accurate diagnoses. Although the study set out to explore the possible enhancements that AI chatbots could offer to medical professionals, the results showed that OpenAI’s GPT-4-powered chatbot significantly outperformed doctors when operating independently.
ChatGPT Outperforms Doctors in Diagnosing Diseases
The study, featured in the JAMA Network Open journal, was carried out at the Beth Israel Deaconess Medical Center in Boston by a team of researchers exploring the efficacy of AI in improving diagnostic accuracy beyond traditional methodologies.
According to a report by the New York Times report, the study involved 50 doctors, including both residents and attending physicians, who were recruited from various large hospital systems across the United States. Participants were presented with six patient case histories and were asked to propose diagnoses along with explanations for their reasoning. Additionally, their final diagnoses were assessed for accuracy.
Medical experts were enlisted as graders to evaluate the performance of the participants. While the graders were aware of the responses, they did not know whether the answers originated from a doctor utilizing AI assistance, a doctor working independently, or solely from ChatGPT.
To ensure the authenticity of the case histories, the researchers selected real patient cases that had been utilized in academic circles for decades but had never been published. This approach is crucial, as ChatGPT’s training does not include data that has not been made public.
The study’s findings were unexpected. Physicians who did not rely on AI for their diagnoses achieved an average accuracy of 74 percent, while those who incorporated the chatbot into their assessments scored slightly higher at 76 percent. In stark contrast, the analysis performed solely by ChatGPT yielded an impressive average accuracy of 90 percent.
Despite various factors potentially influencing the study’s outcomes, such as the doctors’ experience levels and inherent biases toward specific diagnoses, the researchers argue that the findings underscore the significant potential of AI technology within medical settings.
