The New York Eye and Ear Infirmary of Mount Sinai (NYEE) demonstrated how GPT-4 can meet or exceed human ophthalmologists in diagnosing and treating eye diseases. 

The findings, detailed in a study published in JAMA Ophthalmology, discuss how AI can assist eye specialists of their decision-making processes.

The research team at Mount Sinai involved 12 attending specialists and three senior trainees from the Department of Ophthalmology on the Icahn School of Medicine. 

They compared the responses of each the AI and human specialists to a set of questions and patient cases related to glaucoma and retina disorders. GPT-4’s responses were evaluated and were found to match or surpass those of the human specialists, particularly in glaucoma. 

Here’s a bit of more detail about the way it worked:

  • Study setup: The research was conducted with a team of 15 ophthalmology professionals on the Mount Sinai Department of Ophthalmology, consisting of attending physicians and senior trainees specialized in glaucoma and retina diseases.
  • Data compilation: For the aim of evaluation, the team used a well-rounded set of 20 ophthalmology questions (split evenly between glaucoma and retina topics) and 20 patient cases that were anonymized to take care of privacy. These were chosen to reflect a variety of cases taken from Mount Sinai clinics.
  • Use of AI: OpenAI’s GPT-4 was prompted to reply the questions and analyze the patient cases. The AI aimed to reply like a practicing ophthalmologist, using clinical shorthand where appropriate to mirror the concise style typical of clinical notes.
  • Evaluation: The study employed a rating system to evaluate each the accuracy and completeness of the responses from GPT-4 and the human specialists. This allowed for a direct comparison of the AI’s performance against that of trained professionals.
  • Glaucoma results: For glaucoma, GPT-4 provided highly accurate and more comprehensive responses than the human specialists. This indicates GPT-4’s strong capability in understanding and advising on glaucoma cases, potentially offering priceless support to ophthalmologists on this subspecialty.
  • Retina results: For retina-related inquiries and patient scenarios, GPT-4 matched human specialists, demonstrating its ability to diagnose and recommend treatments for retina conditions accurately. Further, GPT-4 often provided more elaborate responses, suggesting an intensive evaluation and understanding of the cases, which could possibly be especially useful in handling more complex or nuanced patient situations.

Dr. Andy Huang, an ophthalmology resident at NYEE and the study’s lead creator, shared his insights, stating, “The performance of GPT-4 in our study was quite eye-opening. We recognized the large potential of this AI system from the moment we began testing it and were fascinated to watch that GPT-4 couldn’t only assist but, in some cases, match or exceed, the expertise of seasoned ophthalmic specialists.”

Dr. Louis R. Pasquale, Deputy Chair for Ophthalmology Research and a senior creator of the study, was also impressed by the outcomes, stating, “AI was particularly surprising in its proficiency in handling each glaucoma and retina patient cases, matching the accuracy and completeness of diagnoses and treatment suggestions made by human doctors in a clinical note format.”

“Just because the AI application Grammarly can teach us the right way to be higher writers, GPT-4 may give us priceless guidance on the right way to be higher clinicians, especially by way of how we document findings of patient exams.”

Dr. Huang envisions a future for AI in ophthalmology, noting its potential to support eye specialists by providing diagnostic assistance and reducing their workload, particularly in complex cases. 

AI has taken well to applications in eye health, comparable to in 2018 to 2020 when DeepMind used machine learning to accurately analyze high-resolution three-dimensional optical coherence tomography (OCT) to diagnose quite a few conditions. 

In 2023, researchers developed an AI model to accurately detect early signs of Parkinson’s disease from eye scans.

Another was trained to detect Parkinson’s, heart disease, and other diseases, again from eye scans, and one other to detect certain diseases in infants, again from retinal scans.

The post GPT-4 meets or exceed human eye specialists at diagnostic questions appeared first on DailyAI.

This article was originally published at