0

Edward Ariel M. Tadea

Zamboanga City Medical Center, Philippines

Presentation Title:

The clinical diagnostic accuracy of zero-shot prompt vs. Structured prompt vs. Iterative prompt in chat generative pretrained transformer version 3.5 among admitted patients in the pediatric emergency room of Zamboanga City Medical Center

Abstract

Background: Artificial intelligence (AI) creates machines, and one of its branches is natural language processing (NLP), which allows computers to analyze human language. Chat generative pre-trained transformer (ChatGPT) is a form of NLP that uses prompt engineering to improve its performance. The comparison of three (3) types of manual prompting in pediatrics for diagnostic accuracy remains unexplored. 

Objective: To determine the clinical diagnostic accuracy of ChatGPT version 3.5 compared to the diagnoses made by ZCMC senior pediatric resident and pediatric consultants among admitted patients in the pediatric emergency room. 

Methods: This was a cross-sectional analytical study. The computed sample size for the study was 28. 

Results: A total of 95 respondents were analyzed in the study. Zero-shot prompt had the highest rate of exactly correct diagnoses, with 20% for admitting diagnoses and 16% for final diagnoses. Zero-shot prompt also exhibited the highest concordance rate, 20% for admitting diagnoses and 15% for final diagnoses. The concordance rate of zero-shot prompt was significantly higher compared to iterative prompt (20% vs. 7%, p= 0.11) in determining admitting diagnoses but not in determining the final diagnoses. 

Conclusion: ChatGPT version 3.5 may function only as an adjunct but not as a primary means of generating clinical diagnosis. Future studies should focus on the application of newer versions of ChatGPT and other types of AI models in the clinical practice.

Biography

Dr. Edward Ariel M. Tadea, MPH, is a dedicated pediatrician whose career is marked by excellence in clinical practice and a deep commitment to public health. He graduated with a Doctor of Medicine from the Ateneo de Zamboanga University School of Medicine in 2016 and was awarded the Community Commitment Award for his dedication and selfless commitment to empowering the people of his adopted community. In 2017, he completed his Master of Public Health at the same institution. From March 2020 to December 2023, Dr. Tadea pursued his Pediatric Residency at the Zamboanga City Medical Center, where he served as Chief Resident in his final year and received the Exemplary Chief Resident Award upon graduating in May 2024. His commitment to patient care and leadership has been a constant driving force in his work, providing compassionate care to children and their communities.