
Edward Ariel M. Tadea
Zamboanga City Medical Center, PhilippinesPresentation Title:
The clinical diagnostic accuracy of zero-shot prompt vs. Structured prompt vs. Iterative prompt in chat generative pretrained transformer version 3.5 among admitted patients in the pediatric emergency room of Zamboanga City Medical Center
Abstract
Background: Artificial intelligence (AI) creates machines, and one of its branches is natural
language processing (NLP), which allows computers to analyze human language. Chat generative
pre-trained transformer (ChatGPT) is a form of NLP that uses prompt engineering to improve its
performance. The comparison of three (3) types of manual prompting in pediatrics for diagnostic
accuracy remains unexplored.
Objective: To determine the clinical diagnostic accuracy of ChatGPT version 3.5 compared to the
diagnoses made by ZCMC senior pediatric resident and pediatric consultants among admitted
patients in the pediatric emergency room.
Methods: This was a cross-sectional analytical study. The computed sample size for the study was
28.
Results: A total of 95 respondents were analyzed in the study. Zero-shot prompt had the highest
rate of exactly correct diagnoses, with 20% for admitting diagnoses and 16% for final diagnoses.
Zero-shot prompt also exhibited the highest concordance rate, 20% for admitting diagnoses and
15% for final diagnoses. The concordance rate of zero-shot prompt was significantly higher
compared to iterative prompt (20% vs. 7%, p= 0.11) in determining admitting diagnoses but not
in determining the final diagnoses.
Conclusion: ChatGPT version 3.5 may function only as an adjunct but not as a primary means of
generating clinical diagnosis. Future studies should focus on the application of newer versions of
ChatGPT and other types of AI models in the clinical practice.
Biography
Dr. Edward Ariel M. Tadea, MPH, is a dedicated pediatrician whose career is marked by excellence in clinical practice and a deep commitment to public health. He graduated with a Doctor of Medicine from the Ateneo de Zamboanga University School of Medicine in 2016 and was awarded the Community Commitment Award for his dedication and selfless commitment to empowering the people of his adopted community. In 2017, he completed his Master of Public Health at the same institution. From March 2020 to December 2023, Dr. Tadea pursued his Pediatric Residency at the Zamboanga City Medical Center, where he served as Chief Resident in his final year and received the Exemplary Chief Resident Award upon graduating in May 2024. His commitment to patient care and leadership has been a constant driving force in his work, providing compassionate care to children and their communities.