Back to 2025 Abstracts
Comparing Accuracy and Completeness of Google Search vs. ChatGPT-4 Responses to Questions Patients Have Regarding Common Craniofacial Conditions
Ishan Patel
1, Amanda N Awad
2, Hanaa Shihadeh
2, Avi Sura
1, Dev Dwivedi
1, Pratik Vangal
1, Dzifa S Kpodzo
2, Stephanie MC Bray
2
1 Albany Medial College, Albany, NY, USA 2. Albany Medical Center, Albany NY, USA
Background: Patients often use Google as a source of quick medical information, although it has the potential to be unreliable for certain search queries. ChatGPT is rising as a search instrument that patients can use to obtain important medical information. The purpose of this study is to compare the ease of readability and accuracy of ChatGPT to Google as a search instrument for the most common patient questions regarding craniosynostosis and cleft palate.
Methods: "Craniosynostosis" and "Cleft Palate" were input into Google and the top 10 questions according to Google Analytics for each search were recorded. Each question was then input into both Google and ChatGPT, and the responses from each were recorded. The ease of readability for each response was determined by the Flesch-Kincaid instrument. The questions and answers were compiled, randomized, and blinded, and the accuracy and completeness of each response were evaluated on a 3 point scale (1=fully incorrect, 2=partially incorrect, 3=correct).
Results: A total of 20 questions were evaluated in this study. Google responses to cleft palate questions were at a significantly lower reading level compared with ChatGPT responses (9.95 vs.13.2, p=0.006). However, the reading level for craniosynostosis questions was not significantly different between Google and ChatGPT (14.6 vs. 14.7, p=0.467). Overall, ChatGPT responses were both significantly more complete and accurate when compared to Google responses (2.4 vs.1.7, p<0.001; 2.75 vs.1.95, p<0.001). This holds true when comparing craniosynostosis and cleft palate questions (p<0.01).
Conclusion: ChatGPT can provide more accurate and holistic information surrounding patients' questions regarding craniosynostosis without sacrificing readability. Patients can use this information in order to inform their future searches in order to obtain the most accurate information about their diagnoses. Further studies evaluating the information learned by patients from both search engines can help clinicians guide patients toward resources that best fit their individual care.
Back to 2025 Abstracts
