1
|
Ying L, Li S, Chen C, Yang F, Li X, Chen Y, Ding Y, Chang G, Li J, Wang X. Screening/diagnosis of pediatric endocrine disorders through the artificial intelligence model in different language settings. Eur J Pediatr 2024:10.1007/s00431-024-05527-1. [PMID: 38502320 DOI: 10.1007/s00431-024-05527-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/20/2024] [Revised: 03/12/2024] [Accepted: 03/14/2024] [Indexed: 03/21/2024]
Abstract
This study is aimed at examining the impact of ChatGPT on pediatric endocrine and metabolic conditions, particularly in the areas of screening and diagnosis, in both Chinese and English modes. A 40-question questionnaire covering the four most common pediatric endocrine and metabolic conditions was posed to ChatGPT in both Chinese and English three times each. Six pediatric endocrinologists evaluated the responses. ChatGPT performed better when responding to questions in English, with an unreliable rate of 7.5% compared to 27.5% for Chinese questions, indicating a more consistent response pattern in English. Among the reliable questions, the answers were more comprehensive and satisfactory in the English mode. We also found disparities in ChatGPT's performance when interacting with different target groups and diseases, with improved performance for questions posed by clinicians in English and better performance for questions related to diabetes and overweight/obesity in Chinese for both clinicians and patients. Language comprehension, providing incomprehensive answers, and errors in key data were the main contributors to the low scores, according to reviewer feedback. CONCLUSION Despite these limitations, as ChatGPT continues to evolve and expand its network, it has significant potential as a practical and effective tool for clinical diagnosis and treatment. WHAT IS KNOWN • The deep learning-based large-language model ChatGPT holds great promise for improving clinical practice for both physicians and patients and has the potential to increase the speed and accuracy of disease screening and diagnosis, as well as enhance the overall efficiency of the medical process. However, the reliability and appropriateness of AI model responses in specific field remains unclear. • This study focused on the reliability and appropriateness of AI model responses to straightforward and fundamental questions related to the four most prevalent pediatric endocrine and metabolic disorders, for both healthcare providers and patients, in different language scenarios. WHAT IS NEW • The AI model performed better when responding to questions in English, with more consistent, as well as more comprehensive and satisfactory responses. In addition, we also found disparities in ChatGPT's performance when interacting with different target groups and different diseases. • Despite these limitations, as ChatGPT continues to evolve and expand its network, it has significant potential as a practical and effective tool for clinical diagnosis and treatment.
Collapse
Affiliation(s)
- Lingwen Ying
- Department of Endocrinology and Metabolism, Shanghai Children's Medical Center, School of Medicine, Shanghai Jiao Tong University, Shanghai, 200127, China
| | - Sichen Li
- Department of Neurosurgery, Huashan Hospital, Shanghai Medical College, Fudan University, Shanghai, 200040, China
- National Center for Neurological Disorders, Shanghai Key Laboratory of Brain Function and Restoration and Neural Regeneration, Neurosurgical Institute of Fudan University, Shanghai Clinical Medical Center of Neurosurgery, Shanghai, 200040, China
| | - Chunyang Chen
- Faculty of Information Technology, Monash University, Clayton, VIC, 3800, Australia
| | - Fan Yang
- Department of Endocrinology and Metabolism, Shanghai Children's Medical Center, School of Medicine, Shanghai Jiao Tong University, Shanghai, 200127, China
| | - Xin Li
- Department of Endocrinology and Metabolism, Shanghai Children's Medical Center, School of Medicine, Shanghai Jiao Tong University, Shanghai, 200127, China
| | - Yao Chen
- Department of Endocrinology and Metabolism, Shanghai Children's Medical Center, School of Medicine, Shanghai Jiao Tong University, Shanghai, 200127, China
| | - Yu Ding
- Department of Endocrinology and Metabolism, Shanghai Children's Medical Center, School of Medicine, Shanghai Jiao Tong University, Shanghai, 200127, China
| | - Guoying Chang
- Department of Endocrinology and Metabolism, Shanghai Children's Medical Center, School of Medicine, Shanghai Jiao Tong University, Shanghai, 200127, China.
| | - Juan Li
- Department of Endocrinology and Metabolism, Shanghai Children's Medical Center, School of Medicine, Shanghai Jiao Tong University, Shanghai, 200127, China
| | - Xiumin Wang
- Department of Endocrinology and Metabolism, Shanghai Children's Medical Center, School of Medicine, Shanghai Jiao Tong University, Shanghai, 200127, China.
| |
Collapse
|
2
|
Hoversten LJ, Traxler MJ. Zooming in on zooming out: Partial selectivity and dynamic tuning of bilingual language control during reading. Cognition 2019; 195:104118. [PMID: 31790961 DOI: 10.1016/j.cognition.2019.104118] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/01/2019] [Revised: 10/21/2019] [Accepted: 10/25/2019] [Indexed: 11/25/2022]
Abstract
Prominent models of bilingual visual word recognition posit a bottom-up nonselective view of lexical processing with parallel access to lexical candidates of both languages. However, these accounts do not accommodate recent findings of top-down effects on the relative global activation level of each language during bilingual reading. We conducted two eye-tracking experiments to systematically assess the degree of accessibility of each language in different global language contexts. When critical words were presented overtly in Experiment 1, code switches disrupted reading early during lexical processing, but not as much as pseudowords did. Participants zoomed out of the target language with increasing exposure to language switches. In Experiment 2, a monolingual language context was created by presenting critical words covertly as parafoveal previews. Here, code-switched words were treated like pseudowords, and participants remained zoomed in to the target language throughout the experiment. Switch direction analyses confirmed and extended these interpretations to provide further support for the role of global language control on lexical access, above and beyond effects due to proficiency differences across languages. Together, these data provide strong evidence for dynamic top-down adjustment of the degree of language selectivity during bilingual reading.
Collapse
Affiliation(s)
| | - Matthew J Traxler
- University of California, Davis, Department of Psychology, Center for Mind and Brain, United States
| |
Collapse
|