Try Google Translate to overcome language barriers

Performance of ChatGPT and Google Translate for Pediatric Discharge Instruction Translation

BACKGROUND AND OBJECTIVES

Patients who speak languages other than English face barriers to equitable healthcare delivery. Machine translation systems, including emerging large language models, have the potential to expand access to translation services, but their merits and limitations in clinical practice remain poorly defined. We aimed to assess the performance of Google Translate and ChatGPT for multilingual translation of pediatric discharge instructions.

METHODS

Twenty standardized discharge instructions for pediatric conditions were translated into Spanish, Brazilian Portuguese, and Haitian Creole by professional translation services, Google Translate and ChatGPT-4.0, and evaluated for adequacy (preserved information), fluency (grammatical correctness), meaning (preserved connotation), and severity (clinical harm), along with assessment of overall preference. Domain-level ratings and preferred translation source were summarized with descriptive statistics and compared with professional translations.

RESULTS

Google Translate and ChatGPT demonstrated similar domain-level ratings to professional translations for Spanish and Portuguese. For Haitian Creole, compared with both Google Translate and ChatGPT, professional translations demonstrated significantly greater adequacy, fluency meaning, and severity scores. ChatGPT (33.3%, P < .001) and Google Translate (23.3%, P = .024) contained more potentially clinically significant errors (severity score ≤3) for Haitian Creole than professional translations (8.3%). Professional Haitian Creole (48.3%) and Portuguese (43.3%), but not Spanish (15%), translations were most frequently preferred among translation sources.

CONCLUSIONS

Machine translation platforms have comparable performance to professional translations for Spanish and Portuguese but shortcomings in quality, accuracy, and preference persist for Haitian Creole. Diverse multilingual training data are needed, along with regulations ensuring safe and equitable applications of machine translation in clinical practice.

A Pragmatic Assessment of Google Translate for Emergency Department Instructions

BACKGROUND

Because many hospitals have no mechanism for written translation, ED providers resort to the use of automated translation software, such as Google Translate (GT) for patient instructions. A recent study of discharge instructions in Spanish and Chinese suggested that accuracy rates of Google Translate (GT) were high.

STUDY OBJECTIVE

To perform a pragmatic assessment of GT for the written translation of commonly used ED discharge instructions in seven commonly spoken languages.

METHODS

A prospective assessment of the accuracy of GT for 20 commonly used ED discharge instruction phrases, as evaluated by a convenience sample of native speakers of seven commonly spoken languages (Spanish, Chinese, Vietnamese, Tagalog, Korean, Armenian, and Farsi). Translations were evaluated using a previously validated matrix for scoring machine translation, containing 5-point Likert scales for fluency, adequacy, meaning, and severity, in addition to a dichotomous assessment of retention of the overall meaning.

RESULTS

Twenty volunteers evaluated 400 google translated discharge statements. Volunteers were 50% female and spoke Spanish (5), Armenian (2), Chinese (3), Tagalog (4), Korean (2), and Farsi (2). The overall meaning was retained for 82.5% (330/400) of the translations. Spanish had the highest accuracy rate (94%), followed by Tagalog (90%), Korean (82.5%), Chinese (81.7%), Farsi (67.5%), and Armenian (55%). Mean Likert scores (on a 5-point scale) were high for fluency (4.2), adequacy (4.4), meaning (4.3), and severity (4.3) but also varied.

CONCLUSION

GT for discharge instructions in the ED is inconsistent between languages and should not be relied on for patient instructions.

Validation of the French version of the Cannabis Use Disorder Identification Test-Revised and comparison with the Cannabis Abuse Screening Test for screening cannabis use disorder in a psychiatric sample

INTRODUCTION

Several questionnaires have been developed for screening cannabis use disorder in clinical populations, but very few studies have compared the screening abilities of the different instruments. Here, we aimed to confirm the psychometric properties of a French version of the Cannabis Use Disorder Identification Test-Revised (CUDIT-R), and to compare its screening abilities with those of the Cannabis Abuse Screening Test (CAST), in subjects consulting in mental health settings.

METHODS

Two hundred and thirteen cannabis smokers who sought treatment for any type of mental disorder, recruited in four French centres, completed the French CUDIT-R (CUDIT-R-Fr) and the full version of the CAST, and were assessed for Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition criteria of cannabis use disorder by an addiction specialist. They were retested with the CUDIT-R-Fr after approximately a week. The factorial construct validity, internal consistency and test-retest reliability of the CUDIT-R-Fr were assessed. The compared sensitivity and specificity of the CAST and CUDIT-R-Fr were explored, using the clinician assessment as the reference.

RESULTS

The French CUDIT-R showed a good internal consistency (Cronbach's alpha = 0.89) and an excellent test-retest reliability (ρ = 0.97). The sensitivity and specificity for screening cannabis use disorder were 0.81 and 0.77 for the CUDIT-R, and 0.92 and 0.63 for the CAST, respectively.

CONCLUSIONS

Based on the recommended cut-offs, the CAST appeared more sensitive, while the CUDIT-R was more specific, for screening cannabis use disorder in a population of cannabis users with heterogeneous types of mental health disorders.

A prospective observational study of machine translation software to overcome the challenge of including ethnic diversity in healthcare research

AIM

This study investigates whether machine translation could help with the challenge of enabling the inclusion of ethnic diversity in healthcare research.

DESIGN

A two phase, prospective observational study.

METHODS

Two machine translators, Google Translate and Babylon 9, were tested. Translation of the Strengths and Difficulties Questionnaire (SDQ) from 24 languages into English and translation of an English information sheet into Spanish and Chinese were quality scored. Quality was assessed using the Translation Assessment Quality Tool.

RESULTS

Only six of the 48 translations of the SDQ were rated as acceptable, all from Google Translate. The mean number of acceptably translated sentences was higher (P = 0·001) for Google Translate 17·1 (sd 7·2) than for Babylon 9 11 (sd 7·9). Translation by Google Translate was better for Spanish and Chinese, although no score was in the acceptable range. Machine translation is not currently sufficiently accurate without editing to provide translation of materials for use in healthcare research.