Applications of Artificial Intelligence and Machine Learning in Disasters and Public Health Emergencies

AI-Enhanced evaluation of YouTube content on post-surgical incontinence following pelvic cancer treatment

Background

Several pelvic area cancers exhibit high incidence rates, and their surgical treatment can result in adverse effects such as urinary and fecal incontinence, significantly impacting patients' quality of life. Post-surgery incontinence is a significant concern, with prevalence rates ranging from 25 to 45% for urinary incontinence and 9-68% for fecal incontinence. Cancer survivors are increasingly turning to YouTube as a platform to connect with others, yet caution is warranted as misinformation is prevalent.

Objective

This study aims to evaluate the information quality in YouTube videos about post-surgical incontinence after pelvic area cancer surgery.

Methods

A YouTube search for "Incontinence after cancer surgery" yielded 108 videos, which were subsequently analyzed. To evaluate these videos, several quality assessment tools were utilized, including DISCERN, GQS, JAMA, PEMAT, and MQ-VET. Statistical analyses, such as descriptive statistics and intercorrelation tests, were employed to assess various video attributes, including characteristics, popularity, educational value, quality, and reliability. Also, artificial intelligence techniques like PCA, t-SNE, and UMAP were used for data analysis. HeatMap and Hierarchical Clustering Dendrogram techniques validated the Machine Learning results.

Results

The quality scales presented a high level of correlation one with each other (p < 0.01) and the Artificial Intelligence-based techniques presented clear clustering representations of the dataset samples, which were reinforced by the Heat Map and Hierarchical Clustering Dendrogram.

Conclusions

YouTube videos on "Incontinence after Cancer Surgery" present a "High" quality across multiple scales. The use of AI tools, like PCA, t-SNE, and UMAP, is highlighted for clustering large health datasets, improving data visualization, pattern recognition, and complex healthcare analysis.

Applying Machine Learning Techniques to Implementation Science

Machine learning (ML) approaches could expand the usefulness and application of implementation science methods in clinical medicine and public health settings. The aim of this viewpoint is to introduce a roadmap for applying ML techniques to address implementation science questions, such as predicting what will work best, for whom, under what circumstances, and with what predicted level of support, and what and when adaptation or deimplementation are needed. We describe how ML approaches could be used and discuss challenges that implementation scientists and methodologists will need to consider when using ML throughout the stages of implementation.

Chatbot Reliability in Managing Thoracic Surgical Clinical Scenarios

BACKGROUND

Chatbot use in medicine is growing, and concerns have been raised regarding their accuracy. This study assessed the performance of 4 different chatbots in managing thoracic surgical clinical scenarios.

METHODS

Topic domains were identified and clinical scenarios were developed within each domain. Each scenario included 3 stems using Key Feature methods related to diagnosis, evaluation, and treatment. Twelve scenarios were presented to ChatGPT-4 (OpenAI), Bard (recently renamed Gemini; Google), Perplexity (Perplexity AI), and Claude 2 (Anthropic) in 3 separate runs. Up to 1 point was awarded for each stem, yielding a potential of 3 points per scenario. Critical failures were identified before scoring; if they occurred, the stem and overall scenario scores were adjusted to 0. We arbitrarily established a threshold of ≥2 points mean adjusted score per scenario as a passing grade and established a critical fail rate of ≥30% as failure to pass.

RESULTS

The bot performances varied considerably within each run, and their overall performance was a fail on all runs (critical mean scenario fails of 83%, 71%, and 71%). The bots trended toward "learning" from the first to the second run, but without improvement in overall raw (1.24 ± 0.47 vs 1.63 ± 0.76 vs 1.51 ± 0.60; P = .29) and adjusted (0.44 ± 0.54 vs 0.80 ± 0.94 vs 0.76 ± 0.81; P = .48) scenario scores after all runs.

CONCLUSIONS

Chatbot performance in managing clinical scenarios was insufficient to provide reliable assistance. This is a cautionary note against reliance on the current accuracy of chatbots in complex thoracic surgery medical decision making.

ChatGPT's performance before and after teaching in mass casualty incident triage

Since its initial launching, ChatGPT has gained significant attention from the media, with many claiming that ChatGPT's arrival is a transformative milestone in the advancement of the AI revolution. Our aim was to assess the performance of ChatGPT before and after teaching the triage of mass casualty incidents by utilizing a validated questionnaire specifically designed for such scenarios. In addition, we compared the triage performance between ChatGPT and medical students. Our cross-sectional study employed a mixed-methods analysis to assess the performance of ChatGPT in mass casualty incident triage, pre- and post-teaching of Simple Triage And Rapid Treatment (START) triage. After teaching the START triage algorithm, ChatGPT scored an overall triage accuracy of 80%, with only 20% of cases being over-triaged. The mean accuracy of medical students on the same questionnaire yielded 64.3%. Qualitative analysis on pre-determined themes on 'walking-wounded', 'respiration', 'perfusion', and 'mental status' on ChatGPT showed similar performance in pre- and post-teaching of START triage. Additional themes on 'disclaimer', 'prediction', 'management plan', and 'assumption' were identified during the thematic analysis. ChatGPT exhibited promising results in effectively responding to mass casualty incident questionnaires. Nevertheless, additional research is necessary to ensure its safety and efficacy before clinical implementation.

Lessons Learned from Natural Disasters around Digital Health Technologies and Delivering Quality Healthcare

As climate change drives increased intensity, duration and severity of weather-related events that can lead to natural disasters and mass casualties, innovative approaches are needed to develop climate-resilient healthcare systems that can deliver safe, quality healthcare under non-optimal conditions, especially in remote or underserved areas. Digital health technologies are touted as a potential contributor to healthcare climate change adaptation and mitigation, through improved access to healthcare, reduced inefficiencies, reduced costs, and increased portability of patient information. Under normal operating conditions, these systems are employed to deliver personalised healthcare and better patient and consumer involvement in their health and well-being. During the COVID-19 pandemic, digital health technologies were rapidly implemented on a mass scale in many settings to deliver healthcare in compliance with public health interventions, including lockdowns. However, the resilience and effectiveness of digital health technologies in the face of the increasing frequency and severity of natural disasters remain to be determined. In this review, using the mixed-methods review methodology, we seek to map what is known about digital health resilience in the context of natural disasters using case studies to demonstrate what works and what does not and to propose future directions to build climate-resilient digital health interventions.