Augmenting diagnostic vision with AI

Quality and Dependability of ChatGPT and DingXiangYuan Forums for Remote Orthopedic Consultations: Comparative Analysis

BACKGROUND

The widespread use of artificial intelligence, such as ChatGPT (OpenAI), is transforming sectors, including health care, while separate advancements of the internet have enabled platforms such as China's DingXiangYuan to offer remote medical services.

OBJECTIVE

This study evaluates ChatGPT-4's responses against those of professional health care providers in telemedicine, assessing artificial intelligence's capability to support the surge in remote medical consultations and its impact on health care delivery.

METHODS

We sourced remote orthopedic consultations from "Doctor DingXiang," with responses from its certified physicians as the control and ChatGPT's responses as the experimental group. In all, 3 blindfolded, experienced orthopedic surgeons assessed responses against 7 criteria: "logical reasoning," "internal information," "external information," "guiding function," "therapeutic effect," "medical knowledge popularization education," and "overall satisfaction." We used Fleiss κ to measure agreement among multiple raters.

RESULTS

Initially, consultation records for a cumulative count of 8 maladies (equivalent to 800 cases) were gathered. We ultimately included 73 consultation records by May 2023, following primary and rescreening, in which no communication records containing private information, images, or voice messages were transmitted. After statistical scoring, we discovered that ChatGPT's "internal information" score (mean 4.61, SD 0.52 points vs mean 4.66, SD 0.49 points; P=.43) and "therapeutic effect" score (mean 4.43, SD 0.75 points vs mean 4.55, SD 0.62 points; P=.32) were lower than those of the control group, but the differences were not statistically significant. ChatGPT showed better performance with a higher "logical reasoning" score (mean 4.81, SD 0.36 points vs mean 4.75, SD 0.39 points; P=.38), "external information" score (mean 4.06, SD 0.72 points vs mean 3.92, SD 0.77 points; P=.25), and "guiding function" score (mean 4.73, SD 0.51 points vs mean 4.72, SD 0.54 points; P=.96), although the differences were not statistically significant. Meanwhile, the "medical knowledge popularization education" score of ChatGPT was better than that of the control group (mean 4.49, SD 0.67 points vs mean 3.87, SD 1.01 points; P<.001), and the difference was statistically significant. In terms of "overall satisfaction," the difference was not statistically significant between the groups (mean 8.35, SD 1.38 points vs mean 8.37, SD 1.24 points; P=.92). According to how Fleiss κ values were interpreted, 6 of the control group's score points were classified as displaying "fair agreement" (P<.001), and 1 was classified as showing "substantial agreement" (P<.001). In the experimental group, 3 points were classified as indicating "fair agreement," while 4 suggested "moderate agreement" (P<.001).

CONCLUSIONS

ChatGPT-4 matches the expertise found in DingXiangYuan forums' paid consultations, excelling particularly in scientific education. It presents a promising alternative for remote health advice. For health care professionals, it could act as an aid in patient education, while patients may use it as a convenient tool for health inquiries.

AI-enabled remote monitoring of vital signs for COVID-19: methods, prospects and challenges

The COVID-19 pandemic has overwhelmed the existing healthcare infrastructure in many parts of the world. Healthcare professionals are not only over-burdened but also at a high risk of nosocomial transmission from COVID-19 patients. Screening and monitoring the health of a large number of susceptible or infected individuals is a challenging task. Although professional medical attention and hospitalization are necessary for high-risk COVID-19 patients, home isolation is an effective strategy for low and medium risk patients as well as for those who are at risk of infection and have been quarantined. However, this necessitates effective techniques for remotely monitoring the patients’ symptoms. Recent advances in Machine Learning (ML) and Deep Learning (DL) have strengthened the power of imaging techniques and can be used to remotely perform several tasks that previously required the physical presence of a medical professional. In this work, we study the prospects of vital signs monitoring for COVID-19 infected as well as quarantined individuals by using DL and image/signal-processing techniques, many of which can be deployed using simple cameras and sensors available on a smartphone or a personal computer, without the need of specialized equipment. We demonstrate the potential of ML-enabled workflows for several vital signs such as heart and respiratory rates, cough, blood pressure, and oxygen saturation. We also discuss the challenges involved in implementing ML-enabled techniques.

Human vs Artificial Intelligence-Based Echocardiography Analysis as Predictor of Outcomes: An analysis from the World Alliance Societies of Echocardiography COVID study

Background

Transthoracic echocardiography is the leading cardiac imaging modality for patients admitted with COVID-19, a condition of high short-term mortality. The aim of this study was to test the hypothesis that artificial intelligence (AI)–based analysis of echocardiographic images could predict mortality more accurately than conventional analysis by a human expert.

Methods

Patients admitted to 13 hospitals for acute COVID-19 who underwent transthoracic echocardiography were included. Left ventricular ejection fraction (LVEF) and left ventricular longitudinal strain (LVLS) were obtained manually by multiple expert readers and by automated AI software. The ability of the manual and AI analyses to predict all-cause mortality was compared.

Results

In total, 870 patients were enrolled. The mortality rate was 27.4% after a mean follow-up period of 230 ± 115 days. AI analysis had lower variability than manual analysis for both LVEF (P = .003) and LVLS (P = .005). AI-derived LVEF and LVLS were predictors of mortality in univariable and multivariable regression analysis (odds ratio, 0.974 [95% CI, 0.956-0.991; P = .003] for LVEF; odds ratio, 1.060 [95% CI, 1.019-1.105; P = .004] for LVLS), but LVEF and LVLS obtained by manual analysis were not. Direct comparison of the predictive value of AI versus manual measurements of LVEF and LVLS showed that AI was significantly better (P = .005 and P = .003, respectively). In addition, AI-derived LVEF and LVLS had more significant and stronger correlations to other objective biomarkers of acute disease than manual reads.

Conclusions

AI-based analysis of LVEF and LVLS had similar feasibility as manual analysis, minimized variability, and consequently increased the statistical power to predict mortality. AI-based, but not manual, analyses were a significant predictor of in-hospital and follow-up mortality.

What works where and how for uptake and impact of artificial intelligence in pathology: A review of theories for a realist evaluation (Preprint)

BACKGROUND

There is increasing interest in the use of artificial intelligence (AI) in pathology to increase accuracy and efficiency. To date, studies of clinicians' perceptions of AI have found only moderate acceptability, suggesting the need for further research regarding how to integrate it into clinical practice.

OBJECTIVE

The aim of the study was to determine contextual factors that may support or constrain the uptake of AI in pathology.

METHODS

To go beyond a simple listing of barriers and facilitators, we drew on the approach of realist evaluation and undertook a review of the literature to elicit stakeholders' theories of how, for whom, and in what circumstances AI can provide benefit in pathology. Searches were designed by an information specialist and peer-reviewed by a second information specialist. Searches were run on the arXiv.org repository, MEDLINE, and the Health Management Information Consortium, with additional searches undertaken on a range of websites to identify gray literature. In line with a realist approach, we also made use of relevant theory. Included documents were indexed in NVivo 12, using codes to capture different contexts, mechanisms, and outcomes that could affect the introduction of AI in pathology. Coded data were used to produce narrative summaries of each of the identified contexts, mechanisms, and outcomes, which were then translated into theories in the form of context-mechanism-outcome configurations.

RESULTS

A total of 101 relevant documents were identified. Our analysis indicates that the benefits that can be achieved will vary according to the size and nature of the pathology department's workload and the extent to which pathologists work collaboratively; the major perceived benefit for specialist centers is in reducing workload. For uptake of AI, pathologists' trust is essential. Existing theories suggest that if pathologists are able to "make sense" of AI, engage in the adoption process, receive support in adapting their work processes, and can identify potential benefits to its introduction, it is more likely to be accepted.

CONCLUSIONS

For uptake of AI in pathology, for all but the most simple quantitative tasks, measures will be required that either increase confidence in the system or provide users with an understanding of the performance of the system. For specialist centers, efforts should focus on reducing workload rather than increasing accuracy. Designers also need to give careful thought to usability and how AI is integrated into pathologists' workflow.

A Comprehensive Explanation Framework for Biomedical Time Series Classification

In this study, we propose a post-hoc explainability framework for deep learning models applied to quasi-periodic biomedical time-series classification. As a case study, we focus on the problem of atrial fibrillation (AF) detection from electrocardiography signals, which has strong clinical relevance. Starting from a state-of-the-art pretrained model, we tackle the problem from two different perspectives: global and local explanation. With global explanation, we analyze the model behavior by looking at entire classes of data, showing which regions of the input repetitive patterns have the most influence for a specific outcome of the model. Our explanation results align with the expectations of clinical experts, showing that features crucial for AF detection contribute heavily to the final decision. These features include R-R interval regularity, absence of the P-wave or presence of electrical activity in the isoelectric period. On the other hand, with local explanation, we analyze specific input signals and model outcomes. We present a comprehensive analysis of the network facing different conditions, whether the model has correctly classified the input signal or not. This enables a deeper understanding of the network's behavior, showing the most informative regions that trigger the classification decision and highlighting possible causes of misbehavior.

Deep-learning-assisted analysis of echocardiographic videos improves predictions of all-cause mortality

Machine learning promises to assist physicians with predictions of mortality and of other future clinical events by learning complex patterns from historical data, such as longitudinal electronic health records. Here we show that a convolutional neural network trained on raw pixel data in 812,278 echocardiographic videos from 34,362 individuals provides superior predictions of one-year all-cause mortality. The model's predictions outperformed the widely used pooled cohort equations, the Seattle Heart Failure score (measured in an independent dataset of 2,404 patients with heart failure who underwent 3,384 echocardiograms), and a machine learning model involving 58 human-derived variables from echocardiograms and 100 clinical variables derived from electronic health records. We also show that cardiologists assisted by the model substantially improved the sensitivity of their predictions of one-year all-cause mortality by 13% while maintaining prediction specificity. Large unstructured datasets may enable deep learning to improve a wide range of clinical prediction models.

Role of Artificial Intelligence Applications in Real-Life Clinical Practice: Systematic Review

BACKGROUND

Artificial intelligence (AI) applications are growing at an unprecedented pace in health care, including disease diagnosis, triage or screening, risk analysis, surgical operations, and so forth. Despite a great deal of research in the development and validation of health care AI, only few applications have been actually implemented at the frontlines of clinical practice.

OBJECTIVE

The objective of this study was to systematically review AI applications that have been implemented in real-life clinical practice.

METHODS

We conducted a literature search in PubMed, Embase, Cochrane Central, and CINAHL to identify relevant articles published between January 2010 and May 2020. We also hand searched premier computer science journals and conferences as well as registered clinical trials. Studies were included if they reported AI applications that had been implemented in real-world clinical settings.

RESULTS

We identified 51 relevant studies that reported the implementation and evaluation of AI applications in clinical practice, of which 13 adopted a randomized controlled trial design and eight adopted an experimental design. The AI applications targeted various clinical tasks, such as screening or triage (n=16), disease diagnosis (n=16), risk analysis (n=14), and treatment (n=7). The most commonly addressed diseases and conditions were sepsis (n=6), breast cancer (n=5), diabetic retinopathy (n=4), and polyp and adenoma (n=4). Regarding the evaluation outcomes, we found that 26 studies examined the performance of AI applications in clinical settings, 33 studies examined the effect of AI applications on clinician outcomes, 14 studies examined the effect on patient outcomes, and one study examined the economic impact associated with AI implementation.

CONCLUSIONS

This review indicates that research on the clinical implementation of AI applications is still at an early stage despite the great potential. More research needs to assess the benefits and challenges associated with clinical AI applications through a more rigorous methodology.

Unity Is Intelligence: A Collective Intelligence Experiment on ECG Reading to Improve Diagnostic Performance in Cardiology

Medical errors have a huge impact on clinical practice in terms of economic and human costs. As a result, technology-based solutions, such as those grounded in artificial intelligence (AI) or collective intelligence (CI), have attracted increasing interest as a means of reducing error rates and their impacts. Previous studies have shown that a combination of individual opinions based on rules, weighting mechanisms, or other CI solutions could improve diagnostic accuracy with respect to individual doctors. We conducted a study to investigate the potential of this approach in cardiology and, more precisely, in electrocardiogram (ECG) reading. To achieve this aim, we designed and conducted an experiment involving medical students, recent graduates, and residents, who were asked to annotate a collection of 10 ECGs of various complexity and difficulty. For each ECG, we considered groups of increasing size (from three to 30 members) and applied three different CI protocols. In all cases, the results showed a statistically significant improvement (ranging from 9% to 88%) in terms of diagnostic accuracy when compared to the performance of individual readers; this difference held for not only large groups, but also smaller ones. In light of these results, we conclude that CI approaches can support the tasks mentioned above, and possibly other similar ones as well. We discuss the implications of applying CI solutions to clinical settings, such as cases of augmented ‘second opinions’ and decision-making.

[Artificial intelligence in cardiology : Relevance, current applications, and future developments]

Big data and applications of artificial intelligence (AI), such as machine learning or deep learning, will enrich healthcare in the future and become increasingly important. Among other things, they have the potential to avoid unnecessary examinations as well as diagnostic and therapeutic errors. They could enable improved, early and accelerated decision-making. In the article, the authors provide an overview of current AI-based applications in cardiology. The examples describe innovative solutions for risk assessment, diagnosis and therapy support up to patient self-management. Big data and AI serve as a basis for efficient, predictive, preventive and personalised medicine. However, the examples also show that research is needed to further develop the solutions for the benefit of the patient and the medical profession, to demonstrate the effectiveness and benefits in health care and to establish legal and ethical standards.

Use of Artificial Intelligence in Dermatology

Artificial intelligence (AI) has emerged as a major frontier in computer science research. Although AI has been available for some time and found its application in many fields of medicine, its use in dermatology is comparatively new and limited. A sound understanding of the concepts of AI is essential for dermatologists as skin conditions with their abundant clinical and dermatoscopic data and images can potentially be the next big thing in the application of AI in medicine. There are already a number of artificial intelligence studies focusing on skin disorders, such as skin cancer, psoriasis, atopic dermatitis and onychomycosis. This article presents an overview of AI and new developments relevant to dermatology, examining both its current applications and future potential.

On the Effectiveness of Deep Representation Learning: the Atrial Fibrillation Case

The automatic and unsupervised analysis of biomedical time series is of primary importance for diagnostic and preventive medicine, enabling fast and reliable data processing to reveal clinical insights without the need for human intervention. Representation learning (RL) methods perform an automatic extraction of meaningful features that can be used, e.g., for a subsequent classification of the measured data. The goal of this study is to explore and quantify the benefits of RL techniques of varying degrees of complexity, focusing on modern deep learning (DL) architectures. We focus on the automatic classification of atrial fibrillation (AF) events from noisy single-lead electrocardiographic signals (ECG) obtained from wireless sensors. This is an important task as it allows the detection of sub-clinical AF which is hard to diagnose with a short in-clinic 12-lead ECG. The effectiveness of the considered architectures is quantified and discussed in terms of classification performance, memory/data efficiency and computational complexity.

AI in Health: State of the Art, Challenges, and Future Directions

INTRODUCTION

Artificial intelligence (AI) technologies continue to attract interest from a broad range of disciplines in recent years, including health. The increase in computer hardware and software applications in medicine, as well as digitization of health-related data together fuel progress in the development and use of AI in medicine. This progress provides new opportunities and challenges, as well as directions for the future of AI in health.

OBJECTIVE

The goals of this survey are to review the current state of AI in health, along with opportunities, challenges, and practical implications. This review highlights recent developments over the past five years and directions for the future.

METHODS

Publications over the past five years reporting the use of AI in health in clinical and biomedical informatics journals, as well as computer science conferences, were selected according to Google Scholar citations. Publications were then categorized into five different classes, according to the type of data analyzed.

RESULTS

The major data types identified were multi-omics, clinical, behavioral, environmental and pharmaceutical research and development (R&D) data. The current state of AI related to each data type is described, followed by associated challenges and practical implications that have emerged over the last several years. Opportunities and future directions based on these advances are discussed.

CONCLUSION

Technologies have enabled the development of AI-assisted approaches to healthcare. However, there remain challenges. Work is currently underway to address multi-modal data integration, balancing quantitative algorithm performance and qualitative model interpretability, protection of model security, federated learning, and model bias.

Reflecting health: smart mirrors for personalized medicine

Inexpensive embedded computing and the related Internet of Things technologies enable the recent development of smart products that can respond to human needs and improve everyday tasks in an attempt to make traditional environments more “intelligent”. Several projects have augmented mirrors for a range of smarter applications in automobiles and homes. The opportunity to apply smart mirror technology to healthcare to predict and to monitor aspects of health and disease is a natural but mostly underdeveloped idea. We envision that smart mirrors comprising a combination of intelligent hardware and software could identify subtle, yet clinically relevant changes in physique and appearance. Similarly, a smart mirror could record and evaluate body position and motion to identify posture and movement issues, as well as offer feedback for corrective actions. Successful development and implementation of smart mirrors for healthcare applications will require overcoming new challenges in engineering, machine learning, computer vision, and biomedical research. This paper examines the potential uses of smart mirrors in healthcare and explores how this technology might benefit users in various medical environments. We also provide a brief description of the state-of-the-art, including a functional prototype concept developed by our group, and highlight the directions to make this device more mainstream in health-related applications.

Clinical judgement in the era of big data and predictive analytics

Clinical judgement is a central and longstanding issue in the philosophy of medicine which has generated significant interest over the past few decades. In this article, we explore different approaches to clinical judgement articulated in the literature, focusing in particular on data-driven, mathematical approaches which we contrast with narrative, virtue-based approaches to clinical reasoning. We discuss the tension between these different clinical epistemologies and further explore the implications of big data and machine learning for a philosophy of clinical judgement. We argue for a pluralistic, integrative approach, and demonstrate how narrative, virtue-based clinical reasoning will remain indispensable in an era of big data and predictive analytics.

Advanced machine learning in action: identification of intracranial hemorrhage on computed tomography scans of the head with clinical workflow integration

Intracranial hemorrhage (ICH) requires prompt diagnosis to optimize patient outcomes. We hypothesized that machine learning algorithms could automatically analyze computed tomography (CT) of the head, prioritize radiology worklists and reduce time to diagnosis of ICH. 46,583 head CTs (~2 million images) acquired from 2007–2017 were collected from several facilities across Geisinger. A deep convolutional neural network was trained on 37,074 studies and subsequently evaluated on 9499 unseen studies. The predictive model was implemented prospectively for 3 months to re-prioritize “routine” head CT studies as “stat” on realtime radiology worklists if an ICH was detected. Time to diagnosis was compared between the re-prioritized “stat” and “routine” studies. A neuroradiologist blinded to the study reviewed false positive studies to determine whether the dictating radiologist overlooked ICH. The model achieved an area under the ROC curve of 0.846 (0.837–0.856). During implementation, 94 of 347 “routine” studies were re-prioritized to “stat”, and 60/94 had ICH identified by the radiologist. Five new cases of ICH were identified, and median time to diagnosis was significantly reduced (p < 0.0001) from 512 to 19 min. In particular, one outpatient with vague symptoms on anti-coagulation was found to have an ICH which was treated promptly with reversal of anticoagulation, resulting in a good clinical outcome. Of the 34 false positives, the blinded over-reader identified four probable ICH cases overlooked in original interpretation. In conclusion, an artificial intelligence algorithm can prioritize radiology worklists to reduce time to diagnosis of new outpatient ICH by 96% and may also identify subtle ICH overlooked by radiologists. This demonstrates the positive impact of advanced machine learning in radiology workflow optimization.

A computer program that automatically analyzes brain images from patients undergoing CT scans of the head can reliably flag those with signs of hemorrhage. A team of researchers from Geisinger in Danville, Pennsylvania, USA, trained and tested a machine-learning algorithm using 46,583 computed tomography imaging studies of the head. Subsequently, they implemented the model into routine care for 3 months to help prioritize radiology worklists. Of 347 routine cases, the computer identified 94 as having an intracranial hemorrhage, two-thirds of which were confirmed by a radiologist, including five among patients who had a new diagnosis of a brain bleed. The algorithm reduced the average time in which a radiologist diagnosed these patients from around 8.5 h to just 19 min, demonstrating the positive impact of incorporating artificial intelligence into radiology workflow.

The future of radiology augmented with Artificial Intelligence: A strategy for success

The rapid development of Artificial Intelligence/deep learning technology and its implementation into routine clinical imaging will cause a major transformation to the practice of radiology. Strategic positioning will ensure the successful transition of radiologists into their new roles as augmented clinicians. This paper describes an overall vision on how to achieve a smooth transition through the practice of augmented radiology where radiologists-in-the-loop ensure the safe implementation of Artificial Intelligence systems.

Home Monitoring of Blood Pressure: Short-Term Changes During Serial Measurements for 56398 Subjects

Hypertension is one of the greatest contributors to premature morbidity and mortality worldwide. It has been demonstrated that lowering blood pressure (BP) by just a few mmHg can bring substantial clinical benefits, reducing the risk of stroke and ischemic heart disease. Properly managing high BP is one of the most pressing global health issues, but accurate methods to continuously monitoring BP at home are still under discussion. Indeed, the BP for any given individual can fluctuate significantly during intervals as short as a few minutes. In clinical settings, the guidelines suggest to wait for 5 or 10 minutes in seated rest before taking the measure, in order to alleviate the effect of the stress induced by the clinical environment. Alternatively, BP measured in the home environment is thought to provide a more accurate measure free of the stress of a clinical environment, but there is currently a lack of extensive studies on the trajectory of serial BP measurements over minutes in the home setting. In this paper, we aim at filling this gap by analyzing a large dataset of more than 16 million BP measurements taken at home with commercial BP monitoring devices. In particular, we propose new techniques to analyze this dataset, taking into account the limitations due to the uncontrolled data collection, and we study the characteristics of the BP trajectory for consecutive measures over several minutes. We show that the BP values significantly decrease after 10 minutes minutes from the initial measurement (4.1 and 6.6 mmHg for the diastolic and systolic BP, respectively), and continue to decrease for about 25 minutes. We also describe statistically the clinical relevance of this change, observing more than 50% misclassifications for measurements in the hypertension region. We then propose a model to study the inter-subject variability, showing significant variations in the expected decrease in systolic BP. These results may provide the initial evidence for future large clinical studies using participant-monitored BP.