The impact of machine learning on patient care: A systematic review

Artificial Intelligence Applied to Forced Spirometry in Primary Care

Introduction

This study aims to create an artificial intelligence (AI) based machine learning (ML) model capable of predicting a spirometric obstructive pattern using variables with the highest predictive power derived from an active case-finding program for COPD in primary care.

Material and methods

A total of 1190 smokers, aged 30-80 years old with no prior history of respiratory disease, underwent spirometry with bronchodilation. The sample was analyzed using AI tools. Based on an exploratory data analysis (EDA), independent variables (according to mutual information analysis) were trained using a gradient boosting algorithm (GBT) and validated through cross-validation.

Results

With an area under the curve close to unity, the model predicted a spirometric obstructive pattern using variables with the highest predictive power: FEV1_theoretical_pre values. Sensitivity: 93%. Positive predictive value: 94%. Specificity: 97%. Negative predictive value: 96%. Accuracy: 95%. Precision: 94%.

Conclusion

An ML model can predict the presence of an obstructive pattern in spirometry in a primary care smoking population with no prior diagnosis of respiratory disease using the FEV1_theoretical_pre values with an accuracy and precision exceeding 90%. Further studies including clinical data and strategies for integrating AI into clinical workflow are needed.

Machine Learning in Electroconvulsive Therapy: A Systematic Review

ABSTRACT

Despite years of research, we are still not able to reliably predict who might benefit from electroconvulsive therapy (ECT) treatment. As we exhaust what is possible using traditional statistical analysis, ECT remains a good candidate for machine learning approaches due to the large data sets with data captured through electroencephalography (EEG) and other objective measures. A systematic review of 6 databases led to the full-text examination of 26 articles using machine learning approaches in examining data predicting response to ECT treatment. The identified articles used a wide variety of data types covering structural and functional imaging data (n = 15), clinical data (n = 5), a combination of clinical and imaging data (n = 2), EEG (n = 3), and social media posts (n = 1). The clinical indications in which response prediction was assessed were depression (n = 21) and psychosis (n = 4). Changes in multiple anatomical regions in the brain were identified as holding a predictive value for response to ECT. These primarily centered on the limbic system and associated networks. Clinical features predicting good response to ECT in depression included shorter duration, lower severity, higher medication dose, psychotic features, low cortisol levels, and positive family history. It has also been possible to predict the likelihood of relapse of readmission with psychosis after ECT treatment, including a better response if higher transfer entropy was calculated from EEG signals. A transdisciplinary approach with an international consortium collecting a wide range of retrospective and prospective data may help to refine and extend these outcomes and translate them into clinical practice.

Predicting kidney allograft survival with explainable machine learning

INTRODUCTION

Despite significant progress over the last decades in the survival of kidney allografts, several risk factors remain contributing to worsening kidney function or even loss of transplants. We aimed to evaluate a new machine learning method to identify these variables which may predict the early graft loss in kidney transplant patients and to assess their usefulness for improving clinical decisions.

MATERIAL AND METHODS

A retrospective cohort study was carried out with 627 kidney transplant patients followed at least three months. All these data were pre-processed, and their selected features were used to develop an automatically working a machine learning algorithm; this algorithm was then applied for training and parameterization of the model; and finally, the tested model was then used for the analysis of patients' features that were the most impactful for the prediction of clinical outcomes. Our models were evaluated using the Area Under the Curve (AUC), and the SHapley Additive exPlanations (SHAP) algorithm was used to interpret its predictions.

RESULTS

The final selected model achieved a precision of 0.81, a sensitivity of 0.61, a specificity of 0.89, and an AUC value of 0.84. In our model, serum creatinine levels of kidney transplant patients, evaluated at the hospital discharge, proved to be the most important factor in the decision-making for the allograft loss. Patients with a weight equivalent to a BMI closer to the normal range prior to a kidney transplant are less likely to experience graft loss compared to patients with a BMI below the normal range. The age of patients at transplantation and Polyomavirus (BKPyV) infection had significant impact on clinical outcomes in our model.

CONCLUSIONS

Our algorithm suggests that the main characteristics that impacted early allograft loss were serum creatinine levels at the hospital discharge, as well as the pre-transplant values such as body weight, age of patients, and their BKPyV infection. We propose that machine learning tools can be developed to effectively assist medical decision-making in kidney transplantation.

EEG-based machine learning models for the prediction of phenoconversion time and subtype in isolated rapid eye movement sleep behavior disorder

STUDY OBJECTIVES

Isolated rapid eye movement sleep behavior disorder (iRBD) is a prodromal stage of α-synucleinopathies and eventually phenoconverts to overt neurodegenerative diseases including Parkinson's disease (PD), dementia with Lewy bodies (DLB), and multiple system atrophy (MSA). Associations of baseline resting-state electroencephalography (EEG) with phenoconversion have been reported. In this study, we aimed to develop machine learning models to predict phenoconversion time and subtype using baseline EEG features in patients with iRBD.

METHODS

At baseline, resting-state EEG and neurological assessments were performed on patients with iRBD. Calculated EEG features included spectral power, weighted phase lag index, and Shannon entropy. Three models were used for survival prediction, and four models were used for α-synucleinopathy subtype prediction. The models were externally validated using data from a different institution.

RESULTS

A total of 236 iRBD patients were followed up for up to 8 years (mean 3.5 years), and 31 patients converted to α-synucleinopathies (16 PD, 9 DLB, 6 MSA). The best model for survival prediction was the random survival forest model with an integrated Brier score of 0.114 and a concordance index of 0.775. The K-nearest neighbor model was the best model for subtype prediction with an area under the receiver operating characteristic curve of 0.901. Slowing of the EEG was an important feature for both models.

CONCLUSIONS

Machine learning models using baseline EEG features can be used to predict phenoconversion time and its subtype in patients with iRBD. Further research including large sample data from many countries is needed to make a more robust model.

Artificial Intelligence and IBD: Where are We Now and Where Will We Be in the Future?

PURPOSE OF REVIEW

Artificial intelligence (AI) is quickly demonstrating the ability to address problems and challenges in the care of IBD. This review with commentary will highlight today's advancements in AI applications for IBD in image analysis, understanding text, and replicating clinical knowledge and experience.

RECENT FINDINGS

Advancements in machine learning methods, availability of high-performance computing, and increasing digitization of medical data are providing opportunities for AI to assist in IBD care. Multiple groups have demonstrated the ability of AI to replicate expert endoscopic scoring in IBD, with expansion into automated capsule endoscopy, enterography, and histologic interpretations. Further, AI image analysis is being used to develop new endoscopic scoring with more granularity and detail than is possible using conventional methods. Advancements in natural language processing are proving to reduce laborious tasks required in the care of IBD, including documentation, information searches, and chart review. Finally, large language models and chatbots that can understand language and generate human-like replies are beginning to exhibit clinical intelligence that will revolutionize how we deliver IBD care. Today, AI is being deployed to replicate expert judgement in specific tasks where disagreement, subjectivity, and bias are common. However, the near future will herald contributions of AI doing what we cannot, including new detailed measures of IBD, enhanced analysis of images, and perhaps even fully automating care. As we speculate on future technologic capabilities that may improve how we care for IBD, this review will also consider how we will implement and fairly use AI in practice.

Use of Machine Learning for Early Detection of Maternal Cardiovascular Conditions: Retrospective Study Using Electronic Health Record Data

BACKGROUND

Cardiovascular conditions (eg, cardiac and coronary conditions, hypertensive disorders of pregnancy, and cardiomyopathies) were the leading cause of maternal mortality between 2017 and 2019. The United States has the highest maternal mortality rate of any high-income nation, disproportionately impacting those who identify as non-Hispanic Black or Hispanic. Novel clinical approaches to the detection and diagnosis of cardiovascular conditions are therefore imperative. Emerging research is demonstrating that machine learning (ML) is a promising tool for detecting patients at increased risk for hypertensive disorders during pregnancy. However, additional studies are required to determine how integrating ML and big data, such as electronic health records (EHRs), can improve the identification of obstetric patients at higher risk of cardiovascular conditions.

OBJECTIVE

This study aimed to evaluate the capability and timing of a proprietary ML algorithm, Healthy Outcomes for all Pregnancy Experiences-Cardiovascular-Risk Assessment Technology (HOPE-CAT), to detect maternal-related cardiovascular conditions and outcomes.

METHODS

Retrospective data from the EHRs of a large health care system were investigated by HOPE-CAT in a virtual server environment. Deidentification of EHR data and standardization enabled HOPE-CAT to analyze data without pre-existing biases. The ML algorithm assessed risk factors selected by clinical experts in cardio-obstetrics, and the algorithm was iteratively trained using relevant literature and current standards of risk identification. After refinement of the algorithm's learned risk factors, risk profiles were generated for every patient including a designation of standard versus high risk. The profiles were individually paired with clinical outcomes pertaining to cardiovascular pregnancy conditions and complications, wherein a delta was calculated between the date of the risk profile and the actual diagnosis or intervention in the EHR.

RESULTS

In total, 604 pregnancies resulting in birth had records or diagnoses that could be compared against the risk profile; the majority of patients identified as Black (n=482, 79.8%) and aged between 21 and 34 years (n=509, 84.4%). Preeclampsia (n=547, 90.6%) was the most common condition, followed by thromboembolism (n=16, 2.7%) and acute kidney disease or failure (n=13, 2.2%). The average delta was 56.8 (SD 69.7) days between the identification of risk factors by HOPE-CAT and the first date of diagnosis or intervention of a related condition reported in the EHR. HOPE-CAT showed the strongest performance in early risk detection of myocardial infarction at a delta of 65.7 (SD 81.4) days.

CONCLUSIONS

This study provides additional evidence to support ML in obstetrical patients to enhance the early detection of cardiovascular conditions during pregnancy. ML can synthesize multiday patient presentations to enhance provider decision-making and potentially reduce maternal health disparities.

A Real-Time Automated Machine Learning Algorithm for Predicting Mortality in Trauma Patients: Survey Says it's Ready for Prime-Time

BACKGROUND

Though artificial intelligence ("AI") has been increasingly applied to patient care, many of these predictive models are retrospective and not readily available for real-time decision-making. This survey-based study aims to evaluate implementation of a new, validated mortality risk calculator (Parkland Trauma Index of Mortality, "PTIM") embedded in our electronic healthrecord ("EHR") that calculates hourly predictions of mortality with high sensitivity and specificity.

METHODS

This is a prospective, survey-based study performed at a level 1 trauma center. An anonymous survey was sent to surgical providers and regarding PTIM implementation. The PTIM score evaluates 23 variables including Glasgow Coma Score (GCS), vital signs, and laboratory data.

RESULTS

Of the 40 completed surveys, 35 reported using PTIM in decision-making. Prior to reviewing PTIM, providers identified perceived top 3 predictors of mortality, including GCS (22/38, 58%), age (18/35, 47%), and maximum heart rate (17/35, 45%). Most providers reported the PTIM assisted their treatment decisions (27/35, 77%) and timing of operative intervention (23/35, 66%). Many providers agreed that PTIM integrated into rounds and patient assessment (22/36, 61%) and that it improved efficiency in assessing patients' potential mortality (21/36, 58%).

CONCLUSIONS

Artificial intelligence algorithms are mostly retrospective and lag in real-time prediction of mortality. To our knowledge, this is the first real-time, automated algorithm predicting mortality in trauma patients. In this small survey-based study, we found PTIM assists in decision-making, timing of intervention, and improves accuracy in assessing mortality. Next steps include evaluating the short- and long-term impact on patient outcomes.

Trust and acceptability of data-driven clinical recommendations in everyday practice: A scoping review

BACKGROUND

Increasing attention is being given to the analysis of large health datasets to derive new clinical decision support systems (CDSS). However, few data-driven CDSS are being adopted into clinical practice. Trust in these tools is believed to be fundamental for acceptance and uptake but to date little attention has been given to defining or evaluating trust in clinical settings.

OBJECTIVES

A scoping review was conducted to explore how and where acceptability and trustworthiness of data-driven CDSS have been assessed from the health professional's perspective.

METHODS

Medline, Embase, PsycInfo, Web of Science, Scopus, ACM Digital, IEEE Xplore and Google Scholar were searched in March 2022 using terms expanded from: "data-driven" AND "clinical decision support" AND "acceptability". Included studies focused on healthcare practitioner-facing data-driven CDSS, relating directly to clinical care. They included trust or a proxy as an outcome, or in the discussion. The preferred reporting items for systematic reviews and meta-analyses extension for scoping reviews (PRISMA-ScR) is followed in the reporting of this review.

RESULTS

3291 papers were screened, with 85 primary research studies eligible for inclusion. Studies covered a diverse range of clinical specialisms and intended contexts, but hypothetical systems (24) outnumbered those in clinical use (18). Twenty-five studies measured trust, via a wide variety of quantitative, qualitative and mixed methods. A further 24 discussed themes of trust without it being explicitly evaluated, and from these, themes of transparency, explainability, and supporting evidence were identified as factors influencing healthcare practitioner trust in data-driven CDSS.

CONCLUSION

There is a growing body of research on data-driven CDSS, but few studies have explored stakeholder perceptions in depth, with limited focused research on trustworthiness. Further research on healthcare practitioner acceptance, including requirements for transparency and explainability, should inform clinical implementation.

Machine learning and deep learning-based approach in smart healthcare: Recent advances, applications, challenges and opportunities

In recent years, machine learning (ML) and deep learning (DL) have been the leading approaches to solving various challenges, such as disease predictions, drug discovery, medical image analysis, etc., in intelligent healthcare applications. Further, given the current progress in the fields of ML and DL, there exists the promising potential for both to provide support in the realm of healthcare. This study offered an exhaustive survey on ML and DL for the healthcare system, concentrating on vital state of the art features, integration benefits, applications, prospects and future guidelines. To conduct the research, we found the most prominent journal and conference databases using distinct keywords to discover scholarly consequences. First, we furnished the most current along with cutting-edge progress in ML-DL-based analysis in smart healthcare in a compendious manner. Next, we integrated the advancement of various services for ML and DL, including ML-healthcare, DL-healthcare, and ML-DL-healthcare. We then offered ML and DL-based applications in the healthcare industry. Eventually, we emphasized the research disputes and recommendations for further studies based on our observations.

Harnessing the potential of data-driven strategies to optimise transfusion practice

No one doubts the significant variation in the practice of transfusion medicine. Common examples are the variability in transfusion thresholds and the use of tranexamic acid for surgery with likely high blood loss despite evidence-based standards. There is a long history of applying different strategies to address this variation, including education, clinical guidelines, audit and feedback, but the effectiveness and cost-effectiveness of these initiatives remains unclear. Advances in computerised decision support systems and the application of novel electronic capabilities offer alternative approaches to improving transfusion practice. In England, the National Institute for Health and Care Research funded a Blood and Transplant Research Unit (BTRU) programme focussing on 'A data-enabled programme of research to improve transfusion practices'. The overarching aim of the BTRU is to accelerate the development of data-driven methods to optimise the use of blood and transfusion alternatives, and to integrate them within routine practice to improve patient outcomes. One particular area of focus is implementation science to address variation in practice.

Artificial intelligence in physical rehabilitation: A systematic review

BACKGROUND

Physical disabilities become more common with advancing age. Rehabilitation restores function, maintaining independence for longer. However, the poor availability and accessibility of rehabilitation limits its clinical impact. Artificial Intelligence (AI) guided interventions have improved many domains of healthcare, but whether rehabilitation can benefit from AI remains unclear.

METHODS

We conducted a systematic review of AI-supported physical rehabilitation technology tested in the clinical setting to understand: 1) availability of AI-supported physical rehabilitation technology; 2) its clinical effect; 3) and the barriers and facilitators to implementation. We searched in MEDLINE, EMBASE, CINAHL, Science Citation Index (Web of Science), CIRRIE (now NARIC), and OpenGrey.

RESULTS

We identified 9054 articles and included 28 projects. AI solutions spanned five categories: App-based systems, robotic devices that replace function, robotic devices that restore function, gaming systems and wearables. We identified five randomised controlled trials (RCTs), which evaluated outcomes relating to physical function, activity, pain, and health-related quality of life. The clinical effects were inconsistent. Implementation barriers included technology literacy, reliability, and user fatigue. Enablers included greater access to rehabilitation programmes, remote monitoring of progress, reduction in manpower requirements and lower cost.

CONCLUSION

Application of AI in physical rehabilitation is a growing field, but clinical effects have yet to be studied rigorously. Developers must strive to conduct robust clinical evaluations in the real-world setting and appraise post implementation experiences.

The applications of machine learning techniques in medical data processing based on distributed computing and the Internet of Things

Medical data processing has grown into a prominent topic in the latest decades with the primary goal of maintaining patient data via new information technologies, including the Internet of Things (IoT) and sensor technologies, which generate patient indexes in hospital data networks. Innovations like distributed computing, Machine Learning (ML), blockchain, chatbots, wearables, and pattern recognition can adequately enable the collection and processing of medical data for decision-making in the healthcare era. Particularly, to assist experts in the disease diagnostic process, distributed computing is beneficial by digesting huge volumes of data swiftly and producing personalized smart suggestions. On the other side, the current globe is confronting an outbreak of COVID-19, so an early diagnosis technique is crucial to lowering the fatality rate. ML systems are beneficial in aiding radiologists in examining the incredible amount of medical images. Nevertheless, they demand a huge quantity of training data that must be unified for processing. Hence, developing Deep Learning (DL) confronts multiple issues, such as conventional data collection, quality assurance, knowledge exchange, privacy preservation, administrative laws, and ethical considerations. In this research, we intend to convey an inclusive analysis of the most recent studies in distributed computing platform applications based on five categorized platforms, including cloud computing, edge, fog, IoT, and hybrid platforms. So, we evaluated 27 articles regarding the usage of the proposed framework, deployed methods, and applications, noting the advantages, drawbacks, and the applied dataset and screening the security mechanism and the presence of the Transfer Learning (TL) method. As a result, it was proved that most recent research (about 43%) used the IoT platform as the environment for the proposed architecture, and most of the studies (about 46%) were done in 2021. In addition, the most popular utilized DL algorithm was the Convolutional Neural Network (CNN), with a percentage of 19.4%. Hence, despite how technology changes, delivering appropriate therapy for patients is the primary aim of healthcare-associated departments. Therefore, further studies are recommended to develop more functional architectures based on DL and distributed environments and better evaluate the present healthcare data analysis models.

Automatic identification of schizophrenia employing EEG records analyzed with deep learning algorithms

Electroencephalography is a method of detecting and analyzing electrical activity in the brain. This electrical activity can be recorded and processed to aid in the clinical diagnosis of mental disorders. In this study, a novel system for classifying schizophrenia patients from EEG recordings is presented. The developed algorithm decomposes the EEG signals into a system of radial basis functions using the method of fuzzy means. This decomposition helps to obtain the information from the various electrodes of the EEG and allows separating between healthy controls and patients with schizophrenia. The proposed method has been compared with classical machine learning algorithms, such as, K-Nearest Neighbor, Adaboost, Support Vector Machine, and Bayesian Linear Discriminant Analysis. The results show that the proposed method obtains the highest values in terms of balanced accuracy, recall, precision and F1 score, close to 93 % in all cases. The model developed in this study can be implemented in brain activity analysis systems that help in the prediction of patients with schizophrenia.

Using Visual Patient to Show Vital Sign Predictions, a Computer-Based Mixed Quantitative and Qualitative Simulation Study

BACKGROUND

Machine learning can analyze vast amounts of data and make predictions for events in the future. Our group created machine learning models for vital sign predictions. To transport the information of these predictions without numbers and numerical values and make them easily usable for human caregivers, we aimed to integrate them into the Philips Visual-Patient-avatar, an avatar-based visualization of patient monitoring.

METHODS

We conducted a computer-based simulation study with 70 participants in 3 European university hospitals. We validated the vital sign prediction visualizations by testing their identification by anesthesiologists and intensivists. Each prediction visualization consisted of a condition (e.g., low blood pressure) and an urgency (a visual indication of the timespan in which the condition is expected to occur). To obtain qualitative user feedback, we also conducted standardized interviews and derived statements that participants later rated in an online survey.

RESULTS

The mixed logistic regression model showed 77.9% (95% CI 73.2-82.0%) correct identification of prediction visualizations (i.e., condition and urgency both correctly identified) and 93.8% (95% CI 93.7-93.8%) for conditions only (i.e., without considering urgencies). A total of 49 out of 70 participants completed the online survey. The online survey participants agreed that the prediction visualizations were fun to use (32/49, 65.3%), and that they could imagine working with them in the future (30/49, 61.2%). They also agreed that identifying the urgencies was difficult (32/49, 65.3%).

CONCLUSIONS

This study found that care providers correctly identified >90% of the conditions (i.e., without considering urgencies). The accuracy of identification decreased when considering urgencies in addition to conditions. Therefore, in future development of the technology, we will focus on either only displaying conditions (without urgencies) or improving the visualizations of urgency to enhance usability for human users.

A Systematic Review of the Barriers to the Implementation of Artificial Intelligence in Healthcare

Artificial intelligence (AI) is expected to improve healthcare outcomes by facilitating early diagnosis, reducing the medical administrative burden, aiding drug development, personalising medical and oncological management, monitoring healthcare parameters on an individual basis, and allowing clinicians to spend more time with their patients. In the post-pandemic world where there is a drive for efficient delivery of healthcare and manage long waiting times for patients to access care, AI has an important role in supporting clinicians and healthcare systems to streamline the care pathways and provide timely and high-quality care for the patients. Despite AI technologies being used in healthcare for some decades, and all the theoretical potential of AI, the uptake in healthcare has been uneven and slower than anticipated and there remain a number of barriers, both overt and covert, which have limited its incorporation. This literature review highlighted barriers in six key areas: ethical, technological, liability and regulatory, workforce, social, and patient safety barriers. Defining and understanding the barriers preventing the acceptance and implementation of AI in the setting of healthcare will enable clinical staff and healthcare leaders to overcome the identified hurdles and incorporate AI technologies for the benefit of patients and clinical staff.

Novel method for predicting nonvisible symptoms using machine learning in cancer palliative care

End-of-life patients with cancer may find expressing their symptoms difficult if they can no longer communicate verbally because of deteriorating health. In this study, we assessed these symptoms using machine learning, which has excellent predictive capabilities and has recently been applied in healthcare. We performed a retrospective clinical survey involving 213 patients with cancer from August 2015 to August 2016. We divided the reported symptoms into two groups-visible and nonvisible symptoms. We used decision tree analysis, an analytical machine learning method that organizes and analyzes information in the form of a tree diagram to visually represent the information structure. Our machine learning model used patient background data and visible symptoms to predict nonvisible symptoms: pain, dyspnea, fatigue, drowsiness, anxiety, delirium, inadequate informed consent, and spiritual issues. The highest and/or lowest values for prediction accuracy, sensitivity, and specificity were 88.0%/55.5%, 84.9%/3.3%, and 96.7%/24.1%, respectively. This work will facilitate better assessment and management of symptoms in patients with cancer. This study was the first to predict nonvisible symptoms using decision tree analyses for patients with cancer receiving palliative care. Notably, applications based on our results may assess symptoms to the same extent as healthcare professionals.

Developing and Validating a Prediction Model For Death or Critical Illness in Hospitalized Adults, an Opportunity for Human-Computer Collaboration

Hospital early warning systems that use machine learning (ML) to predict clinical deterioration are increasingly being used to aid clinical decision-making. However, it is not known how ML predictions complement physician and nurse judgment. Our objective was to train and validate a ML model to predict patient deterioration and compare model predictions with real-world physician and nurse predictions.

DESIGN

Retrospective and prospective cohort study.

SETTING

Academic tertiary care hospital.

PATIENTS

Adult general internal medicine hospitalizations.

MEASUREMENTS AND MAIN RESULTS

We developed and validated a neural network model to predict in-hospital death and ICU admission in 23,528 hospitalizations between April 2011 and April 2019. We then compared model predictions with 3,374 prospectively collected predictions from nurses, residents, and attending physicians about their own patients in 960 hospitalizations between April 30, and August 28, 2019. ML model predictions achieved clinician-level accuracy for predicting ICU admission or death (ML median F1 score 0.32 [interquartile range (IQR) 0.30-0.34], AUC 0.77 [IQ 0.76-0.78]; clinicians median F1-score 0.33 [IQR 0.30-0.35], AUC 0.64 [IQR 0.63-0.66]). ML predictions were more accurate than clinicians for ICU admission. Of all ICU admissions and deaths, 36% occurred in hospitalizations where the model and clinicians disagreed. Combining human and model predictions detected 49% of clinical deterioration events, improving sensitivity by 16% compared with clinicians alone and 24% compared with the model alone while maintaining a positive predictive value of 33%, thus keeping false alarms at a clinically acceptable level.

CONCLUSIONS

ML models can complement clinician judgment to predict clinical deterioration in hospital. These findings demonstrate important opportunities for human-computer collaboration to improve prognostication and personalized medicine in hospital.

The Deterioration Risk Index: Developing and Piloting a Machine Learning Algorithm to Reduce Pediatric Inpatient Deterioration

OBJECTIVES

Develop and deploy a disease cohort-based machine learning algorithm for timely identification of hospitalized pediatric patients at risk for clinical deterioration that outperforms our existing situational awareness program.

DESIGN

Retrospective cohort study.

SETTING

Nationwide Children's Hospital, a freestanding, quaternary-care, academic children's hospital in Columbus, OH.

PATIENTS

All patients admitted to inpatient units participating in the preexisting situational awareness program from October 20, 2015, to December 31, 2019, excluding patients over 18 years old at admission and those with a neonatal ICU stay during their hospitalization.

INTERVENTIONS

We developed separate algorithms for cardiac, malignancy, and general cohorts via lasso-regularized logistic regression. Candidate model predictors included vital signs, supplemental oxygen, nursing assessments, early warning scores, diagnoses, lab results, and situational awareness criteria. Model performance was characterized in clinical terms and compared with our previous situational awareness program based on a novel retrospective validation approach. Simulations with frontline staff, prior to clinical implementation, informed user experience and refined interdisciplinary workflows. Model implementation was piloted on cardiology and hospital medicine units in early 2021.

MEASUREMENTS AND MAIN RESULTS

The Deterioration Risk Index (DRI) was 2.4 times as sensitive as our existing situational awareness program (sensitivities of 53% and 22%, respectively; p < 0.001) and required 2.3 times fewer alarms per detected event (121 DRI alarms per detected event vs 276 for existing program). Notable improvements were a four-fold sensitivity gain for the cardiac diagnostic cohort (73% vs 18%; p < 0.001) and a three-fold gain (81% vs 27%; p < 0.001) for the malignancy diagnostic cohort. Postimplementation pilot results over 18 months revealed a 77% reduction in deterioration events (three events observed vs 13.1 expected, p = 0.001).

CONCLUSIONS

The etiology of pediatric inpatient deterioration requires acknowledgement of the unique pathophysiology among cardiology and oncology patients. Selection and weighting of diverse candidate risk factors via machine learning can produce a more sensitive early warning system for clinical deterioration. Leveraging preexisting situational awareness platforms and accounting for operational impacts of model implementation are key aspects to successful bedside translation.

Predicting Social Determinants of Health in Patient Navigation: Case Study

BACKGROUND

Patient navigation (PN) programs have demonstrated efficacy in improving health outcomes for marginalized populations across a range of clinical contexts by addressing barriers to health care, including social determinants of health (SDoHs). However, it can be challenging for navigators to identify SDoHs by asking patients directly because of many factors, including patients' reluctance to disclose information, communication barriers, and the variable resources and experience levels of patient navigators. Navigators could benefit from strategies that augment their ability to gather SDoH data. Machine learning can be leveraged as one of these strategies to identify SDoH-related barriers. This could further improve health outcomes, particularly in underserved populations.

OBJECTIVE

In this formative study, we explored novel machine learning-based approaches to predict SDoHs in 2 Chicago area PN studies. In the first approach, we applied machine learning to data that include comments and interaction details between patients and navigators, whereas the second approach augmented patients' demographic information. This paper presents the results of these experiments and provides recommendations for data collection and the application of machine learning techniques more generally to the problem of predicting SDoHs.

METHODS

We conducted 2 experiments to explore the feasibility of using machine learning to predict patients' SDoHs using data collected from PN research. The machine learning algorithms were trained on data collected from 2 Chicago area PN studies. In the first experiment, we compared several machine learning algorithms (logistic regression, random forest, support vector machine, artificial neural network, and Gaussian naive Bayes) to predict SDoHs from both patient demographics and navigator's encounter data over time. In the second experiment, we used multiclass classification with augmented information, such as transportation time to a hospital, to predict multiple SDoHs for each patient.

RESULTS

In the first experiment, the random forest classifier achieved the highest accuracy among the classifiers tested. The overall accuracy to predict SDoHs was 71.3%. In the second experiment, multiclass classification effectively predicted a few patients' SDoHs based purely on demographic and augmented data. The best accuracy of these predictions overall was 73%. However, both experiments yielded high variability in individual SDoH predictions and correlations that become salient among SDoHs.

CONCLUSIONS

To our knowledge, this study is the first approach to applying PN encounter data and multiclass learning algorithms to predict SDoHs. The experiments discussed yielded valuable lessons, including the awareness of model limitations and bias, planning for standardization of data sources and measurement, and the need to identify and anticipate the intersectionality and clustering of SDoHs. Although our focus was on predicting patients' SDoHs, machine learning can have a broad range of applications in the field of PN, from tailoring intervention delivery (eg, supporting PN decision-making) to informing resource allocation for measurement, and PN supervision.

Artificial intelligence in colposcopic examination: A promising tool to assist junior colposcopists

Introduction

Well-trained colposcopists are in huge shortage worldwide, especially in low-resource areas. Here, we aimed to evaluate the Colposcopic Artificial Intelligence Auxiliary Diagnostic System (CAIADS) to detect abnormalities based on digital colposcopy images, especially focusing on its role in assisting junior colposcopist to correctly identify the lesion areas where biopsy should be performed.

Materials and methods

This is a hospital-based retrospective study, which recruited the women who visited colposcopy clinics between September 2021 to January 2022. A total of 366 of 1,146 women with complete medical information recorded by a senior colposcopist and valid histology results were included. Anonymized colposcopy images were reviewed by CAIADS and a junior colposcopist separately, and the junior colposcopist reviewed the colposcopy images with CAIADS results (named CAIADS-Junior). The diagnostic accuracy and biopsy efficiency of CAIADS and CAIADS-Junior were assessed in detecting cervical intraepithelial neoplasia grade 2 or worse (CIN2+), CIN3+, and cancer in comparison with the senior and junior colposcipists. The factors influencing the accuracy of CAIADS were explored.

Results

For CIN2 + and CIN3 + detection, CAIADS showed a sensitivity at ~80%, which was not significantly lower than the sensitivity achieved by the senior colposcopist (for CIN2 +: 80.6 vs. 91.3%, p = 0.061 and for CIN3 +: 80.0 vs. 90.0%, p = 0.189). The sensitivity of the junior colposcopist was increased significantly with the assistance of CAIADS (for CIN2 +: 95.1 vs. 79.6%, p = 0.002 and for CIN3 +: 97.1 vs. 85.7%, p = 0.039) and was comparable to those of the senior colposcopists (for CIN2 +: 95.1 vs. 91.3%, p = 0.388 and for CIN3 +: 97.1 vs. 90.0%, p = 0.125). In detecting cervical cancer, CAIADS achieved the highest sensitivity at 100%. For all endpoints, CAIADS showed the highest specificity (55–64%) and positive predictive values compared to both senior and junior colposcopists. When CIN grades became higher, the average biopsy numbers decreased for the subspecialists and CAIADS required a minimum number of biopsies to detect per case (2.2–2.6 cut-points). Meanwhile, the biopsy sensitivity of the junior colposcopist was the lowest, but the CAIADS-assisted junior colposcopist achieved a higher biopsy sensitivity.

Conclusion

Colposcopic Artificial Intelligence Auxiliary Diagnostic System could assist junior colposcopists to improve diagnostic accuracy and biopsy efficiency, which might be a promising solution to improve the quality of cervical cancer screening in low-resource settings.

Artificial intelligence as a diagnostic aid in cross-sectional radiological imaging of surgical pathology in the abdominopelvic cavity: a systematic review

OBJECTIVES

There is emerging use of artificial intelligence (AI) models to aid diagnostic imaging. This review examined and critically appraised the application of AI models to identify surgical pathology from radiological images of the abdominopelvic cavity, to identify current limitations and inform future research.

DESIGN

Systematic review.

DATA SOURCES

Systematic database searches (Medline, EMBASE, Cochrane Central Register of Controlled Trials) were performed. Date limitations (January 2012 to July 2021) were applied.

ELIGIBILITY CRITERIA

Primary research studies were considered for eligibility using the PIRT (participants, index test(s), reference standard and target condition) framework. Only publications in the English language were eligible for inclusion in the review.

DATA EXTRACTION AND SYNTHESIS

Study characteristics, descriptions of AI models and outcomes assessing diagnostic performance were extracted by independent reviewers. A narrative synthesis was performed in accordance with the Synthesis Without Meta-analysis guidelines. Risk of bias was assessed (Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2)).

RESULTS

Fifteen retrospective studies were included. Studies were diverse in surgical specialty, the intention of the AI applications and the models used. AI training and test sets comprised a median of 130 (range: 5-2440) and 37 (range: 10-1045) patients, respectively. Diagnostic performance of models varied (range: 70%-95% sensitivity, 53%-98% specificity). Only four studies compared the AI model with human performance. Reporting of studies was unstandardised and often lacking in detail. Most studies (n=14) were judged as having overall high risk of bias with concerns regarding applicability.

CONCLUSIONS

AI application in this field is diverse. Adherence to reporting guidelines is warranted. With finite healthcare resources, future endeavours may benefit from targeting areas where radiological expertise is in high demand to provide greater efficiency in clinical care. Translation to clinical practice and adoption of a multidisciplinary approach should be of high priority.

PROSPERO REGISTRATION NUMBER

CRD42021237249.

Machine Learning Models Using Routinely Collected Clinical Data Offer Robust and Interpretable Predictions of 90-Day Unplanned Acute Care Use for Cancer Immunotherapy Patients

PURPOSE

Clinical management of patients receiving immune checkpoint inhibitors (ICIs) could be informed using accurate predictive tools to identify patients at risk of short-term acute care utilization (ACU). We used routinely collected data to develop and assess machine learning (ML) algorithms to predict unplanned ACU within 90 days of ICI treatment initiation.

METHODS

We used aggregated electronic health record data from 7,960 patients receiving ICI treatments to train and assess eight ML algorithms. We developed the models using pre-SARS-COV-19 COVID-19 data generated between January 2016 and February 2020. We validated our algorithms using data collected between March 2020 and June 2022 (peri-COVID-19 sample). We assessed performance using area under the receiver operating characteristic curves (AUROC), sensitivity, specificity, and calibration plots. We derived intuitive explanations of predictions using variable importance and Shapley additive explanation analyses. We assessed the marginal performance of ML models compared with that of univariate and multivariate logistic regression (LR) models.

RESULTS

Most algorithms significantly outperformed the univariate and multivariate LR models. The extreme gradient boosting trees (XGBT) algorithm demonstrated the best overall performance (AUROC, 0.70; sensitivity, 0.53; specificity, 0.74) on the peri-COVID-19 sample. The algorithm performance was stable across both pre- and peri-COVID-19 samples, as well as ICI regimen and cancer groups. Type of ICI agents, oxygen saturation, diastolic blood pressure, albumin level, platelet count, immature granulocytes, absolute monocyte, chloride level, red cell distribution width, and alcohol intake were the top 10 key predictors used by the XGBT algorithm.

CONCLUSION

Machine learning algorithms trained using routinely collected data outperformed traditional statistical models when predicting 90-day ACU. The XGBT algorithm has the potential to identify high-ACU risk patients and enable preventive interventions to avoid ACU.

Machine Learning Augmented Interpretation of Chest X-rays: A Systematic Review

Limitations of the chest X-ray (CXR) have resulted in attempts to create machine learning systems to assist clinicians and improve interpretation accuracy. An understanding of the capabilities and limitations of modern machine learning systems is necessary for clinicians as these tools begin to permeate practice. This systematic review aimed to provide an overview of machine learning applications designed to facilitate CXR interpretation. A systematic search strategy was executed to identify research into machine learning algorithms capable of detecting >2 radiographic findings on CXRs published between January 2020 and September 2022. Model details and study characteristics, including risk of bias and quality, were summarized. Initially, 2248 articles were retrieved, with 46 included in the final review. Published models demonstrated strong standalone performance and were typically as accurate, or more accurate, than radiologists or non-radiologist clinicians. Multiple studies demonstrated an improvement in the clinical finding classification performance of clinicians when models acted as a diagnostic assistance device. Device performance was compared with that of clinicians in 30% of studies, while effects on clinical perception and diagnosis were evaluated in 19%. Only one study was prospectively run. On average, 128,662 images were used to train and validate models. Most classified less than eight clinical findings, while the three most comprehensive models classified 54, 72, and 124 findings. This review suggests that machine learning devices designed to facilitate CXR interpretation perform strongly, improve the detection performance of clinicians, and improve the efficiency of radiology workflow. Several limitations were identified, and clinician involvement and expertise will be key to driving the safe implementation of quality CXR machine learning systems.

The Use and Structure of Emergency Nurses' Triage Narrative Data: Scoping Review

BACKGROUND

Emergency departments use triage to ensure that patients with the highest level of acuity receive care quickly and safely. Triage is typically a nursing process that is documented as structured and unstructured (free text) data. Free-text triage narratives have been studied for specific conditions but never reviewed in a comprehensive manner.

OBJECTIVE

The objective of this paper was to identify and map the academic literature that examines triage narratives. The paper described the types of research conducted, identified gaps in the research, and determined where additional review may be warranted.

METHODS

We conducted a scoping review of unstructured triage narratives. We mapped the literature, described the use of triage narrative data, examined the information available on the form and structure of narratives, highlighted similarities among publications, and identified opportunities for future research.

RESULTS

We screened 18,074 studies published between 1990 and 2022 in CINAHL, MEDLINE, Embase, Cochrane, and ProQuest Central. We identified 0.53% (96/18,074) of studies that directly examined the use of triage nurses' narratives. More than 12 million visits were made to 2438 emergency departments included in the review. In total, 82% (79/96) of these studies were conducted in the United States (43/96, 45%), Australia (31/96, 32%), or Canada (5/96, 5%). Triage narratives were used for research and case identification, as input variables for predictive modeling, and for quality improvement. Overall, 31% (30/96) of the studies offered a description of the triage narrative, including a list of the keywords used (27/96, 28%) or more fulsome descriptions (such as word counts, character counts, abbreviation, etc; 7/96, 7%). We found limited use of reporting guidelines (8/96, 8%).

CONCLUSIONS

The breadth of the identified studies suggests that there is widespread routine collection and research use of triage narrative data. Despite the use of triage narratives as a source of data in studies, the narratives and nurses who generate them are poorly described in the literature, and data reporting is inconsistent. Additional research is needed to describe the structure of triage narratives, determine the best use of triage narratives, and improve the consistent use of triage-specific data reporting guidelines.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.1136/bmjopen-2021-055132.

Federated Learning in Health care Using Structured Medical Data

The success of machine learning-based studies is largely subjected to accessing a large amount of data. However, accessing such data is typically not feasible within a single health system/hospital. Although multicenter studies are the most effective way to access a vast amount of data, sharing data outside the institutes involves legal, business, and technical challenges. Federated learning (FL) is a newly proposed machine learning framework for multicenter studies, tackling data-sharing issues across participant institutes. The promise of FL is simple. FL facilitates multicenter studies without losing data access control and allows the construction of a global model by aggregating local models trained from participant institutes. This article reviewed recently published studies that utilized FL in clinical studies with structured medical data. In addition, challenges and open questions in FL in clinical studies with structured medical data were discussed.

Detecting 3D syndromic faces as outliers using unsupervised normalizing flow models

Many genetic syndromes are associated with distinctive facial features. Several computer-assisted methods have been proposed that make use of facial features for syndrome diagnosis. Training supervised classifiers, the most common approach for this purpose, requires large, comprehensive, and difficult to collect databases of syndromic facial images. In this work, we use unsupervised, normalizing flow-based manifold and density estimation models trained entirely on unaffected subjects to detect syndromic 3D faces as statistical outliers. Furthermore, we demonstrate a general, user-friendly, gradient-based interpretability mechanism that enables clinicians and patients to understand model inferences. 3D facial surface scans of 2471 unaffected subjects and 1629 syndromic subjects representing 262 different genetic syndromes were used to train and evaluate the models. The flow-based models outperformed unsupervised comparison methods, with the best model achieving an ROC-AUC of 86.3% on a challenging, age and sex diverse data set. In addition to highlighting the viability of outlier-based syndrome screening tools, our methods generalize and extend previously proposed outlier scores for 3D face-based syndrome detection, resulting in improved performance for unsupervised syndrome detection.

Diagnostic accuracy of artificial intelligence for detecting gastrointestinal luminal pathologies: A systematic review and meta-analysis

Background

Artificial Intelligence (AI) holds considerable promise for diagnostics in the field of gastroenterology. This systematic review and meta-analysis aims to assess the diagnostic accuracy of AI models compared with the gold standard of experts and histopathology for the diagnosis of various gastrointestinal (GI) luminal pathologies including polyps, neoplasms, and inflammatory bowel disease.

Methods

We searched PubMed, CINAHL, Wiley Cochrane Library, and Web of Science electronic databases to identify studies assessing the diagnostic performance of AI models for GI luminal pathologies. We extracted binary diagnostic accuracy data and constructed contingency tables to derive the outcomes of interest: sensitivity and specificity. We performed a meta-analysis and hierarchical summary receiver operating characteristic curves (HSROC). The risk of bias was assessed using Quality Assessment for Diagnostic Accuracy Studies-2 (QUADAS-2) tool. Subgroup analyses were conducted based on the type of GI luminal disease, AI model, reference standard, and type of data used for analysis. This study is registered with PROSPERO (CRD42021288360).

Findings

We included 73 studies, of which 31 were externally validated and provided sufficient information for inclusion in the meta-analysis. The overall sensitivity of AI for detecting GI luminal pathologies was 91.9% (95% CI: 89.0–94.1) and specificity was 91.7% (95% CI: 87.4–94.7). Deep learning models (sensitivity: 89.8%, specificity: 91.9%) and ensemble methods (sensitivity: 95.4%, specificity: 90.9%) were the most commonly used models in the included studies. Majority of studies (n = 56, 76.7%) had a high risk of selection bias while 74% (n = 54) studies were low risk on reference standard and 67% (n = 49) were low risk for flow and timing bias.

Interpretation

The review suggests high sensitivity and specificity of AI models for the detection of GI luminal pathologies. There is a need for large, multi-center trials in both high income countries and low- and middle- income countries to assess the performance of these AI models in real clinical settings and its impact on diagnosis and prognosis.

Systematic review registration

[https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=288360], identifier [CRD42021288360].

From compute to care: Lessons learned from deploying an early warning system into clinical practice

Background

Deploying safe and effective machine learning models is essential to realize the promise of artificial intelligence for improved healthcare. Yet, there remains a large gap between the number of high-performing ML models trained on healthcare data and the actual deployment of these models. Here, we describe the deployment of CHARTwatch, an artificial intelligence-based early warning system designed to predict patient risk of clinical deterioration.

Methods

We describe the end-to-end infrastructure that was developed to deploy CHARTwatch and outline the process from data extraction to communicating patient risk scores in real-time to physicians and nurses. We then describe the various challenges that were faced in deployment, including technical issues (e.g., unstable database connections), process-related challenges (e.g., changes in how a critical lab is measured), and challenges related to deploying a clinical system in the middle of a pandemic. We report various measures to quantify the success of the deployment: model performance, adherence to workflows, and infrastructure uptime/downtime. Ultimately, success is driven by end-user adoption and impact on relevant clinical outcomes. We assess our deployment process by evaluating how closely we followed existing guidance for good machine learning practice (GMLP) and identify gaps that are not addressed in this guidance.

Results

The model demonstrated strong and consistent performance in real-time in the first 19 months after deployment (AUC 0.76) as in the silent deployment heldout test data (AUC 0.79). The infrastructure remained online for >99% of time in the first year of deployment. Our deployment adhered to all 10 aspects of GMLP guiding principles. Several steps were crucial for deployment but are not mentioned or are missing details in the GMLP principles, including the need for a silent testing period, the creation of robust downtime protocols, and the importance of end-user engagement. Evaluation for impacts on clinical outcomes and adherence to clinical protocols is underway.

Conclusion

We deployed an artificial intelligence-based early warning system to predict clinical deterioration in hospital. Careful attention to data infrastructure, identifying problems in a silent testing period, close monitoring during deployment, and strong engagement with end-users were critical for successful deployment.

Randomized Clinical Trials of Machine Learning Interventions in Health Care: A Systematic Review

IMPORTANCE

Despite the potential of machine learning to improve multiple aspects of patient care, barriers to clinical adoption remain. Randomized clinical trials (RCTs) are often a prerequisite to large-scale clinical adoption of an intervention, and important questions remain regarding how machine learning interventions are being incorporated into clinical trials in health care.

OBJECTIVE

To systematically examine the design, reporting standards, risk of bias, and inclusivity of RCTs for medical machine learning interventions.

EVIDENCE REVIEW

In this systematic review, the Cochrane Library, Google Scholar, Ovid Embase, Ovid MEDLINE, PubMed, Scopus, and Web of Science Core Collection online databases were searched and citation chasing was done to find relevant articles published from the inception of each database to October 15, 2021. Search terms for machine learning, clinical decision-making, and RCTs were used. Exclusion criteria included implementation of a non-RCT design, absence of original data, and evaluation of nonclinical interventions. Data were extracted from published articles. Trial characteristics, including primary intervention, demographics, adherence to the CONSORT-AI reporting guideline, and Cochrane risk of bias were analyzed.

FINDINGS

Literature search yielded 19 737 articles, of which 41 RCTs involved a median of 294 participants (range, 17-2488 participants). A total of 16 RCTS (39%) were published in 2021, 21 (51%) were conducted at single sites, and 15 (37%) involved endoscopy. No trials adhered to all CONSORT-AI standards. Common reasons for nonadherence were not assessing poor-quality or unavailable input data (38 trials [93%]), not analyzing performance errors (38 [93%]), and not including a statement regarding code or algorithm availability (37 [90%]). Overall risk of bias was high in 7 trials (17%). Of 11 trials (27%) that reported race and ethnicity data, the median proportion of participants from underrepresented minority groups was 21% (range, 0%-51%).

CONCLUSIONS AND RELEVANCE

This systematic review found that despite the large number of medical machine learning-based algorithms in development, few RCTs for these technologies have been conducted. Among published RCTs, there was high variability in adherence to reporting standards and risk of bias and a lack of participants from underrepresented minority groups. These findings merit attention and should be considered in future RCT design and reporting.

A machine learning framework supporting prospective clinical decisions applied to risk prediction in oncology

We present a general framework for developing a machine learning (ML) tool that supports clinician assessment of patient risk using electronic health record-derived real-world data and apply the framework to a quality improvement use case in an oncology setting to identify patients at risk for a near-term (60 day) emergency department (ED) visit who could potentially be eligible for a home-based acute care program. Framework steps include defining clinical quality improvement goals, model development and validation, bias assessment, retrospective and prospective validation, and deployment in clinical workflow. In the retrospective analysis for the use case, 8% of patient encounters were associated with a high risk (pre-defined as predicted probability ≥20%) for a near-term ED visit by the patient. Positive predictive value (PPV) and negative predictive value (NPV) for future ED events was 26% and 91%, respectively. Odds ratio (OR) of ED visit (high- vs. low-risk) was 3.5 (95% CI: 3.4–3.5). The model appeared to be calibrated across racial, gender, and ethnic groups. In the prospective analysis, 10% of patients were classified as high risk, 76% of whom were confirmed by clinicians as eligible for home-based acute care. PPV and NPV for future ED events was 22% and 95%, respectively. OR of ED visit (high- vs. low-risk) was 5.4 (95% CI: 2.6–11.0). The proposed framework for an ML-based tool that supports clinician assessment of patient risk is a stepwise development approach; we successfully applied the framework to an ED visit risk prediction use case.

Six Drivers to Face the XXI Century Challenges and Build the New Healthcare System: "La Salute in Movimento" Manifesto

The aging of the population, the burden of chronic diseases, possible new pandemics are among the challenges for healthcare in the XXI century. To face them, technological innovations and the national recovery and resilience plan within the European Union can represent opportunities to implement changes and renovate the current healthcare system in Italy, in an effort to guarantee equal access to health services. Considering such scenario, a panel of Italian experts gathered in a multidisciplinary Think Tank to discuss possible design of concepts at the basis of a new healthcare system. These ideas were summarized in a manifesto with six drivers for change: vision, governance, competence, intelligence, humanity and relationship. Each driver was linked to an action to actively move toward a new healthcare system based on trust between science, citizens and institutions.

Artificial Intelligence Based on Machine Learning in Pharmacovigilance: A Scoping Review

INTRODUCTION

Artificial intelligence based on machine learning has made large advancements in many fields of science and medicine but its impact on pharmacovigilance is yet unclear.

OBJECTIVE

The present study conducted a scoping review of the use of artificial intelligence based on machine learning to understand how it is used for pharmacovigilance tasks, characterize differences with other fields, and identify opportunities to improve pharmacovigilance through the use of machine learning.

DESIGN

The PubMed, Embase, Web of Science, and IEEE Xplore databases were searched to identify articles pertaining to the use of machine learning in pharmacovigilance published from the year 2000 to September 2021. After manual screening of 7744 abstracts, a total of 393 papers met the inclusion criteria for further analysis. Extraction of key data on study design, data sources, sample size, and machine learning methodology was performed. Studies with the characteristics of good machine learning practice were defined and manual review focused on identifying studies that fulfilled these criteria and results that showed promise.

RESULTS

The majority of studies (53%) were focused on detecting safety signals using traditional statistical methods. Of the studies that used more recent machine learning methods, 61% used off-the-shelf techniques with minor modifications. Temporal analysis revealed that newer methods such as deep learning have shown increased use in recent years. We found only 42 studies (10%) that reflect current best practices and trends in machine learning. In the subset of 154 papers that focused on data intake and ingestion, 30 (19%) were found to incorporate the same best practices.

CONCLUSION

Advances from artificial intelligence have yet to fully penetrate pharmacovigilance, although recent studies show signs that this may be changing.

Prediction of Hidden Coronary Artery Disease Using Machine Learning in Patients With Acute Ischemic Stroke

BACKGROUND AND OBJECTIVES

A machine learning technique for identifying hidden coronary artery disease (CAD) might be useful. We developed and validated machine learning models to predict patients with hidden CAD and assess long-term outcomes in patients with acute ischemic stroke.

METHODS

Multidetector coronary computed tomography was performed for patients without known history of CAD. Primary outcomes were defined as having any degree of CAD and having obstructive CAD (≥50% stenosis). Demographic variables, risk factors, laboratory results, Trial of ORG 10172 in Acute Stroke Treatment (TOAST) classification, NIH Stroke Scale score, blood pressure, and carotid artery stenosis were used to develop and validate machine learning models to predict CAD. Area under the receiver operating characteristic curves (AUC) was calculated for performance analysis, and Kaplan-Meier and Cox survival analyses of long-term outcomes were performed. Major adverse cardiovascular events (MACE) were defined as ischemic stroke, myocardial infarction, unstable angina, urgent coronary revascularization, and cardiovascular mortality.

RESULTS

Overall, 1,710 patients were included for the training dataset and 348 patients for the validation dataset. An Extreme Gradient Boosting model was developed to predict any degree of CAD, which showed an AUC of 0.763 (95% CI 0.711-0.814) on validation. A logistic regression model was used to predict obstructive CAD and had an AUC of 0.714 (95% CI 0.692-0.799). During the first 5 years of follow-up, MACE occurred more frequently when predicted of any CAD (P = 0.022) or obstructive CAD (P < 0.001). Cox proportional analysis showed that the hazard ratio of MACE was 1.5 (95% CI 1.1-2.2; P = 0.016) when predicted of any CAD, whereas it was 1.9 (95% CI 1.3-2.6; P < 0.001) for obstructive CAD.

DISCUSSION

We demonstrated that machine learning may help identify hidden CAD in patients with acute ischemic stroke. Long-term outcomes were also associated with prediction results.

CLASSIFICATION OF EVIDENCE

This study provides Class II evidence that in patients with acute ischemic stroke with CAD risk factors but no known history of CAD, a machine learning model predicts CAD on multidetector coronary computed tomography with an AUC of 0.763 (95% CI 0.711-0.814).

The Use of Artificial Intelligence in Complementary and Alternative Medicine: A Systematic Scoping Review

Background: The development of artificial intelligence (AI) in the medical field has been growing rapidly. As AI models have been introduced in complementary and alternative medicine (CAM), a systematized review must be performed to understand its current status.

Objective: To categorize and seek the current usage of AI in CAM.

Method: A systematic scoping review was conducted based on the method proposed by the Joanna Briggs Institute. The three databases, PubMed, Embase, and Cochrane Library, were used to find studies regarding AI and CAM. Only English studies from 2000 were included. Studies without mentioning either AI techniques or CAM modalities were excluded along with the non-peer-reviewed studies. A broad-range search strategy was applied to locate all relevant studies.

Results: A total of 32 studies were identified, and three main categories were revealed: 1) acupuncture treatment, 2) tongue and lip diagnoses, and 3) herbal medicine. Other CAM modalities were music therapy, meditation, pulse diagnosis, and TCM syndromes. The majority of the studies utilized AI models to predict certain patterns and find reliable computerized models to assist physicians.

Conclusion: Although the results from this review have shown the potential use of AI models in CAM, future research ought to focus on verifying and validating the models by performing a large-scale clinical trial to better promote AI in CAM in the era of digital health.

Machine learning and artificial intelligence in research and healthcare✰,✰✰

Artificial intelligence (AI) is a broad term referring to the application of computational algorithms that can analyze large data sets to classify, predict, or gain useful conclusions. Under the umbrella of AI is machine learning (ML). ML is the process of building or learning statistical models using previously observed real world data to predict outcomes, or categorize observations based on 'training' provided by humans. These predictions are then applied to future data, all the while folding in the new data into its perpetually improving and calibrated statistical model. The future of AI and ML in healthcare research is exciting and expansive. AI and ML are becoming cornerstones in the medical and healthcare-research domains and are integral in our continued processing and capitalization of robust patient EMR data. Considerations for the use and application of ML in healthcare settings include assessing the quality of data inputs and decision-making that serve as the foundations of the ML model, ensuring the end-product is interpretable, transparent, and ethical concerns are considered throughout the development process. The current and future applications of ML include improving the quality and quantity of data collected from EMRs to improve registry data, utilizing these robust datasets to improve and standardized research protocols and outcomes, clinical decision-making applications, natural language processing and improving the fundamentals of value-based care, to name only a few.

Governing Data and Artificial Intelligence for Health Care: Developing an International Understanding

Background

Although advanced analytical techniques falling under the umbrella heading of artificial intelligence (AI) may improve health care, the use of AI in health raises safety and ethical concerns. There are currently no internationally recognized governance mechanisms (policies, ethical standards, evaluation, and regulation) for developing and using AI technologies in health care. A lack of international consensus creates technical and social barriers to the use of health AI while potentially hampering market competition.

Objective

The aim of this study is to review current health data and AI governance mechanisms being developed or used by Global Digital Health Partnership (GDHP) member countries that commissioned this research, identify commonalities and gaps in approaches, identify examples of best practices, and understand the rationale for policies.

Methods

Data were collected through a scoping review of academic literature and a thematic analysis of policy documents published by selected GDHP member countries. The findings from this data collection and the literature were used to inform semistructured interviews with key senior policy makers from GDHP member countries exploring their countries’ experience of AI-driven technologies in health care and associated governance and inform a focus group with professionals working in international health and technology to discuss the themes and proposed policy recommendations. Policy recommendations were developed based on the aggregated research findings.

Results

As this is an empirical research paper, we primarily focused on reporting the results of the interviews and the focus group. Semistructured interviews (n=10) and a focus group (n=6) revealed 4 core areas for international collaborations: leadership and oversight, a whole systems approach covering the entire AI pipeline from data collection to model deployment and use, standards and regulatory processes, and engagement with stakeholders and the public. There was a broad range of maturity in health AI activity among the participants, with varying data infrastructure, application of standards across the AI life cycle, and strategic approaches to both development and deployment. A demand for further consistency at the international level and policies was identified to support a robust innovation pipeline. In total, 13 policy recommendations were developed to support GDHP member countries in overcoming core AI governance barriers and establishing common ground for international collaboration.

Conclusions

AI-driven technology research and development for health care outpaces the creation of supporting AI governance globally. International collaboration and coordination on AI governance for health care is needed to ensure coherent solutions and allow countries to support and benefit from each other’s work. International bodies and initiatives have a leading role to play in the international conversation, including the production of tools and sharing of practical approaches to the use of AI-driven technologies for health care.

Implementation Frameworks for Artificial Intelligence Translation Into Health Care Practice: Scoping Review

Background

Significant efforts have been made to develop artificial intelligence (AI) solutions for health care improvement. Despite the enthusiasm, health care professionals still struggle to implement AI in their daily practice.

Objective

This paper aims to identify the implementation frameworks used to understand the application of AI in health care practice.

Methods

A scoping review was conducted using the Cochrane, Evidence Based Medicine Reviews, Embase, MEDLINE, and PsycINFO databases to identify publications that reported frameworks, models, and theories concerning AI implementation in health care. This review focused on studies published in English and investigating AI implementation in health care since 2000. A total of 2541 unique publications were retrieved from the databases and screened on titles and abstracts by 2 independent reviewers. Selected articles were thematically analyzed against the Nilsen taxonomy of implementation frameworks, and the Greenhalgh framework for the nonadoption, abandonment, scale-up, spread, and sustainability (NASSS) of health care technologies.

Results

In total, 7 articles met all eligibility criteria for inclusion in the review, and 2 articles included formal frameworks that directly addressed AI implementation, whereas the other articles provided limited descriptions of elements influencing implementation. Collectively, the 7 articles identified elements that aligned with all the NASSS domains, but no single article comprehensively considered the factors known to influence technology implementation. New domains were identified, including dependency on data input and existing processes, shared decision-making, the role of human oversight, and ethics of population impact and inequality, suggesting that existing frameworks do not fully consider the unique needs of AI implementation.

Conclusions

This literature review demonstrates that understanding how to implement AI in health care practice is still in its early stages of development. Our findings suggest that further research is needed to provide the knowledge necessary to develop implementation frameworks to guide the future implementation of AI in clinical practice and highlight the opportunity to draw on existing knowledge from the field of implementation science.

Clinical deployment environments: Five pillars of translational machine learning for health

Machine Learning for Health (ML4H) has demonstrated efficacy in computer imaging and other self-contained digital workflows, but has failed to substantially impact routine clinical care. This is no longer because of poor adoption of Electronic Health Records Systems (EHRS), but because ML4H needs an infrastructure for development, deployment and evaluation within the healthcare institution. In this paper, we propose a design pattern called a Clinical Deployment Environment (CDE). We sketch the five pillars of the CDE: (1) real world development supported by live data where ML4H teams can iteratively build and test at the bedside (2) an ML-Ops platform that brings the rigour and standards of continuous deployment to ML4H (3) design and supervision by those with expertise in AI safety (4) the methods of implementation science that enable the algorithmic insights to influence the behaviour of clinicians and patients and (5) continuous evaluation that uses randomisation to avoid bias but in an agile manner. The CDE is intended to answer the same requirements that bio-medicine articulated in establishing the translational medicine domain. It envisions a transition from "real-world" data to "real-world" development.

Machine learning methods applied to triage in emergency services: A systematic review

BACKGROUND

In emergency services is important to accurately assess and classify symptoms, which may be improved with the help of technology. One mechanism that could help and improve predictions from health records or patient flow is machine learning (ML).

AIM

To analyse the effectiveness of ML systems in triage for making predictions at the emergency department in comparison with other triage scales/scores.

METHODS

Following the PRISMA recommendations, a systematic review was conducted using CINAHL, Cochrane, Cuiden, Medline and Scopus databases with the search equation "Machine learning AND triage AND emergency".

RESULTS

Eleven studies were identified. The studies show that the use of ML methods consistently predict important outcomes like mortality, critical care outcomes and admission, and the need for hospitalization in comparison with scales like Emergency Severity Index or others. Among the ML models considered, XGBoost and Deep Neural Networks obtained the highest levels of prediction accuracy, while Logistic Regression performed obtained the worst values.

CONCLUSIONS

Machine learning methods can be a good instrument for helping triage process with the prediction of important emergency variables like mortality or the need for critical care or hospitalization.

Artificial intelligence, machine learning, and deep learning for clinical outcome prediction

AI is a broad concept, grouping initiatives that use a computer to perform tasks that would usually require a human to complete. AI methods are well suited to predict clinical outcomes. In practice, AI methods can be thought of as functions that learn the outcomes accompanying standardized input data to produce accurate outcome predictions when trialed with new data. Current methods for cleaning, creating, accessing, extracting, augmenting, and representing data for training AI clinical prediction models are well defined. The use of AI to predict clinical outcomes is a dynamic and rapidly evolving arena, with new methods and applications emerging. Extraction or accession of electronic health care records and combining these with patient genetic data is an area of present attention, with tremendous potential for future growth. Machine learning approaches, including decision tree methods of Random Forest and XGBoost, and deep learning techniques including deep multi-layer and recurrent neural networks, afford unique capabilities to accurately create predictions from high dimensional, multimodal data. Furthermore, AI methods are increasing our ability to accurately predict clinical outcomes that previously were difficult to model, including time-dependent and multi-class outcomes. Barriers to robust AI-based clinical outcome model deployment include changing AI product development interfaces, the specificity of regulation requirements, and limitations in ensuring model interpretability, generalizability, and adaptability over time.

Registries: Big data, bigger problems?

Patient registries have grown in size and number along with general computing power and digitization of the healthcare world. In contrast to databases, registries are typically patient data systematically created and collected for the express purpose of answering health-related questions. Registries can be disease-, procedure-, pathology-, or product-based in nature. Registry-based studies typically fit into Level II or III in the hierarchy of evidence-based medicine. However, a recent advent in the use of registry data has been the development and execution of registry-based trials, such as the TASTE trial, which may elevate registry-based studies into the realm of Level I evidence. Some strengths of registries include the sheer volume of data, the inclusion of a diverse set of participants, and their ability to be linked to other registries and databases. Limitations of registries include variable quality of the collected data, and a lack of active follow-up (which may underestimate rates of adverse events). As with any study type, the intended design does not automatically lead to a study of a certain quality. While no specific tool exists for assessing the quality of a registry-based study, some important considerations include ensuring the registry is appropriate for the question being asked, whether the patient population is representative, the presence of an appropriate comparison group, and the validity and generalizability of the registry in question. The future of clinical registries remains to be seen, but the incorporation of big data and machine learning algorithms will certainly play an important role.