Development and validation of a prognostic COVID-19 severity assessment (COSA) score and machine learning models for patient triage at a tertiary hospital

Development and Implementation of Automated Referral Triaging System for Spinal Cord Stimulation Procedure in Pain Medicine

Effective referral triaging enhances patient service outcomes, experience and access to care especially for specialized procedures. This study presents the development and implementation of an automated triaging system to predict patients who would benefit from Spinal Cord Stimulation (SCS) procedure for their pain management. The proposed triage system aims to improve the triage process by reducing unnecessary appointments before SCS assessment, ensuring appropriate pain management care. It compares various machine learning techniques for the prediction while addressing the class imbalance and overlap challenges inherent in the data. Both data-level and algorithm-level approaches were explored. Two years of patient data was collected including patient characteristics, diagnosis history, pain symptoms, appointment history, medication history, and concepts from clinical notes extracted using Natural Language Processing. EasyEnsemble with Ada Boosting method, an algorithm-level approach, showed the most promising results. The tenfold validation indicated the average area under curve of 0.82, true positive rate (TPR) of 77.3%, and true negative rate (TNR) of 73.0%. The probability threshold was adjusted to 0.575 to meet practice expectation of 15% or less on false positive rate (FPR). The implementation pipeline for the selected model was designed to be applicable to real clinical settings. The one-year implementation results showed TPR of 64.7% and TNR of 87.2%, which reduced FPR by 12.8% while reduced TPR by 12.6%. The trade-off was acceptable to practice. The proposed triage system demonstrated promising accuracy, leading to the enhancement of scheduling systems, patient care, and the reduction of unnecessary appointments in a pain medicine setting.

Augmentation strategies for an imbalanced learning problem on a novel COVID-19 severity dataset

Since the beginning of the COVID-19 pandemic, many different machine learning models have been developed to detect and verify COVID-19 pneumonia based on chest X-ray images. Although promising, binary models have only limited implications for medical treatment, whereas the prediction of disease severity suggests more suitable and specific treatment options. In this study, we publish severity scores for the 2358 COVID-19 positive images in the COVIDx8B dataset, creating one of the largest collections of publicly available COVID-19 severity data. Furthermore, we train and evaluate deep learning models on the newly created dataset to provide a first benchmark for the severity classification task. One of the main challenges of this dataset is the skewed class distribution, resulting in undesirable model performance for the most severe cases. We therefore propose and examine different augmentation strategies, specifically targeting majority and minority classes. Our augmentation strategies show significant improvements in precision and recall values for the rare and most severe cases. While the models might not yet fulfill medical requirements, they serve as an appropriate starting point for further research with the proposed dataset to optimize clinical resource allocation and treatment.

Predicting adverse outcomes in pregnant patients positive for SARS-CoV-2: a machine learning approach- a retrospective cohort study

BACKGROUND

Pregnant people are particularly vulnerable to SARS-CoV-2 infection and to ensuing severe illness. Predicting adverse maternal and perinatal outcomes could aid clinicians in deciding on hospital admission and early initiation of treatment in affected individuals, streamlining the triaging processes.

METHODS

An international repository of 1501 SARS-CoV-2-positive cases in pregnancy was created, consisting of demographic variables, patient comorbidities, laboratory markers, respiratory parameters, and COVID-19-related symptoms. Data were filtered, preprocessed, and feature selection methods were used to obtain the optimal feature subset for training a variety of machine learning models to predict maternal or fetal/neonatal death or critical illness.

RESULTS

The Random Forest model demonstrated the best performance among the trained models, correctly identifying 83.3% of the high-risk patients and 92.5% of the low-risk patients, with an overall accuracy of 89.0%, an AUC of 0.90 (95% Confidence Interval 0.83 to 0.95), and a recall, precision, and F1 score of 0.85, 0.94, and 0.89, respectively. This was achieved using a feature subset of 25 features containing patient characteristics, symptoms, clinical signs, and laboratory markers. These included maternal BMI, gravidity, parity, existence of pre-existing conditions, nicotine exposure, anti-hypertensive medication administration, fetal malformations, antenatal corticosteroid administration, presence of dyspnea, sore throat, fever, fatigue, duration of symptom phase, existence of COVID-19-related pneumonia, need for maternal oxygen administration, disease-related inpatient treatment, and lab markers including sFLT-1/PlGF ratio, platelet count, and LDH.

CONCLUSIONS

We present the first COVID-19 prognostication pipeline specifically for pregnant patients while utilizing a large SARS-CoV-2 in pregnancy data repository. Our model accurately identifies those at risk of severe illness or clinical deterioration, presenting a promising tool for advancing personalized medicine in pregnant patients with COVID-19.

Identification of Multisystem Inflammatory Syndrome in Children Classes and Development of Hyperinflammation Score in Pediatric COVID-19

The aim of this study is to describe characteristics and hospital course of children admitted with COVID-19 to a tertiary care pediatric center in Southeastern United States, and to present the frequency of three classes of multisystem inflammatory syndrome in children (MIS-C) and develop pediatric COVID-19 associated hyperinflammation score (PcHIS). A retrospective cohort study of 68 children was performed. Critical illness was defined as any child requiring respiratory or cardiovascular support or renal replacement therapy. PcHIS was developed by using six variables: fever, hematological dysfunction, coagulopathy, hepatic injury, macrophage activation, and cytokinemia. Centers for Disease Control and Prevention criteria were used to identify MIS-C, and three classes of MIS-C were identified based on the findings of recently published latent class analysis (Class 1: MIS-C without Kawasaki like disease, Class 2: MIS-C with respiratory disease, and Class 3: MIS-C with Kawasaki like disease). The median age was 6.4 years. Fever, respiratory, and gastrointestinal were common presenting symptoms. MIS-C was present in 32 (47%), critical COVID-19 illness in 11 (16%), and 17 (25%) were admitted to the PICU. Children with critical illness were adolescents with elevated body mass index and premorbid conditions. PcHIS score of 3 had a sensitivity of 100% and a specificity of 77% for predicting critical COVID-19 illness. Among MIS-C patients, 15 (47%) were in Class 1, 8 (25%) were in Class 2, and 9 (28%) were in Class 3. We conclude that most children with COVID-19 have mild-to-moderate illness. Critical COVID-19 is mainly seen in obese adolescents with premorbid conditions. Three Classes of MIS-C are identifiable based on clinical features. Validation and clinical implication of inflammation score in pediatric COVID-19 need further investigation.

A machine learning and explainable artificial intelligence triage-prediction system for COVID-19

COVID-19 is a respiratory disease caused by the SARS-CoV-2 contagion, severely disrupted the healthcare infrastructure. Various countries have developed COVID-19 vaccines that have effectively prevented the severe symptoms caused by the virus to a certain extent. However, a small section of people continues to perish. Artificial intelligence advances have revolutionized healthcare diagnosis and prognosis infrastructure. In this study, we predict the severity of COVID-19 using heterogenous Machine Learning and Deep Learning algorithms by considering clinical markers, vital signs, and other critical factors. This study extensively reviews various classifier architectures to predict the COVID-19 severity. We built and evaluated multiple pipelines entailing combinations of five state-of-the-art data-balancing techniques (Synthetic Minority Oversampling Technique (SMOTE), Adaptive Synthetic, Borderline SMOTE, SMOTE with Tomek links, and SMOTE with Edited Nearest Neighbor (ENN)) and twelve heterogeneous classifiers such as Logistic Regression, Decision Tree, Random Forest, Support Vector Machine, K-Nearest Neighbors, Naïve Bayes, Xgboost, Extratrees, Adaboost, Light GBM, Catboost, and 1-D Convolution Neural Network. The best-performing pipeline consists of Random Forest trained on Borderline SMOTE balanced data that produced the highest recall of 83%. We deployed Explainable Artificial Intelligence tools such as Shapley Additive Explanations and Local Interpretable Model-agnostic Explanations, ELI5, Qlattice, Anchor, and Feature Importance to demystify complex tree-based ensemble models. These tools provide valuable insights into the significance of critical features in the severity prediction of a COVID-19 patient. It was observed that changes in respiratory rate, blood pressure, lactate, and calcium values were the primary contributors to the increase in severity of a COVID-19 patient. This architecture aims to be an explainable decision-support triaging system for medical professionals in countries lacking advanced medical technology and infrastructure to reduce fatalities.

Prognostic models in COVID-19 infection that predict severity: a systematic review

Current evidence on COVID-19 prognostic models is inconsistent and clinical applicability remains controversial. We performed a systematic review to summarize and critically appraise the available studies that have developed, assessed and/or validated prognostic models of COVID-19 predicting health outcomes. We searched six bibliographic databases to identify published articles that investigated univariable and multivariable prognostic models predicting adverse outcomes in adult COVID-19 patients, including intensive care unit (ICU) admission, intubation, high-flow nasal therapy (HFNT), extracorporeal membrane oxygenation (ECMO) and mortality. We identified and assessed 314 eligible articles from more than 40 countries, with 152 of these studies presenting mortality, 66 progression to severe or critical illness, 35 mortality and ICU admission combined, 17 ICU admission only, while the remaining 44 studies reported prediction models for mechanical ventilation (MV) or a combination of multiple outcomes. The sample size of included studies varied from 11 to 7,704,171 participants, with a mean age ranging from 18 to 93 years. There were 353 prognostic models investigated, with area under the curve (AUC) ranging from 0.44 to 0.99. A great proportion of studies (61.5%, 193 out of 314) performed internal or external validation or replication. In 312 (99.4%) studies, prognostic models were reported to be at high risk of bias due to uncertainties and challenges surrounding methodological rigor, sampling, handling of missing data, failure to deal with overfitting and heterogeneous definitions of COVID-19 and severity outcomes. While several clinical prognostic models for COVID-19 have been described in the literature, they are limited in generalizability and/or applicability due to deficiencies in addressing fundamental statistical and methodological concerns. Future large, multi-centric and well-designed prognostic prospective studies are needed to clarify remaining uncertainties.

Telehealth use by walk-in patients who subsequently visited an academic emergency department: a repeated cross-sectional study during the COVID-19 summer 2021

AIM OF THE STUDY

During the COVID-19 pandemic, telehealth use increased worldwide in a variety of medical specialities and reached new population groups. A baseline survey of telehealth use prior to admission to the emergency department (ED) conducted before COVID-19 concluded that predominantly well-educated men used telehealth. It is unclear how COVID-19 changed the use of telehealth in Swiss emergency patients. We therefore aimed to investigate (i) the frequency of telehealth use during the pandemic, and (ii) how the pandemic has influenced telehealth use and users.

MATERIALS AND METHODS

A repeated cross-sectional study was conducted among ED walk-in patients at a tertiary university hospital in Switzerland. The study took place one and a half years after the first confirmed COVID-19 case, during 30 shifts from 8 to 29 July 2021 and compared with the baseline survey conducted in 2019. Eligible patients were questioned about their use of, and attitudes to telehealth.

RESULTS

A total of 1020 patients were screened for the COVID survey and 443 complete questionnaires were evaluated. A trend towards a general increase (+6.4%) in telehealth use was demonstrated (50.3%, n = 223 COVID survey vs 43.9%, n = 183 baseline survey; p = 0.058), with a shift to more female patients using telehealth in the COVID survey (female 54.9%, n = 124 vs 45.1%, n = 102; p = 0.052). During the pandemic, first use of telehealth was reported by 12.2% (n = 54) of patients, with a significant increase among patients with low educational status, and the latter patients often indicated that they did not plan to use telehealth after the pandemic. The perceived usefulness of telehealth and adherence to recommendations increased in the COVID survey compared with the baseline survey (adherence 90.3%, n = 149, vs 78.0%, n = 131; p = 0.002).

CONCLUSION

We found a trend towards increased use of telehealth among Swiss ED patients. First-time users of telehealth were predominantly less educated and inclusion of these user groups may not be sustainable, as was indicated by the patients. COVID-19 led to greater adherence to telehealth recommendations and higher perceived usefulness. This could be due to the limited access to healthcare providers due to pandemic precautions. When offering telehealth, the needs of all patient groups must be considered, in order to ensure that telehealth provides the greatest benefit with lower barriers to use.

Pediatric COVID-19: Correlations between Clinical and Imaging Perspectives

Background

Although SARS-CoV-2 infection primarily affects adults, the increasing emergence of infected pediatric patients has been recently reported. However, there is a paucity of data regarding the value of imaging in relation to the clinical severity of this pandemic emergency.

Objectives

To demonstrate the relationships between clinical and radiological COVID-19 findings and to determine the most effective standardized pediatric clinical and imaging strategies predicting the disease severity. Patients and Methods. This observational study enrolled eighty pediatric patients with confirmed COVID-19 infection. The studied patients were categorized according to the disease severity and the presence of comorbidities. Patients' clinical findings, chest X-ray, and CT imaging results were analyzed. Patients' evaluations using several clinical and radiological severity scores were recorded. The relations between clinical and radiological severities were examined.

Results

Significant associations were found between severe-to-critical illness and abnormal radiological findings (p = 0.009). In addition, chest X-ray score, chest CT severity score, and rapid evaluation of anamnesis, PO2, imaging disease, and dyspnea-COVID (RAPID-COVID) score were significantly higher among patients with severe infection (p < 0.001, <0.001, and 0.001) and those with comorbidities (p = 0.005, 0.002, and <0.001).

Conclusions

Chest imaging of pediatric patients with COVID-19 infection may be of value during the evaluation of severe cases of infected pediatric patients and in those with underlying comorbid conditions, especially during the early stage of infection. Moreover, the combined use of specific clinical and radiological COVID-19 scores are likely to be a successful measure of the extent of disease severity.

Machine learning models to predict the maximum severity of COVID-19 based on initial hospitalization record

Background

As the worldwide spread of coronavirus disease 2019 (COVID-19) continues for a long time, early prediction of the maximum severity is required for effective treatment of each patient.

Objective

This study aimed to develop predictive models for the maximum severity of hospitalized COVID-19 patients using artificial intelligence (AI)/machine learning (ML) algorithms.

Methods

The medical records of 2,263 COVID-19 patients admitted to 10 hospitals in Daegu, Korea, from February 18, 2020, to May 19, 2020, were comprehensively reviewed. The maximum severity during hospitalization was divided into four groups according to the severity level: mild, moderate, severe, and critical. The patient's initial hospitalization records were used as predictors. The total dataset was randomly split into a training set and a testing set in a 2:1 ratio, taking into account the four maximum severity groups. Predictive models were developed using the training set and were evaluated using the testing set. Two approaches were performed: using four groups based on original severity levels groups (i.e., 4-group classification) and using two groups after regrouping the four severity level into two (i.e., binary classification). Three variable selection methods including randomForestSRC were performed. As AI/ML algorithms for 4-group classification, GUIDE and proportional odds model were used. For binary classification, we used five AI/ML algorithms, including deep neural network and GUIDE.

Results

Of the four maximum severity groups, the moderate group had the highest percentage (1,115 patients; 49.5%). As factors contributing to exacerbation of maximum severity, there were 25 statistically significant predictors through simple analysis of linear trends. As a result of model development, the following three models based on binary classification showed high predictive performance: (1) Mild vs. Above Moderate, (2) Below Moderate vs. Above Severe, and (3) Below Severe vs. Critical. The performance of these three binary models was evaluated using AUC values 0.883, 0.879, and, 0.887, respectively. Based on results for each of the three predictive models, we developed web-based nomograms for clinical use (http://statgen.snu.ac.kr/software/nomogramDaeguCovid/).

Conclusions

We successfully developed web-based nomograms predicting the maximum severity. These nomograms are expected to help plan an effective treatment for each patient in the clinical field.

Drug-Disease Severity and Target-Disease Severity Interaction Networks in COVID-19 Patients

Drug interactions with other drugs are a well-known phenomenon. Similarly, however, pre-existing drug therapy can alter the course of diseases for which it has not been prescribed. We performed network analysis on drugs and their respective targets to investigate whether there are drugs or targets with protective effects in COVID-19, making them candidates for repurposing. These networks of drug-disease interactions (DDSIs) and target-disease interactions (TDSIs) revealed a greater share of patients with diabetes and cardiac co-morbidities in the non-severe cohort treated with dipeptidyl peptidase-4 (DPP4) inhibitors. A possible protective effect of DPP4 inhibitors is also plausible on pathophysiological grounds, and our results support repositioning efforts of DPP4 inhibitors against SARS-CoV-2. At target level, we observed that the target location might have an influence on disease progression. This could potentially be attributed to disruption of functional membrane micro-domains (lipid rafts), which in turn could decrease viral entry and thus disease severity.

Identification of COVID-19 patients at risk of hospital admission and mortality: a European multicentre retrospective analysis of mid-regional pro-adrenomedullin

Background

Mid-Regional pro-Adrenomedullin (MR-proADM) is an inflammatory biomarker that improves the prognostic assessment of patients with sepsis, septic shock and organ failure. Previous studies of MR-proADM have primarily focussed on bacterial infections. A limited number of small and monocentric studies have examined MR-proADM as a prognostic factor in patients infected with SARS-CoV-2, however there is need for multicenter validation. An evaluation of its utility in predicting need for hospitalisation in viral infections was also performed.

Methods

An observational retrospective analysis of 1861 patients, with SARS-CoV-2 confirmed by RT-qPCR, from 10 hospitals across Europe was performed. Biomarkers, taken upon presentation to Emergency Departments (ED), clinical scores, patient demographics and outcomes were collected. Multiclass random forest classifier models were generated as well as calculation of area under the curve analysis. The primary endpoint was hospital admission with and without death.

Results

Patients suitable for safe discharge from Emergency Departments could be identified through an MR-proADM value of ≤ 1.02 nmol/L in combination with a CRP (C-Reactive Protein) of ≤ 20.2 mg/L and age ≤ 64, or in combination with a SOFA (Sequential Organ Failure Assessment) score < 2 if MR-proADM was ≤ 0.83 nmol/L regardless of age. Those at an increased risk of mortality could be identified upon presentation to secondary care with an MR-proADM value of > 0.85 nmol/L, in combination with a SOFA score ≥ 2 and LDH > 720 U/L, or in combination with a CRP > 29.26 mg/L and age ≤ 64, when MR-proADM was > 1.02 nmol/L.

Conclusions

This international study suggests that for patients presenting to the ED with confirmed SARS-CoV-2 infection, MR-proADM in combination with age and CRP or with the patient’s SOFA score could identify patients at low risk where outpatient treatment may be safe.

A Comprehensive Review of Machine Learning Used to Combat COVID-19

Coronavirus disease (COVID-19) has had a significant impact on global health since the start of the pandemic in 2019. As of June 2022, over 539 million cases have been confirmed worldwide with over 6.3 million deaths as a result. Artificial Intelligence (AI) solutions such as machine learning and deep learning have played a major part in this pandemic for the diagnosis and treatment of COVID-19. In this research, we review these modern tools deployed to solve a variety of complex problems. We explore research that focused on analyzing medical images using AI models for identification, classification, and tissue segmentation of the disease. We also explore prognostic models that were developed to predict health outcomes and optimize the allocation of scarce medical resources. Longitudinal studies were conducted to better understand COVID-19 and its effects on patients over a period of time. This comprehensive review of the different AI methods and modeling efforts will shed light on the role that AI has played and what path it intends to take in the fight against COVID-19.

A Hybrid Feature Selection Approach to Screen a Novel Set of Blood Biomarkers for Early COVID-19 Mortality Prediction

The increase in coronavirus disease 2019 (COVID-19) infection caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has placed pressure on healthcare services worldwide. Therefore, it is crucial to identify critical factors for the assessment of the severity of COVID-19 infection and the optimization of an individual treatment strategy. In this regard, the present study leverages a dataset of blood samples from 485 COVID-19 individuals in the region of Wuhan, China to identify essential blood biomarkers that predict the mortality of COVID-19 individuals. For this purpose, a hybrid of filter, statistical, and heuristic-based feature selection approach was used to select the best subset of informative features. As a result, minimum redundancy maximum relevance (mRMR), a two-tailed unpaired t-test, and whale optimization algorithm (WOA) were eventually selected as the three most informative blood biomarkers: International normalized ratio (INR), platelet large cell ratio (P-LCR), and D-dimer. In addition, various machine learning (ML) algorithms (random forest (RF), support vector machine (SVM), extreme gradient boosting (EGB), naïve Bayes (NB), logistic regression (LR), and k-nearest neighbor (KNN)) were trained. The performance of the trained models was compared to determine the model that assist in predicting the mortality of COVID-19 individuals with higher accuracy, F1 score, and area under the curve (AUC) values. In this paper, the best performing RF-based model built using the three most informative blood parameters predicts the mortality of COVID-19 individuals with an accuracy of 0.96 ± 0.062, F1 score of 0.96 ± 0.099, and AUC value of 0.98 ± 0.024, respectively on the independent test data. Furthermore, the performance of our proposed RF-based model in terms of accuracy, F1 score, and AUC was significantly better than the known blood biomarkers-based ML models built using the Pre_Surv_COVID_19 data. Therefore, the present study provides a novel hybrid approach to screen the most informative blood biomarkers to develop an RF-based model, which accurately and reliably predicts in-hospital mortality of confirmed COVID-19 individuals, during surge periods. An application based on our proposed model was implemented and deployed at Heroku.

CT-based severity assessment for COVID-19 using weakly supervised non-local CNN

Evaluating patient criticality is the foremost step in administering appropriate COVID-19 treatment protocols. Learning an Artificial Intelligence (AI) model from clinical data for automatic risk-stratification enables accelerated response to patients displaying critical indicators. Chest CT manifestations including ground-glass opacities and consolidations are a reliable indicator for prognostic studies and show variability with patient condition. To this end, we propose a novel attention framework to estimate COVID-19 severity as a regression score from a weakly annotated CT scan dataset. It takes a non-locality approach that correlates features across different parts and spatial scales of the 3D scan. An explicit guidance mechanism from limited infection labeling drives attention refinement and feature modulation. The resulting encoded representation is further enriched through cross-channel attention. The attention model also infuses global contextual awareness into the deep voxel features by querying the base CT scan to mine relevant features. Consequently, it learns to effectively localize its focus region and chisel out the infection precisely. Experimental validation on the MosMed dataset shows that the proposed architecture has significant potential in augmenting existing methods as it achieved a 0.84 R-squared score and 0.133 mean absolute difference.

The Accuracy of Machine Learning Approaches Using Non-image Data for the Prediction of COVID-19: A Meta-Analysis

Objective

COVID-19 is a novel, severely contagious disease with enormous negative impact on humanity as well as the world economy. An expeditious, feasible tool for detecting COVID-19 remains yet elusive. Recently, there has been a surge of interest in applying machine learning techniques to predict COVID-19 using non-image data. We have therefore undertaken a meta-analysis to quantify the diagnostic performance of machine learning models facilitating the prediction of COVID-19.

Materials and methods

A comprehensive electronic database search for the period between January 1st, 2021 and December 3rd, 2021 was undertaken in order to identify eligible studies relevant to this meta-analysis. Summary sensitivity, specificity, and the area under receiver operating characteristic curves were used to assess potential diagnostic accuracy. Risk of bias was assessed by means of a revised Quality Assessment of Diagnostic Studies.

Results

A total of 30 studies, including 34 models, met all of the inclusion criteria. Summary sensitivity, specificity, and area under receiver operating characteristic curves were 0.86, 0.86, and 0.91, respectively. The purpose of machine learning models, class imbalance, and feature selection are significant covariates useful in explaining the between-study heterogeneity, in terms of both sensitivity and specificity.

Conclusions

Our study findings show that non-image data can be used to predict COVID-19 with an acceptable performance. Further, class imbalance and feature selection are suggested to be incorporated whenever building models for the prediction of COVID-19, thus improving further diagnostic performance.

Accuracy of the Veterans Health Administration COVID-19 (VACO) Index for predicting short-term mortality among 1307 US academic medical centre inpatients and 427 224 US Medicare patients

BACKGROUND

The Veterans Health Administration COVID-19 (VACO) Index predicts 30-day all-cause mortality in patients with COVID-19 using age, sex and pre-existing comorbidity diagnoses. The VACO Index was initially developed and validated in a nationwide cohort of US veterans-we now assess its accuracy in an academic medical centre and a nationwide US Medicare cohort.

METHODS

With measures and weights previously derived and validated in US national Veterans Health Administration (VA) inpatients and outpatients (n=13 323), we evaluated the accuracy of the VACO Index for estimating 30-day all-cause mortality using area under the receiver operating characteristic curve (AUC) and calibration plots of predicted versus observed mortality in inpatients at a single US academic medical centre (n=1307) and in Medicare inpatients and outpatients aged 65+ (n=427 224).

RESULTS

30-day mortality varied by data source: VA 8.5%, academic medical centre 17.5%, Medicare 16.0%. The VACO Index demonstrated similar discrimination in VA (AUC=0.82) and academic medical centre inpatient population (AUC=0.80), and when restricted to patients aged 65+ in VA (AUC=0.69) and Medicare inpatient and outpatient data (AUC=0.67). The Index modestly overestimated risk in VA and Medicare data and underestimated risk in Yale New Haven Hospital data.

CONCLUSIONS

The VACO Index estimates risk of short-term mortality across a wide variety of patients with COVID-19 using data available prior to or at the time of diagnosis. The VACO Index could help inform primary and booster vaccination prioritisation, and indicate who among outpatients testing positive for SARS-CoV-2 should receive greater clinical attention or scarce treatments.

Prediction models for severe manifestations and mortality due to COVID-19: A systematic review

BACKGROUND

Throughout 2020, the coronavirus disease 2019 (COVID-19) has become a threat to public health on national and global level. There has been an immediate need for research to understand the clinical signs and symptoms of COVID-19 that can help predict deterioration including mechanical ventilation, organ support, and death. Studies thus far have addressed the epidemiology of the disease, common presentations, and susceptibility to acquisition and transmission of the virus; however, an accurate prognostic model for severe manifestations of COVID-19 is still needed because of the limited healthcare resources available.

OBJECTIVE

This systematic review aims to evaluate published reports of prediction models for severe illnesses caused COVID-19.

METHODS

Searches were developed by the primary author and a medical librarian using an iterative process of gathering and evaluating terms. Comprehensive strategies, including both index and keyword methods, were devised for PubMed and EMBASE. The data of confirmed COVID-19 patients from randomized control studies, cohort studies, and case-control studies published between January 2020 and May 2021 were retrieved. Studies were independently assessed for risk of bias and applicability using the Prediction Model Risk Of Bias Assessment Tool (PROBAST). We collected study type, setting, sample size, type of validation, and outcome including intubation, ventilation, any other type of organ support, or death. The combination of the prediction model, scoring system, performance of predictive models, and geographic locations were summarized.

RESULTS

A primary review found 445 articles relevant based on title and abstract. After further review, 366 were excluded based on the defined inclusion and exclusion criteria. Seventy-nine articles were included in the qualitative analysis. Inter observer agreement on inclusion 0.84 (95%CI 0.78-0.89). When the PROBAST tool was applied, 70 of the 79 articles were identified to have high or unclear risk of bias, or high or unclear concern for applicability. Nine studies reported prediction models that were rated as low risk of bias and low concerns for applicability.

CONCLUSION

Several prognostic models for COVID-19 were identified, with varying clinical score performance. Nine studies that had a low risk of bias and low concern for applicability, one from a general public population and hospital setting. The most promising and well-validated scores include Clift et al.,¹⁵ and Knight et al.,¹⁸ which seem to have accurate prediction models that clinicians can use in the public health and emergency department setting.

Potential applications and performance of machine learning techniques and algorithms in clinical practice: A systematic review

PURPOSE

The advent of clinically adapted machine learning algorithms can solve numerous problems ranging from disease diagnosis and prognosis to therapy recommendations. This systematic review examines the performance of machine learning (ML) algorithms and evaluates the progress made to date towards their implementation in clinical practice.

METHODS

Systematic searching of databases (PubMed, MEDLINE, Scopus, Google Scholar, Cochrane Library and WHO Covid-19 database) to identify original articles published between January 2011 and October 2021. Studies reporting ML techniques in clinical practice involving humans and ML algorithms with a performance metric were considered.

RESULTS

Of 873 unique articles identified, 36 studies were eligible for inclusion. The XGBoost (extreme gradient boosting) algorithm showed the highest potential for clinical applications (n = 7 studies); this was followed jointly by random forest algorithm, logistic regression, and the support vector machine, respectively (n = 5 studies). Prediction of outcomes (n = 33), in particular Inflammatory diseases (n = 7) received the most attention followed by cancer and neuropsychiatric disorders (n = 5 for each) and Covid-19 (n = 4). Thirty-three out of the thirty-six included studies passed more than 50% of the selected quality assessment criteria in the TRIPOD checklist. In contrast, none of the studies could achieve an ideal overall bias rating of 'low' based on the PROBAST checklist. In contrast, only three studies showed evidence of the deployment of ML algorithm(s) in clinical practice.

CONCLUSIONS

ML is potentially a reliable tool for clinical decision support. Although advocated widely in clinical practice, work is still in progress to validate clinically adapted ML algorithms. Improving quality standards, transparency, and interpretability of ML models will further lower the barriers to acceptability.

Does COVID-19 Clinical Status Associate with Outcome Severity? An Unsupervised Machine Learning Approach for Knowledge Extraction

Since the beginning of the COVID-19 pandemic, 195 million people have been infected and 4.2 million have died from the disease or its side effects. Physicians, healthcare scientists and medical staff continuously try to deal with overloaded hospital admissions, while in parallel, they try to identify meaningful correlations between the severity of infected patients with their symptoms, comorbidities and biomarkers. Artificial intelligence (AI) and machine learning (ML) have been used recently in many areas related to COVID-19 healthcare. The main goal is to manage effectively the wide variety of issues related to COVID-19 and its consequences. The existing applications of ML to COVID-19 healthcare are based on supervised classifications which require a labeled training dataset, serving as reference point for learning, as well as predefined classes. However, the existing knowledge about COVID-19 and its consequences is still not solid and the points of common agreement among different scientific communities are still unclear. Therefore, this study aimed to follow an unsupervised clustering approach, where prior knowledge is not required (tabula rasa). More specifically, 268 hospitalized patients at the First Propaedeutic Department of Internal Medicine of AHEPA University Hospital of Thessaloniki were assessed in terms of 40 clinical variables (numerical and categorical), leading to a high-dimensionality dataset. Dimensionality reduction was performed by applying a principal component analysis (PCA) on the numerical part of the dataset and a multiple correspondence analysis (MCA) on the categorical part of the dataset. Then, the Bayesian information criterion (BIC) was applied to Gaussian mixture models (GMM) in order to identify the optimal number of clusters under which the best grouping of patients occurs. The proposed methodology identified four clusters of patients with similar clinical characteristics. The analysis revealed a cluster of asymptomatic patients that resulted in death at a rate of 23.8%. This striking result forces us to reconsider the relationship between the severity of COVID-19 clinical symptoms and the patient's mortality.

A catalogue of tools and variables from crisis and routine care to support decision-making about allocation of intensive care beds and ventilator treatment during pandemics: Scoping review

PURPOSE

This scoping review sought to identify objective factors to assist clinicians and policy-makers in making consistent, objective and ethically sound decisions about resource allocation when healthcare rationing is inevitable.

MATERIALS AND METHODS

Review of guidelines and tools used in ICUs, hospital wards and emergency departments on how to best allocate intensive care beds and ventilators either during routine care or developed during previous epidemics, and association with patient outcomes during and after hospitalisation.

RESULTS

Eighty publications from 20 countries reporting accuracy or validity of prognostic tools/algorithms, or significant correlation between prognostic variables and clinical outcomes met our eligibility criteria: twelve pandemic guidelines/triage protocols/consensus statements, twenty-two pandemic algorithms, and 46 prognostic tools/variables from non-crisis situations. Prognostic indicators presented here can be combined to create locally-relevant triage algorithms for clinicians and policy makers deciding about allocation of ICU beds and ventilators during a pandemic. No consensus was found on the ethical issues to incorporate in the decision to admit or triage out of intensive care.

CONCLUSIONS

This review provides a unique reference intended as a discussion starter for clinicians and policy makers to consider formalising an objective a locally-relevant triage consensus document that enhances confidence in decision-making during healthcare rationing of critical care and ventilator resources.

A Proposed Framework for Machine Learning-Aided Triage in Public Specialty Ophthalmology Clinics in Hong Kong

The public specialty ophthalmic clinics in Hong Kong, under the Hospital Authority, receive tens of thousands of referrals each year. Triaging these referrals incurs a significant workload for practitioners and the other clinical duties. It is well-established that Hong Kong is currently facing a shortage of healthcare workers. Thus a more efficient system in triaging will not only free up resources for better use but also improve the satisfaction of both practitioners and patients. Machine learning (ML) has been shown to improve the efficiency of various medical workflows, including triaging, by both reducing the workload and increasing accuracy in some cases. Despite a myriad of studies on medical artificial intelligence, there is no specific framework for a triaging algorithm in ophthalmology clinics. This study proposes a general framework for developing, deploying and evaluating an ML-based triaging algorithm in a clinical setting. Through literature review, this study identifies good practices in various facets of developing such a network and protocols for maintenance and evaluation of the impact concerning clinical utility and external validity out of the laboratory. We hope this framework, albeit not exhaustive, can act as a foundation to accelerate future pilot studies and deployments.

A machine-learning parsimonious multivariable predictive model of mortality risk in patients with Covid-19

The COVID-19 pandemic is impressively challenging the healthcare system. Several prognostic models have been validated but few of them are implemented in daily practice. The objective of the study was to validate a machine-learning risk prediction model using easy-to-obtain parameters to help to identify patients with COVID-19 who are at higher risk of death. The training cohort included all patients admitted to Fondazione Policlinico Gemelli with COVID-19 from March 5, 2020, to November 5, 2020. Afterward, the model was tested on all patients admitted to the same hospital with COVID-19 from November 6, 2020, to February 5, 2021. The primary outcome was in-hospital case-fatality risk. The out-of-sample performance of the model was estimated from the training set in terms of Area under the Receiving Operator Curve (AUROC) and classification matrix statistics by averaging the results of fivefold cross validation repeated 3-times and comparing the results with those obtained on the test set. An explanation analysis of the model, based on the SHapley Additive exPlanations (SHAP), is also presented. To assess the subsequent time evolution, the change in paO2/FiO2 (P/F) at 48 h after the baseline measurement was plotted against its baseline value. Among the 921 patients included in the training cohort, 120 died (13%). Variables selected for the model were age, platelet count, SpO2, blood urea nitrogen (BUN), hemoglobin, C-reactive protein, neutrophil count, and sodium. The results of the fivefold cross-validation repeated 3-times gave AUROC of 0.87, and statistics of the classification matrix to the Youden index as follows: sensitivity 0.840, specificity 0.774, negative predictive value 0.971. Then, the model was tested on a new population (n = 1463) in which the case-fatality rate was 22.6%. The test model showed AUROC 0.818, sensitivity 0.813, specificity 0.650, negative predictive value 0.922. Considering the first quartile of the predicted risk score (low-risk score group), the case-fatality rate was 1.6%, 17.8% in the second and third quartile (high-risk score group) and 53.5% in the fourth quartile (very high-risk score group). The three risk score groups showed good discrimination for the P/F value at admission, and a positive correlation was found for the low-risk class to P/F at 48 h after admission (adjusted R-squared = 0.48). We developed a predictive model of death for people with SARS-CoV-2 infection by including only easy-to-obtain variables (abnormal blood count, BUN, C-reactive protein, sodium and lower SpO2). It demonstrated good accuracy and high power of discrimination. The simplicity of the model makes the risk prediction applicable for patients in the Emergency Department, or during hospitalization. Although it is reasonable to assume that the model is also applicable in not-hospitalized persons, only appropriate studies can assess the accuracy of the model also for persons at home.

Hypotension Prediction Index with non-invasive continuous arterial pressure waveforms (ClearSight): clinical performance in Gynaecologic Oncologic Surgery

Intraoperative hypotension (IOH) is common during major surgery and is associated with a poor postoperative outcome. Hypotension Prediction Index (HPI) is an algorithm derived from machine learning that uses the arterial waveform to predict IOH. The aim of this study was to assess the diagnostic ability of HPI working with non-invasive ClearSight system in predicting impending hypotension in patients undergoing major gynaecologic oncologic surgery (GOS). In this retrospective analysis hemodynamic data were downloaded from an Edwards Lifesciences HemoSphere platform and analysed. Receiver operating characteristic curves were constructed to evaluate the performance of HPI working on the ClearSight pressure waveform in predicting hypotensive events, defined as mean arterial pressure < 65 mmHg for > 1 min. Sensitivity, specificity, positive predictive value and negative predictive value were computed at a cutpoint (the value which minimizes the difference between sensitivity and specificity). Thirty-one patients undergoing GOS were included in the analysis, 28 of which had complete data set. The HPI predicted hypotensive events with a sensitivity of 0.85 [95% confidence interval (CI) 0.73-0.94] and specificity of 0.85 (95% CI 0.74-0.95) 15 min before the event [area under the curve (AUC) 0.95 (95% CI 0.89-0.99)]; with a sensitivity of 0.82 (95% CI 0.71-0.92) and specificity of 0.83 (95% CI 0.71-0.93) 10 min before the event [AUC 0.9 (95% CI 0.83-0.97)]; and with a sensitivity of 0.86 (95% CI 0.78-0.93) and specificity 0.86 (95% CI 0.77-0.94) 5 min before the event [AUC 0.93 (95% CI 0.89-0.97)]. HPI provides accurate and continuous prediction of impending IOH before its occurrence in patients undergoing GOS in general anesthesia.

Machine learning application for the prediction of SARS-CoV-2 infection using blood tests and chest radiograph

Triaging and prioritising patients for RT-PCR test had been essential in the management of COVID-19 in resource-scarce countries. In this study, we applied machine learning (ML) to the task of detection of SARS-CoV-2 infection using basic laboratory markers. We performed the statistical analysis and trained an ML model on a retrospective cohort of 5148 patients from 24 hospitals in Hong Kong to classify COVID-19 and other aetiology of pneumonia. We validated the model on three temporal validation sets from different waves of infection in Hong Kong. For predicting SARS-CoV-2 infection, the ML model achieved high AUCs and specificity but low sensitivity in all three validation sets (AUC: 89.9-95.8%; Sensitivity: 55.5-77.8%; Specificity: 91.5-98.3%). When used in adjunction with radiologist interpretations of chest radiographs, the sensitivity was over 90% while keeping moderate specificity. Our study showed that machine learning model based on readily available laboratory markers could achieve high accuracy in predicting SARS-CoV-2 infection.

Diagnosis and prediction of COVID-19 severity: can biochemical tests and machine learning be used as prognostic indicators?

OBJECTIVE

This study aimed to implement and evaluate machine learning based-models to predict COVID-19' diagnosis and disease severity.

METHODS

COVID-19 test samples (positive or negative results) from patients who attended a single hospital were evaluated. Patients diagnosed with COVID-19 were categorised according to the severity of the disease. Data were submitted to exploratory analysis (principal component analysis, PCA) to detect outlier samples, recognise patterns, and identify important variables. Based on patients' laboratory tests results, machine learning models were implemented to predict disease positivity and severity. Artificial neural networks (ANN), decision trees (DT), partial least squares discriminant analysis (PLS-DA), and K nearest neighbour algorithm (KNN) models were used. The four models were validated based on the accuracy (area under the ROC curve).

RESULTS

The first subset of data had 5,643 patient samples (5,086 negatives and 557 positives for COVID-19). The second subset included 557 COVID-19 positive patients. The ANN, DT, PLS-DA, and KNN models allowed the classification of negative and positive samples with >84% accuracy. It was also possible to classify patients with severe and non-severe disease with an accuracy >86%. The following were associated with the prediction of COVID-19 diagnosis and severity: hyperferritinaemia, hypocalcaemia, pulmonary hypoxia, hypoxemia, metabolic and respiratory acidosis, low urinary pH, and high levels of lactate dehydrogenase.

CONCLUSION

Our analysis shows that all the models could assist in the diagnosis and prediction of COVID-19 severity.

Identification of Variable Importance for Predictions of Mortality From COVID-19 Using AI Models for Ontario, Canada

The Severe Acute Respiratory Syndrome Coronavirus 2 pandemic has challenged medical systems to the brink of collapse around the globe. In this paper, logistic regression and three other artificial intelligence models (XGBoost, Artificial Neural Network and Random Forest) are described and used to predict mortality risk of individual patients. The database is based on census data for the designated area and co-morbidities obtained using data from the Ontario Health Data Platform. The dataset consisted of more than 280,000 COVID-19 cases in Ontario for a wide-range of age groups; 0–9, 10–19, 20–29, 30–39, 40–49, 50–59, 60–69, 70–79, 80–89, and 90+. Findings resulting from using logistic regression, XGBoost, Artificial Neural Network and Random Forest, all demonstrate excellent discrimination (area under the curve for all models exceeded 0.948 with the best performance being 0.956 for an XGBoost model). Based on SHapley Additive exPlanations values, the importance of 24 variables are identified, and the findings indicated the highest importance variables are, in order of importance, age, date of test, sex, and presence/absence of chronic dementia. The findings from this study allow the identification of out-patients who are likely to deteriorate into severe cases, allowing medical professionals to make decisions on timely treatments. Furthermore, the methodology and results may be extended to other public health regions.

Gray level co-occurrence matrix and extreme learning machine for Covid-19 diagnosis

Background

Chest CT is considered to be a more accurate method for diagnosing suspected patients. However, with the spread of the epidemic, traditional diagnostic methods have been unable to meet the requirements of efficiency and speed. Therefore, it is necessary to use artificial intelligence to help people make efficient and accurate judgments. A number of studies have shown that it is feasible to use deep learning methods to help people diagnose COVID-19. However, most of the existing methods are single-layer neural network structures, and their accuracy and efficiency need to be improved.

Method

In this scheme, a hybrid model is adopted. Firstly, the gray co-occurrence matrix is used to extract the features of the images, and then the extreme learning machine is used for classification.

Results

The experimental results show that the model proposed in this paper is feasible and can help medical staff to accurately determine suspected patients for subsequent isolation and treatment.

SARS-CoV-2, hemoglobin and protoporphyrin IX: Interactions and perspectives

Background

SARS-CoV-2 attacks hemoglobin through its structural protein ORF3a, dissociating the iron from the heme, as iron is necessary by cell machinery for virus replication. In this process protoporphyrin (PpIX) is released.

Methods

The decrease in the hemoglobin levels observed in patients with Covid-19 is frequently accompanied by an increase in PpIX levels. This evidence was confirmed by the quantification of PpIX by high-performance liquid chromatography (HPLC). PpIX emission is observed in its two characteristic bands at approximately 635 nm and 705 nm.

Results

This paper searches to understand the role of heme and PpIX inside the cells. Perspectives on the use of PpIX fluorescence as a sensor to monitor the presence of SARS-CoV-2 in the tissue, blood, urine, or feces to map the evolution and severity of the disease or to monitor the response of the Covid-19 treatment modalities were described.

Conclusion

Fluorescence spectroscopy could be adopted as an excellent diagnostic technique for Covid-19, of low cost and high sensitivity. This method can potentially be used as a marker to monitor the response to the treatments. Photodynamic and sonodynamic therapies using the endogenous PpIX increased in the acute phase of the disease, could be employed for Covid-19 treatment.

Machine Learning Based Clinical Decision Support System for Early COVID-19 Mortality Prediction

The coronavirus disease 2019 (COVID-19), caused by the virus SARS-CoV-2, is an acute respiratory disease that has been classified as a pandemic by the World Health Organization (WHO). The sudden spike in the number of infections and high mortality rates have put immense pressure on the public healthcare systems. Hence, it is crucial to identify the key factors for mortality prediction to optimize patient treatment strategy. Different routine blood test results are widely available compared to other forms of data like X-rays, CT-scans, and ultrasounds for mortality prediction. This study proposes machine learning (ML) methods based on blood tests data to predict COVID-19 mortality risk. A powerful combination of five features: neutrophils, lymphocytes, lactate dehydrogenase (LDH), high-sensitivity C-reactive protein (hs-CRP), and age helps to predict mortality with 96% accuracy. Various ML models (neural networks, logistic regression, XGBoost, random forests, SVM, and decision trees) have been trained and performance compared to determine the model that achieves consistently high accuracy across the days that span the disease. The best performing method using XGBoost feature importance and neural network classification, predicts with an accuracy of 90% as early as 16 days before the outcome. Robust testing with three cases based on days to outcome confirms the strong predictive performance and practicality of the proposed model. A detailed analysis and identification of trends was performed using these key biomarkers to provide useful insights for intuitive application. This study provide solutions that would help accelerate the decision-making process in healthcare systems for focused medical treatments in an accurate, early, and reliable manner.

Digital Transformation in Personalized Medicine with Artificial Intelligence and the Internet of Medical Things

Digital transformation is impacting every facet of science and society, not least because there is a growing need for digital services and products with the COVID-19 pandemic. But the need for digital transformation in diagnostics and personalized medicine field cuts deeper. In the past, personalized/precision medicine initiatives have been unable to capture the patients' experiences and clinical outcomes in real-time and in real-world settings. The availability of wearable smart sensors, wireless connectivity, artificial intelligence, and the Internet of Medical Things is changing the personalized/precision medicine research and implementation landscape. Digital transformation in poised to accelerate personalized/precision medicine and systems science in multiple fronts such as deep real-time phenotyping with patient-reported outcomes, high-throughput association studies between omics and highly granular phenotypic variation, digital clinical trials, among others. The present expert review offers an analysis of these systems science frontiers with a view to future applications at the intersection of digital health and personalized medicine, or put in other words, signaling the rise of "digital personalized medicine."

Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal

OBJECTIVE

To review and appraise the validity and usefulness of published and preprint reports of prediction models for diagnosing coronavirus disease 2019 (covid-19) in patients with suspected infection, for prognosis of patients with covid-19, and for detecting people in the general population at increased risk of covid-19 infection or being admitted to hospital with the disease.

DESIGN

Living systematic review and critical appraisal by the COVID-PRECISE (Precise Risk Estimation to optimise covid-19 Care for Infected or Suspected patients in diverse sEttings) group.

DATA SOURCES

PubMed and Embase through Ovid, up to 1 July 2020, supplemented with arXiv, medRxiv, and bioRxiv up to 5 May 2020.

STUDY SELECTION

Studies that developed or validated a multivariable covid-19 related prediction model.

DATA EXTRACTION

At least two authors independently extracted data using the CHARMS (critical appraisal and data extraction for systematic reviews of prediction modelling studies) checklist; risk of bias was assessed using PROBAST (prediction model risk of bias assessment tool).

RESULTS

37 421 titles were screened, and 169 studies describing 232 prediction models were included. The review identified seven models for identifying people at risk in the general population; 118 diagnostic models for detecting covid-19 (75 were based on medical imaging, 10 to diagnose disease severity); and 107 prognostic models for predicting mortality risk, progression to severe disease, intensive care unit admission, ventilation, intubation, or length of hospital stay. The most frequent types of predictors included in the covid-19 prediction models are vital signs, age, comorbidities, and image features. Flu-like symptoms are frequently predictive in diagnostic models, while sex, C reactive protein, and lymphocyte counts are frequent prognostic factors. Reported C index estimates from the strongest form of validation available per model ranged from 0.71 to 0.99 in prediction models for the general population, from 0.65 to more than 0.99 in diagnostic models, and from 0.54 to 0.99 in prognostic models. All models were rated at high or unclear risk of bias, mostly because of non-representative selection of control patients, exclusion of patients who had not experienced the event of interest by the end of the study, high risk of model overfitting, and unclear reporting. Many models did not include a description of the target population (n=27, 12%) or care setting (n=75, 32%), and only 11 (5%) were externally validated by a calibration plot. The Jehi diagnostic model and the 4C mortality score were identified as promising models.

CONCLUSION

Prediction models for covid-19 are quickly entering the academic literature to support medical decision making at a time when they are urgently needed. This review indicates that almost all pubished prediction models are poorly reported, and at high risk of bias such that their reported predictive performance is probably optimistic. However, we have identified two (one diagnostic and one prognostic) promising models that should soon be validated in multiple cohorts, preferably through collaborative efforts and data sharing to also allow an investigation of the stability and heterogeneity in their performance across populations and settings. Details on all reviewed models are publicly available at https://www.covprecise.org/. Methodological guidance as provided in this paper should be followed because unreliable predictions could cause more harm than benefit in guiding clinical decisions. Finally, prediction model authors should adhere to the TRIPOD (transparent reporting of a multivariable prediction model for individual prognosis or diagnosis) reporting guideline.

SYSTEMATIC REVIEW REGISTRATION

Protocol https://osf.io/ehc47/, registration https://osf.io/wy245.

READERS' NOTE

This article is a living systematic review that will be updated to reflect emerging evidence. Updates may occur for up to two years from the date of original publication. This version is update 3 of the original article published on 7 April 2020 (BMJ 2020;369:m1328). Previous updates can be found as data supplements (https://www.bmj.com/content/369/bmj.m1328/related#datasupp). When citing this paper please consider adding the update number and date of access for clarity.