Development of An Individualized Risk Prediction Model for COVID-19 Using Electronic Health Record Data

Prediction model to identify infectious COVID-19 patients in the emergency department

Background

Real-time reverse-transcriptase polymerase chain reaction (RT-PCR) has been the gold standard for diagnosing coronavirus disease 2019 (COVID-19) but has a lag time for the results. An effective prediction algorithm for infectious COVID-19, utilized at the emergency department (ED), may reduce the risk of healthcare-associated COVID-19.

Objective

To develop a prototypic prediction model for infectious COVID-19 at the time of presentation to the ED.

Material and methods

Retrospective cohort study of all adult patients admitted to Singapore General Hospital (SGH) through ED between March 15, 2020, and December 31, 2022, with admission of COVID-19 RT-PCR results. Two prediction models were developed and evaluated using area under the curve (AUC) of receiver operating characteristics (ROC) to identify infectious COVID-19 patients (cycle threshold (Ct) of <25).

Results

Total of 78,687 patients were admitted to SGH through ED during study period. 6,132 of them tested severe acute respiratory coronavirus 2 positive on RT-PCR. Nearly 70% (4,226 of 6,132) of the patients had infectious COVID-19 (Ct<25). Model that included demographics, clinical history, symptom and laboratory variables had AUROC of 0.85 with sensitivity and specificity of 80.0% & 72.1% respectively. When antigen rapid test results at ED were available and added to the model for a subset of the study population, AUROC reached 0.97 with sensitivity and specificity of 95.0% and 92.8% respectively. Both models maintained respective sensitivity and specificity results when applied to validation data.

Conclusion

Clinical predictive models based on available information at ED can be utilized for identification of infectious COVID-19 patients and may enhance infection prevention efforts.

Individualized Prediction of SARS-CoV-2 Infection in Mexico City Municipality during the First Six Waves of the Pandemic

After COVID-19 emerged, alternative methods to laboratory tests for the individualized prediction of SARS-CoV-2 were developed in several world regions. The objective of this investigation was to develop models for the individualized prediction of SARS-CoV-2 infection in a large municipality of Mexico. The study included data from 36,949 patients with suspected SARS-CoV-2 infection who received a diagnostic tested at health centers of the Alvaro Obregon Jurisdiction in Mexico City registered in the Epidemiological Surveillance System for Viral Respiratory Diseases (SISVER-SINAVE). The variables that were different between a positive test and a negative test were used to generate multivariate binary logistic regression models. There was a large variation in the prediction variables for the models of different pandemic waves. The models obtained an overall accuracy of 73% (63-82%), sensitivity of 52% (18-71%), and specificity of 84% (71-92%). In conclusion, the individualized prediction models of a positive COVID-19 test based on SISVER-SINAVE data had good performance. The large variation in the prediction variables for the models of different pandemic waves highlights the continuous change in the factors that influence the spread of COVID-19. These prediction models could be applied in early case identification strategies, especially in vulnerable populations.

Synthetic Health Data Can Augment Community Research Efforts to Better Inform the Public During Emerging Pandemics

The COVID-19 pandemic had disproportionate effects on the Veteran population due to the increased prevalence of medical and environmental risk factors. Synthetic electronic health record (EHR) data can help meet the acute need for Veteran population-specific predictive modeling efforts by avoiding the strict barriers to access, currently present within Veteran Health Administration (VHA) datasets. The U.S. Food and Drug Administration (FDA) and the VHA launched the precisionFDA COVID-19 Risk Factor Modeling Challenge to develop COVID-19 diagnostic and prognostic models; identify Veteran population-specific risk factors; and test the usefulness of synthetic data as a substitute for real data. The use of synthetic data boosted challenge participation by providing a dataset that was accessible to all competitors. Models trained on synthetic data showed similar but systematically inflated model performance metrics to those trained on real data. The important risk factors identified in the synthetic data largely overlapped with those identified from the real data, and both sets of risk factors were validated in the literature. Tradeoffs exist between synthetic data generation approaches based on whether a real EHR dataset is required as input. Synthetic data generated directly from real EHR input will more closely align with the characteristics of the relevant cohort. This work shows that synthetic EHR data will have practical value to the Veterans' health research community for the foreseeable future.

Predicting SARS-CoV-2 infection among hemodialysis patients using multimodal data

Background

The coronavirus disease 2019 (COVID-19) pandemic has created more devastation among dialysis patients than among the general population. Patient-level prediction models for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection are crucial for the early identification of patients to prevent and mitigate outbreaks within dialysis clinics. As the COVID-19 pandemic evolves, it is unclear whether or not previously built prediction models are still sufficiently effective.

Methods

We developed a machine learning (XGBoost) model to predict during the incubation period a SARS-CoV-2 infection that is subsequently diagnosed after 3 or more days. We used data from multiple sources, including demographic, clinical, treatment, laboratory, and vaccination information from a national network of hemodialysis clinics, socioeconomic information from the Census Bureau, and county-level COVID-19 infection and mortality information from state and local health agencies. We created prediction models and evaluated their performances on a rolling basis to investigate the evolution of prediction power and risk factors.

Result

From April 2020 to August 2020, our machine learning model achieved an area under the receiver operating characteristic curve (AUROC) of 0.75, an improvement of over 0.07 from a previously developed machine learning model published by Kidney360 in 2021. As the pandemic evolved, the prediction performance deteriorated and fluctuated more, with the lowest AUROC of 0.6 in December 2021 and January 2022. Over the whole study period, that is, from April 2020 to February 2022, fixing the false-positive rate at 20%, our model was able to detect 40% of the positive patients. We found that features derived from local infection information reported by the Centers for Disease Control and Prevention (CDC) were the most important predictors, and vaccination status was a useful predictor as well. Whether or not a patient lives in a nursing home was an effective predictor before vaccination, but became less predictive after vaccination.

Conclusion

As found in our study, the dynamics of the prediction model are frequently changing as the pandemic evolves. County-level infection information and vaccination information are crucial for the success of early COVID-19 prediction models. Our results show that the proposed model can effectively identify SARS-CoV-2 infections during the incubation period. Prospective studies are warranted to explore the application of such prediction models in daily clinical practice.

Establishment of an International Evidence Sharing Network Through Common Data Model for Cardiovascular Research

A retrospective observational study is one of the most widely used research methods in medicine. However, evidence postulated from a single data source likely contains biases such as selection bias, information bias, and confounding bias. Acquiring enough data from multiple institutions is one of the most effective methods to overcome the limitations. However, acquiring data from multiple institutions from many countries requires enormous effort because of financial, technical, ethical, and legal issues as well as standardization of data structure and semantics. The Observational Health Data Sciences and Informatics (OHDSI) research network standardized 928 million unique records or 12% of the world's population into a common structure and meaning and established a research network of 453 data partners from 41 countries around the world. OHDSI is a distributed research network wherein researchers do not own or directly share data but only analyzed results. However, sharing evidence without sharing data is difficult to understand. In this review, we will look at the basic principles of OHDSI, common data model, distributed research networks, and some representative studies in the cardiovascular field using the network. This paper also briefly introduces a Korean distributed research network named FeederNet.

A Calculator for COVID-19 Severity Prediction Based on Patient Risk Factors and Number of Vaccines Received

Vaccines have allowed for a significant decrease in COVID-19 risk, and new antiviral medications can prevent disease progression if given early in the course of the disease. The rapid and accurate estimation of the risk of severe disease in new patients is needed to prioritize the treatment of high-risk patients and maximize lives saved. We used electronic health records from 101,039 individuals infected with SARS-CoV-2, since the beginning of the pandemic and until 30 November 2021, in a national healthcare organization in Israel to build logistic models estimating the probability of subsequent hospitalization and death of newly infected patients based on a few major risk factors (age, sex, body mass index, hemoglobin A1C, kidney function, and the presence of hypertension, pulmonary disease, and malignancy) and the number of BNT162b2 mRNA vaccine doses received. The model’s performance was assessed by 10-fold cross-validation: the area under the curve was 0.889 for predicting hospitalization and 0.967 for predicting mortality. A total of 50%, 80%, and 90% of death events could be predicted with respective specificities of 98.6%, 95.2%, and 91.2%. These models enable the rapid identification of individuals at high risk for hospitalization and death when infected, and they can be used to prioritize patients to receive scarce medications or booster vaccination. The calculator is available online.

Predicting COVID-19 severity using major risk factors and received vaccines

BACKGROUND

Vaccines are highly effective in preventing severe disease and death from COVID-19, and new medications that can reduce severity of disease have been approved. However, many countries are facing limited supply of vaccine doses and medications. A model estimating the probabilities for hospitalization and mortality according to individual risk factors and vaccine doses received could help prioritize vaccination and yet scarce medications to maximize lives saved and reduce the burden on hospitalization facilities.

METHODS

Electronic health records from 101,039 individuals infected with SARS-CoV-2, since the beginning of the pandemic and until November 30, 2021 were extracted from a national healthcare organization in Israel. Logistic regression models were built to estimate the risk for subsequent hospitalization and death based on the number of BNT162b2 mRNA vaccine doses received and few major risk factors (age, sex, body mass index, hemoglobin A1C, kidney function, and presence of hypertension, pulmonary disease and malignancy).

RESULTS

The models built predict the outcome of newly infected individuals with remarkable accuracy: area under the curve was 0.889 for predicting hospitalization, and 0.967 for predicting mortality. Even when a breakthrough infection occurs, having received three vaccination doses significantly reduces the risk of hospitalization by 66% (OR=0.339) and of death by 78% (OR=0.223).

CONCLUSIONS

The models enable rapid identification of individuals at high risk for hospitalization and death when infected. These patients can be prioritized to receive booster vaccination and the yet scarce medications. A calculator based on these models is made publicly available on http://covidest.web.app.

Real-World Evidence of COVID-19 Patients' Data Quality in the Electronic Health Records

Despite the importance of electronic health records data, less attention has been given to data quality. This study aimed to evaluate the quality of COVID-19 patients' records and their readiness for secondary use. We conducted a retrospective chart review study of all COVID-19 inpatients in an academic healthcare hospital for the year 2020, which were identified using ICD-10 codes and case definition guidelines. COVID-19 signs and symptoms were higher in unstructured clinical notes than in structured coded data. COVID-19 cases were categorized as 218 (66.46%) "confirmed cases", 10 (3.05%) "probable cases", 9 (2.74%) "suspected cases", and 91 (27.74%) "no sufficient evidence". The identification of "probable cases" and "suspected cases" was more challenging than "confirmed cases" where laboratory confirmation was sufficient. The accuracy of the COVID-19 case identification was higher in laboratory tests than in ICD-10 codes. When validating using laboratory results, we found that ICD-10 codes were inaccurately assigned to 238 (72.56%) patients' records. "No sufficient evidence" records might indicate inaccurate and incomplete EHR data. Data quality evaluation should be incorporated to ensure patient safety and data readiness for secondary use research and predictive analytics. We encourage educational and training efforts to motivate healthcare providers regarding the importance of accurate documentation at the point-of-care.