Development and validation of self-monitoring auto-updating prognostic models of survival for hospitalized COVID-19 patients

Seeing the random forest through the decision trees. Supporting learning health systems from histopathology with machine learning models: Challenges and opportunities

This paper discusses some overlooked challenges faced when working with machine learning models for histopathology and presents a novel opportunity to support "Learning Health Systems" with them. Initially, the authors elaborate on these challenges after separating them according to their mitigation strategies: those that need innovative approaches, time, or future technological capabilities and those that require a conceptual reappraisal from a critical perspective. Then, a novel opportunity to support "Learning Health Systems" by integrating hidden information extracted by ML models from digitalized histopathology slides with other healthcare big data is presented.

European and US Guideline-Based Statin Eligibility, Genetically Predicted Coronary Artery Disease, and the Risk of Major Coronary Events

BACKGROUND

A study was designed to investigate whether the coronary artery disease polygenic risk score (CAD-PRS) may guide lipid-lowering treatment initiation as well as deferral in primary prevention beyond established clinical risk scores.

METHODS AND RESULTS

Participants were 311 799 individuals from the UK Biobank free of atherosclerotic cardiovascular disease, diabetes, chronic kidney disease, and lipid-lowering treatment at baseline. Participants were categorized as statin indicated, statin indication unclear, or statin not indicated as defined by the European and US guidelines on statin use. For a median of 11.9 (11.2-12.6) years, 8196 major coronary events developed. CAD-PRS added to European-Systematic Coronary Risk Evaluation 2 (European-SCORE2) and US-Pooled Cohort Equation (US-PCE) identified 18% and 12% of statin-indication-unclear individuals whose risk of major coronary events were the same as or higher than the average risk of statin-indicated individuals and 16% and 12% of statin-indicated individuals whose major coronary event risks were the same as or lower than the average risk of statin-indication-unclear individuals. For major coronary and atherosclerotic cardiovascular disease events, CAD-PRS improved C-statistics greater among statin-indicated or statin-indication-unclear than statin-not-indicated individuals. For atherosclerotic cardiovascular disease events, CAD-PRS added to the European evaluation and US equation resulted in a net reclassification improvement of 13.6% (95% CI, 11.8-15.5) and 14.7% (95% CI, 13.1-16.3) among statin-indicated, 10.8% (95% CI, 9.6-12.0) and 15.3% (95% CI, 13.2-17.5) among statin-indication-unclear, and 0.9% (95% CI, 0.6-1.3) and 3.6% (95% CI, 3.0-4.2) among statin-not-indicated individuals.

CONCLUSIONS

CAD-PRS may guide statin initiation as well as deferral among statin-indication-unclear or statin-indicated individuals as defined by the European and US guidelines. CAD-PRS had little clinical utility among statin-not-indicated individuals.

Empirical data drift detection experiments on real-world medical imaging data

While it is common to monitor deployed clinical artificial intelligence (AI) models for performance degradation, it is less common for the input data to be monitored for data drift - systemic changes to input distributions. However, when real-time evaluation may not be practical (eg., labeling costs) or when gold-labels are automatically generated, we argue that tracking data drift becomes a vital addition for AI deployments. In this work, we perform empirical experiments on real-world medical imaging to evaluate three data drift detection methods' ability to detect data drift caused (a) naturally (emergence of COVID-19 in X-rays) and (b) synthetically. We find that monitoring performance alone is not a good proxy for detecting data drift and that drift-detection heavily depends on sample size and patient features. Our work discusses the need and utility of data drift detection in various scenarios and highlights gaps in knowledge for the practical application of existing methods.

Longitudinal dynamic clinical phenotypes of in-hospital COVID-19 patients across three dominant virus variants in New York

BACKGROUND

COVID-19 is a challenging disease to characterize given its wide-ranging heterogeneous symptomatology. Several studies have attempted to extract clinical phenotypes but often relied on data from small patient cohorts, usually limited to only one viral variant and utilizing a static snapshot of patient data.

OBJECTIVE

This study aimed to identify clinical phenotypes of hospitalized COVID-19 patients and investigate their longitudinal dynamics throughout the pandemic, with the goal to relate these phenotypes to clinical outcomes and treatment strategies.

METHODS

We utilized routinely collected demographic and clinical data throughout the hospitalization of 38,077 patients admitted between 3/2020 to 5/2022, in 12 New York hospitals. Uniform Manifold Approximation and Projection and agglomerative hierarchical clustering were used to derive the clusters, followed by exploratory data analysis to compare the prevalence of comorbidities and treatments per cluster.

RESULTS

4 distinct clinical phenotypes remained robust in multi-site validation and were associated with different mortality rates. The temporal progression of these phenotypes throughout the COVID-19 pandemic demonstrated increased variability across the waves of the three dominant viral variants (alpha, delta, omicron). Longitudinal analysis evaluating changes in clinical phenotypes of each patient throughout the course of a 4-week hospital stay exemplified the dynamic nature of the disease progression. Factors such as sex, race/ethnicity and specific treatment modalities revealed significant and clinically relevant differences between the observed phenotypes.

CONCLUSIONS

Our proposed methodology has the potential of enabling clinicians and policy makers to draw evidence-based conclusions for guiding treatment modalities in a dynamic fashion.

Automatic correction of performance drift under acquisition shift in medical image classification

Image-based prediction models for disease detection are sensitive to changes in data acquisition such as the replacement of scanner hardware or updates to the image processing software. The resulting differences in image characteristics may lead to drifts in clinically relevant performance metrics which could cause harm in clinical decision making, even for models that generalise in terms of area under the receiver-operating characteristic curve. We propose Unsupervised Prediction Alignment, a generic automatic recalibration method that requires no ground truth annotations and only limited amounts of unlabelled example images from the shifted data distribution. We illustrate the effectiveness of the proposed method to detect and correct performance drift in mammography-based breast cancer screening and on publicly available histopathology data. We show that the proposed method can preserve the expected performance in terms of sensitivity/specificity under various realistic scenarios of image acquisition shift, thus offering an important safeguard for clinical deployment.

The association of clinically relevant variables with chest radiograph lung disease burden quantified in real-time by radiologists upon initial presentation in individuals hospitalized with COVID-19

Objectives

We aimed to correlate lung disease burden on presentation chest radiographs (CXR), quantified at the time of study interpretation, with clinical presentation in patients hospitalized with coronavirus disease 2019 (COVID-19).

Material and methods

This retrospective cross-sectional study included 5833 consecutive adult patients, aged 18 and older, hospitalized with a diagnosis of COVID-19 with a CXR quantified in real-time while hospitalized in 1 of 12 acute care hospitals across a multihospital integrated healthcare network between March 24, 2020, and May 22, 2020. Lung disease burden was quantified in real-time by 118 radiologists on 5833 CXR at the time of exam interpretation with each lung annotated by the degree of lung opacity as clear (0%), mild (1–33%), moderate (34–66%), or severe (67–100%). CXR findings were classified as (1) clear versus disease, (2) unilateral versus bilateral, (3) symmetric versus asymmetric, or (4) not severe versus severe. Lung disease burden was characterized on initial presentation by patient demographics, co-morbidities, vital signs, and lab results with chi-square used for univariate analysis and logistic regression for multivariable analysis.

Results

Patients with severe lung disease were more likely to have oxygen impairment, an elevated respiratory rate, low albumin, high lactate dehydrogenase, and high ferritin compared to non-severe lung disease. A lack of opacities in COVID-19 was associated with a low estimated glomerular filtration rate, hypernatremia, and hypoglycemia.

Conclusions

COVID-19 lung disease burden quantified in real-time on presentation CXR was characterized by demographics, comorbidities, emergency severity index, Charlson Comorbidity Index, vital signs, and lab results on 5833 patients. This novel approach to real-time quantified chest radiograph lung disease burden by radiologists needs further research to understand how this information can be incorporated to improve clinical care for pulmonary-related diseases.. An absence of opacities in COVID-19 may be associated with poor oral intake and a prerenal state as evidenced by the association of clear CXRs with a low eGFR, hypernatremia, and hypoglycemia.

US and Dutch Perspectives on the Use of COVID-19 Clinical Prediction Models: Findings from a Qualitative Analysis

INTRODUCTION

Clinical prediction models (CPMs) for coronavirus disease 2019 (COVID-19) may support clinical decision making, treatment, and communication. However, attitudes about using CPMs for COVID-19 decision making are unknown.

METHODS

Online focus groups and interviews were conducted among health care providers, survivors of COVID-19, and surrogates (i.e., loved ones/surrogate decision makers) in the United States and the Netherlands. Semistructured questions explored experiences about clinical decision making in COVID-19 care and facilitators and barriers for implementing CPMs.

RESULTS

In the United States, we conducted 4 online focus groups with 1) providers and 2) surrogates and survivors of COVID-19 between January 2021 and July 2021. In the Netherlands, we conducted 3 focus groups and 4 individual interviews with 1) providers and 2) surrogates and survivors of COVID-19 between May 2021 and July 2021. Providers expressed concern about CPM validity and the belief that patients may interpret CPM predictions as absolute. They described CPMs as potentially useful for resource allocation, triaging, education, and research. Several surrogates and people who had COVID-19 were not given prognostic estimates but believed this information would have supported and influenced their decision making. A limited number of participants felt the data would not have applied to them and that they or their loved ones may not have survived, as poor prognosis may have suggested withdrawal of treatment.

CONCLUSIONS

Many providers had reservations about using CPMs for people with COVID-19 due to concerns about CPM validity and patient-level interpretation of the outcome predictions. However, several people who survived COVID-19 and their surrogates indicated that they would have found this information useful for decision making. Therefore, information provision may be needed to improve provider-level comfort and patient and surrogate understanding of CPMs.

HIGHLIGHTS

While clinical prediction models (CPMs) may provide an objective means of assessing COVID-19 prognosis, provider concerns about CPM validity and the interpretation of CPM predictions may limit their clinical use.Providers felt that CPMs may be most useful for resource allocation, triage, research, or educational purposes for COVID-19.Several survivors of COVID-19 and their surrogates felt that CPMs would have been informative and may have aided them in making COVID-19 treatment decisions, while others felt the data would not have applied to them.

Prediction of intrapartum fever using continuously monitored vital signs and heart rate variability

Objectives

Neonatal early onset sepsis (EOS), bacterial infection during the first seven days of life, is difficult to diagnose because presenting signs are non-specific, but early diagnosis before birth can direct life-saving treatment for mother and baby. Specifically, maternal fever during labor from placental infection is the strongest predictor of EOS. Alterations in maternal heart rate variability (HRV) may precede development of intrapartum fever, enabling incipient EOS detection. The objective of this work was to build a predictive model for intrapartum fever.

Methods

Continuously measured temperature, heart rate, and beat-to-beat RR intervals were obtained from wireless sensors on women (n = 141) in labor; traditional manual vital signs were taken every 3-6 hours. Validated measures of HRV were calculated in moving 5-minute windows of RR intervals: standard deviation of normal-to-normal intervals (SDNN) and root mean square of successive differences (RMSSD) between normal heartbeats.

Results

Fever (>38.0 °C) was detected by manual or continuous measurements in 48 women. Compared to afebrile mothers, average SDNN and RMSSD in febrile mothers decreased significantly (p < 0.001) at 2 and 3 hours before fever onset, respectively. This observed HRV divergence and raw recorded vitals were applied to a logistic regression model at various time horizons, up to 4-5 hours before fever onset. Model performance increased with decreasing time horizons, and a model built using continuous vital signs as input variables consistently outperformed a model built from episodic vital signs.

Conclusions

HRV-based predictive models could identify mothers at risk for fever and infants at risk for EOS, guiding maternal antibiotic prophylaxis and neonatal monitoring.