Clinical Phenotyping with an Outcomes-driven Mixture of Experts for Patient Matching and Risk Estimation

Observational medical data present unique opportunities for analysis of medical outcomes and treatment decision making. However, because these datasets do not contain the strict pairing of randomized control trials, matching techniques are to draw comparisons among patients. A key limitation to such techniques is verification that the variables used to model treatment decision making are also relevant in identifying the risk of major adverse events. This article explores a deep mixture of experts approach to jointly learn how to match patients and model the risk of major adverse events in patients. Although trained with information regarding treatment and outcomes, after training, the proposed model is decomposable into a network that clusters patients into phenotypes from information available before treatment. This model is validated on a dataset of patients with acute myocardial infarction complicated by cardiogenic shock. The mixture of experts approach can predict the outcome of mortality with an area under the receiver operating characteristic curve of 0.85 ± 0.01 while jointly discovering five potential phenotypes of interest. The technique and interpretation allow for identifying clinically relevant phenotypes that may be used both for outcomes modeling as well as potentially evaluating individualized treatment effects.

A prediction model for asthma exacerbations after stopping asthma biologics

BACKGROUND

Little is known regarding the prediction of the risks of asthma exacerbation after stopping asthma biologics.

OBJECTIVE

To develop and validate a predictive model for the risk of asthma exacerbations after stopping asthma biologics using machine learning models.

METHODS

We identified 3057 people with asthma who stopped asthma biologics in the OptumLabs Database Warehouse and considered a wide range of demographic and clinical risk factors to predict subsequent outcomes. The primary outcome used to assess success after stopping was having no exacerbations in the 6 months after stopping the biologic. Elastic-net logistic regression (GLMnet), random forest, and gradient boosting machine models were used with 10-fold cross-validation within a development (80%) cohort and validation cohort (20%).

RESULTS

The mean age of the total cohort was 47.1 (SD, 17.1) years, 1859 (60.8%) were women, 2261 (74.0%) were White, and 1475 (48.3%) were in the Southern region of the United States. The elastic-net logistic regression model yielded an area under the curve (AUC) of 0.75 (95% confidence interval [CI], 0.71-0.78) in the development and an AUC of 0.72 in the validation cohort. The random forest model yielded an AUC of 0.75 (95% CI, 0.68-0.79) in the development cohort and an AUC of 0.72 in the validation cohort. The gradient boosting machine model yielded an AUC of 0.76 (95% CI, 0.72-0.80) in the development cohort and an AUC of 0.74 in the validation cohort.

CONCLUSION

Outcomes after stopping asthma biologics can be predicted with moderate accuracy using machine learning methods.

Impact of labor characteristics on maternal and neonatal outcomes of labor: A machine-learning model

Introduction

Since Friedman’s seminal publication on laboring women, numerous publications have sought to define normal labor progress. However, there is paucity of data on contemporary labor cervicometry incorporating both maternal and neonatal outcomes. The objective of this study is to establish intrapartum prediction models of unfavorable labor outcomes using machine-learning algorithms.

Materials and methods

Consortium on Safe Labor is a large database consisting of pregnancy and labor characteristics from 12 medical centers in the United States. Outcomes, including maternal and neonatal outcomes, were retrospectively collected. We defined primary outcome as the composite of following unfavorable outcomes: cesarean delivery in active labor, postpartum hemorrhage, intra-amniotic infection, shoulder dystocia, neonatal morbidity, and mortality. Clinical and obstetric parameters at admission and during labor progression were used to build machine-learning risk-prediction models based on the gradient boosting algorithm.

Results

Of 228,438 delivery episodes, 66,586 were eligible for this study. Mean maternal age was 26.95 ± 6.48 years, mean parity was 0.92 ± 1.23, and mean gestational age was 39.35 ± 1.13 weeks. Unfavorable labor outcome was reported in 14,439 (21.68%) deliveries. Starting at a cervical dilation of 4 cm, the area under receiver operating characteristics curve (AUC) of prediction models increased from 0.75 (95% confidence interval, 0.75–0.75) to 0.89 (95% confidence interval, 0.89–0.90) at a dilation of 10 cm. Baseline labor risk score was above 35% in patients with unfavorable outcomes compared to women with favorable outcomes, whose score was below 25%.

Conclusion

Labor risk score is a machine-learning–based score that provides individualized and dynamic alternatives to conventional labor charts. It predicts composite of adverse birth, maternal, and neonatal outcomes as labor progresses. Therefore, it can be deployed in clinical practice to monitor labor progress in real time and support clinical decisions.

Identifying Patients With Advanced Heart Failure Using Administrative Data

Objective

To develop algorithms to identify patients with advanced heart failure (HF) that can be applied to administrative data.

Patients and Methods

In a population-based cohort of all residents of Olmsted County, Minnesota, with greater than or equal to 1 HF billing code 2007-2017 (n=8657), we identified all patients with advanced HF (n=847) by applying the gold standard European Society of Cardiology advanced HF criteria via manual medical review by an HF cardiologist. The advanced HF index date was the date the patient first met all European Society of Cardiology criteria. We subsequently developed candidate algorithms to identify advanced HF using administrative data (billing codes and prescriptions relevant to HF or comorbidities that affect HF outcomes), applied them to the HF cohort, and assessed their ability to identify patients with advanced HF on or after their advanced HF index date.

Results

A single hospitalization for HF or ventricular arrhythmias identified all patients with advanced HF (sensitivity, 100%); however, the positive predictive value (PPV) was low (36.4%). More stringent definitions, including additional hospitalizations and/or other signs of advanced HF (hyponatremia, acute kidney injury, hypotension, or high-dose diuretic use), decreased the sensitivity but improved the specificity and PPV. For example, 2 hospitalizations plus 1 sign of advanced HF had a sensitivity of 72.7%, specificity of 89.8%, and PPV of 60.5%. Negative predictive values were high for all algorithms evaluated.

Conclusion

Algorithms using administrative data can identify patients with advanced HF with reasonable performance.

Estimation of manual wheelchair-based activities in the free-living environment using a neural network model with inertial body-worn sensors

Shoulder pain is common in manual wheelchair (MWC) users. Overuse is thought to be a major cause, but little is known about exposure to activities of daily living (ADLs). The study goal was to develop a method to estimate three conditions in the field: (1) non-propulsion activity, (2) MWC propulsion, and (3) static time using an inertial measurement unit (IMU). Upper arm IMU data were collected as ten MWC users performed lab-based MWC-related ADLs. A neural network model was developed to classify data as non-propulsion activity, propulsion, or static, and validated for the lab-based data collection by video comparison. Six of the participants' free-living IMU data were collected and the lab-based model was applied to estimate daily non-propulsion activity, propulsion, and static time. The neural network model yielded lab-based validity measures ≥0.87 for differentiating non-propulsion activity, propulsion, and static time. A quasi-validation of one participant's field-based data yielded validity measures ≥0.66 for identifying propulsion. Participants' estimated mean daily non-propulsion activity, propulsion, and static time ranged from 158 to 409, 13 to 25, and 367 to 609 min, respectively. The preliminary results suggest the model may be able to accurately identify MWC users' field-based activities. The inclusion of field-based IMU data in the model could further improve field-based classification.

Longitudinal cohorts for harnessing the electronic health record for disease prediction in a US population

PURPOSE

The depth and breadth of clinical data within electronic health record (EHR) systems paired with innovative machine learning methods can be leveraged to identify novel risk factors for complex diseases. However, analysing the EHR is challenging due to complexity and quality of the data. Therefore, we developed large electronic population-based cohorts with comprehensive harmonised and processed EHR data.

PARTICIPANTS

All individuals 30 years of age or older who resided in Olmsted County, Minnesota on 1 January 2006 were identified for the discovery cohort. Algorithms to define a variety of patient characteristics were developed and validated, thus building a comprehensive risk profile for each patient. Patients are followed for incident diseases and ageing-related outcomes. Using the same methods, an independent validation cohort was assembled by identifying all individuals 30 years of age or older who resided in the largely rural 26-county area of southern Minnesota and western Wisconsin on 1 January 2013.

FINDINGS TO DATE

For the discovery cohort, 76 255 individuals (median age 49; 53% women) were identified from which a total of 9 644 221 laboratory results; 9 513 840 diagnosis codes; 10 924 291 procedure codes; 1 277 231 outpatient drug prescriptions; 966 136 heart rate measurements and 1 159 836 blood pressure (BP) measurements were retrieved during the baseline time period. The most prevalent conditions in this cohort were hyperlipidaemia, hypertension and arthritis. For the validation cohort, 333 460 individuals (median age 54; 52% women) were identified and to date, a total of 19 926 750 diagnosis codes, 10 527 444 heart rate measurements and 7 356 344 BP measurements were retrieved during baseline.

FUTURE PLANS

Using advanced machine learning approaches, these electronic cohorts will be used to identify novel sex-specific risk factors for complex diseases. These approaches will allow us to address several challenges with the use of EHR.

Use of Mechanical Circulatory Support Devices Among Patients With Acute Myocardial Infarction Complicated by Cardiogenic Shock

IMPORTANCE

Mechanical circulatory support (MCS) devices, including intravascular microaxial left ventricular assist devices (LVADs) and intra-aortic balloon pumps (IABPs), are used in patients who undergo percutaneous coronary intervention (PCI) for acute myocardial infarction (AMI) complicated by cardiogenic shock despite limited evidence of their clinical benefit.

OBJECTIVE

To examine trends in the use of MCS devices among patients who underwent PCI for AMI with cardiogenic shock, hospital-level use variation, and factors associated with use.

DESIGN, SETTING, AND PARTICIPANTS

This cross-sectional study used the CathPCI and Chest Pain-MI Registries of the American College of Cardiology National Cardiovascular Data Registry. Patients who underwent PCI for AMI complicated by cardiogenic shock between October 1, 2015, and December 31, 2017, were identified from both registries. Data were analyzed from October 2018 to August 2020.

EXPOSURES

Therapies to provide hemodynamic support were categorized as intravascular microaxial LVAD, IABP, TandemHeart, extracorporeal membrane oxygenation, LVAD, other devices, combined IABP and intravascular microaxial LVAD, combined IABP and other device (defined as TandemHeart, extracorporeal membrane oxygenation, LVAD, or another MCS device), or medical therapy only.

MAIN OUTCOMES AND MEASURES

Use of MCS devices overall and specific MCS devices, including intravascular microaxial LVAD, at both patient and hospital levels and variables associated with use.

RESULTS

Among the 28 304 patients included in the study, the mean (SD) age was 65.4 (12.6) years and 18 968 were men (67.0%). The overall MCS device use was constant from the fourth quarter of 2015 to the fourth quarter of 2017, although use of intravascular microaxial LVADs significantly increased (from 4.1% to 9.8%; P < .001), whereas use of IABPs significantly decreased (from 34.8% to 30.0%; P < .001). A significant hospital-level variation in MCS device use was found. The median (interquartile range [IQR]) proportion of patients who received MCS devices was 42% (30%-54%), and the median proportion of patients who received intravascular microaxial LVADs was 1% (0%-10%). In multivariable analyses, cardiac arrest at first medical contact or during hospitalization (odds ratio [OR], 1.82; 95% CI, 1.58-2.09) and severe left main and/or proximal left anterior descending coronary artery stenosis (OR, 1.36; 95% CI, 1.20-1.54) were patient characteristics that were associated with higher odds of receiving intravascular microaxial LVADs only compared with IABPs only.

CONCLUSIONS AND RELEVANCE

This study found that, among patients who underwent PCI for AMI complicated by cardiogenic shock, overall use of MCS devices was constant, and a 2.5-fold increase in intravascular microaxial LVAD use was found along with a corresponding decrease in IABP use and a significant hospital-level variation in MCS device use. These trends were observed despite limited clinical trial evidence of improved outcomes associated with device use.

Comparative Effectiveness and Safety of Oral Anticoagulants Across Kidney Function in Patients With Atrial Fibrillation

BACKGROUND

Patients with atrial fibrillation and severely decreased kidney function were excluded from the pivotal non-vitamin K antagonist oral anticoagulants (NOAC) trials, thereby raising questions about comparative safety and effectiveness in patients with reduced kidney function. The study aimed to compare oral anticoagulants across the range of kidney function in patients with atrial fibrillation.

METHODS AND RESULTS

Using a US administrative claims database with linked laboratory data, 34 569 new users of oral anticoagulants with atrial fibrillation and estimated glomerular filtration rate ≥15 mL/(min·1.73 m²) were identified between October 1, 2010 to November 29, 2017. The proportion of patients using NOACs declined with decreasing kidney function-73.5%, 69.6%, 65.4%, 59.5%, and 45.0% of the patients were prescribed a NOAC in estimated glomerular filtration rate ≥90, 60 to 90, 45 to 60, 30 to 45, 15 to 30 mL/min per 1.73 m² groups, respectively. Stabilized inverse probability of treatment weighting was used to balance 4 treatment groups (apixaban, dabigatran, rivaroxaban, and warfarin) on 66 baseline characteristics. In comparison to warfarin, apixaban was associated with a lower risk of stroke (hazard ratio [HR], 0.57 [0.43-0.75]; P<0.001), major bleeding (HR, 0.51 [0.44-0.61]; P<0.001), and mortality (HR, 0.68 [0.56-0.83]; P<0.001); dabigatran was associated with a similar risk of stroke but a lower risk of major bleeding (HR, 0.57 [0.43-0.75]; P<0.001) and mortality (HR, 0.68 [0.48-0.98]; P=0.04); rivaroxaban was associated with a lower risk of stroke (HR, 0.69 [0.51-0.94]; P=0.02), major bleeding (HR, 0.84 [0.72-0.99]; P=0.04), and mortality (HR, 0.73 [0.58-0.91]; P=0.006). There was no significant interaction between treatment and estimated glomerular filtration rate categories for any outcome. When comparing one NOAC to another NOAC, there was no significant difference in mortality, but some differences existed for stroke or major bleeding. No relationship between treatments and falsification end points was found, suggesting no evidence for substantial residual confounding.

CONCLUSIONS

Relative to warfarin, NOACs are used less frequently as kidney function declines. However, NOACs appears to have similar or better comparative effectiveness and safety across the range of kidney function.

Improved Interpretability of Machine Learning Model Using Unsupervised Clustering: Predicting Time to First Treatment in Chronic Lymphocytic Leukemia

PURPOSE

Time to event is an important aspect of clinical decision making. This is particularly true when diseases have highly heterogeneous presentations and prognoses, as in chronic lymphocytic lymphoma (CLL). Although machine learning methods can readily learn complex nonlinear relationships, many methods are criticized as inadequate because of limited interpretability. We propose using unsupervised clustering of the continuous output of machine learning models to provide discrete risk stratification for predicting time to first treatment in a cohort of patients with CLL.

PATIENTS AND METHODS

A total of 737 treatment-naïve patients with CLL diagnosed at Mayo Clinic were included in this study. We compared predictive abilities for two survival models (Cox proportional hazards and random survival forest) and four classification methods (logistic regression, support vector machines, random forest, and gradient boosting machine). Probability of treatment was then stratified.

RESULTS

Machine learning methods did not yield significantly more accurate predictions of time to first treatment. However, automated risk stratification provided by clustering was able to better differentiate patients who were at risk for treatment within 1 year than models developed using standard survival analysis techniques.

CONCLUSION

Clustering the posterior probabilities of machine learning models provides a way to better interpret machine learning models.

Abstract 11: Comparative Effectiveness and Safety of Oral Anticoagulants across Baseline Kidney Function

Background: Stroke prevention using warfarin is challenging in AF patients with CKD, due to high bleeding risk and difficulties in the INR control. NOACs provide alternative options, but all have greater degrees of renal clearance. This study aimed to compare the outcomes of apixaban, dabigatran, rivaroxaban, and warfarin across the range of kidney function in patients with AF.

Methods: Using a US administrative database including private insurance or Medicare Advantage patients with linked claims and laboratory data, we identified 34,569 new users of oral anticoagulants with AF and eGFR ≥15 between 10/1/2010-11/29/2017. Stabilized IPTW balanced four treatment groups on 66 baseline characteristics. The primary outcomes included stroke, major bleeding, and mortality. Weighted Cox proportional hazards models compared treatments in the overall population and in each eGFR subgroup, with mortality as a competing risk for stroke and major bleeding.

Results: The proportion of patients using warfarin increased as the kidney function declined - 26.5%, 30.4%, 34.6%, 40.5%, and 55.0% of patients were prescribed warfarin in eGFR ≥90, 60-90, 45-60, 30-45, 15-30 groups, respectively. In comparison to warfarin, apixaban was associated with a lower risk of stroke, major bleeding, and mortality; dabigatran was associated with a similar risk of stroke, and a lower risk of major bleeding and mortality; rivaroxaban was associated with a lower risk of stroke, major bleeding, and mortality (Figure). When comparing one NOAC to another NOAC, apixaban and dabigatran were associated with a lower risk of major bleeding than rivaroxaban (HR 0.61 [0.51-0.73], p<0.001 for apixaban versus rivaroxaban; HR 0.67 [0.50-0.90], p=0.007 for dabigatran versus rivaroxaban); dabigatran was associated with a higher risk of stroke than apixaban (HR 1.65 [1.11-2.46], p=0.01); there was no difference in mortality. There was no significant interaction between treatment and eGFR categories for any outcome, but the number of patients with low eGFR was small.

Conclusions: In practice, relative to warfarin, NOACs are progressively less commonly used with increasing degree of renal dysfunction. However, each NOAC was consistently associated with at least equivalent effectiveness and safety compared with warfarin across the range of kidney function.

Automated Diabetes Case Identification Using Electronic Health Record Data at a Tertiary Care Facility

Objective

To develop and validate a phenotyping algorithm for the identification of patients with type 1 and type 2 diabetes mellitus (DM) preoperatively using routinely available clinical data from electronic health records.

Patients and Methods

We used first-order logic rules (if-then-else rules) to imply the presence or absence of DM types 1 and 2. The “if” clause of each rule is a conjunction of logical and, or predicates that provides evidence toward or against the presence of DM. The rule includes International Classification of Diseases, Ninth Revision, Clinical Modification diagnostic codes, outpatient prescription information, laboratory values, and positive annotation of DM in patients’ clinical notes. This study was conducted from March 2, 2015, through February 10, 2016. The performance of our rule-based approach and similar approaches proposed by other institutions was evaluated with a reference standard created by an expert reviewer and implemented for routine clinical care at an academic medical center.

Results

A total of 4208 surgical patients (mean age, 52 years; males, 48%) were analyzed to develop the phenotyping algorithm. Expert review identified 685 patients (16.28% of the full cohort) as having DM. Our proposed method identified 684 patients (16.25%) as having DM. The algorithm performed well—99.70% sensitivity, 99.97% specificity—and compared favorably with previous approaches.

Conclusion

Among patients undergoing surgery, determination of DM can be made with high accuracy using simple, computationally efficient rules. Knowledge of patients’ DM status before surgery may alter physicians’ care plan and reduce postsurgical complications. Nevertheless, future efforts are necessary to determine the effect of first-order logic rules on clinical processes and patient outcomes.