Evaluating Algorithmic Bias in 30-Day Hospital Readmission Models: Retrospective Analysis

BACKGROUND

The adoption of predictive algorithms in health care comes with the potential for algorithmic bias, which could exacerbate existing disparities. Fairness metrics have been proposed to measure algorithmic bias, but their application to real-world tasks is limited.

OBJECTIVE

This study aims to evaluate the algorithmic bias associated with the application of common 30-day hospital readmission models and assess the usefulness and interpretability of selected fairness metrics.

METHODS

We used 10.6 million adult inpatient discharges from Maryland and Florida from 2016 to 2019 in this retrospective study. Models predicting 30-day hospital readmissions were evaluated: LACE Index, modified HOSPITAL score, and modified Centers for Medicare & Medicaid Services (CMS) readmission measure, which were applied as-is (using existing coefficients) and retrained (recalibrated with 50% of the data). Predictive performances and bias measures were evaluated for all, between Black and White populations, and between low- and other-income groups. Bias measures included the parity of false negative rate (FNR), false positive rate (FPR), 0-1 loss, and generalized entropy index. Racial bias represented by FNR and FPR differences was stratified to explore shifts in algorithmic bias in different populations.

RESULTS

The retrained CMS model demonstrated the best predictive performance (area under the curve: 0.74 in Maryland and 0.68-0.70 in Florida), and the modified HOSPITAL score demonstrated the best calibration (Brier score: 0.16-0.19 in Maryland and 0.19-0.21 in Florida). Calibration was better in White (compared to Black) populations and other-income (compared to low-income) groups, and the area under the curve was higher or similar in the Black (compared to White) populations. The retrained CMS and modified HOSPITAL score had the lowest racial and income bias in Maryland. In Florida, both of these models overall had the lowest income bias and the modified HOSPITAL score showed the lowest racial bias. In both states, the White and higher-income populations showed a higher FNR, while the Black and low-income populations resulted in a higher FPR and a higher 0-1 loss. When stratified by hospital and population composition, these models demonstrated heterogeneous algorithmic bias in different contexts and populations.

CONCLUSIONS

Caution must be taken when interpreting fairness measures' face value. A higher FNR or FPR could potentially reflect missed opportunities or wasted resources, but these measures could also reflect health care use patterns and gaps in care. Simply relying on the statistical notions of bias could obscure or underplay the causes of health disparity. The imperfect health data, analytic frameworks, and the underlying health systems must be carefully considered. Fairness measures can serve as a useful routine assessment to detect disparate model performances but are insufficient to inform mechanisms or policy changes. However, such an assessment is an important first step toward data-driven improvement to address existing health disparities.

Predicting pressure injury risk in hospitalised patients using machine learning with electronic health records: a US multilevel cohort study

OBJECTIVE

To predict the risk of hospital-acquired pressure injury using machine learning compared with standard care.

DESIGN

We obtained electronic health records (EHRs) to structure a multilevel cohort of hospitalised patients at risk for pressure injury and then calibrate a machine learning model to predict future pressure injury risk. Optimisation methods combined with multilevel logistic regression were used to develop a predictive algorithm of patient-specific shifts in risk over time. Machine learning methods were tested, including random forests, to identify predictive features for the algorithm. We reported the results of the regression approach as well as the area under the receiver operating characteristics (ROC) curve for predictive models.

SETTING

Hospitalised inpatients.

PARTICIPANTS

EHRs of 35 001 hospitalisations over 5 years across 2 academic hospitals.

MAIN OUTCOME MEASURE

Longitudinal shifts in pressure injury risk.

RESULTS

The predictive algorithm with features generated by machine learning achieved significantly improved prediction of pressure injury risk (p<0.001) with an area under the ROC curve of 0.72; whereas standard care only achieved an area under the ROC curve of 0.52. At a specificity of 0.50, the predictive algorithm achieved a sensitivity of 0.75.

CONCLUSIONS

These data could help hospitals conserve resources within a critical period of patient vulnerability of hospital-acquired pressure injury which is not reimbursed by US Medicare; thus, conserving between 30 000 and 90 000 labour-hours per year in an average 500-bed hospital. Hospitals can use this predictive algorithm to initiate a quality improvement programme for pressure injury prevention and further customise the algorithm to patient-specific variation by facility.

Use of artificial intelligence in critical care: opportunities and obstacles

BACKGROUND

Perhaps nowhere else in the healthcare system than in the intensive care unit environment are the challenges to create useful models with direct time-critical clinical applications more relevant and the obstacles to achieving those goals more massive. Machine learning-based artificial intelligence (AI) techniques to define states and predict future events are commonplace activities of modern life. However, their penetration into acute care medicine has been slow, stuttering and uneven. Major obstacles to widespread effective application of AI approaches to the real-time care of the critically ill patient exist and need to be addressed.

MAIN BODY

Clinical decision support systems (CDSSs) in acute and critical care environments support clinicians, not replace them at the bedside. As will be discussed in this review, the reasons are many and include the immaturity of AI-based systems to have situational awareness, the fundamental bias in many large databases that do not reflect the target population of patient being treated making fairness an important issue to address and technical barriers to the timely access to valid data and its display in a fashion useful for clinical workflow. The inherent "black-box" nature of many predictive algorithms and CDSS makes trustworthiness and acceptance by the medical community difficult. Logistically, collating and curating in real-time multidimensional data streams of various sources needed to inform the algorithms and ultimately display relevant clinical decisions support format that adapt to individual patient responses and signatures represent the efferent limb of these systems and is often ignored during initial validation efforts. Similarly, legal and commercial barriers to the access to many existing clinical databases limit studies to address fairness and generalizability of predictive models and management tools.

CONCLUSIONS

AI-based CDSS are evolving and are here to stay. It is our obligation to be good shepherds of their use and further development.

Large language models for science and medicine

Large language models (LLMs) are a type of machine learning model that learn statistical patterns over text, such as predicting the next words in a sequence of text. Both general purpose and task-specific LLMs have demonstrated potential across diverse applications. Science and medicine have many data types that are highly suitable for LLMs, such as scientific texts (publications, patents and textbooks), electronic medical records, large databases of DNA and protein sequences and chemical compounds. Carefully validated systems that can understand and reason across all these modalities may maximize benefits. Despite the inevitable limitations and caveats of any new technology and some uncertainties specific to LLMs, LLMs have the potential to be transformative in science and medicine.

A Nationwide Network of Health AI Assurance Laboratories

Importance

Given the importance of rigorous development and evaluation standards needed of artificial intelligence (AI) models used in health care, nationwide accepted procedures to provide assurance that the use of AI is fair, appropriate, valid, effective, and safe are urgently needed.

Observations

While there are several efforts to develop standards and best practices to evaluate AI, there is a gap between having such guidance and the application of such guidance to both existing and new AI models being developed. As of now, there is no publicly available, nationwide mechanism that enables objective evaluation and ongoing assessment of the consequences of using health AI models in clinical care settings.

Conclusion and Relevance

The need to create a public-private partnership to support a nationwide health AI assurance labs network is outlined here. In this network, community best practices could be applied for testing health AI models to produce reports on their performance that can be widely shared for managing the lifecycle of AI models over time and across populations and sites where these models are deployed.

Research gaps and opportunities in precision nutrition: an NIH workshop report

Precision nutrition is an emerging concept that aims to develop nutrition recommendations tailored to different people's circumstances and biological characteristics. Responses to dietary change and the resulting health outcomes from consuming different diets may vary significantly between people based on interactions between their genetic backgrounds, physiology, microbiome, underlying health status, behaviors, social influences, and environmental exposures. On 11-12 January 2021, the National Institutes of Health convened a workshop entitled "Precision Nutrition: Research Gaps and Opportunities" to bring together experts to discuss the issues involved in better understanding and addressing precision nutrition. The workshop proceeded in 3 parts: part I covered many aspects of genetics and physiology that mediate the links between nutrient intake and health conditions such as cardiovascular disease, Alzheimer disease, and cancer; part II reviewed potential contributors to interindividual variability in dietary exposures and responses such as baseline nutritional status, circadian rhythm/sleep, environmental exposures, sensory properties of food, stress, inflammation, and the social determinants of health; part III presented the need for systems approaches, with new methods and technologies that can facilitate the study and implementation of precision nutrition, and workforce development needed to create a new generation of researchers. The workshop concluded that much research will be needed before more precise nutrition recommendations can be achieved. This includes better understanding and accounting for variables such as age, sex, ethnicity, medical history, genetics, and social and environmental factors. The advent of new methods and technologies and the availability of considerably more data bring tremendous opportunity. However, the field must proceed with appropriate levels of caution and make sure the factors listed above are all considered, and systems approaches and methods are incorporated. It will be important to develop and train an expanded workforce with the goal of reducing health disparities and improving precision nutritional advice for all Americans.

A prospective birth cohort study of maternal prenatal cigarette smoking assessed by self-report and biomarkers on childhood risk of overweight or obesity

Background

Most studies on the association of in utero exposure to cigarette smoking and childhood overweight or obesity (OWO) were based on maternal self-reported smoking status, and few were based on objective biomarkers. The concordance of self-report smoking, and maternal and cord blood biomarkers of cigarette smoking as well as their effects on children's long-term risk of overweight and obesity are unclear.

Methods

In this study, we analyzed data from 2351 mother-child pairs in the Boston Birth Cohort, a sample of US predominantly Black, indigenous, and people of color (BIPOC) that enrolled children at birth and followed prospectively up to age 18 years. In utero smoking exposure was measured by maternal self-report and by maternal and cord plasma biomarkers of smoking: cotinine and hydroxycotinine. We assessed the individual and joint associations of each smoking exposure measure and maternal OWO with childhood OWO using multinomial logistic regressions. We used nested logistic regressions to investigate the childhood OWO prediction performance when adding maternal and cord plasma biomarkers as input covariates on top of self-reported data.

Results

Our results demonstrated that in utero cigarette smoking exposure defined by self-report and by maternal or cord metabolites was consistently associated with increased risk of long-term child OWO. Children with cord hydroxycotinine in the fourth quartile (vs. first quartile) had 1.66 (95% confidence interval [CI] 1.03-2.66) times the odds for overweight and 1.57 (95% CI 1.05-2.36) times the odds for obesity. The combined effect of maternal OWO and smoking on offspring risk of obesity is 3.66 (95% CI 2.37-5.67) if using self-reported smoking. Adding maternal and cord plasma biomarker information to self-reported data improved the prediction accuracy of long-term child OWO risk.

Conclusions

This longitudinal birth cohort study of US BIPOC underscored the role of maternal smoking as an obesogen for offspring OWO risk. Our findings call for public health intervention strategies to focus on maternal smoking - as a highly modifiable target, including smoking cessation and countermeasures (such as optimal nutrition) that may alleviate the increasing obesity burden in the United States and globally.

A prospective birth cohort study of maternal prenatal cigarette smoking assessed by self-report and biomarkers on childhood risk of overweight or obesity

Background

Most studies on the association of in utero exposure to cigarette smoking and childhood overweight or obesity (OWO) were based on maternal self-reported smoking status, and few were based on objective biomarkers.

Objective

We aim to assess the concordance of self-report smoking, and maternal and cord blood biomarkers of cigarette smoking as well as to quantify the in utero cigarette smoking on child long-term risk of overweight and obesity.

Methods

In this study, we analyzed data from 2351 mother-child pairs in the Boston Birth Cohort, a sample of US predominantly Black, indigenous, and people of color (BIPOC) that enrolled children at birth and followed prospectively up to age 18 years. In utero smoking exposure was measured by maternal self-report and by maternal and cord plasma biomarkers of smoking: cotinine and hydroxycotinine. We assessed the individual and joint associations of each smoking exposure measure and maternal OWO with childhood OWO using multinomial logistic regressions. We used nested logistic regressions to investigate the childhood OWO prediction performance when adding maternal and cord plasma biomarkers as input covariates on top of self-reported data.

Results

Our results demonstrated that in utero cigarette smoking exposure defined by self-report and by maternal or cord metabolites was consistently associated with increased risk of long-term child OWO. Children with cord hydroxycotinine in the 4th quartile (vs. 1st quartile) had 1.66 (95% CI 1.03-2.66) times the odds for overweight and 1.57 (95% CI 1.05-2.36) times the odds for obesity. The combined effect of maternal overweight or obesity and smoking on offspring risk of obesity is 3.66 (95% CI 2.37-5.67) if using self-reported smoking. Adding maternal and cord plasma biomarker information to self-reported data improved the prediction accuracy of long-term child OWO risk.

Conclusions

This longitudinal birth cohort study of US BIPOC underscored the role of maternal smoking as an obesogen for offspring OWO risk. Our findings call for public health intervention strategies to focus on maternal smoking - as a highly modifiable target, including smoking cessation and countermeasures (such as optimal nutrition) that may alleviate the increasing obesity burden in the U.S. and globally.

Medicine 2032: The future of cardiovascular disease prevention with machine learning and digital health technology

Machine learning (ML) refers to computational algorithms that iteratively improve their ability to recognize patterns in data. The digitization of our healthcare infrastructure is generating an abundance of data from electronic health records, imaging, wearables, and sensors that can be analyzed by ML algorithms to generate personalized risk assessments and promote guideline-directed medical management. ML's strength in generating insights from complex medical data to guide clinical decisions must be balanced with the potential to adversely affect patient privacy, safety, health equity, and clinical interpretability. This review provides a primer on key advances in ML for cardiovascular disease prevention and how they may impact clinical practice.

A Random Forest Genomic Classifier for Tumor Agnostic Prediction of Response to Anti-PD1 Immunotherapy

Tumor mutational burden (TMB), a surrogate for tumor neoepitope burden, is used as a pan-tumor biomarker to identify patients who may benefit from anti-program cell death 1 (PD1) immunotherapy, but it is an imperfect biomarker. Multiple additional genomic characteristics are associated with anti-PD1 responses, but the combined predictive value of these features and the added informativeness of each respective feature remains unknown. We evaluated whether machine learning (ML) approaches using proposed determinants of anti-PD1 response derived from whole exome sequencing (WES) could improve prediction of anti-PD1 responders over TMB alone. Random forest classifiers were trained on publicly available anti-PD1 data (n = 104), and subsequently tested on an independent anti-PD1 cohort (n = 69). Both the training and test datasets included a range of cancer types such as non-small cell lung cancer (NSCLC), head and neck squamous cell carcinoma (HNSCC), melanoma, and smaller numbers of patients from other tumor types. Features used include summaries such as TMB and number of frameshift mutations, as well as more gene-level features such as counts of mutations associated with immune checkpoint response and resistance. Both ML algorithms demonstrated area under the receiver-operator curves (AUC) that exceeded TMB alone (AUC 0.63 "human-guided," 0.64 "cluster," and 0.58 TMB alone). Mutations within oncogenes disproportionately modulate anti-PD1 responses relative to their overall contribution to tumor neoepitope burden. The use of a ML algorithm evaluating multiple proposed genomic determinants of anti-PD1 responses modestly improves performance over TMB alone, highlighting the need to integrate other biomarkers to further improve model performance.

Human-machine teaming is key to AI adoption: clinicians' experiences with a deployed machine learning system

While a growing number of machine learning (ML) systems have been deployed in clinical settings with the promise of improving patient care, many have struggled to gain adoption and realize this promise. Based on a qualitative analysis of coded interviews with clinicians who use an ML-based system for sepsis, we found that, rather than viewing the system as a surrogate for their clinical judgment, clinicians perceived themselves as partnering with the technology. Our findings suggest that, even without a deep understanding of machine learning, clinicians can build trust with an ML system through experience, expert endorsement and validation, and systems designed to accommodate clinicians’ autonomy and support them across their entire workflow.

Prospective, multi-site study of patient outcomes after implementation of the TREWS machine learning-based early warning system for sepsis

Early recognition and treatment of sepsis are linked to improved patient outcomes. Machine learning-based early warning systems may reduce the time to recognition, but few systems have undergone clinical evaluation. In this prospective, multi-site cohort study, we examined the association between patient outcomes and provider interaction with a deployed sepsis alert system called the Targeted Real-time Early Warning System (TREWS). During the study, 590,736 patients were monitored by TREWS across five hospitals. We focused our analysis on 6,877 patients with sepsis who were identified by the alert before initiation of antibiotic therapy. Adjusting for patient presentation and severity, patients in this group whose alert was confirmed by a provider within 3 h of the alert had a reduced in-hospital mortality rate (3.3%, confidence interval (CI) 1.7, 5.1%, adjusted absolute reduction, and 18.7%, CI 9.4, 27.0%, adjusted relative reduction), organ failure and length of stay compared with patients whose alert was not confirmed by a provider within 3 h. Improvements in mortality rate (4.5%, CI 0.8, 8.3%, adjusted absolute reduction) and organ failure were larger among those patients who were additionally flagged as high risk. Our findings indicate that early warning systems have the potential to identify sepsis patients early and improve patient outcomes and that sepsis patients who would benefit the most from early treatment can be identified and prioritized at the time of the alert.

Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI

A growing number of artificial intelligence (AI)-based clinical decision support systems are showing promising performance in preclinical, in silico evaluation, but few have yet demonstrated real benefit to patient care. Early-stage clinical evaluation is important to assess an AI system's actual clinical performance at small scale, ensure its safety, evaluate the human factors surrounding its use and pave the way to further large-scale trials. However, the reporting of these early studies remains inadequate. The present statement provides a multi-stakeholder, consensus-based reporting guideline for the Developmental and Exploratory Clinical Investigations of DEcision support systems driven by Artificial Intelligence (DECIDE-AI). We conducted a two-round, modified Delphi process to collect and analyze expert opinion on the reporting of early clinical evaluation of AI systems. Experts were recruited from 20 pre-defined stakeholder categories. The final composition and wording of the guideline was determined at a virtual consensus meeting. The checklist and the Explanation & Elaboration (E&E) sections were refined based on feedback from a qualitative evaluation process. In total, 123 experts participated in the first round of Delphi, 138 in the second round, 16 in the consensus meeting and 16 in the qualitative evaluation. The DECIDE-AI reporting guideline comprises 17 AI-specific reporting items (made of 28 subitems) and ten generic reporting items, with an E&E paragraph provided for each. Through consultation and consensus with a range of stakeholders, we developed a guideline comprising key items that should be reported in early-stage clinical studies of AI-based decision support systems in healthcare. By providing an actionable checklist of minimal reporting items, the DECIDE-AI guideline will facilitate the appraisal of these studies and replicability of their findings.

OUP accepted manuscript

Objective

Health care providers increasingly rely upon predictive algorithms when making important treatment decisions, however, evidence indicates that these tools can lead to inequitable outcomes across racial and socio-economic groups. In this study, we introduce a bias evaluation checklist that allows model developers and health care providers a means to systematically appraise a model’s potential to introduce bias.

Materials and Methods

Our methods include developing a bias evaluation checklist, a scoping literature review to identify 30-day hospital readmission prediction models, and assessing the selected models using the checklist.

Results

We selected 4 models for evaluation: LACE, HOSPITAL, Johns Hopkins ACG, and HATRIX. Our assessment identified critical ways in which these algorithms can perpetuate health care inequalities. We found that LACE and HOSPITAL have the greatest potential for introducing bias, Johns Hopkins ACG has the most areas of uncertainty, and HATRIX has the fewest causes for concern.

Discussion

Our approach gives model developers and health care providers a practical and systematic method for evaluating bias in predictive models. Traditional bias identification methods do not elucidate sources of bias and are thus insufficient for mitigation efforts. With our checklist, bias can be addressed and eliminated before a model is fully developed or deployed.

Conclusion

The potential for algorithms to perpetuate biased outcomes is not isolated to readmission prediction models; rather, we believe our results have implications for predictive models across health care. We offer a systematic method for evaluating potential bias with sufficient flexibility to be utilized across models and applications.

Digital Endpoints: Definition, Benefits, and Current Barriers in Accelerating Development and Adoption

The assessment of health and disease requires a set of criteria to define health status and progression. These health measures are referred to as "endpoints." A "digital endpoint" is defined by its use of sensor-generated data often collected outside of a clinical setting such as in a patient's free-living environment. Applicable sensors exist in an array of devices and can be applied in a diverse set of contexts. For example, a smartphone's microphone might be used to diagnose or predict mild cognitive impairment due to Alzheimer's disease or a wrist-worn activity monitor (such as those found in smartwatches) may be used to measure a drug's effect on the nocturnal activity of patients with sickle cell disease. Digital endpoints are generating considerable excitement because they permit a more authentic assessment of the patient's experience, reveal formerly untold realities of disease burden, and can cut drug discovery costs in half. However, before these benefits can be realized, effort must be applied not only to the technical creation of digital endpoints but also to the environment that allows for their development and application. The future of digital endpoints rests on meaningful interdisciplinary collaboration, sufficient evidence that digital endpoints can realize their promise, and the development of an ecosystem in which the vast quantities of data that digital endpoints generate can be analyzed. The fundamental nature of health care is changing. With coronavirus disease 2019 serving as a catalyst, there has been a rapid expansion of home care models, telehealth, and remote patient monitoring. The increasing adoption of these health-care innovations will expedite the requirement for a digital characterization of clinical status as current assessment tools often rely upon direct interaction with patients and thus are not fit for purpose to be administered remotely. With the ubiquity of relatively inexpensive sensors, digital endpoints are positioned to drive this consequential change. It is therefore not surprising that regulators, physicians, researchers, and consultants have each offered their assessment of these novel tools. However, as we further describe later, the broad adoption of digital endpoints will require a cooperative effort. In this article, we present an analysis of the current state of digital endpoints. We also attempt to unify the perspectives of the parties involved in the development and deployment of these tools. We conclude with an interdependent list of challenges that must be collaboratively addressed before these endpoints are widely adopted.

Will Artificial Intelligence Replace the Movement Disorders Specialist for Diagnosing and Managing Parkinson's Disease?

The use of artificial intelligence (AI) to help diagnose and manage disease is of increasing interest to researchers and clinicians. Volumes of health data are generated from smartphones and ubiquitous inexpensive sensors. By using these data, AI can offer otherwise unobtainable insights about disease burden and patient status in a free-living environment. Moreover, from clinical datasets AI can improve patient symptom monitoring and global epidemiologic efforts. While these applications are exciting, it is necessary to examine both the utility and limitations of these novel analytic methods. The most promising uses of AI remain aspirational. For example, defining the molecular subtypes of Parkinson's disease will be assisted by future applications of AI to relevant datasets. This will allow clinicians to match patients to molecular therapies and will thus help launch precision medicine. Until AI proves its potential in pushing the frontier of precision medicine, its utility will primarily remain in individualized monitoring, complementing but not replacing movement disorders specialists.

Development and validation of high definition phenotype-based mortality prediction in critical care units

Objectives

The objectives of this study are to construct the high definition phenotype (HDP), a novel time-series data structure composed of both primary and derived parameters, using heterogeneous clinical sources and to determine whether different predictive models can utilize the HDP in the neonatal intensive care unit (NICU) to improve neonatal mortality prediction in clinical settings.

Materials and Methods

A total of 49 primary data parameters were collected from July 2018 to May 2020 from eight level-III NICUs. From a total of 1546 patients, 757 patients were found to contain sufficient fixed, intermittent, and continuous data to create HDPs. Two different predictive models utilizing the HDP, one a logistic regression model (LRM) and the other a deep learning long–short-term memory (LSTM) model, were constructed to predict neonatal mortality at multiple time points during the patient hospitalization. The results were compared with previous illness severity scores, including SNAPPE, SNAPPE-II, CRIB, and CRIB-II.

Results

A HDP matrix, including 12 221 536 minutes of patient stay in NICU, was constructed. The LRM model and the LSTM model performed better than existing neonatal illness severity scores in predicting mortality using the area under the receiver operating characteristic curve (AUC) metric. An ablation study showed that utilizing continuous parameters alone results in an AUC score of >80% for both LRM and LSTM, but combining fixed, intermittent, and continuous parameters in the HDP results in scores >85%. The probability of mortality predictive score has recall and precision of 0.88 and 0.77 for the LRM and 0.97 and 0.85 for the LSTM.

Conclusions and Relevance

The HDP data structure supports multiple analytic techniques, including the statistical LRM approach and the machine learning LSTM approach used in this study. LRM and LSTM predictive models of neonatal mortality utilizing the HDP performed better than existing neonatal illness severity scores. Further research is necessary to create HDP–based clinical decision tools to detect the early onset of neonatal morbidities.

Development and Validation of a Machine Learning Model to Predict Near-Term Risk of Iatrogenic Hypoglycemia in Hospitalized Patients

Importance

Accurate clinical decision support tools are needed to identify patients at risk for iatrogenic hypoglycemia, a potentially serious adverse event, throughout hospitalization.

Objective

To predict the risk of iatrogenic hypoglycemia within 24 hours after each blood glucose (BG) measurement during hospitalization using a machine learning model.

Design, Setting, and Participants

This retrospective cohort study, conducted at 5 hospitals within the Johns Hopkins Health System, included 54 978 admissions of 35 147 inpatients who had at least 4 BG measurements and received at least 1 U of insulin during hospitalization between December 1, 2014, and July 31, 2018. Data from the largest hospital were split into a 70% training set and 30% test set. A stochastic gradient boosting machine learning model was developed using the training set and validated on internal and external validation.

Exposures

A total of 43 clinical predictors of iatrogenic hypoglycemia were extracted from the electronic medical record, including demographic characteristics, diagnoses, procedures, laboratory data, medications, orders, anthropomorphometric data, and vital signs.

Main Outcomes and Measures

Iatrogenic hypoglycemia was defined as a BG measurement less than or equal to 70 mg/dL occurring within the pharmacologic duration of action of administered insulin, sulfonylurea, or meglitinide.

Results

This cohort study included 54 978 admissions (35 147 inpatients; median [interquartile range] age, 66.0 [56.0-75.0] years; 27 781 [50.5%] male; 30 429 [55.3%] White) from 5 hospitals. Of 1 612 425 index BG measurements, 50 354 (3.1%) were followed by iatrogenic hypoglycemia in the subsequent 24 hours. On internal validation, the model achieved a C statistic of 0.90 (95% CI, 0.89-0.90), a positive predictive value of 0.09 (95% CI, 0.08-0.09), a positive likelihood ratio of 4.67 (95% CI, 4.59-4.74), a negative predictive value of 1.00 (95% CI, 1.00-1.00), and a negative likelihood ratio of 0.22 (95% CI, 0.21-0.23). On external validation, the model achieved C statistics ranging from 0.86 to 0.88, positive predictive values ranging from 0.12 to 0.13, negative predictive values of 0.99, positive likelihood ratios ranging from 3.09 to 3.89, and negative likelihood ratios ranging from 0.23 to 0.25. Basal insulin dose, coefficient of variation of BG, and previous hypoglycemic episodes were the strongest predictors.

Conclusions and Relevance

These findings suggest that iatrogenic hypoglycemia can be predicted in a short-term prediction horizon after each BG measurement during hospitalization. Further studies are needed to translate this model into a real-time informatics alert and evaluate its effectiveness in reducing the incidence of inpatient iatrogenic hypoglycemia.

Machine Learning-Based Automatic Classification of Video Recorded Neonatal Manipulations and Associated Physiological Parameters: A Feasibility Study

Our objective in this study was to determine if machine learning (ML) can automatically recognize neonatal manipulations, along with associated changes in physiological parameters. A retrospective observational study was carried out in two Neonatal Intensive Care Units (NICUs) between December 2019 to April 2020. Both the video and physiological data (heart rate (HR) and oxygen saturation (SpO₂)) were captured during NICU hospitalization. The proposed classification of neonatal manipulations was achieved by a deep learning system consisting of an Inception-v3 convolutional neural network (CNN), followed by transfer learning layers of Long Short-Term Memory (LSTM). Physiological signals prior to manipulations (baseline) were compared to during and after manipulations. The validation of the system was done using the leave-one-out strategy with input of 8 s of video exhibiting manipulation activity. Ten neonates were video recorded during an average length of stay of 24.5 days. Each neonate had an average of 528 manipulations during their NICU hospitalization, with the average duration of performing these manipulations varying from 28.9 s for patting, 45.5 s for a diaper change, and 108.9 s for tube feeding. The accuracy of the system was 95% for training and 85% for the validation dataset. In neonates <32 weeks’ gestation, diaper changes were associated with significant changes in HR and SpO₂, and, for neonates ≥32 weeks’ gestation, patting and tube feeding were associated with significant changes in HR. The presented system can classify and document the manipulations with high accuracy. Moreover, the study suggests that manipulations impact physiological parameters.

Predictors of the start of declining eGFR in patients with systemic lupus erythematosus

OBJECTIVE

To characterize the longitudinal trajectory of estimated glomerular filtration rate (eGFR) in patients with systemic lupus erythematosus (SLE) and identify predictors of the change in eGFR trajectory.

METHODS

The longitudinal eGFR levels of patients in the Hopkins Lupus Cohort were modelled by piecewise linear regression to evaluate the slope of different line segments. The slopes were classified into declining (≤-4 mL/min/1.73 m² per year), stable (-4 to 4 mL/min/1.73 m² per year), and increasing (≥4 mL/min/1.73 m² per year) states. The transition rate between states and the impact of clinical parameters were estimated by a Markov model.

RESULTS

The analysis was based on 494 SLE patients. At a mean follow-up of 8.8 years, 347 (70.2%), 107 (21.7%), 33 (6.7%), and 7 (1.4%) patients had zero, one, two, and three state transitions, respectively. In patients with no transition, 37 (10.7%), 308 (88.8%), and 2 (0.6%) were in declining, stable, and increasing state, respectively. In patients with one transition, 43 (40.2%) changed from declining to stable state while 29 (27.1%) changed from stable to declining state. When patients were in a non-declining GFR state, those who were younger and African Americans were more likely to transition to a declining GFR state. In adjusted analyses, high blood pressure, C4 and low hematocrit were associated with change from non-declining to declining state. High urine protein-to-creatinine ratio also tended to be associated with change from non-declining to declining state. African American patients were less likely to move from declining to non-declining state. Use of prednisone was associated with change from declining to non-declining state.

CONCLUSIONS

Patients with high blood pressure, low complement C4, low haematocrit, and high urine protein-to-creatinine ratio are more likely to have a declining eGFR trajectory, while the use of prednisone stabilizes the declining eGFR trajectory.

Reporting and Implementing Interventions Involving Machine Learning and Artificial Intelligence

Increasingly, interventions aimed at improving care are likely to use such technologies as machine learning and artificial intelligence. However, health care has been relatively late to adopt them. This article provides clinical examples in which machine learning and artificial intelligence are already in use in health care and appear to deliver benefit. Three key bottlenecks toward increasing the pace of diffusion and adoption are methodological issues in evaluation of artificial intelligence-based interventions, reporting standards to enable assessment of model performance, and issues that need to be addressed for an institution to adopt these interventions. Methodological best practices will include external validation, ideally at a different site; use of proactive learning algorithms to correct for site-specific biases and increase robustness as algorithms are deployed across multiple sites; addressing subgroup performance; and communicating to providers the uncertainty of predictions. Regarding reporting, especially important issues are the extent to which implementing standardized approaches for introducing clinical decision support has been followed, describing the data sources, reporting on data assumptions, and addressing biases. Although most health care organizations in the United States have adopted electronic health records, they may be ill prepared to adopt machine learning and artificial intelligence. Several steps can enable this: preparing data, developing tools to get suggestions to clinicians in useful ways, and getting clinicians engaged in the process. Open challenges and the role of regulation in this area are briefly discussed. Although these techniques have enormous potential to improve care and personalize recommendations for individuals, the hype regarding them is tremendous. Organizations will need to approach this domain carefully with knowledgeable partners to obtain the hoped-for benefits and avoid failures.

Comparison of Automated Activity Recognition to Provider Observations of Patient Mobility in the ICU

OBJECTIVES

To compare noninvasive mobility sensor patient motion signature to direct observations by physicians and nurses.

DESIGN

Prospective, observational study.

SETTING

Academic hospital surgical ICU.

PATIENTS AND MEASUREMENTS

A total of 2,426 1-minute clips from six ICU patients (development dataset) and 4,824 1-minute clips from five patients (test dataset).

INTERVENTIONS

None.

MAIN RESULTS

Noninvasive mobility sensor achieved a minute-level accuracy of 94.2% (2,138/2,272) and an hour-level accuracy of 81.4% (70/86).

CONCLUSIONS

The automated noninvasive mobility sensor system represents a significant departure from current manual measurement and reporting used in clinical care, lowering the burden of measurement and documentation on caregivers.

An Individualized, Data-Driven Digital Approach for Precision Behavior Change

Chronic disease now affects approximately half of the US population, causes 7 in 10 deaths, and accounts for roughly 80% of US health care expenditure. Because the root causes of chronic diseases are largely behavioral, effective therapies require frequent, individualized interventions that extend beyond the hospital and clinic to reach patients in their day-to-day lives. However, a mismatch currently exists between what the health care system is equipped to provide and the interventions necessary to effectively address the chronic disease burden. To remedy this health crisis, we present an individualized, data-driven digital approach for chronic disease management and prevention through precision behavior change. The rapid growth of information, biological, and communication technologies makes this an opportune time to develop digital tools that deliver precision interventions for health behavior change to address the chronic disease crisis. Building on this rapid growth, we propose a framework that includes the precise targeting of risk-producing behaviors using real-time sensing technology, machine learning data analysis to identify the most effective intervention, and delivery of that intervention with health-reinforcing feedback to provide real-time, individualized support to empower sustainable health behavior change.

Deep Phenotyping of Parkinson's Disease

Phenotype is the set of observable traits of an organism or condition. While advances in genetics, imaging, and molecular biology have improved our understanding of the underlying biology of Parkinson's disease (PD), clinical phenotyping of PD still relies primarily on history and physical examination. These subjective, episodic, categorical assessments are valuable for diagnosis and care but have left gaps in our understanding of the PD phenotype. Sensors can provide objective, continuous, real-world data about the PD clinical phenotype, increase our knowledge of its pathology, enhance evaluation of therapies, and ultimately, improve patient care. In this paper, we explore the concept of deep phenotyping-the comprehensive assessment of a condition using multiple clinical, biological, genetic, imaging, and sensor-based tools-for PD. We discuss the rationale for, outline current approaches to, identify benefits and limitations of, and consider future directions for deep clinical phenotyping.

Using Smartphones and Machine Learning to Quantify Parkinson Disease Severity: The Mobile Parkinson Disease Score

Importance

Current Parkinson disease (PD) measures are subjective, rater-dependent, and assessed in clinic. Smartphones can measure PD features, yet no smartphone-derived rating score exists to assess motor symptom severity in real-world settings.

Objectives

To develop an objective measure of PD severity and test construct validity by evaluating the ability of the measure to capture intraday symptom fluctuations, correlate with current standard PD outcome measures, and respond to dopaminergic therapy.

Design, Setting, and Participants

This observational study assessed individuals with PD who remotely completed 5 tasks (voice, finger tapping, gait, balance, and reaction time) on the smartphone application. We used a novel machine-learning-based approach to generate a mobile Parkinson disease score (mPDS) that objectively weighs features derived from each smartphone activity (eg, stride length from the gait activity) and is scaled from 0 to 100 (where higher scores indicate greater severity). Individuals with and without PD additionally completed standard in-person assessments of PD with smartphone assessments during a period of 6 months.

Main Outcomes and Measures

Ability of the mPDS to detect intraday symptom fluctuations, the correlation between the mPDS and standard measures, and the ability of the mPDS to respond to dopaminergic medication.

Results

The mPDS was derived from 6148 smartphone activity assessments from 129 individuals (mean [SD] age, 58.7 [8.6] years; 56 [43.4%] women). Gait features contributed most to the total mPDS (33.4%). In addition, 23 individuals with PD (mean [SD] age, 64.6 [11.5] years; 11 [48%] women) and 17 without PD (mean [SD] age 54.2 [16.5] years; 12 [71%] women) completed in-clinic assessments. The mPDS detected symptom fluctuations with a mean (SD) intraday change of 13.9 (10.3) points on a scale of 0 to 100. The measure correlated well with the Movement Disorder Society Unified Parkinson Disease's Rating Scale total (r = 0.81; P < .001) and part III only (r = 0.88; P < .001), the Timed Up and Go assessment (r = 0.72; P = .002), and the Hoehn and Yahr stage (r = 0.91; P < .001). The mPDS improved by a mean (SD) of 16.3 (5.6) points in response to dopaminergic therapy.

Conclusions and Relevance

Using a novel machine-learning approach, we created and demonstrated construct validity of an objective PD severity score derived from smartphone assessments. This score complements standard PD measures by providing frequent, objective, real-world assessments that could enhance clinical care and evaluation of novel therapeutics.

Metadata Concepts for Advancing the Use of Digital Health Technologies in Clinical Research

Digital health technologies (smartphones, smartwatches, and other body-worn sensors) can act as novel tools to aid in the diagnosis and remote objective monitoring of an individual's disease symptoms, both in clinical care and in research. Nonetheless, such digital health technologies have yet to widely demonstrate value in clinical research due to insufficient data interpretability and lack of regulatory acceptance. Metadata, i.e., data that accompany and describe the primary data, can be utilized to better understand the context of the sensor data and can assist in data management, data sharing, and subsequent data analysis. The need for data and metadata standards for digital health technologies has been raised in academic and industry research communities and has also been noted by regulatory authorities. Therefore, to address this unmet need, we here propose a metadata set that reflects regulatory guidelines and that can serve as a conceptual map to (1) inform researchers on the metadata they should collect in digital health studies, aiming to increase the interpretability and exchangeability of their data, and (2) direct standard development organizations on how to extend their existing standards to incorporate digital health technologies. The proposed metadata set is informed by existing standards pertaining to clinical trials and medical devices, in addition to existing schemas that have supported digital health technology studies. We illustrate this specifically in the context of Parkinson's disease, as a model for a wide range of other chronic conditions for which remote monitoring would be useful in both care and science. We invite the scientific and clinical research communities to apply the proposed metadata set to ongoing and planned research. Where the proposed metadata fall short, we ask users to contribute to its ongoing revision so that an adequate degree of consensus can be maintained in a rapidly evolving technology landscape.

Author Correction: Do no harm: a roadmap for responsible machine learning for health care

An amendment to this paper has been published and can be accessed via a link at the top of the paper.

Do no harm: a roadmap for responsible machine learning for health care

Interest in machine-learning applications within medicine has been growing, but few studies have progressed to deployment in patient care. We present a framework, context and ultimately guidelines for accelerating the translation of machine-learning-based interventions in health care. To be successful, translation will require a team of engaged stakeholders and a systematic process from beginning (problem formulation) to end (widespread deployment).

Better medicine through machine learning: What's real, and what's artificial?

Machine Learning Special Issue Guest Editors Suchi Saria, Atul Butte, and Aziz Sheikh cut through the hyperbole with an accessible and accurate portrayal of the forefront of machine learning in clinical translation.

Scalable Joint Models for Reliable Uncertainty-Aware Event Prediction

Missing data and noisy observations pose significant challenges for reliably predicting events from irregularly sampled multivariate time series (longitudinal) data. Imputation methods, which are typically used for completing the data prior to event prediction, lack a principled mechanism to account for the uncertainty due to missingness. Alternatively, state-of-the-art joint modeling techniques can be used for jointly modeling the longitudinal and event data and compute event probabilities conditioned on the longitudinal observations. These approaches, however, make strong parametric assumptions and do not easily scale to multivariate signals with many observations. Our proposed approach consists of several key innovations. First, we develop a flexible and scalable joint model based upon sparse multiple-output Gaussian processes. Unlike state-of-the-art joint models, the proposed model can explain highly challenging structure including non-Gaussian noise while scaling to large data. Second, we derive an optimal policy for predicting events using the distribution of the event occurrence estimated by the joint model. The derived policy trades-off the cost of a delayed detection versus incorrect assessments and abstains from making decisions when the estimated event probability does not satisfy the derived confidence criteria. Experiments on a large dataset show that the proposed framework significantly outperforms state-of-the-art techniques in event prediction.

Factors associated with physicians' prescriptions for rheumatoid arthritis drugs not filled by patients

Background

This study estimated the extent and predictors of primary nonadherence (i.e., prescriptions made by physicians but not initiated by patients) to methotrexate and to biologics or tofacitinib in rheumatoid arthritis (RA) patients who were newly prescribed these medications.

Methods

Using administrative claims linked with electronic health records (EHRs) from multiple healthcare provider organizations in the USA, RA patients who received a new prescription for methotrexate or biologics/tofacitinib were identified from EHRs. Claims data were used to ascertain filling or administration status. A logistic regression model for predicting primary nonadherence was developed and tested in training and test samples. Predictors were selected based on clinical judgment and LASSO logistic regression.

Results

A total of 36.8% of patients newly prescribed methotrexate failed to initiate methotrexate within 2 months; 40.6% of patients newly prescribed biologics/tofacitinib failed to initiate within 3 months. Factors associated with methotrexate primary nonadherence included age, race, region, body mass index, count of active drug ingredients, and certain previously diagnosed and treated conditions at baseline. Factors associated with biologics/tofacitinib primary nonadherence included age, insurance, and certain previously treated conditions at baseline. The area under the receiver operating characteristic curve of the logistic regression model estimated in the training sample and applied to the independent test sample was 0.86 and 0.78 for predicting primary nonadherence to methotrexate and to biologics/tofacitinib, respectively.

Conclusions

This study confirmed that failure to initiate new prescriptions for methotrexate and biologics/tofacitinib was common in RA patients. It is feasible to predict patients at high risk of primary nonadherence to methotrexate and to biologics/tofacitinib and to target such patients for early interventions to promote adherence.

Development and validation of a prediction model for insulin-associated hypoglycemia in non-critically ill hospitalized adults

OBJECTIVE

To develop and validate a multivariable prediction model for insulin-associated hypoglycemia in non-critically ill hospitalized adults.

RESEARCH DESIGN AND METHODS

We collected pharmacologic, demographic, laboratory, and diagnostic data from 128 657 inpatient days in which at least 1 unit of subcutaneous insulin was administered in the absence of intravenous insulin, total parenteral nutrition, or insulin pump use (index days). These data were used to develop multivariable prediction models for biochemical and clinically significant hypoglycemia (blood glucose (BG) of ≤70 mg/dL and <54 mg/dL, respectively) occurring within 24 hours of the index day. Split-sample internal validation was performed, with 70% and 30% of index days used for model development and validation, respectively.

RESULTS

Using predictors of age, weight, admitting service, insulin doses, mean BG, nadir BG, BG coefficient of variation (CVBG), diet status, type 1 diabetes, type 2 diabetes, acute kidney injury, chronic kidney disease (CKD), liver disease, and digestive disease, our model achieved a c-statistic of 0.77 (95% CI 0.75 to 0.78), positive likelihood ratio (+LR) of 3.5 (95% CI 3.4 to 3.6) and negative likelihood ratio (-LR) of 0.32 (95% CI 0.30 to 0.35) for prediction of biochemical hypoglycemia. Using predictors of sex, weight, insulin doses, mean BG, nadir BG, CVBG, diet status, type 1 diabetes, type 2 diabetes, CKD stage, and steroid use, our model achieved a c-statistic of 0.80 (95% CI 0.78 to 0.82), +LR of 3.8 (95% CI 3.7 to 4.0) and -LR of 0.2 (95% CI 0.2 to 0.3) for prediction of clinically significant hypoglycemia.

CONCLUSIONS

Hospitalized patients at risk of insulin-associated hypoglycemia can be identified using validated prediction models, which may support the development of real-time preventive interventions.

Process Monitoring in the Intensive Care Unit: Assessing Patient Mobility Through Activity Analysis with a Non-Invasive Mobility Sensor

Throughout a patient's stay in the Intensive Care Unit (ICU), accurate measurement of patient mobility, as part of routine care, is helpful in understanding the harmful effects of bedrest [1]. However, mobility is typically measured through observation by a trained and dedicated observer, which is extremely limiting. In this work, we present a video-based automated mobility measurement system called NIMS: Non-Invasive Mobility Sensor . Our main contributions are: (1) a novel multi-person tracking methodology designed for complex environments with occlusion and pose variations, and (2) an application of human-activity attributes in a clinical setting. We demonstrate NIMS on data collected from an active patient room in an adult ICU and show a high inter-rater reliability using a weighted Kappa statistic of 0.86 for automatic prediction of the highest level of patient mobility as compared to clinical experts.

A targeted real-time early warning score (TREWScore) for septic shock

Sepsis is a leading cause of death in the United States, with mortality highest among patients who develop septic shock. Early aggressive treatment decreases morbidity and mortality. Although automated screening tools can detect patients currently experiencing severe sepsis and septic shock, none predict those at greatest risk of developing shock. We analyzed routinely available physiological and laboratory data from intensive care unit patients and developed "TREWScore," a targeted real-time early warning score that predicts which patients will develop septic shock. TREWScore identified patients before the onset of septic shock with an area under the ROC (receiver operating characteristic) curve (AUC) of 0.83 [95% confidence interval (CI), 0.81 to 0.85]. At a specificity of 0.67, TREWScore achieved a sensitivity of 0.85 and identified patients a median of 28.2 [interquartile range (IQR), 10.6 to 94.2] hours before onset. Of those identified, two-thirds were identified before any sepsis-related organ dysfunction. In comparison, the Modified Early Warning Score, which has been used clinically for septic shock prediction, achieved a lower AUC of 0.73 (95% CI, 0.71 to 0.76). A routine screening protocol based on the presence of two of the systemic inflammatory response syndrome criteria, suspicion of infection, and either hypotension or hyperlactatemia achieved a lower sensitivity of 0.74 at a comparable specificity of 0.64. Continuous sampling of data from the electronic health records and calculation of TREWScore may allow clinicians to identify patients at risk for septic shock and provide earlier interventions that would prevent or mitigate the associated morbidity and mortality.

Consensus Statement on Electronic Health Predictive Analytics: A Guiding Framework to Address Challenges

Context:

The recent explosion in available electronic health record (EHR) data is motivating a rapid expansion of electronic health care predictive analytic (e-HPA) applications, defined as the use of electronic algorithms that forecast clinical events in real time with the intent to improve patient outcomes and reduce costs. There is an urgent need for a systematic framework to guide the development and application of e-HPA to ensure that the field develops in a scientifically sound, ethical, and efficient manner.

Objectives:

Building upon earlier frameworks of model development and utilization, we identify the emerging opportunities and challenges of e-HPA, propose a framework that enables us to realize these opportunities, address these challenges, and motivate e-HPA stakeholders to both adopt and continuously refine the framework as the applications of e-HPA emerge.

Methods:

To achieve these objectives, 17 experts with diverse expertise including methodology, ethics, legal, regulation, and health care delivery systems were assembled to identify emerging opportunities and challenges of e-HPA and to propose a framework to guide the development and application of e-HPA.

Findings:

The framework proposed by the panel includes three key domains where e-HPA differs qualitatively from earlier generations of models and algorithms (Data Barriers, Transparency, and Ethics) and areas where current frameworks are insufficient to address the emerging opportunities and challenges of e-HPA (Regulation and Certification; and Education and Training). The following list of recommendations summarizes the key points of the framework:

Data Barriers: Establish mechanisms within the scientific community to support data sharing for predictive model development and testing.

Transparency: Set standards around e-HPA validation based on principles of scientific transparency and reproducibility.

Ethics: Develop both individual-centered and society-centered risk-benefit approaches to evaluate e-HPA.

Regulation and Certification: Construct a self-regulation and certification framework within e-HPA.

Education and Training: Make significant changes to medical, nursing, and paraprofessional curricula by including training for understanding, evaluating, and utilizing predictive models.

A Probabilistic Graphical Model for Individualizing Prognosis in Chronic, Complex Diseases

Making accurate prognoses in chronic, complex diseases is challenging due to the wide variation in expression across individuals. In many such diseases, the notion of subtypes-subpopulations that share similar symptoms and patterns of progression-have been proposed. We develop a probabilistic model that exploits the concept of subtypes to individualize prognoses of disease trajectories. These subtypes are learned automatically from data. On a new individual, our model incorporates static and time-varying markers to dynamically update predictions of subtype membership and provide individualized predictions of disease trajectory. We use our model to tackle the problem of predicting lung function trajectories in scleroderma, an autoimmune disease, and demonstrate improved predictive performance over existing approaches.

Learning a Severity Score for Sepsis: A Novel Approach based on Clinical Comparisons

Sepsis is one of the leading causes of death in the United States. Early administration of treatment has been shown to decrease sepsis-related mortality and morbidity. Existing scoring systems such as the Acute Physiology and Chronic Health Evaluation (APACHE II) and Sequential Organ Failure Assessment scores (SOFA) achieve poor sensitivity in distinguishing between the different stages of sepsis. Recently, we proposed the Disease Severity Score Learning (DSSL) framework that automatically derives a severity score from data based on clinical comparisons - pairs of disease states ordered by their severity. In this paper, we test the feasibility of using DSSL to develop a sepsis severity score. We show that the learned score significantly outperforms APACHE-II and SOFA in distinguishing between the different stages of sepsis. Additionally, the learned score is sensitive to changes in severity leading up to septic shock and post treatment administration.

Dissecting an online intervention for cancer survivors: four exploratory analyses of internet engagement and its effects on health status and health behaviors

The Internet has been used extensively to offer health education content and also for social support. More recently, we have seen the advent of Internet-based health education interventions that combine content with structured social networking. In many ways this is the Internet equivalent to small group interventions. While we have some knowledge about the efficacy of these interventions, few studies have examined how participants engage with programs and how that might affect outcomes. This study seeks to explore (a) the content of posts and (b) the nature of participant engagement with an online, 6-week workshop for cancer survivors and how such engagement may affect health outcomes. Using methodologies related to computational linguistics (latent Dirichlet allocation) and more standard statistical approaches, we identified (a) discussion board themes; (b) the relationship between reading and posting messages and outcomes; (c) how making, completing, or not completing action plans is related to outcome; and (d) how self-tailoring relates to outcomes. When considering all posts, emotional support is a key theme. However, different sets of themes are expressed in the first workshop post where participants are asked to express their primary concern. Writing posts was related to improved outcomes, but reading posts was less important. Completing, but not merely making, action plans and self-tailoring are statistically associated with future positive health outcomes. The findings from these exploratory studies can be considered when shaping future electronically mediated social networking interventions. In addition, the methods used here can be used in analyzing other large electronically mediated social-networking interventions.

Perinatal risk factors for severe injury in neonates treated with whole-body hypothermia for encephalopathy

OBJECTIVE

Our objective was to identify perinatal risk factors that are available within 1 hour of birth that are associated with severe brain injury after hypothermia treatment for suspected hypoxic-ischemic encephalopathy.

STUDY DESIGN

One hundred nine neonates at ≥35 weeks' gestation who were admitted from January 2007 to September 2012 with suspected hypoxic-ischemic encephalopathy were treated with whole-body hypothermia; 98 of them (90%) underwent brain magnetic resonance imaging (MRI) at 7-10 days of life. Eight neonates died before brain imaging. Neonates who had severe brain injury, which was defined as death or abnormal MRI results (cases), were compared with surviving neonates with normal MRI (control subjects). Logistic regression models were used to identify risk factors that were predictive of severe injury.

RESULTS

Cases and control subjects did not differ with regard to gestational age, birthweight, mode of delivery, or diagnosis of nonreassuring fetal heart rate before delivery. Cases were significantly (P < .05) more likely to have had an abruption, a cord and neonatal arterial gas level that showed metabolic acidosis, lower platelet counts, lower glucose level, longer time to spontaneous respirations, intubation, chest compressions in the delivery room, and seizures. In multivariable logistic regression, lower initial neonatal arterial pH (P = .004), spontaneous respiration at >30 minutes of life (P = .002), and absence of exposure to oxytocin (P = .033) were associated independently with severe injury with 74.3% sensitivity and 74.4% specificity.

CONCLUSION

Worsening metabolic acidosis at birth, longer time to spontaneous respirations, and lack of exposure to oxytocin correlated with severe brain injury in neonates who were treated with whole-body hypothermia. These risk factors may help quickly identify neonatal candidates for time-sensitive investigational therapies for brain neuroprotection.

Developing predictive models using electronic medical records: challenges and pitfalls

While Electronic Medical Records (EMR) contain detailed records of the patient-clinician encounter - vital signs, laboratory tests, symptoms, caregivers' notes, interventions prescribed and outcomes - developing predictive models from this data is not straightforward. These data contain systematic biases that violate assumptions made by off-the-shelf machine learning algorithms, commonly used in the literature to train predictive models. In this paper, we discuss key issues and subtle pitfalls specific to building predictive models from EMR. We highlight the importance of carefully considering both the special characteristics of EMR as well as the intended clinical use of the predictive model and show that failure to do so could lead to developing models that are less useful in practice. Finally, we describe approaches for training and evaluating models on EMR using early prediction of septic shock as our example application.

Early detection of poor adherers to statins: applying individualized surveillance to pay for performance

BACKGROUND

Medication nonadherence costs $300 billion annually in the US. Medicare Advantage plans have a financial incentive to increase medication adherence among members because the Centers for Medicare and Medicaid Services (CMS) now awards substantive bonus payments to such plans, based in part on population adherence to chronic medications. We sought to build an individualized surveillance model that detects early which beneficiaries will fall below the CMS adherence threshold.

METHODS

This was a retrospective study of over 210,000 beneficiaries initiating statins, in a database of private insurance claims, from 2008-2011. A logistic regression model was constructed to use statin adherence from initiation to day 90 to predict beneficiaries who would not meet the CMS measure of proportion of days covered 0.8 or above, from day 91 to 365. The model controlled for 15 additional characteristics. In a sensitivity analysis, we varied the number of days of adherence data used for prediction.

RESULTS

Lower adherence in the first 90 days was the strongest predictor of one-year nonadherence, with an odds ratio of 25.0 (95% confidence interval 23.7-26.5) for poor adherence at one year. The model had an area under the receiver operating characteristic curve of 0.80. Sensitivity analysis revealed that predictions of comparable accuracy could be made only 40 days after statin initiation. When members with 30-day supplies for their first statin fill had predictions made at 40 days, and members with 90-day supplies for their first fill had predictions made at 100 days, poor adherence could be predicted with 86% positive predictive value.

CONCLUSIONS

To preserve their Medicare Star ratings, plan managers should identify or develop effective programs to improve adherence. An individualized surveillance approach can be used to target members who would most benefit, recognizing the tradeoff between improved model performance over time and the advantage of earlier detection.