How to develop a more accurate risk prediction model when there are few events

Associations of cardiovascular biomarkers and plasma albumin with exceptional survival to the highest ages

Supercentenarians (those aged ≥110 years) are approaching the current human longevity limit by preventing or surviving major illness. Identifying specific biomarkers conducive to exceptional survival might provide insights into counter-regulatory mechanisms against aging-related disease. Here, we report associations between cardiovascular disease-related biomarkers and survival to the highest ages using a unique dataset of 1,427 oldest individuals from three longitudinal cohort studies, including 36 supercentenarians, 572 semi-supercentenarians (105–109 years), 288 centenarians (100–104 years), and 531 very old people (85–99 years). During follow-up, 1,000 participants (70.1%) died. Overall, N-terminal pro-B-type natriuretic peptide (NT-proBNP), interleukin-6, cystatin C and cholinesterase are associated with all-cause mortality independent of traditional cardiovascular risk factors and plasma albumin. Of these, low NT-proBNP levels are statistically associated with a survival advantage to supercentenarian age. Only low albumin is associated with high mortality across age groups. These findings expand our knowledge on the biology of human longevity.

Supercentenarians are approaching the current longevity limit by avoiding or surviving major illness, thus identifying biomarkers for exceptional survival might provide insights into the protection against disease of aging. Here, the authors show low NT-proBNP and high albumin in plasma are the biological correlates of survival to the highest ages.

Methodology of a Novel Risk Stratification Algorithm for Patients with Multiple Myeloma in the Relapsed Setting

INTRODUCTION

Risk stratification tools provide valuable information to inform treatment decisions. Existing algorithms for patients with multiple myeloma (MM) were based on patients with newly diagnosed disease, and these have not been validated in the relapsed setting or in routine clinical practice. We developed a risk stratification algorithm (RSA) for patients with MM at initiation of second-line (2L) treatment, based on data from the Czech Registry of Monoclonal Gammopathies.

METHODS

Predictors of overall survival (OS) at 2L treatment were identified using Cox proportional hazards models and backward selection. Risk scores were obtained by multiplying the hazard ratios for each predictor. The K-adaptive partitioning for survival (KAPS) algorithm defined four groups of stratification based on individual risk scores.

RESULTS

Performance of the RSA was assessed using Nagelkerke's R² test and Harrell's concordance index through Kaplan-Meier analysis of OS data. Prognostic groups were successfully defined based on real-world data. Use of a multiplicative score based on Cox modeling and KAPS to define cut-off values was effective.

CONCLUSION

Through innovative methods of risk assessment and collaboration between physicians and statisticians, the RSA was capable of stratifying patients at 2L treatment by survival expectations. This approach can be used to develop clinical decision-making tools in other disease areas to improve patient management.

FUNDING

Amgen Europe GmbH.

Prediction models for prostate cancer to be used in the primary care setting: a systematic review

OBJECTIVE

To identify risk prediction models for prostate cancer (PCa) that can be used in the primary care and community health settings.

DESIGN

Systematic review.

DATA SOURCES

MEDLINE and Embase databases combined from inception and up to the end of January 2019.

ELIGIBILITY

Studies were included based on satisfying all the following criteria: (i) presenting an evaluation of PCa risk at initial biopsy in patients with no history of PCa, (ii) studies not incorporating an invasive clinical assessment or expensive biomarker/genetic tests, (iii) inclusion of at least two variables with prostate-specific antigen (PSA) being one of them, and (iv) studies reporting a measure of predictive performance. The quality of the studies and risk of bias was assessed by using the Prediction model Risk Of Bias ASsessment Tool (PROBAST).

DATA EXTRACTION AND SYNTHESIS

Relevant information extracted for each model included: the year of publication, source of data, type of model, number of patients, country, age, PSA range, mean/median PSA, other variables included in the model, number of biopsy cores to assess outcomes, study endpoint(s), cancer detection, model validation and model performance.

RESULTS

An initial search yielded 109 potential studies, of which five met the set criteria. Four studies were cohort-based and one was a case-control study. PCa detection rate was between 20.6% and 55.8%. Area under the curve (AUC) was reported in four studies and ranged from 0.65 to 0.75. All models showed significant improvement in predicting PCa compared with being based on PSA alone. The difference in AUC between extended models and PSA alone was between 0.06 and 0.21.

CONCLUSION

Only a few PCa risk prediction models have the potential to be readily used in the primary healthcare or community health setting. Further studies are needed to investigate other potential variables that could be integrated into models to improve their clinical utility for PCa testing in a community setting.

The association of lung ultrasound images with COVID-19 infection in an emergency room cohort

Lung ultrasound could facilitate the triage of patients with suspected COVID‐19 infection admitted to the emergency room. We developed a predictive model for COVID‐19 diagnosis based on lung ultrasound and clinical features. We used ultrasound to image the lung bilaterally at two anterior sites, one and two hands below each clavicle, and a posterolateral site that was the posterior transverse continuation from the lower anterior site. We studied 100 patients, 31 of whom had a COVID‐19 positive reverse transcriptase polymerase chain reaction. A positive test was independently associated with: quick sequential organ failure assessment score ≥1; ≥3 B‐lines at the upper site; consolidation and thickened pleura at the lower site; and thickened pleura line at the posterolateral site. The model discrimination was an area (95%CI) under the receiver operating characteristic curve of 0.82 (0.75–0.90). The characteristics (95%CI) of the model’s diagnostic threshold, applied to the population from which it was derived, were: sensitivity, 97% (83–100%); specificity, 62% (50–74%); positive predictive value, 54% (41–98%); and negative predictive value, 98% (88–99%). This model may facilitate triage of patients with suspected COVID‐19 infection admitted to the emergency room.

Machine learning to predict early recurrence after oesophageal cancer surgery

BACKGROUND

Early cancer recurrence after oesophagectomy is a common problem, with an incidence of 20-30 per cent despite the widespread use of neoadjuvant treatment. Quantification of this risk is difficult and existing models perform poorly. This study aimed to develop a predictive model for early recurrence after surgery for oesophageal adenocarcinoma using a large multinational cohort and machine learning approaches.

METHODS

Consecutive patients who underwent oesophagectomy for adenocarcinoma and had neoadjuvant treatment in one Dutch and six UK oesophagogastric units were analysed. Using clinical characteristics and postoperative histopathology, models were generated using elastic net regression (ELR) and the machine learning methods random forest (RF) and extreme gradient boosting (XGB). Finally, a combined (ensemble) model of these was generated. The relative importance of factors to outcome was calculated as a percentage contribution to the model.

RESULTS

A total of 812 patients were included. The recurrence rate at less than 1 year was 29·1 per cent. All of the models demonstrated good discrimination. Internally validated areas under the receiver operating characteristic (ROC) curve (AUCs) were similar, with the ensemble model performing best (AUC 0·791 for ELR, 0·801 for RF, 0·804 for XGB, 0·805 for ensemble). Performance was similar when internal-external validation was used (validation across sites, AUC 0·804 for ensemble). In the final model, the most important variables were number of positive lymph nodes (25·7 per cent) and lymphovascular invasion (16·9 per cent).

CONCLUSION

The model derived using machine learning approaches and an international data set provided excellent performance in quantifying the risk of early recurrence after surgery, and will be useful in prognostication for clinicians and patients.

Development and Validation of a County-Level Social Determinants of Health Risk Assessment Tool for Cardiovascular Disease

PURPOSE

Social determinants of health (SDoH) have been linked to a variety of health conditions, but there are no multivariate measures of these determinants to estimate the risk of morbidity or mortality in a community. We developed a score derived from multivariate measures of SDoH that predicts county-level cardiovascular disease (CVD) mortality.

METHODS

Using county-level data from 3,026 US counties, we developed a score considering variables of neighborhood socioeconomic status, food/lifestyle environment, and health care resource availability and accessibility to predict the 3-year average (2015-2017) age-adjusted county-level mortality rate for all CVD. We used one 50% random sample to develop the score and the other to validate the score. A Poisson regression model was developed to estimate parameters of variables while accounting for intrastate correlation.

RESULTS

The index score was based on 7 SDoH factors: percentage of the population of minority (nonwhite) race, poverty rate, percentage of the population without a high school diploma, grocery store ratio, fast-food restaurant ratio, after-tax soda price, and primary care physician supply. The area under the curve for the development and validation groups was similar, 0.851 (95% CI, 0.829-0.872) and 0.840 (95% CI, 0.817-0.863), respectively, indicating excellent discriminative ability. The index had better predictive performance for CVD burden than other area-level indexes: poverty only (area under the curve= 0.808, P <.001); the Centers for Disease Control and Prevention's Social Vulnerability Index (CDC-SVI) (area under the curve =0.786, P <.001); and the Agency for Healthcare Research and Quality's Socioeconomic Status (AHRQ-SES) index (area under the curve =0.835, P = .03).

CONCLUSIONS

Our validated multivariate SDoH index score accurately classifies counties with high CVD burden and therefore has the potential to improve CVD risk prediction for vulnerable populations and interventions for CVD at the county level.

Using big data to retrospectively validate the COMPASS-CAT risk assessment model: considerations on methodology

External validation is a prerequisite in order for a prediction model to be introduced into clinical practice. Nonetheless, methodologically intact external validation studies are a scarce finding. Utilization of big datasets can help overcome several causes of methodological failure. However, transparent reporting is needed to standardize the methods, assess the risk of bias and synthesize multiple validation studies in order to infer model generalizability. We describe the methodological challenges faced when using multiple big datasets to perform the first retrospective external validation study of the Prospective Comparison of Methods for thromboembolic risk assessment with clinical Perceptions and AwareneSS in real life patients-Cancer Associated Thrombosis (COMPASS-CAT) Risk Assessment Model for predicting venous thromboembolism in patients with cancer. The challenges included choosing the starting point, defining time sensitive variables that serve both as risk factors and outcome variables and using non-research oriented databases to form validated definitions from administrative codes. We also present the structured plan we used so as to overcome those obstacles and reduce bias with the target of producing an external validation study that successfully complies with prediction model reporting guidelines.

Identifying predictive factors for neuropathic pain after breast cancer surgery using machine learning

INTRODUCTION

Neuropathic pain (NP) remains a major debilitating condition affecting more than 26% of breast cancer survivors worldwide. NP is diagnosed using a validated 10-items Douleur Neuropathique - 4 screening questionnaire which is administered 3 months after surgery and requires patient-doctor interaction. To develop an effective prognosis model admissible soon after surgery, without the need for patient-doctor interaction, we sought to [1] identify specific pain characteristics that can help determine which patients may be susceptible to NP after BC surgery, and 2) assess the utility of machine learning models developed in objective [1] as a knowledge discovery tool for downstream analysis.

METHODS

The dataset is from a prospective cohort study of female patients scheduled to undergo breast cancer surgery for the first time at the Jewish General Hospital, Montreal, Canada between November 2014 and March 2019. NP was assessed at 3 months after surgery using Douleur Neuropathique - 4 interview scores (in short, DN4-interview; range: 0-7). For the primary analysis, we constructed six ML algorithms (least square, ridge, elastic net, random forest, gradient boosting, and neural net) to identify the most relevant predictors for DN4-interview score; and compared model performance based on root mean square error (RMSE). For the secondary analysis, we built a logistic classification model for neuropathic pain (DN4-interview score ≥ 3 versus DN4-interview score < 3) using the relevant-consensus-predictors from the primary analysis.

RESULTS

Anxiety, type of surgery, preoperative baseline pain and acute pain on movement were identified as the most relevant predictors for DN4 - interview score. The least square regression model (RMSE = 1.43) is comparable in performance with random forest (RMSE = 1.39) and neural network model (RMSE = 1.50). The Gradient boosting model (RMSE = 1.16) outperformed the models compared including the penalized regression models (ridge regressions, RMSE = 1.28; and elastic net, RMSE = 1.31). In the secondary analysis, the preferred logistic regression classier for NP had an area under the curve (AUC) of 0.68 (95% CI = 0.57 to 0.79). Anxiety was significantly associated with the likelihood of NP (odds ratio = 2.18; 95% CI = 1.05-4.49). In comparison to their counterparts, the odds of NP were higher in participants with acute pain on movement or with present preoperative baseline pain or participants who performed total mastectomy surgery, but the differences were not statistically significant.

CONCLUSIONS

Modern machine learning models show improvements over traditional least square regression in predicting of DN4-interview score. Penalized regression methods and the Gradient boosting model out-perform other models. As a predictor discovery tool, machine learning algorithms identify relevant predictors for DN4-interview score that remain statistically significant indicators of neuropathic pain in the classification model. Anxiety, type of surgery and acute pain on movement remain the most useful predictors for neuropathic pain.

Inpatient stroke rehabilitation: prediction of clinical outcomes using a machine-learning approach

BACKGROUND

In clinical practice, therapists often rely on clinical outcome measures to quantify a patient's impairment and function. Predicting a patient's discharge outcome using baseline clinical information may help clinicians design more targeted treatment strategies and better anticipate the patient's assistive needs and discharge care plan. The objective of this study was to develop predictive models for four standardized clinical outcome measures (Functional Independence Measure, Ten-Meter Walk Test, Six-Minute Walk Test, Berg Balance Scale) during inpatient rehabilitation.

METHODS

Fifty stroke survivors admitted to a United States inpatient rehabilitation hospital participated in this study. Predictors chosen for the clinical discharge scores included demographics, stroke characteristics, and scores of clinical tests at admission. We used the Pearson product-moment and Spearman's rank correlation coefficients to calculate correlations among clinical outcome measures and predictors, a cross-validated Lasso regression to develop predictive equations for discharge scores of each clinical outcome measure, and a Random Forest based permutation analysis to compare the relative importance of the predictors.

RESULTS

The predictive equations explained 70-77% of the variance in discharge scores and resulted in a normalized error of 13-15% for predicting the outcomes of new patients. The most important predictors were clinical test scores at admission. Additional variables that affected the discharge score of at least one clinical outcome were time from stroke onset to rehabilitation admission, age, sex, body mass index, race, and diagnosis of dysphasia or speech impairment.

CONCLUSIONS

The models presented in this study could help clinicians and researchers to predict the discharge scores of clinical outcomes for individuals enrolled in an inpatient stroke rehabilitation program that adheres to U.S. Medicare standards.

Development and Validation of a Nomogram for Predicting the 6-Year Risk of Cognitive Impairment Among Chinese Older Adults

OBJECTIVE

Although some people with mild cognitive impairment may not suffer from dementia lifelong, about 5% of them will progress to dementia within 1 year in community settings. However, a general tool for predicting the risk of cognitive impairment was not adequately studied among older adults.

DESIGN

Prospective cohort study.

SETTING

Community-living, older adults from 22 provinces in China.

PARTICIPANTS

We included 10,066 older adults aged 65 years and above (mean age, 83.2 ± 11.1 years), with normal cognition at baseline in the 2002-2008 cohort and 9354 older adults (mean age, 83.5 ± 10.8 years) in the 2008-2014 cohort of the Chinese Longitudinal Healthy Longevity Survey.

METHODS

We measured cognitive function using the Chinese version of the Mini-Mental State Examination. Demographic, medical, and lifestyle information was used to develop the nomogram via a Lasso selection procedure using a Cox proportional hazards regression model. We validated the nomogram internally with 2000 bootstrap resamples and externally in a later cohort. The predictive accuracy and discriminative ability of the nomogram were measured by area-under-the-curves and calibration curves, respectively.

RESULTS

Eight factors were identified with which to construct the nomogram: age, baseline of the Mini-Mental State Examination, activities of daily living and instrumental activities of daily living score, chewing ability, visual function, history of stroke, watching TV or listening to the radio, and growing flowers or raising pets. The area-under-the-curves for internal and external validation were 0.891 and 0.867, respectively, for predicting incident cognitive impairment. The calibration curves showed good consistency between nomogram-based predictions and observations.

CONCLUSIONS AND IMPLICATIONS

The nomogram-based prediction yielded consistent results in 2 separate large cohorts. This feasible prognostic nomogram constructed using readily ascertained information may assist public health practitioners or physicians to provide preventive interventions of cognitive impairment.

Genetic Architecture Associated With Familial Short Stature

CONTEXT

Human height is an inheritable, polygenic trait under complex and multilocus genetic regulation. Familial short stature (FSS; also called genetic short stature) is the most common type of short stature and is insufficiently known.

OBJECTIVE

To investigate the FSS genetic profile and develop a polygenic risk predisposition score for FSS risk prediction.

DESIGN AND SETTING

The FSS participant group of Han Chinese ancestry was diagnosed by pediatric endocrinologists in Taiwan.

PATIENTS AND INTERVENTIONS

The genetic profiles of 1163 participants with FSS were identified by using a bootstrapping subsampling and genome-wide association studies (GWAS) method.

MAIN OUTCOME MEASURES

Genetic profile, polygenic risk predisposition score for risk prediction.

RESULTS

Ten novel genetic single nucleotide polymorphisms (SNPs) and 9 reported GWAS human height-related SNPs were identified for FSS risk. These 10 novel SNPs served as a polygenic risk predisposition score for FSS risk prediction (area under the curve: 0.940 in the testing group). This FSS polygenic risk predisposition score was also associated with the height reduction regression tendency in the general population.

CONCLUSION

A polygenic risk predisposition score composed of 10 genetic SNPs is useful for FSS risk prediction and the height reduction tendency. Thus, it might contribute to FSS risk in the Han Chinese population from Taiwan.

Refining prediction of major bleeding on antiplatelet treatment after transient ischaemic attack or ischaemic stroke

Introduction

Bleeding is the main safety concern of treatment with antiplatelet drugs. We aimed to refine prediction of major bleeding on antiplatelet treatment after a transient ischaemic attack (TIA) or stroke by assessing the added value of new predictors to the existing S₂TOP-BLEED score.

Patients and methods

We used Cox regression analysis to study the association between candidate predictors and major bleeding among 2072 patients with a transient ischaemic attack or ischaemic stroke included in a population-based study (Oxford Vascular Study – OXVASC). An updated model was proposed and validated in 1094 patients with a myocardial infarction included in OXVASC. Models were compared with c-statistics, calibration plots, and net reclassification improvement.

Results

Independent predictors for major bleeding on top of S₂TOP-BLEED variables were peptic ulcer (hazard ratio (HR): 1.72; 1.04–2.86), cancer (HR: 2.40; 1.57–3.68), anaemia (HR: 1.55; 0.99–2.44) and renal failure (HR: 2.20; 1.57–4.28). Addition of those variables improved discrimination from 0.69 (0.64–0.73) to 0.73 (0.69–0.78) in the TIA/stroke cohort (p = 0.01). Performance improved particularly for upper gastro-intestinal bleeds (0.70; 0.64–0.75 to 0.77; 0.72–0.82). Net reclassification improved over the entire range of the score (net reclassification improvement: 0.56; 0.36–0.76). In the validation cohort, discriminatory performance improved from 0.68 (0.62–0.74) to 0.70 (0.64–0.76).

Discussion and Conclusion

Peptic ulcer, cancer, anaemia and renal failure improve predictive performance of the S₂TOP-BLEED score for major bleeding after stroke. Future external validation studies will be required to confirm the value of the STOP-BLEED+ score in transient ischaemic attack/stroke patients.

Screening for preeclampsia in low-risk twin pregnancies at early gestation

INTRODUCTION

Preeclampsia affects about 10% of twin pregnancies and significantly increases the risk of adverse pregnancy outcomes. However, screening models for preeclampsia in twin pregnancies remain elusive. The present study aimed to evaluate the performance of a multi-marker first trimester preeclampsia screening model in low-risk twin pregnancies.

MATERIAL AND METHODS

Between 2014 and 2017, we prospectively assessed first trimester biomarkers for preeclampsia in a 'low-risk' twin pregnancy cohort at a single center. Multiple logistic regression was used to determine significant predictors for early preeclampsia (occurring prior to 34 weeks) and late preeclampsia (occurring after 34 weeks). The performance of the screening models fitted using the significant predictors was calculated using receiver operating characteristics curves, and internal validation was performed using bootstrapping.

RESULTS

A total of 769 twin pregnancies were included in the study. Early preeclampsia and late preeclampsia developed in 27 (3.5%) and 59 (7.7%) cases, respectively. Logistic regression analyses showed that maternal age, body mass index, mean artery pressure and placental growth factor were significant predictors for early preeclampsia. Maternal age, body mass index, mean artery pressure and pregnancy-associated plasma protein A were significant for late preeclampsia. Uterine artery pulsatility index was not predictive of either early or late preeclampsia. For the fitted screening model of early and late preeclampsia, the areas under receiver operating characteristics curves were 0.82 (95% confidence interval [CI] 0.76-0.88) and 0.66 (95% CI 0.59-0.73), which were expected to decrease to 0.77 and 0.60, respectively, based on bootstrapping; the positive predictive values were 10.2% and 12.5%; and the estimated detection rates were 40.7% and 22.0%, respectively, at a false-positive rate of 10%.

CONCLUSIONS

A multi-marker screening model for preeclampsia in low-risk twin pregnancies, using a modified version of Fetal Medicine Foundation predictors in singletons, does not perform well. Uterine artery pulsatility index is of little value in screening for preeclampsia in low-risk twin pregnancies.

Regression shrinkage methods for clinical prediction models do not guarantee improved performance: Simulation study

When developing risk prediction models on datasets with limited sample size, shrinkage methods are recommended. Earlier studies showed that shrinkage results in better predictive performance on average. This simulation study aimed to investigate the variability of regression shrinkage on predictive performance for a binary outcome. We compared standard maximum likelihood with the following shrinkage methods: uniform shrinkage (likelihood-based and bootstrap-based), penalized maximum likelihood (ridge) methods, LASSO logistic regression, adaptive LASSO, and Firth's correction. In the simulation study, we varied the number of predictors and their strength, the correlation between predictors, the event rate of the outcome, and the events per variable. In terms of results, we focused on the calibration slope. The slope indicates whether risk predictions are too extreme (slope < 1) or not extreme enough (slope > 1). The results can be summarized into three main findings. First, shrinkage improved calibration slopes on average. Second, the between-sample variability of calibration slopes was often increased relative to maximum likelihood. In contrast to other shrinkage approaches, Firth's correction had a small shrinkage effect but showed low variability. Third, the correlation between the estimated shrinkage and the optimal shrinkage to remove overfitting was typically negative, with Firth's correction as the exception. We conclude that, despite improved performance on average, shrinkage often worked poorly in individual datasets, in particular when it was most needed. The results imply that shrinkage methods do not solve problems associated with small sample size or low number of events per variable.

Plasma metabolite biomarkers predictive of radiation induced cardiotoxicity

PURPOSE

Although advancements in cancer treatments using radiation therapy (RT) have led to improved outcomes, radiation-induced heart disease (RIHD) remains a significant source of morbidity and mortality in survivors of cancers in the chest. Currently, there are no diagnostic tests in clinical use due to a lack of understanding of the natural history and mechanisms of RIHD development. Few studies have examined the utility of using metabolomics to prospectively identify cancer survivors who are at risk of developing cardiotoxicity.

METHODS

We analyzed plasma and left ventricle heart tissue samples collected from a cohort of male Sprague Dawley (SD) rats that were either sham irradiated or received fractionated doses (9 Gy per day × 5 days) of targeted X-ray radiation to the heart. Metabolomic and lipidomic analyses were utilized as a correlative approach for delineation of novel biomarkers associated with radiation-induced cardiac toxicity. Additionally, we used high-resolution mass spectrometry to examine the metabolomic profiles of plasma samples obtained from patients receiving high dose thoracic RT for esophageal cancer.

RESULTS

Metabolic alterations in the rat model and patient plasma profiles, showed commonalities of radiation response that included steroid hormone biosynthesis and vitamin E metabolism. Alterations in patient plasma profiles were used to develop classification algorithms predictive of patients at risk of developing RIHD.

CONCLUSION

Herein, we report the feasibility of developing a metabolomics-based biomarker panel that is associated with adverse outcomes of cardiac function in patients who received RT for the treatment of esophageal cancer.

Developing and validating a multivariable prediction model for in-hospital mortality of pneumonia with advanced chronic kidney disease patients: a retrospective analysis using a nationwide database in Japan

BACKGROUND

The prognosis of pneumonia in patients with advanced stage chronic kidney disease (CKD) remains unimproved for years. We attempt to develop a simple and more useful scoring system for predicting in-hospital mortality for advanced CKD patients with pneumonia.

METHODS

Using the Diagnosis Procedure Combination database, we identified the in-hospital adult patients both with a record of pneumonia and stage 5 or 5D CKD as a comorbidity on admission between April 1, 2012 and March 31, 2016. Predictive variable selection was analyzed by multivariable logistic regression analysis, stepwise method, LASSO method and random forest method, and then develop a new simple scoring system seeking for highest c-statistics combination of variables in one sample data set for model development. Finally, we compared c-statistics of univariate logistic regression about new scoring system with c-statistics about "A-DROP" in the other sample data set.

RESULT

We identified 8402 patients in 707 hospitals, and the total in-hospital mortality was 11.0% (437 patients) in development data set. Seven variables were selected, which includes age (male ≥ 70 years, female ≥ 75 years), respiratory failure, orientation disturbance, low blood pressure, the need of assistance in feeding or bowel control, severe or moderate thinness and CRP 200 mg/L or extent of consolidation on chest X-ray ≥ 2/3 of one lung. The c-statistics of univariate logistic regression was 0.8017 using seven variables, while that was 0.7372 using "A-DROP" CONCLUSION: In advanced CKD patients, if we select appropriate variables for predicting in-hospital mortality, simple scoring system may have better discrimination than "A-DROP".

Neurocritical Care: Bench to Bedside (Eds. Claude Hemphill, Michael James) Integrating and Using Big Data in Neurocritical Care

The critical care environment drives huge volumes of data, and clinicians are tasked with quickly processing this data and responding to it urgently. The neurocritical care environment increasingly involves EEG, multimodal intracranial monitoring, and complex imaging which preclude comprehensive human synthesis, and requires new concepts to integrate data into clinical care. By definition, Big Data is data that cannot be handled using traditional infrastructures and is characterized by the volume, variety, velocity, and variability of the data being produced. Big Data in the neurocritical care unit requires rethinking of data storage infrastructures and the development of tools and analytics to drive advancements in the field. Preprocessing, feature extraction, statistical inference, and analytic tools are required in order to achieve the primary goals of Big Data for clinical use: description, prediction, and prescription. Barriers to its use at bedside include a lack of infrastructure development within the healthcare industry, lack of standardization of data inputs, and ultimately existential and scientific concerns about the outputs that result from the use of tools such as artificial intelligence. However, as implied by the fundamental theorem of biomedical informatics, physicians remain central to the development and utility of Big Data to improve patient care.

Checking the predictive accuracy of basic symptoms against ultra high-risk criteria and testing of a multivariable prediction model: Evidence from a prospective three-year observational study of persons at clinical high-risk for psychosis

Background:

The aim of this study was to critically examine the prognostic validity of various clinical high-risk (CHR) criteria alone and in combination with additional clinical characteristics.

Methods:

A total of 188 CHR positive persons from the region of Zurich, Switzerland (mean age 20.5 years; 60.2% male), meeting ultra high-risk (UHR) and/or basic symptoms (BS) criteria, were followed over three years. The test battery included the Structured Interview for Prodromal Syndromes (SIPS), verbal IQ and many other screening tools. Conversion to psychosis was defined according to ICD-10 criteria for schizophrenia (F20) or brief psychotic disorder (F23).

Results:

Altogether n = 24 persons developed manifest psychosis within three years and according to Kaplan–Meier survival analysis, the projected conversion rate was 17.5%. The predictive accuracy of UHR was statistically significant but poor (area under the curve [AUC] = 0.65, P < .05), whereas BS did not predict psychosis beyond mere chance (AUC = 0.52, P = .730). Sensitivity and specificity were 0.83 and 0.47 for UHR, and 0.96 and 0.09 for BS. UHR plus BS achieved an AUC = 0.66, with sensitivity and specificity of 0.75 and 0.56. In comparison, baseline antipsychotic medication yielded a predictive accuracy of AUC = 0.62 (sensitivity = 0.42; specificity = 0.82). A multivariable prediction model comprising continuous measures of positive symptoms and verbal IQ achieved a substantially improved prognostic accuracy (AUC = 0.85; sensitivity = 0.86; specificity = 0.85; positive predictive value = 0.54; negative predictive value = 0.97).

Conclusions:

We showed that BS have no predictive accuracy beyond chance, while UHR criteria poorly predict conversion to psychosis. Combining BS with UHR criteria did not improve the predictive accuracy of UHR alone. In contrast, dimensional measures of both positive symptoms and verbal IQ showed excellent prognostic validity. A critical re-thinking of binary at-risk criteria is necessary in order to improve the prognosis of psychotic disorders.

Development and Evaluation of a Machine Learning Prediction Model for Flap Failure in Microvascular Breast Reconstruction

BACKGROUND

Despite high success rates, flap failure remains an inherent risk in microvascular breast reconstruction. Identifying patients who are at high risk for flap failure would enable us to recommend alternative reconstructive techniques. However, as flap failure is a rare event, identification of risk factors is statistically challenging. Machine learning is a form of artificial intelligence that automates analytical model building. It has been proposed that machine learning can build superior prediction models when the outcome of interest is rare.

METHODS

In this study we evaluate machine learning resampling and decision-tree classification models for the prediction of flap failure in a large retrospective cohort of microvascular breast reconstructions.

RESULTS

A total of 1012 patients were included in the study. Twelve patients (1.1%) experienced flap failure. The ROSE informed oversampling technique and decision-tree classification resulted in a strong prediction model (AUC 0.95) with high sensitivity and specificity. In the testing cohort, the model maintained acceptable specificity and predictive power (AUC 0.67), but sensitivity was reduced. The model identified four high-risk patient groups. Obesity, comorbidities and smoking were found to contribute to flap loss. The flap failure rate in high-risk patients was 7.8% compared with 0.44% in the low-risk cohort (p = 0.001).

CONCLUSIONS

This machine-learning risk prediction model suggests that flap failure may not be a random event. The algorithm indicates that flap failure is multifactorial and identifies a number of potential contributing factors that warrant further investigation.

Predictive tools to determine risk of infection in kidney transplant recipients

Introduction: Infection represents a major complication after kidney transplantation (KT). Therapeutic drug monitoring is essentially the only approach for the adjustment of immunosuppression in current practice, with suboptimal results. The implementation of immune monitoring strategies may contribute to minimizing the risk of adverse events attributable to over-immunosuppression without compromising graft outcomes.Areas covered: The present review (based on PubMed/MEDLINE searches from database inception to November 2019) is focused on immune biomarkers with no antigen specificity (non-pathogen-specific), including serum levels of immunoglobulins and complement factors, peripheral blood lymphocyte subpopulations, soluble CD30, intracellular ATP production by stimulated CD4⁺ T-cells, and other cell-based immune assays. We also summarized recent advances in the use of replication kinetics of latent viruses to assess the functionality of T-cell immunity, with focus on the nonpathogenic anelloviruses. Finally, the composite risk scores reported in the literature are critically discussed.Expert opinion: Notable efforts have been made to develop an enlarging repertoire of immune biomarkers and prediction models, although most of them still lack technical standardization and external validation. Preventive interventions based on these tools (prolongation of prophylaxis, tapering of immunosuppression, or immunoglobulin replacement therapy in hypogammaglobulinemic patients) remain to be defined, ideally in the context of controlled trials.

Systematic review: risk prediction models for recurrence of hepatocellular carcinoma after liver transplantation

Recurrence of hepatocellular carcinoma (HCC) after liver transplantation (LT) is a significant clinical problem associated with poor surgical outcomes. This study aims to summarize the current evidence on risk prediction models of HCC recurrence after LT. PubMed and EMBASE were searched to May 25, 2019, for relevant articles. Studies originally designed to develop or validate a risk prediction model for HCC recurrence after LT were included. Two independent authors summarized the study characteristics and evaluated the risk of bias and applicability concerns in the included studies. From 26 included studies, 18 original risk prediction models were determined, but only five models were externally validated. The average number of predictors involved in the construction of risk models was three. The most frequently employed predictors were alpha-fetoprotein, tumor size, vascular invasion, tumor number, tumor differentiation, and neutrophil-lymphocyte ratio. Most studies showed good discriminatory performance (AUC >0.75). The overall quality of the included studies was generally low. Most of the original models lacked the highly recommended external and prospective validation in diverse populations. The AFP model was the well-validated preoperative risk model that can stratify patients into high- and low-risk groups.

Development and Validation of a Prognostic Model for Patients with Advanced Lung Cancer Treated with the Immune Checkpoint Inhibitor Atezolizumab

PURPOSE

Immune checkpoint inhibitors (ICI) are a significant advance to the treatment of advanced non-small cell lung cancer (NSCLC); however, their initiation is associated with heterogeneity in outcomes. This study aimed to develop and validate a prognostic tool of survival in patients with advanced NSCLC treated with ICIs.

EXPERIMENTAL DESIGN

A pretreatment prognostic model was developed and validated using clinicopathologic data. Development data consisted of patients with advanced NSCLC treated with atezolizumab from the randomised trials OAK and POPLAR (n = 751). Data from the single-arm atezolizumab trials BIRCH and FIR (n = 797) were used for external validation. Prognostic scores were categorized into low, intermediate-low, intermediate, intermediate-high, and high-risk prognostic groups. The primary outcome was overall survival (OS), with progression-free survival (PFS) secondary.

RESULTS

Pretreatment C-reactive protein (CRP) was the most predictive variable for OS. The prognostic tool was optimally defined by CRP, lactate dehydrogenase, derived neutrophil-to-lymphocyte ratio, albumin, PD-L1 expression, performance status, time since metastatic diagnosis, and metastatic site count. Prognostic groups had significantly different OS (c-statistic = 0.72), with median OS ranging from >24 to 3 months for the low- to high-risk groups. Performance was maintained on validation (c = 0.76). These findings were similar for PFS, with median PFS ranging from 5 months to 1 month for the low- to high-risk groups. Benefit of atezolizumab (vs. docetaxel) was greatest in the low-risk group (>3 months median OS improvement), with little benefit apparent for the highest risk group.

CONCLUSIONS

A prognostic tool was developed and validated to identify patient groups with distinctly different survival following atezolizumab initiation for advanced NSCLC.

Lactate Measurements and Their Association With Mortality in Pediatric Severe Sepsis in India: Evidence That 6-Hour Level Performs Best

PURPOSE

To investigate association of static and dynamic lactate indices with early mortality (within 48 hours of admission), as well as need for vasopressors and mechanical ventilation in pediatric severe sepsis/shock. To explore optimal cutoffs of lactate indices. We hypothesized that dynamic indices are superior to static indices in predicting early mortality.

METHODS

This prospective cohort study involved children (aged <14 years) admitted in emergency department, tertiary care teaching hospital in North India with severe sepsis/shock (2015-2016). Arterial lactate was measured at admission (X0) and after 6 hours (X6). Primary outcome of the measurement was early mortality. Association between lactate indices- lactate at 0 hours (Lac0), lactate after 6 hours (Lac6), time-weighted average (LacTW), delta (ΔLac), clearance (LacCl%) and early mortality, need for vasopressors, and mechanical ventilation-was assessed using Student t test/Mann-Whitney test. Area under the receiver operating characteristic curve (AUROC) for early mortality deduced for all lactate indices and compared with reference (Lac0). Optimal cutoffs (maximizing both sensitivity and specificity) and their positive predictive value (PPV) and negative predictive value (NPV) were determined.

RESULTS

During the study period, 116 children were assessed. Septic shock was present at admission in 56.9% children; 50% of children died during the next 48 hours. Lac0, Lac6, and LacTW were significantly higher, and LacCl% was lower in nonsurvivors versus survivors (all P < .001). Lac6 (0.837 [0.76-0.91]) had significantly higher AUROC (95% confidence interval) than Lac0 (0.77; P = .03). Abnormal lactate metrics (higher Lac0, Lac6, LacTW, and lower LacCl%) were associated with vasopressors need and mechanical ventilation. On logistic regression, Lac6 emerged as an independent predictor of early mortality as well as vasopressor and mechanical ventilation need. The optimal cutoff of Lac6 for identifying early mortality with good sensitivity, specificity, PPV, and NPV was ≥2.65 (76, 85, 83, 78).

CONCLUSIONS

Lactate6 is the best marker associated with early mortality and higher level of care in severe sepsis/septic shock in resource-poor regions.

Heart rate variability and falls in Huntington's disease

PURPOSE

Persons with Huntington's disease (HD) have a high incidence of falls. Autonomic nervous system dysfunction has been reported even in early stages of this disease. To date, there has been no analysis of the relationship between heart rate variability (HRV) and falls in this patient population. The aim of the study reported here was to evaluate the relationship between HRV and falls in persons with HD.

METHODS

Huntington's disease patients enrolled in a prospective study on fear of falling and falls were assessed using short-term HRV analyses and blood pressure measures in both the resting and standing states. Time-frequency domains and nonlinear parameters were calculated. Data on falls, the risk of falling (RoF) and disease-specific scales were collected at baseline and at the end of the 6-month follow-up.

RESULTS

Of the 24 HD patients who were invited to participate in the study, 20 completed the baseline analysis and 18 completed the 6-month follow-up. At baseline, seven (35%) HD patients reported at least one fall (single fallers) and 13 (65%) reported ≥ 2 falls (recurrent fallers) in the previous 12 months. At baseline, recurrent fallers had lower RMSSD (root mean square of successive RR interval differences) in the resting state (RMSSD-resting), higher LF/HF (low/high frequency) ratio in both states and higher DFA-α1 parameter (detrended fluctuation analyses over the short term) in both states. This association was similar at the 6-month follow-up for recurrent fallers, who showed lower RMSSD-resting and higher LF/HF ratio in the standing state (LF/HF-standing) than single fallers. Significant correlations were found between the number of falls, RMSSD-resting and LF/HF-standing. No differences were found between recurrent and single fallers for any blood pressure measures.

CONCLUSIONS

The observed HRV pattern is consistent with a higher sympathetic prevalence associated with a higher RoF. Reduced parasympathetic HRV values in this patient population predict being a recurrent faller at 6 months of follow-up, independently of orthostatic phenomena.

Medication-related harm in older adults following hospital discharge: development and validation of a prediction tool

OBJECTIVES

To develop and validate a tool to predict the risk of an older adult experiencing medication-related harm (MRH) requiring healthcare use following hospital discharge.

DESIGN, SETTING, PARTICIPANTS

Multicentre, prospective cohort study recruiting older adults (≥65 years) discharged from five UK teaching hospitals between 2013 and 2015.

PRIMARY OUTCOME MEASURE

Participants were followed up for 8 weeks in the community by senior pharmacists to identify MRH (adverse drug reactions, harm from non-adherence, harm from medication error). Three data sources provided MRH and healthcare use information: hospital readmissions, primary care use, participant telephone interview. Candidate variables for prognostic modelling were selected using two systematic reviews, the views of patients with MRH and an expert panel of clinicians. Multivariable logistic regression with backward elimination, based on the Akaike Information Criterion, was used to develop the PRIME tool. The tool was internally validated.

RESULTS

1116 out of 1280 recruited participants completed follow-up (87%). Uncertain MRH cases ('possible' and 'probable') were excluded, leaving a tool derivation cohort of 818. 119 (15%) participants experienced 'definite' MRH requiring healthcare use and 699 participants did not. Modelling resulted in a prediction tool with eight variables measured at hospital discharge: age, gender, antiplatelet drug, sodium level, antidiabetic drug, past adverse drug reaction, number of medicines, living alone. The tool's discrimination C-statistic was 0.69 (0.66 after validation) and showed good calibration. Decision curve analysis demonstrated the potential value of the tool to guide clinical decision making compared with alternative approaches.

CONCLUSIONS

The PRIME tool could be used to identify older patients at high risk of MRH requiring healthcare use following hospital discharge. Prior to clinical use we recommend the tool's evaluation in other settings.

Developing an individualized risk calculator for psychopathology among young people victimized during childhood: A population-representative cohort study

BACKGROUND

Victimized children are at greater risk for psychopathology than non-victimized peers. However, not all victimized children develop psychiatric disorders, and accurately identifying which victimized children are at greatest risk for psychopathology is important to provide targeted interventions. This study sought to develop and internally validate individualized risk prediction models for psychopathology among victimized children.

METHODS

Participants were members of the Environmental Risk (E-Risk) Longitudinal Twin Study, a nationally-representative British birth cohort of 2,232 twins born in 1994-1995. Victimization exposure was measured prospectively between ages 5 and 12 years, alongside a comprehensive range of individual-, family-, and community-level predictors of psychopathology. Structured psychiatric interviews took place at age-18 assessment. Logistic regression models were estimated with Least Absolute Shrinkage and Selection Operator (LASSO) regularization to avoid over-fitting to the current sample, and internally validated using 10-fold nested cross-validation.

RESULTS

26.5% (n = 591) of E-Risk participants had been exposed to at least one form of severe childhood victimization, and 60.4% (n = 334) of victimized children met diagnostic criteria for any psychiatric disorder at age 18. Separate prediction models for any psychiatric disorder, internalizing disorders, and externalizing disorders selected parsimonious subsets of predictors. The three internally validated models showed adequate discrimination, based on area-under-the-curve estimates (range = =0.66-0.73), and good calibration.

LIMITATIONS

External validation in wholly-independent data is needed before clinical implementation.

CONCLUSIONS

Findings offer proof-of-principle evidence that prediction modeling can be useful in supporting identification of victimized children at greatest risk for psychopathology. This has the potential to inform targeted interventions and rational resource allocation.

Risk model for severe postoperative complications after total pancreatectomy based on a nationwide clinical database

BACKGROUND

Total pancreatectomy is required to completely clear tumours that are locally advanced or located in the centre of the pancreas. However, reports describing clinical outcomes after total pancreatectomy are rare. The aim of this retrospective observational study was to assess clinical outcomes following total pancreatectomy using a nationwide registry and to create a risk model for severe postoperative complications.

METHODS

Patients who underwent total pancreatectomy from 2013 to 2017, and who were recorded in the Japan Society of Gastroenterological Surgery and Japanese Society of Hepato-Biliary-Pancreatic Surgery database, were included. Severe complications at 30 days were defined as those with a Clavien-Dindo grade III needing reoperation, or grade IV-V. Occurrence of severe complications was modelled using data from patients treated from 2013 to 2016, and the accuracy of the model tested among patients from 2017 using c-statistics and a calibration plot.

RESULTS

A total of 2167 patients undergoing total pancreatectomy were included. Postoperative 30-day and in-hospital mortality rates were 1·0 per cent (22 of 2167 patients) and 2·7 per cent (58 of 167) respectively, and severe complications developed in 6·0 per cent (131 of 2167). Factors showing a strong positive association with outcome in this risk model were the ASA performance status grade and combined arterial resection. In the test cohort, the c-statistic of the model was 0·70 (95 per cent c.i. 0·59 to 0·81).

CONCLUSION

The risk model may be used to predict severe complications after total pancreatectomy.

Preoperative Prediction of Pathologic Response to Neoadjuvant Chemoradiotherapy in Patients With Esophageal Cancer Using 18F-FDG PET/CT and DW-MRI: A Prospective Multicenter Study

PURPOSE

Accurate preoperative prediction of pathologic response to neoadjuvant chemoradiotherapy (nCRT) in patients with esophageal cancer could enable omission of esophagectomy in patients with a pathologic complete response (pCR). This study aimed to evaluate the individual and combined value of ¹⁸F-fluorodeoxyglucose positron emission tomography with integrated computed tomography (¹⁸F-FDG PET/CT) and diffusion-weighted magnetic resonance imaging (DW-MRI) during and after nCRT to predict pathologic response in patients with esophageal cancer.

METHODS AND MATERIALS

In this multicenter prospective study, patients scheduled to receive nCRT followed by esophagectomy for esophageal cancer underwent ¹⁸F-FDG PET/CT and DW-MRI scanning before the start of nCRT, during nCRT, and before esophagectomy. Response to nCRT was based on histopathologic evaluation of the resection specimen. Relative changes in ¹⁸F-FDG PET/CT and DW-MRI parameters were compared between patients with pCR and non-pCR groups. Multivariable ridge regression analyses with bootstrapped c-indices were performed to evaluate the individual and combined value of ¹⁸F-FDG PET/CT and DW-MRI.

RESULTS

pCR was found in 26.1% of 69 patients. Relative changes in ¹⁸F-FDG PET/CT parameters after nCRT (Δ standardized uptake value [SUV]_mean,postP = .016, and Δ total lesion glycolysis _postP = .024), as well as changes in DW-MRI parameters during nCRT (Δ apparent diffusion coefficient [ADC]_duringP = .008) were significantly different between pCR and non-pCR. A c-statistic of 0.84 was obtained for a model with ΔADC_during, ΔSUV_mean,post, and histology in classifying patients as pCR (versus 0.82 for ΔADC_during and 0.79 for ΔSUV_mean,post alone).

CONCLUSIONS

Changes on ¹⁸F-FDG PET/CT after nCRT and early changes on DW-MRI during nCRT can help identify pCR to nCRT in esophageal cancer. Moreover, ¹⁸F-FDG PET/CT and DW-MRI might be of complementary value in the assessment of pCR.

A novel risk score model for stomach adenocarcinoma based on the expression levels of 10 genes

Stomach adenocarcinoma (STAD) accounts for 95% of cases of malignant gastric cancer, which is the third leading cause of cancer-associated mortality worldwide. The pathogenesis and effective diagnosis of STAD have become popular topics for research in the previous decade. In the present study, high-throughput RNA sequencing expression profiles and clinical data from patients with STAD were obtained from The Cancer Genome Atlas database and were used as a training dataset to screen differentially expressed genes (DEGs). Prognostic DEGs were identified using univariate Cox regression analysis and were further screened by the least absolute shrinkage and selection operator regularization regression algorithm. The resulting genes were used to construct a risk score model, the validation and effectiveness evaluation of which were performed on an independent dataset downloaded from the Gene Expression Omnibus database. Stratified and functional pathway (gene set enrichment) analyses were performed on groups with different estimated prognosis. A total of 92 genes significantly associated with STAD prognosis were obtained by univariate Cox regression analysis, and 10 prognosis-associated DEGs; hemoglobin b, chromosome 4 open reading frame 48, Dickkopf WNT signaling pathway inhibitor 1, coagulation factor V, serpin family E member 1, transmembrane protein 200A, NADPH oxidase organizer 1, C-X-C motif chemokine ligand 3, mannosidase endo-α-like and tripartite motif-containing 31; were selected for the development of the risk score model. The reliability of this prognostic method was verified using a validation set, and the results of multivariate Cox analysis indicated that the risk score may serve as an independent prognostic factor. In functional DEG analysis, eight Kyoto Encyclopedia of Genes and Genomes pathways were identified to be significantly associated with STAD risk factors. Thus, the 10-gene risk score model established in the present study was regarded as credible. This risk assessment tool may help identify patients with a high risk of STAD, and the proposed prognostic mRNAs may be useful in elucidating STAD pathogenesis.

Development and validation of machine learning models to predict gastrointestinal leak and venous thromboembolism after weight loss surgery: an analysis of the MBSAQIP database

BACKGROUND

Postoperative gastrointestinal leak and venous thromboembolism (VTE) are devastating complications of bariatric surgery. The performance of currently available predictive models for these complications remains wanting, while machine learning has shown promise to improve on traditional modeling approaches. The purpose of this study was to compare the ability of two machine learning strategies, artificial neural networks (ANNs), and gradient boosting machines (XGBs) to conventional models using logistic regression (LR) in predicting leak and VTE after bariatric surgery.

METHODS

ANN, XGB, and LR prediction models for leak and VTE among adults undergoing initial elective weight loss surgery were trained and validated using preoperative data from 2015 to 2017 from Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program database. Data were randomly split into training, validation, and testing populations. Model performance was measured by the area under the receiver operating characteristic curve (AUC) on the testing data for each model.

RESULTS

The study cohort contained 436,807 patients. The incidences of leak and VTE were 0.70% and 0.46%. ANN (AUC 0.75, 95% CI 0.73-0.78) was the best-performing model for predicting leak, followed by XGB (AUC 0.70, 95% CI 0.68-0.72) and then LR (AUC 0.63, 95% CI 0.61-0.65, p < 0.001 for all comparisons). In detecting VTE, ANN, and XGB, LR achieved similar AUCs of 0.65 (95% CI 0.63-0.68), 0.67 (95% CI 0.64-0.70), and 0.64 (95% CI 0.61-0.66), respectively; the performance difference between XGB and LR was statistically significant (p = 0.001).

CONCLUSIONS

ANN and XGB outperformed traditional LR in predicting leak. These results suggest that ML has the potential to improve risk stratification for bariatric surgery, especially as techniques to extract more granular data from medical records improve. Further studies investigating the merits of machine learning to improve patient selection and risk management in bariatric surgery are warranted.

Logistic LASSO and Elastic Net to Characterize Vitamin D Deficiency in a Hypertensive Obese Population

Aim: The primary objective of our research was to compare the performance of data analysis to predict vitamin D deficiency using three different regression approaches and to evaluate the usefulness of incorporating machine learning algorithms into the data analysis in a clinical setting. Methods: We included 221 patients from our hypertension unit, whose data were collected from electronic records dated between 2006 and 2017. We used classical stepwise logistic regression, and two machine learning methods [least absolute shrinkage and selection operator (LASSO) and elastic net]. We assessed the performance of these three algorithms in terms of sensitivity, specificity, misclassification error, and area under the curve (AUC). Results: LASSO and elastic net regression performed better than logistic regression in terms of AUC, which was significantly better in both penalized methods, with AUC = 0.76 and AUC = 0.74 for elastic net and LASSO, respectively, than in logistic regression, with AUC = 0.64. In terms of misclassification rate, elastic net (18%) outperformed LASSO (22%) and logistic regression (25%). Conclusion: Compared with a classical logistic regression approach, penalized methods were found to have better performance in predicting vitamin D deficiency. The use of machine learning algorithms such as LASSO and elastic net may significantly improve the prediction of vitamin D deficiency in a hypertensive obese population.

Protocol for the derivation and external validation of a 30-day mortality risk prediction model for older patients having emergency general surgery (PAUSE score-Probability of mortality Associated with Urgent/emergent general Surgery in oldEr patients score)

INTRODUCTION

People 65 years and older represent the fastest growing segment of the surgical population. Older age is associated with doubling of risk when undergoing emergency general surgery (EGS) procedures and often coexists with medical complexity and considerations of end-of-life care, creating prognostic and decisional uncertainty. Combined with the time-sensitive nature of EGS, it is challenging to gauge perioperative risk and ensure that clinical decisions are aligned with the patient values. Current preoperative risk prediction models for older EGS patients have major limitations regarding derivation and validation, and do not address the specific risk profile of older patients. Accurate and externally validated models specific to older patients are needed to inform care and decision making.

METHODS AND ANALYSIS

We will derive, internally and externally validate a multivariable model to predict 30-day mortality in EGS patients >65 years old. Our derivation sample will be individuals enrolled in the National Surgical Quality Improvement Program (NSQIP) database between 2012 and 2016 having 1 of 7 core EGS procedures. Postulated predictor variables have been identified based on previous research, clinical and epidemiological knowledge. Our model will be derived using logistic regression penalised with elastic net regularisation and ensembled using bootstrap aggregation. The resulting model will be internally validated using k-fold cross-validation and bootstrap validation techniques and externally validated using population-based health administrative data. Discrimination and calibration will be reported at each step.

ETHICS AND DISSEMINATION

Ethics for NSQIP data use was obtained from the Ottawa Hospital Research Ethics Board; external validation will use routinely collected anonymised data legally exempt from research ethics review. The final risk score will be published in a peer-reviewed journal. We plan to further disseminate the model as an online calculator or application for clinical use. Future research will be required to test the clinical application of the final model.

Reporting and Methods in Developing Prognostic Prediction Models for Metabolic Syndrome: A Systematic Review and Critical Appraisal

PURPOSE

A prognostic prediction model for metabolic syndrome can calculate the probability of risk of experiencing metabolic syndrome within a specific period for individualized treatment decisions. We aimed to provide a systematic review and critical appraisal on prognostic models for metabolic syndrome.

MATERIALS AND METHODS

Studies were identified through searching in English databases (PubMed, EMBASE, CINAHL, and Web of Science) and Chinese databases (Sinomed, WANFANG, CNKI, and CQVIP). A checklist for critical appraisal and data extraction for systematic reviews of prediction modeling studies (CHARMS) and the prediction model risk of bias assessment tool (PROBAST) were used for the data extraction process and critical appraisal.

RESULTS

From the 29,668 retrieved articles, eleven studies meeting the selection criteria were included in this review. Forty-eight predictors were identified from prognostic prediction models. The c-statistic ranged from 0.67 to 0.95. Critical appraisal has shown that all modeling studies were subject to a high risk of bias in methodological quality mainly driven by outcome and statistical analysis, and six modeling studies were subject to a high risk of bias in applicability.

CONCLUSION

Future model development and validation studies should adhere to the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) statement to improve methodological quality and applicability, thus increasing the transparency of the reporting of a prediction model study. It is not appropriate to adopt any of the identified models in this study for clinical practice since all models are prone to optimism and overfitting.

Pulse Wave Velocity and Machine Learning to Predict Cardiovascular Outcomes in Prediabetic and Diabetic Populations

Few studies have addressed the predictive value of arterial stiffness determined by pulse wave velocity (PWV) in a high-risk population with no prevalent cardiovascular disease and with obesity, hypertension, hyperglycemia, and preserved kidney function. This longitudinal, retrospective study enrolled 88 high-risk patients and had a follow-up time of 12.4 years. We collected clinical and laboratory data, as well as information on arterial stiffness parameters using arterial tonometry and measurements from ambulatory blood pressure monitoring. We considered nonfatal, incident cardiovascular events as the primary outcome. Given the small size of our dataset, we used survival analysis (i.e., Cox proportional hazards model) combined with a machine learning-based algorithm/penalization method to evaluate the data. Our predictive model, calculated with Cox regression and least absolute shrinkage and selection operator (LASSO), included body mass index, diabetes mellitus, gender (male), and PWV. We recorded 16 nonfatal cardiovascular events (5 myocardial infarctions, 5 episodes of heart failure, and 6 strokes). The adjusted hazard ratio for PWV was 1.199 (95% confidence interval: 1.09-1.37, p < 0.001). Arterial stiffness was a predictor of cardiovascular disease development, as determined by PWV in a high-risk population. Thus, in obese, hypertensive, hyperglycemic patients with preserved kidney function, PWV can serve as a prognostic factor for major adverse cardiac events.

Syndecan-1 Predicts Outcome in Patients with ST-Segment Elevation Infarction Independent from Infarct-related Myocardial Injury

Syndecan-1 (sdc1) is a surface protein part of the endothelial glycocalyx (eGC). Soluble sdc1 is derived from shedding and indicates damaged eGC. We assessed the predictive value of plasma sdc1 concentrations for future cardiovascular events in acute reperfused ST-segment elevation myocardial infarction (STEMI) patients. A total of 206 patients admitted for STEMI were included in this study (29% female; age 65 ± 12 years) and followed-up for six months. Plasma samples were obtained post-intervention and analyzed for sdc1 by Enzyme-linked Immunosorbent Assay (ELISA). Primary outcome was six-month-mortality. Sdc1 did not correlate with biomarkers such as creatine kinase (CK) (r = 0.11; p = 0.01) or troponin (r = −0.12; p = 0.09), nor with infarct size (r = −0.04; p = 0.67) and myocardial salvage index (r = 0.11; p = 0.17). Sdc-1 was associated with mortality (changes per 100 ng/mL sdc-1 concentration; HR 1.08 95% 1.03–1.12; p = 0.001). An optimal cut-off was calculated at >120 ng/mL. After correction for known risk factors sdc1 >120 ng/mL was independently associated with mortality after 6 months. In our study, sdc1 is independently associated with six-month-mortality after STEMI. Combining clinical evaluation and different biomarkers assessing both infarct-related myocardial injury and systemic stress response might improve the accuracy of predicting clinical prognosis in STEMI patients.

Development of an individualized risk calculator for poor functioning in young people victimized during childhood: A longitudinal cohort study

BACKGROUND

Childhood victimization elevates the average risk of developing functional impairment in adulthood. However, not all victimized children demonstrate poor outcomes. Although research has described factors that confer vulnerability or resilience, it is unknown if this knowledge can be translated to accurately identify the most vulnerable victimized children.

OBJECTIVE

To build and internally validate a risk calculator to identify those victimized children who are most at risk of functional impairment at age 18 years.

PARTICIPANTS

We utilized data from the Environmental Risk (E-Risk) Longitudinal Twin Study, a nationally-representative birth cohort of 2232 UK children born in 1994-95.

METHODS

Victimization exposure was assessed repeatedly between ages 5 and 12 years along with a range of individual-, family- and community-level predictors. Functional outcomes were assessed at age 18 years. We developed and evaluated a prediction model for psychosocial disadvantage and economic disadvantage using the Least Absolute Shrinkage and Selection Operator (LASSO) regularized regression with nested 10-fold cross-validation.

RESULTS

The model predicting psychosocial disadvantage following childhood victimization retained 12 of 22 predictors, had an area under the curve (AUC) of 0.65, and was well-calibrated within the range of 40-70% predicted risk. The model predicting economic disadvantage retained 10 of 22 predictors, achieved excellent discrimination (AUC = 0.80), and a high degree of calibration.

CONCLUSIONS

Prediction modelling techniques can be applied to estimate individual risk for poor functional outcomes in young adulthood following childhood victimization. Such risk prediction tools could potentially assist practitioners to target interventions, which is particularly useful in a context of scarce resources.

Acute kidney injury: a risk scoring system for general surgical patients

This article describes the development of a scoring system for general surgical patients to highlight those at greater risk of developing acute kidney injury (AKI). Following a search of the literature on current practice, a list of common variables was composed. Hospital Episode Statistics (HES) data from two random hospital trusts was used. With the help of a risk analysis system (CRAB Medical module, CRAB Clinical Informatics Ltd) it was possible to examine the relationship between potential risk factors and the incidence of AKI. Using Analyse-it for Excel a binary logistic model was created, which led to the development of a logistic regression equation and consequently a scoring system. The sensitivity and specificity of the model was tested using the receiver operating characteristic (ROC) curve. There was good correlation across the whole risk spectrum with an area under ROC curve of 0.806 (95% confidence intervals 0.787-0.825). The scoring system was developed into an admission checklist for general surgical patients to highlight a patient's risk of developing AKI. In a ward setting a checklist that immediately assesses the patient and produces a rapid indication as to whether the patient is at high risk or low risk would seem to be the ideal tool.

Assessing the risk of early unplanned rehospitalisation in preterm babies: EPIPAGE 2 study

Background

Gaining a better understanding of the probability, timing and prediction of rehospitalisation amongst preterm babies could help improve outcomes. There is limited research addressing these topics amongst extremely and very preterm babies. In this context, unplanned rehospitalisations constitute an important, potentially modifiable adverse event. We aimed to establish the probability, time-distribution and predictability of unplanned rehospitalisation within 30 days of discharge in a population of French preterm babies.

Methods

This study used data from EPIPAGE 2, a population-based prospective study of French preterm babies. Only those babies discharged home alive and whose parents responded to the one-year survey were eligible for inclusion in our study. For Kaplan-Meier analysis, the outcome was unplanned rehospitalisation censored at 30 days. For predictive modelling, the outcome was binary, recording unplanned rehospitalisation within 30 days of discharge. Predictors included routine clinical variables selected based on expert opinion.

Results

Of 3841 eligible babies, 350 (9.1, 95% CI 8.2–10.1) experienced an unplanned rehospitalisation within 30 days. The probability of rehospitalisation progressed at a consistent rate over the 30 days. There were significant differences in rehospitalisation probability by gestational age. The cross-validated performance of a ten predictor model demonstrated low discrimination and calibration. The area under the receiver operating characteristic curve was 0.62 (95% CI 0.59–0.65).

Conclusions

Unplanned rehospitalisation within 30 days of discharge was infrequent and the probability of rehospitalisation progressed at a consistent rate. Lower gestational age increased the probability of rehospitalisation. Predictive models comprised of clinically important variables had limited predictive ability.

Integrating proteomic, sociodemographic and clinical data to predict future depression diagnosis in subthreshold symptomatic individuals

Individuals with subthreshold depression have an increased risk of developing major depressive disorder (MDD). The aim of this study was to develop a prediction model to predict the probability of MDD onset in subthreshold individuals, based on their proteomic, sociodemographic and clinical data. To this end, we analysed 198 features (146 peptides representing 77 serum proteins (measured using MRM-MS), 22 sociodemographic factors and 30 clinical features) in 86 first-episode MDD patients (training set patient group), 37 subthreshold individuals who developed MDD within two or four years (extrapolation test set patient group), and 86 subthreshold individuals who did not develop MDD within four years (shared reference group). To ensure the development of a robust and reproducible model, we applied feature extraction and model averaging across a set of 100 models obtained from repeated application of group LASSO regression with ten-fold cross-validation on the training set. This resulted in a 12-feature prediction model consisting of six serum proteins (AACT, APOE, APOH, FETUA, HBA and PHLD), three sociodemographic factors (body mass index, childhood trauma and education level) and three depressive symptoms (sadness, fatigue and leaden paralysis). Importantly, the model demonstrated a fair performance in predicting future MDD diagnosis of subthreshold individuals in the extrapolation test set (AUC = 0.75), which involved going beyond the scope of the model. These findings suggest that it may be possible to detect disease indications in subthreshold individuals up to four years prior to diagnosis, which has important clinical implications regarding the identification and treatment of high-risk individuals.

Weight loss thresholds to detect early hypernatremia in newborns

OBJECTIVE

The literature indicates a single universal cut-off point for weight loss after birth for the risk of hypernatremia, without considering other factors. The aim of this study was to construct and internally validate cut-off points for the percentage weight loss associated with the risk of hypernatremia, taking into account risk factors.

METHODS

A prospective study with a three-day follow-up was conducted in 165 neonates with a gestational age ≥35 weeks. The main outcome variable was mild or moderate hypernatremia (serum sodium≥145mmol/L). Secondary variables (risk factors) were maternal and infant variables. A multivariate logistic regression model was constructed to predict hypernatremia, obtaining its probability and the optimal discriminant cut-off point for hypernatremia (receiver operating characteristic analysis). Based on this point, threshold weight loss values were obtained according to the other variables. These values were internally validated by bootstrapping.

RESULTS

There were 51 cases (30.9%) of hypernatremia. The mean percentage weight loss for hypernatremic infants was 8.6% and 6.0% for the rest. Associated variables in the multivariate model included greater weight loss, male gender, higher education level, multiparity, and cesarean delivery. The model had an area under the receiver operating characteristic curve of 0.84 (sensitivity=77.6%; specificity=73.2%). Similar values were obtained in the bootstrapping validation. The lowest percentage weight loss was 4.77%, for cesarean delivery in male infants of mothers with a higher education level.

CONCLUSIONS

The weight loss percentage values depended on the type of delivery, parity, newborn gender, and level of maternal education. External studies are required to validate these values.

Predicting verbal memory decline following temporal lobe resection for epilepsy

OBJECTIVES

The aim of the study was to develop a prediction model for verbal memory decline after temporal lobe resection (TLR) for epilepsy. The model will be used in the preoperative counselling of patients to give individualized information about risk for verbal memory decline.

MATERIALS AND METHODS

A sample of 110 consecutive patients who underwent TLR for epilepsy at Sahlgrenska University Hospital between 1987 and 2011 constituted the basis for the prediction model. They had all gone through a formal neuropsychological assessment before surgery and 2 years after. Penalized regression and 20 × 10-fold cross-validation were used in order to build a reliable model for predicting individual risks.

RESULTS

The final model included four predictors: side of surgery; inclusion or not of the hippocampus in the resection; preoperative verbal memory function; and presence/absence of focal to bilateral tonic-clonic seizures (TCS) the last year prior to the presurgical investigation. The impact of a history of TCS is a new finding which we interpret as a sign of a more widespread network disease which influences neuropsychological function and the cognitive reserve. The model correctly identified 82% of patients with post-operative decline in verbal memory, and the overall accuracy was 70%-85% depending on choice of risk thresholds.

CONCLUSIONS

The model makes it possible to provide patients with individualized prediction regarding the risk of verbal memory decline following TLR. This will help them make more informed decisions regarding treatment, and it will also enable the epilepsy surgery team to prepare them better for the rehabilitation process.

Prospects for predictive modeling of transition cow diseases

Transition cow diseases can negatively impact animal welfare and reduce dairy herd profitability. Transition cow disease incidence has remained relatively stable over time despite monitoring and management efforts aimed to reduce the risk of developing diseases. Dairy cattle disease risk is monitored by assessing multiple factors, including certain biomarker test results, health records, feed intake, body condition score, and milk production. However, these factors, which are used to make herd management decisions, are often reviewed separately without considering the correlation between them. In addition, the biomarkers that are currently used for monitoring may not be representative of the complex physiological changes that occur during the transition period. Predictive modeling, which uses data to predict future or current outcomes, is a method that can be used to combine the most predictive variables and their interactions efficiently. The use of an effective predictive model with relevant predictors for transition cow diseases will result in better targeted interventions, and therefore lower disease incidence. This review will discuss predictive modeling methods and candidate variables in the context of transition cow diseases. The next step is to investigate novel biomarkers and statistical methods that are best suited for the prediction of transition cow diseases.

Cocaine Positivity in ST-Elevation Myocardial Infarction: A True or False Association

INTRODUCTION

Every year, more than 500,000 US Emergency Department visits are associated with cocaine use. People who use cocaine tend to have a lower incidence of true ST-elevation myocardial infarction (STEMI).

OBJECTIVE

To identify the factors associated with true STEMI in patients with cocaine-positive (CPos) findings.

METHODS

We retrospectively analyzed 1144 consecutive patients with STEMI between 2008 and 2013. True STEMI was defined as having a culprit lesion on coronary angiogram. Multivariate and univariate analyses were used to identify risk factors and create a predictive model.

RESULTS

A total of 64 patients with suspected STEMI were CPos (mean age 53.1 ± 11.2 years; male = 80%). True STEMI was diagnosed in 34 patients. Patients with CPos true STEMI were more likely to be uninsured than those with false STEMI (61.8% vs 34.5%, p = 0.03) and have higher peak troponin levels (21.1 ng/mL vs 2.12 ng/mL, p = < 0.01) with no difference in mean age between the 2 groups (p = 0.24). In multivariate analyses, independent predictors of true STEMI in patients with CPos findings included age older than 65 years (odds ratio [OR] = 19.3, 95% confidence iterval [CI] = 1.2-318.3), lack of health insurance (OR = 4.9, 95% CI = 1.2-19.6), and troponin level higher than 0.05 (OR = 24.0, 95% CI = 2.6-216.8) (all p < 0.05). A multivariate risk score created with a C-statistic of 82% (95% CI = 71-93) significantly improved the identification of patients with true STEMI.

CONCLUSION

Among those with suspected STEMI, patients with CPos findings had a higher incidence of false STEMI. Older age, lack of health insurance, and troponin levels outside of defined limits were associated with true STEMI in this group.

Development and Validation of a Prediction Model for Organ-Specific Recurrences After Curative Resection of Colon Cancer

BACKGROUND

Early detection of postoperative recurrence is beneficial for patients with cancer; however, optimal surveillance remains an issue. To optimize the follow-up plan, the estimation of an individual patient's risk of recurrence is indispensable.

OBJECTIVE

This study aimed to establish a statistical model for predicting the risk of organ-specific recurrence after curative resection of colon cancer.

DESIGN

This was a retrospective cohort study at a tertiary referral hospital.

SETTINGS

This study included 1720 patients with colon cancer treated at the University of Tokyo Hospital between 1997 and 2015. Data were retrospectively retrieved from patient medical charts. The risk score was developed using a competing risk model in a derivation cohort (973 patients treated in 1997-2009) and then validated in a validation cohort (747 patients treated in 2010-2015).

PATIENTS

Patients who underwent curative resection for stage I to III colon cancer were included.

MAIN OUTCOME MEASURES

The prediction of the incidence of postoperative liver and lung metastasis of colon cancer was measured.

RESULTS

The factors selected for the prediction model for liver metastasis included differentiation, T category, venous invasion, N category, and preoperative CEA level. The model for lung metastasis included sex, lymphatic invasion, venous invasion, N category, preoperative CEA level, and malignant bowel obstruction. During external validation, the area under the curve at 60 months was 0.78 (95% CI, 0.71-0.84) for liver metastasis and 0.72 (95% CI, 0.64-0.81) for lung metastasis.

LIMITATIONS

The generalizability of the model to different healthcare settings remains to be elucidated.

CONCLUSIONS

We developed a prediction model to estimate the risk of recurrence in the liver and lung after curative resection of colon cancer, which demonstrated good discrimination ability in the external validation cohort. Our model can aid clinicians and patients in customizing postoperative surveillance according to an individual patient's risk of organ-specific recurrence. See Video Abstract at http://links.lww.com/DCR/A977. DESARROLLO Y VALIDACIÓN DE UN MODELO DE PREDICCIÓN PARA RECURRENCIAS ESPECÍFICAS DESPUÉS DE RESECCIÓN CURATIVA DE UN CÁNCER DE COLON: La detección temprana de una recidiva postoperatoria es beneficiosa para los pacientes afectados de cáncer. Sin embargo, la mejor vigilancia sigue siendo un problema. Para optimizar el plan de seguimiento, la estimación del riesgo individual de recurrencia de un paciente es indispensable.

OBJETIVO

Establecer un modelo estadístico para predecir el riesgo de recurrencia en un organo específico luego de la resección curativa de un cáncer de colon. DISEÑO:: Estudio retrospectivo de cohortes en un hospital de referencia terciaria.

AJUSTES

Este estudio incluyó 1720 pacientes con cáncer de colon tratados en el Hospital de la Universidad de Tokio entre 1997 y 2015. Los datos se recuperaron retrospectivamente de las historias clinicas de los pacientes. La puntuación de riesgo fué desarrollada utilizando un modelo de riesgo competitivo en cohortes de derivación (973 pacientes tratados en 1997-2009) y luego se lo validó en cohortes de validación (747 pacientes tratados en 2010-2015).

PACIENTES

Todos aquellos casos que se sometieron a una resección curativa de cáncer de colon en estadio I-III RESULTADOS PRINCIPLES:: La predicción de la incidencia de metástasis hepáticas y pulmonares postoperatorias del cáncer de colon.

RESULTADOS

Los factores seleccionados para el modelo de predicción de metástasis hepáticas incluyeron diferenciación tumoral, categoría T, invasión venosa, categoría N y nivel de antígeno carcinoembrionario preoperatorio. El modelo de predicción de metástasis pulmonar incluyó el sexo del paciente, la invasión linfática, la invasión venosa, la categoría N, el nivel de antígeno carcinoembrionario preoperatorio y la obstrucción intestinal maligna. Durante la validación externa, el área inferior de la curva a 60 meses fue de 0,78 (intervalo de confianza del 95%: 0,71 a 0,84) para las metástasis hepáticas y de 0,72 (intervalo de confianza del 95%: 0,64 a 0,81) para las metástasis pulmonares.

LIMITACIONES

La generalización del presente modelo a diferentes entornos de atención en salud aún no ha podido ser dilucidado.

CONCLUSIONES

Desarrollamos un modelo de predicción para estimar el riesgo de recurrencia en el hígado y el pulmón después de resección curativa de cáncer de colon, éste modelo demostró una buena capacidad de discriminación en las cohortes de validación externa. El modelo puede ayudar a médicos y pacientes a personalizar la vigilancia postoperatoria de acuerdo con el riesgo individual de recurrencia específica en un órgano específico. Vea el Resumen del Video en http://links.lww.com/DCR/A977.

Prediction of significant prostate cancer in biopsy-naïve men: Validation of a novel risk model combining MRI and clinical parameters and comparison to an ERSPC risk calculator and PI-RADS

BACKGROUND

Risk models (RM) need external validation to assess their value beyond the setting in which they were developed. We validated a RM combining mpMRI and clinical parameters for the probability of harboring significant prostate cancer (sPC, Gleason Score ≥ 3+4) for biopsy-naïve men.

MATERIAL AND METHODS

The original RM was based on data of 670 biopsy-naïve men from Heidelberg University Hospital who underwent mpMRI with PI-RADS scoring prior to MRI/TRUS-fusion biopsy 2012-2015. Validity was tested by a consecutive cohort of biopsy-naïve men from Heidelberg (n = 160) and externally by a cohort of 133 men from University College London Hospital (UCLH). Assessment of validity was performed at fusion-biopsy by calibration plots, receiver operating characteristics curve and decision curve analyses. The RM`s performance was compared to ERSPC-RC3, ERSPC-RC3+PI-RADSv1.0 and PI-RADSv1.0 alone.

RESULTS

SPC was detected in 76 men (48%) at Heidelberg and 38 men (29%) at UCLH. The areas under the curve (AUC) were 0.86 for the RM in both cohorts. For ERSPC-RC3+PI-RADSv1.0 the AUC was 0.84 in Heidelberg and 0.82 at UCLH, for ERSPC-RC3 0.76 at Heidelberg and 0.77 at UCLH and for PI-RADSv1.0 0.79 in Heidelberg and 0.82 at UCLH. Calibration curves suggest that prevalence of sPC needs to be adjusted to local circumstances, as the RM overestimated the risk of harboring sPC in the UCLH cohort. After prevalence-adjustment with respect to the prevalence underlying ERSPC-RC3 to ensure a generalizable comparison, not only between the Heidelberg and die UCLH subgroup, the RM`s Net benefit was superior over the ERSPC`s and the mpMRI`s for threshold probabilities above 0.1 in both cohorts.

CONCLUSIONS

The RM discriminated well between men with and without sPC at initial MRI-targeted biopsy but overestimated the sPC-risk at UCLH. Taking prevalence into account, the model demonstrated benefit compared with clinical risk calculators and PI-RADSv1.0 in making the decision to biopsy men at suspicion of PC. However, prevalence differences must be taken into account when using or validating the presented risk model.