Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement

Human-guided deep learning with ante-hoc explainability by convolutional network from non-image data for pregnancy prognostication

BACKGROUND AND OBJECTIVE

Deep learning is applied in medicine mostly due to its state-of-the-art performance for diagnostic imaging. Supervisory authorities also require the model to be explainable, but most explain the model after development (post hoc) instead of incorporating explanation into the design (ante hoc). This study aimed to demonstrate a human-guided deep learning with ante-hoc explainability by convolutional network from non-image data to develop, validate, and deploy a prognostic prediction model for PROM and an estimator of time of delivery using a nationwide health insurance database.

METHODS

To guide modeling, we constructed and verified association diagrams respectively from literatures and electronic health records. Non-image data were transformed into meaningful images utilizing predictor-to-predictor similarities, harnessing the power of convolutional neural network mostly used for diagnostic imaging. The network architecture was also inferred from the similarities.

RESULTS

This resulted the best model for prelabor rupture of membranes (n=883, 376) with the area under curves 0.73 (95% CI 0.72 to 0.75) and 0.70 (95% CI 0.69 to 0.71) respectively by internal and external validations, and outperformed previous models found by systematic review. It was explainable by knowledge-based diagrams and model representation.

CONCLUSIONS

This allows prognostication with actionable insights for preventive medicine.

Radiomics analysis of contrast-enhanced T1W MRI: predicting the recurrence of acute pancreatitis

To investigate the predictive value of radiomics based on T1-weighted contrast-enhanced MRI (CE-MRI) in forecasting the recurrence of acute pancreatitis (AP). A total of 201 patients with first-episode of acute pancreatitis were enrolled retrospectively (140 in the training cohort and 61 in the testing cohort), with 69 and 30 patients who experienced recurrence in each cohort, respectively. Quantitative image feature extraction was obtained from MR contrast-enhanced late arterial-phase images. The optimal radiomics features retained after dimensionality reduction were used to construct the radiomics model through logistic regression analysis, and the clinical characteristics were collected to construct the clinical model. The nomogram model was established by linearly integrating the clinically independent risk factor with the optimal radiomics signature. The five best radiomics features were determined by dimensionality reduction. The radiomics model had a higher area under the receiver operating characteristic curve (AUC) than the clinical model for estimating the recurrence of acute pancreatitis for both the training cohort (0.915 vs. 0.811, p = 0.020) and testing cohort (0.917 vs. 0.681, p = 0.002). The nomogram model showed good performance, with an AUC of 0.943 in the training cohort and 0.906 in the testing cohort. The radiomics model based on CE-MRI showed good performance for optimizing the individualized prediction of recurrent acute pancreatitis, which provides a reference for the prevention and treatment of recurrent pancreatitis.

Development of a Clinical Prediction Model for Diabetes in Chronic Pancreatitis: The PREDICT3c Study

OBJECTIVE

Diabetes that arises from chronic pancreatitis (CP) is associated with increased morbidity and mortality. Methods to predict which patients with CP are at greatest risk for diabetes are urgently needed. We aimed to examine independent risk factors for diabetes in a large cohort of patients with CP.

RESEARCH DESIGN AND METHODS

This cross-sectional study comprised 645 individuals with CP enrolled in the PROCEED study, of whom 276 had diabetes. We conducted univariable and multivariable regression analyses of potential risk factors for diabetes. Model performance was assessed by area under the receiver operating characteristic curve (AUROC) analysis, and accuracy was evaluated by cross validation. Exploratory analyses were stratified according to the timing of development of diabetes relative to the diagnosis of pancreatitis.

RESULTS

Independent correlates of diabetes in CP included risk factors for type 2 diabetes (older age, overweight/obese status, male sex, non-White race, tobacco use) as well as pancreatic disease-related factors (history of acute pancreatitis complications, nonalcoholic etiology of CP, exocrine pancreatic dysfunction, pancreatic calcification, pancreatic atrophy) (AUROC 0.745). Type 2 diabetes risk factors were predominant for diabetes occurring before pancreatitis, and pancreatic disease-related factors were predominant for diabetes occurring after pancreatitis.

CONCLUSIONS

Multiple factors are associated with diabetes in CP, including canonical risk factors for type 2 diabetes and features associated with pancreatitis severity. This study lays the groundwork for the future development of models integrating clinical and nonclinical data to identify patients with CP at risk for diabetes and identifies modifiable risk factors (obesity, smoking) on which to focus for diabetes prevention.

A machine learning approach applied to gynecological ultrasound to predict progression-free survival in ovarian cancer patients

In a growing number of social and clinical scenarios, machine learning (ML) is emerging as a promising tool for implementing complex multi-parametric decision-making algorithms. Regarding ovarian cancer (OC), despite the standardization of features that can support the discrimination of ovarian masses into benign and malignant, there is a lack of accurate predictive modeling based on ultrasound (US) examination for progression-free survival (PFS). This retrospective observational study analyzed patients with epithelial ovarian cancer (EOC) who were followed in a tertiary center from 2018 to 2019. Demographic features, clinical characteristics, information about the surgery and post-surgery histopathology were collected. Additionally, we recorded data about US examinations according to the International Ovarian Tumor Analysis (IOTA) classification. Our study aimed to realize a tool to predict 12 month PFS in patients with OC based on a ML algorithm applied to gynecological ultrasound assessment. Proper feature selection was used to determine an attribute core set. Three different machine learning algorithms, namely Logistic Regression (LR), Random Forest (RFF), and K-nearest neighbors (KNN), were then trained and validated with five-fold cross-validation to predict 12 month PFS. Our analysis included n. 64 patients and 12 month PFS was achieved by 46/64 patients (71.9%). The attribute core set used to train machine learning algorithms included age, menopause, CA-125 value, histotype, FIGO stage and US characteristics, such as major lesion diameter, side, echogenicity, color score, major solid component diameter, presence of carcinosis. RFF showed the best performance (accuracy 93.7%, precision 90%, recall 90%, area under receiver operating characteristic curve (AUROC) 0.92). We developed an accurate ML model to predict 12 month PFS.

Development and Validation of a Machine Learning Approach Leveraging Real-World Clinical Narratives as a Predictor of Survival in Advanced Cancer

PURPOSE

Predicting short-term mortality in patients with advanced cancer remains challenging. Whether digitalized clinical text can be used to build models to enhance survival prediction in this population is unclear.

MATERIALS AND METHODS

We conducted a single-centered retrospective cohort study in patients with advanced solid tumors. Clinical correspondence authored by oncologists at the first patient encounter was extracted from the electronic medical records. Machine learning (ML) models were trained using narratives from the derivation cohort, before being tested on a temporal validation cohort at the same site. Performance was benchmarked against Eastern Cooperative Oncology Group performance status (PS), comparing ML models alone (comparison 1) or in combination with PS (comparison 2), assessed by areas under receiver operating characteristic curves (AUCs) for predicting vital status at 11 time points from 2 to 52 weeks.

RESULTS

ML models were built on the derivation cohort (4,791 patients from 2001 to April 2017) and tested on the validation cohort of 726 patients (May 2017-June 2019). In 441 patients (61%) where clinical narratives were available and PS was documented, ML models outperformed the predictivity of PS (mean AUC improvement, 0.039, P < .001, comparison 1). Inclusion of both clinical text and PS in ML models resulted in further improvement in prediction accuracy over PS with a mean AUC improvement of 0.050 ( P < .001, comparison 2); the AUC was > 0.80 at all assessed time points for models incorporating clinical text. Exploratory analysis of oncologist's narratives revealed recurring descriptors correlating with survival, including referral patterns, mobility, physical functions, and concomitant medications.

CONCLUSION

Applying ML to oncologists' narratives with or without including patient's PS significantly improved survival prediction to 12 months, suggesting the utility of clinical text in building prognostic support tools.

Prediction models of vaginal birth after cesarean delivery: A systematic review

BACKGROUND

Cesarean section rates are rising in the world. Women with a history of cesarean section will select a cesarean section at the next pregnancy. An objective and accurate prediction about the success rate of vaginal delivery after cesarean section can help women to reduce the complications caused by cesarean section, shorten the time spent in the hospital, and effectively plan medical resources.

OBJECTIVE

To systematically review and critically assess the existing prediction models of vaginal delivery after cesarean section.

METHODS

Some databases (PubMed, Web of Science, EMBASE, the Cochrane Library, Cumulative Index to Nursing and Allied Health Literature) were searched from 2000 to 2021 for studies regarding the prediction model of vaginal birth after cesarean delivery. The researchers successively conducted independent literature screening, data extraction and quality evaluation of the included literature, and then utilized the Prediction model Risk of Bias Assessment Tool to assess the methodological quality of the models in the included studies.

RESULTS

A total of 33 studies were included, in which 20 prediction models were identified. Sixteen studies involved external validation of existing models (Grobman's models). In the 20 prediction models, 12 were internally validated, only three had external validation, and seven models were not explicitly reported, with the area under the curve ranging from 0.660 to 0.953; The most common predictors included in the model were body mass index and previous vaginal delivery, followed by maternal age, previous cesarean delivery indication, history of vaginal birth after cesarean, fetal weight, and Bishop's score, gestational age, history of vaginal birth after cesarean, maternal race; The prediction effect of Grobman's model was validated in multiple external populations; The majority of the studies(n = 27) had high risk of bias in the of the Prediction model Risk of Bias Assessment Tool.

CONCLUSIONS

This review provides obstetricians and midwives with important information about the prediction models of vaginal birth after cesarean section, which has been reported optimistic predictive performance and acceptable predictive power. However, the majority of the development studies have methodological limitations, which may hinder the widely application of these models by obstetricians. Further studies are supposed to develop predictive models with low risk of bias, and conduct internal and external validation, providing pragmatic and practical predictions to obstetricians.

PROSPERO REGISTRATION NUMBER

CRD42022299048.

Predicting outcome in acute respiratory admissions using patterns of National Early Warning Scores

AIMS

Accurately predicting risk of patient deterioration is vital. Altered physiology in chronic disease affects the prognostic ability of vital signs based early warning score systems. We aimed to assess the potential of early warning score patterns to improve outcome prediction in patients with respiratory disease.

METHODS

Patients admitted under respiratory medicine between April 2015 and March 2017 had their National Early Warning Score 2 (NEWS2) calculated retrospectively from vital sign observations. Prediction models (including temporal patterns) were constructed and assessed for ability to predict death within 24 hours using all observations collected not meeting exclusion criteria. The best performing model was tested on a validation cohort of admissions from April 2017 to March 2019.

RESULTS

The derivation cohort comprised 7,487 admissions and the validation cohort included 8,739 admissions. Adding the maximum score in the preceding 24 hours to the most recently recorded NEWS2 improved area under the receiver operating characteristic curve for death in 24 hours from 0.888 (95% confidence interval (CI) 0.881-0.895) to 0.902 (95% CI 0.895-0.909) in the overall respiratory population.

CONCLUSION

Combining the most recently recorded score and the maximum NEWS2 score from the preceding 24 hours demonstrated greater accuracy than using snapshot NEWS2. This simple inclusion of a scoring pattern should be considered in future iterations of early warning scoring systems.

Development and Validation of a Prognostic Nomogram to Predict 30-Day Mortality Risk in Patients with Sepsis-Induced Cardiorenal Syndrome

INTRODUCTION

Sepsis-induced cardiorenal syndrome (sepsis-induced CRS) is a devastating medical condition that is frequently associated with a high fatality rate. In this study, we aimed to develop an individualized nomogram that may help clinicians assess 30-day mortality risk in patients diagnosed with sepsis-induced CRS.

METHODS

A total of 340 patients with sepsis-induced CRS admitted from January 2015 to May 2019 in Shanghai Tongji Hospital were used as a training cohort to develop a nomogram prognostic model. The model was constructed using multivariable logistic analyses and was then externally validated by an independent cohort of 103 patients diagnosed with sepsis-induced CRS from June 2019 to December 2020. The prognostic ability of the nomogram was assessed through discrimination, calibration, and accuracy.

RESULTS

Five prognostic factors were determined and included in the nomogram: age, Sequential (sepsis-related) Organ Failure Assessment (SOFA) score, vasopressors, baseline serum creatinine, and the rate of change in myoglobin. Our prognostic nomogram showed well-fitted calibration curves and yielded strong discrimination power with the area under the curve of 0.879 and 0.912 in model development and validation, respectively. In addition, the nomogram prognostic model exhibited an evidently higher predictive accuracy than the SOFA score.

CONCLUSIONS

We developed a prognostic nomogram model for patients with sepsis-induced CRS and externally validated the model in another independent cohort. The nomogram exhibited greater strength in predicting 30-day mortality risk than the SOFA score, which may help clinicians estimate short-term prognosis and modulate therapeutic strategies.

Artificial Neural Networks Can Predict Early Failure of Cementless Total Hip Arthroplasty in Patients With Osteoporosis

BACKGROUND

Total hip arthroplasty (THA) done in the aging population is associated with osteoporosis-related complications. The altered bone density in osteoporotic patients is a risk factor for revision surgery. This study aimed to develop and validate machine learning (ML) models to predict revision surgery in patients with osteoporosis after primary noncemented THA.

METHODS

We retrospectively reviewed a consecutive series of 350 patients with osteoporosis (T-score less than or equal to -2.5) who underwent primary noncemented THA at a tertiary referral center. All patients had a minimum 2-year follow-up (range: 2.1 to 5.6). Four ML algorithms were developed to predict the probability of revision surgery, and these were assessed by discrimination, calibration, and decision curve analysis.

RESULTS

The overall incidence of revision surgery was 5.2% at a mean follow-up of 3.7 years after primary noncemented THA in osteoporotic patients. Revision THA was done because of periprosthetic fracture in nine patients (50%), aseptic loosening/subsidence in five patients (28%), periprosthetic joint infection in two patients (11%) and dislocation in two patients (11%). The strongest predictors for revision surgery in patients after primary noncemented THA were female sex, BMI (>35 kg/m2), age (>70 years), American Society of Anesthesiology score (≥3), and T-score. All four ML models demonstrated good model performance across discrimination (AUC range: 0.78 to 0.81), calibration, and decision curve analysis.

CONCLUSION

The ML models presented in this study demonstrated high accuracy for the prediction of revision surgery in osteoporotic patients after primary noncemented THA. The presented ML models have the potential to be used by orthopaedic surgeons for preoperative patient counseling and optimization to improve the outcomes of primary noncemented THA in osteoporotic patients.

Preoperative Prediction of G1 and G2/3 Grades in Patients With Nonfunctional Pancreatic Neuroendocrine Tumors Using Multimodality Imaging

OBJECTIVES

We aimed to develop and validate a multimodality radiomics model for the preoperative prediction of nonfunctional pancreatic neuroendocrine tumor (NF-pNET) grade (G).

METHODS

This retrospective study assessed 123 patients with surgically resected, pathologically confirmed NF-pNETs who underwent multidetector computed tomography and MRI scans between December 2012 and May 2020. Radiomic features were extracted from multidetector computed tomography and MRI. Wilcoxon rank-sum test and Max-Relevance and Min-Redundancy tests were used to select the features. The linear discriminative analysis (LDA) was used to construct the four models including a clinical model, MRI radiomics model, computed tomography radiomics model, and mixed radiomics model. The performance of the models was assessed using a training cohort (82 patients) and a validation cohort (41 patients), and decision curve analysis was applied for clinical use.

RESULTS

We successfully constructed 4 models to predict the tumor grade of NF- pNETs. Model 4 combined 6 features of T2-weighted imaging radiomics features and 1 arterial-phase computed tomography radiomics feature, and showed better discrimination in the training cohort (AUC = 0.92) and validation cohort (AUC = 0.85) relative to the other models. In the decision curves, if the threshold probability was 0.07-0.87, the use of the radiomics score to distinguish NF-pNET G1 and G2/3 offered more benefit than did the use of a "treat all patients" or a "treat none" scheme in the training cohort of the MRI radiomics model.

CONCLUSION

The LDA classifier combining multimodality images may be a valuable noninvasive tool for distinguishing NF-pNET grades and avoid unnecessary surgery.

Differentiation of Solid Pseudopapillary Tumor and Non-Functional Neuroendocrine Tumors of the Pancreas Based on CT Delayed Imaging: A Propensity Score Analysis

PURPOSE

To evaluate the diagnostic performance of the delayed-phase difference between tumor and pancreas for differentiating solid pseudopapillary tumors (SPTs) from non-functional neuroendocrine tumors (NF-NETs) of the pancreas.

METHODS

This retrospective review included 148 consecutive patients with SPT and 98 consecutive patients with NF-NET confirmed by pathology. Patients with SPT and NF-NET were matched via propensity score matching (PSM). All patients underwent multidetector computed tomography (MDCT). For each patient, the delayed-phase difference between the tumor and pancreas was measured, and the performance of this variable was assessed based on its discriminative ability and clinical utility.

RESULTS

After PSM, 27 patients with SPT and 27 patients with NF-NET were included in the matched analysis. There were no statistically significant differences in clinical and CT characteristics between the resulting two groups (p > 0.05). The delayed-phase difference values between the tumor and pancreas were significantly lower in patients with SPT (median: -0.45; range: -2.05 to 0.73) than in patients with NF-NET (median: 0.71; range: -1.39 to 2.38). The delayed-phase difference between tumor and pancreas had a high diagnostic accuracy (area under the curve=0.88). The best cutoff point based on maximizing the sum of the sensitivity and specificity was -0.23 (sensitivity = 88.89%; specificity = 88.89%; accuracy = 0.89).

CONCLUSIONS

The delayed-phase difference between tumor and pancreas can accurately and noninvasively differentiate SPT from NF-NET.

A Machine Learning Approach for Predicting Sustained Remission in Rheumatoid Arthritis Patients on Biologic Agents

METHODS

In this longitudinal study, patients with RA who started a biological disease-modifying antirheumatic drug (bDMARD) in a tertiary care center were analyzed. Demographic and clinical characteristics were collected at treatment baseline, 12-month, and 24-month follow-up. A wrapper feature selection algorithm was used to determine an attribute core set. Four different ML algorithms, namely, LR, random forest, K-nearest neighbors, and extreme gradient boosting, were then trained and validated with 10-fold cross-validation to predict 24-month sustained DAS28 (Disease Activity Score on 28 joints) remission. The performances of the algorithms were then compared assessing accuracy, precision, and recall.

RESULTS

Our analysis included 367 patients (female 323/367, 88%) with mean age ± SD of 53.7 ± 12.5 years at bDMARD baseline. Sustained DAS28 remission was achieved by 175 (47.2%) of 367 patients. The attribute core set used to train algorithms included acute phase reactant levels, Clinical Disease Activity Index, Health Assessment Questionnaire-Disability Index, as well as several clinical characteristics. Extreme gradient boosting showed the best performance (accuracy, 72.7%; precision, 73.2%; recall, 68.1%), outperforming random forest (accuracy, 65.9%; precision, 65.6%; recall, 59.3%), LR (accuracy, 64.9%; precision, 62.6%; recall, 61.9%), and K-nearest neighbors (accuracy, 63%; precision, 61.5%; recall, 54.8%).

CONCLUSIONS

We showed that ML models can be used to predict sustained remission in RA patients on bDMARDs. Furthermore, our method only relies on a few easy-to-collect patient attributes. Our results are promising but need to be tested on longitudinal cohort studies.

Mastering Prognostic Tools: An Opportunity to Enhance Personalized Care and to Optimize Clinical Outcomes in Physical Therapy

In health care, clinical decision making is typically based on diagnostic findings. Rehabilitation clinicians commonly rely on pathoanatomical diagnoses to guide treatment and define prognosis. Targeting prognostic factors is a promising way for rehabilitation clinicians to enhance treatment decision-making processes, personalize rehabilitation approaches, and ultimately improve patient outcomes. This can be achieved by using prognostic tools that provide accurate estimates of the probability of future outcomes for a patient in clinical practice. Most literature reviews of prognostic tools in rehabilitation have focused on prescriptive clinical prediction rules. These studies highlight notable methodological issues and conclude that these tools are neither valid nor useful for clinical practice. This has raised the need to open the scope of research to understand what makes a quality prognostic tool that can be used in clinical practice. Methodological guidance in prognosis research has emerged in the last decade, encompassing exploratory studies on the development of prognosis and prognostic models. Methodological rigor is essential to develop prognostic tools, because only prognostic models developed and validated through a rigorous methodological process should guide clinical decision making. This Perspective argues that rehabilitation clinicians need to master the identification and use of prognostic tools to enhance their capacity to provide personalized rehabilitation. It is time for prognosis research to look for prognostic models that were developed and validated following a comprehensive process before being simplified into suitable tools for clinical practice. New models, or rigorous validation of current models, are needed. The approach discussed in this Perspective offers a promising way to overcome the limitations of most models and provide clinicians with quality tools for personalized rehabilitation approaches.

IMPACT

Prognostic research can be applied to clinical rehabilitation; this Perspective proposes solutions to develop high-quality prognostic models to optimize patient outcomes.

External Validation of a Neural Network Model in Aneurysmal Subarachnoid Hemorrhage: A Comparison With Conventional Logistic Regression Models

BACKGROUND

Interest in machine learning (ML)-based predictive modeling has led to the development of models predicting outcomes after aneurysmal subarachnoid hemorrhage (aSAH), including the Nijmegen acute subarachnoid hemorrhage calculator (Nutshell). Generalizability of such models to external data remains unclear.

OBJECTIVE

To externally validate the performance of the Nutshell tool while comparing it with the conventional Subarachnoid Hemorrhage International Trialists (SAHIT) models and to review the ML literature on outcome prediction after aSAH and aneurysm treatment.

METHODS

A prospectively maintained database of patients with aSAH presenting consecutively to our institution in the 2013 to 2018 period was used. The web-based Nutshell and SAHIT calculators were used to derive the risks of poor long-term (12-18 months) outcomes and 30-day mortality. Discrimination was evaluated using the area under the curve (AUC), and calibration was investigated using calibration plots. The literature on relevant ML models was surveyed for a synopsis.

RESULTS

In 269 patients with aSAH, the SAHIT models outperformed the Nutshell tool (AUC: 0.786 vs 0.689, P = .025) in predicting long-term functional outcomes. A logistic regression model of the Nutshell variables derived from our data achieved adequate discrimination (AUC = 0.759) of poor outcomes. The SAHIT models outperformed the Nutshell tool in predicting 30-day mortality (AUC: 0.810 vs 0.636, P < .001). Calibration properties were more favorable for the SAHIT models. Most published aneurysm-related ML-based outcome models lack external validation and usable testing platforms.

CONCLUSION

The Nutshell tool demonstrated limited performance on external validation in comparison with the SAHIT models. External validation and the dissemination of testing platforms for ML models must be emphasized.

Radiomics nomogram for the preoperative prediction of lymph node metastasis in pancreatic ductal adenocarcinoma

PURPOSE

To develop and validate a radiomics nomogram for the preoperative prediction of lymph node (LN) metastasis in pancreatic ductal adenocarcinoma (PDAC).

MATERIALS AND METHODS

In this retrospective study, 225 patients with surgically resected, pathologically confirmed PDAC underwent multislice computed tomography (MSCT) between January 2014 and January 2017. Radiomics features were extracted from arterial CT scans. The least absolute shrinkage and selection operator method was used to select the features. Multivariable logistic regression analysis was used to develop the predictive model, and a radiomics nomogram was built and internally validated in 45 consecutive patients with PDAC between February 2017 and December 2017. The performance of the nomogram was assessed in the training and validation cohort. Finally, the clinical usefulness of the nomogram was estimated using decision curve analysis (DCA).

RESULTS

The radiomics signature, which consisted of 13 selected features of the arterial phase, was significantly associated with LN status (p < 0.05) in both the training and validation cohorts. The multivariable logistic regression model included the radiomics signature and CT-reported LN status. The individualized prediction nomogram showed good discrimination in the training cohort [area under the curve (AUC), 0.75; 95% confidence interval (CI), 0.68-0.82] and in the validation cohort (AUC, 0.81; 95% CI, 0.69-0.94) and good calibration. DCA demonstrated that the radiomics nomogram was clinically useful.

CONCLUSIONS

The presented radiomics nomogram that incorporates the radiomics signature and CT-reported LN status is a noninvasive, preoperative prediction tool with favorable predictive accuracy for LN metastasis in patients with PDAC.

Development and Validation of a Mortality Prediction Model in Extremely Low Gestational Age Neonates

INTRODUCTION

Understanding factors that associate with neonatal death may lead to strategies or interventions that can aid clinicians and inform families.

OBJECTIVE

The aim of the study was to develop an early prediction model of neonatal death in extremely low gestational age (ELGA, <28 weeks) neonates.

METHODS

A predictive cohort study of ELGA neonates was derived from the Swedish Neonatal Quality Register between the years 2011 to May 2021. The goal was to use readily available clinical variables, collected within the first hour of birth, to predict in-hospital death. Data were split into a train cohort (80%) to build the model and tested in 20% of randomly selected neonates. Model performance was assessed via area under the receiver operating characteristic curve (AUC) and compared to validated mortality prediction models and an external cohort of neonates.

RESULTS

Among 3,752 live-born extremely preterm infants (46% girls), in-hospital mortality was 18% (n = 685). The median gestational age and birth weight were 25.0 weeks (interquartile range [IQR] 24.0, 27.0) and 780 g (IQR 620, 940), respectively. The proposed model consisted of three variables: birth weight (grams), Apgar score at 5 min of age, and gestational age (weeks). The BAG model had an AUC of 76.9% with a 95% confidence interval (CI) (72.6%, 81.3%), while birth weight and gestational age had an AUC of 73.1% (95% CI: 68.4%,77.9%) and 71.3% (66.3%, 76.2%). In the validation cohort, the BAG model had an AUC of 68.9%.

CONCLUSION

The BAG model is a new mortality prediction model in ELGA neonates that was developed using readily available information.

Contrast-enhanced computed tomography radiomics and multilayer perceptron network classifier: an approach for predicting CD20+ B cells in patients with pancreatic ductal adenocarcinoma

PURPOSE

To develop and validate a machine-learning classifier based on contrast-enhanced computed tomography (CT) for the preoperative prediction of CD20⁺ B lymphocyte expression in patients with pancreatic ductal adenocarcinoma (PDAC).

METHODS

Overall, 189 patients with PDAC (n = 132 and n = 57 in the training and validation sets, respectively) underwent immunohistochemistry and radiomics feature extraction. The X-tile software was used to stratify them into groups with 'high' and 'low' CD20⁺ B lymphocyte expression levels. For each patient, 1409 radiomic features were extracted from volumes of interest and reduced using variance analysis and Spearman correlation analysis. A multilayer perceptron (MLP) network classifier was developed using the training and validation set. Model performance was determined by its discriminative ability, calibration, and clinical utility.

RESULTS

A log-rank test showed that the patients with high CD20⁺ B expression had significantly longer survival than those with low CD20⁺ B expression. The prediction model showed good discrimination in both the training and validation sets. For the training set, the area under the curve (AUC), sensitivity, specificity, accuracy, positive predictive value, and negative predictive value were 0.82 (95% CI 0.74-0.89), 92.42%, 57.58%, 0.75, 0.69, and 0.88, respectively; whereas these values for the validation set were 0.84 (95% CI 0.72-0.93), 86.21%, 78.57%, 0.83, 0.81, and 0.85, respectively.

CONCLUSION

The MLP network classifier based on contrast-enhanced CT can accurately predict CD20⁺ B expression in patients with PDAC.

Dynamic Nomogram for Predicting Thrombocytopenia in Adults with Acute Pancreatitis

Objective

Thrombocytopenia increases the risk of hemorrhage in patients with acute pancreatitis (AP), leading to poor clinical outcomes. Currently, there is no reliable tool for the early assessment of thrombocytopenia in these patients. We aimed to develop a nomogram based on available clinical parameters and validate its efficacy in predicting thrombocytopenia.

Methods

This was a retrospective study. All the data were extracted from an electronic database from May 2018 to May 2019. Patients with a diagnosis of AP and staying in the intensive care unit for more than 3 days were retrospectively analyzed. A clinical signature was built based on reproducible features, using the least absolute shrinkage and selection operator method (LASSO), and logistic regression established the model (P < 0.05). Nomogram performance was determined by its discrimination, calibration, and clinical usefulness.

Results

A total of 594 eligible patients were enrolled, of whom 399 were allocated to the training sets and the 195 in the test sets. The clinical features, including blood urea nitrogen (BUN), fibrinogen (FIB), and antithrombase III, were significantly associated with the incidence of thrombocytopenia after acute pancreatitis (p < 0.05) in training sets. The individualized nomogram showed good discrimination in the training sample (area under the receiver operating characteristic curve [AUC], 0.881) and in the validation sample (AUC, 0.883) with good calibration.

Conclusion

The proposed nomogram has good performance for predicting thrombocytopenia in patients with acute pancreatitis and may facilitate clinical decision-making.

Parental risk factors associated with congenital heart disease in a Thai population: multivariable analysis

Background

Congenital heart diseases (CHDs) are the most common types of birth defects and contribute to a large proportion of infant morbidities and mortalities worldwide. These defects may require multiple surgical interventions impacting the infant's quality of life.

Objectives

To identify risk factors associated with CHD in a population of Thai children.

Methods

We conducted a case-control study of patients attending the Pediatric Clinic, Naresuan University Hospital, Thailand. We included data from pediatric patients diagnosed with CHDs as cases, and patients without cardiovascular abnormalities as controls. Risk data were collected from July 2019 to April 2020 using face-to-face interviews. Multiple logistic regression was used to analyze parental factors associated with CHDs.

Results

We included 249 cases classified into 2 groups according to severity and 304 patients as controls. For those less-severely affected (155 patients, 62.2%), ventricular septal defect (27.7%) was the most prevalent, whereas for those with severe CHDs, tetralogy of Fallot was the most prevalent (14.0%). There was no difference in sex distribution or maternal obstetric history between the groups. In multivariable analysis, a family history of CHDs (adjusted odds ratio [AOR] 4.67, 95% confidence interval (CI) 1.61-13.57, P = 0.005) and maternal exposure to second-hand cigarette smoke (AOR 1.58, 95% CI 1.03-2.42, P = 0.002) were identified as significant risk factors for CHDs.

Conclusion

A family history of CHDs and maternal exposure to second-hand cigarette smoke are associated with having offspring with CHDs in the population studied. These findings help us to encourage affected parents to obtain a fetal echocardiogram.

Multicenter Validation of Individual Preoperative Motor Outcome Prediction for Deep Brain Stimulation in Parkinson's Disease

BACKGROUND

Subthalamic nucleus deep brain stimulation (STN DBS) is an established therapy for Parkinson's disease (PD) patients suffering from motor response fluctuations despite optimal medical treatment, or severe dopaminergic side effects. Despite careful clinical selection and surgical procedures, some patients do not benefit from STN DBS. Preoperative prediction models are suggested to better predict individual motor response after STN DBS. We validate a preregistered model, DBS-PREDICT, in an external multicenter validation cohort.

METHODS

DBS-PREDICT considered eleven, solely preoperative, clinical characteristics and applied a logistic regression to differentiate between weak and strong motor responders. Weak motor response was defined as no clinically relevant improvement on the Unified Parkinson's Disease Rating Scale (UPDRS) II, III, or IV, 1 year after surgery, defined as, respectively, 3, 5, and 3 points or more. Lower UPDRS III and IV scores and higher age at disease onset contributed most to weak response predictions. Individual predictions were compared with actual clinical outcomes.

RESULTS

322 PD patients treated with STN DBS from 6 different centers were included. DBS-PREDICT differentiated between weak and strong motor responders with an area under the receiver operator curve of 0.76 and an accuracy up to 77%.

CONCLUSION

Proving generalizability and feasibility of preoperative STN DBS outcome prediction in an external multicenter cohort is an important step in creating clinical impact in DBS with data-driven tools. Future prospective studies are required to overcome several inherent practical and statistical limitations of including clinical decision support systems in DBS care.

Development and External Validation of a Model for Predicting Sufficient Filter Lifespan in Anticoagulation-Free Continuous Renal Replacement Therapy Patients

Background

Anticoagulation-free continuous renal replacement therapy (CRRT) was recommended by the current clinical guideline for patients with increased bleeding risk and contraindications of citrate. Nevertheless, anticoagulation-free CRRT yielded heterogeneous filter lifespan. Furthermore, the specific cutoff values for traditional coagulation parameters to predict sufficient filter lifespan of anticoagulation-free CRRT have not yet been determined. The purpose of our present study was to develop and validate a model for predicting sufficient filter lifespan in anticoagulation-free CRRT patients.

Methods

Patients who underwent anticoagulation-free CRRT in our center between June 2013 and June 2019 were retrospectively included. The primary outcome was sufficient filter lifespan (≥24 h). Thirty-seven predictors were included for modeling based on their clinical significance and previous reports. The final model was developed by using multivariable logistic regression analysis and was validated in a separate external cohort.

Results

The development cohort included 170 patients. Sufficient filter lifespan was observed in 80 patients. Thirteen variables were independent predictors for sufficient filter lifespan by logistic regression: body temperature, mean arterial pressure, activated partial thromboplastin time, direct bilirubin, alkaline phosphatase, blood urea nitrogen, vasopressor use, body mass index, white blood cell, platelet count, D-dimer, uric acid, and pH. The area under the curve (AUC) of the stepwise model and internal validation model was 0.82 (95% confidence interval [CI] [0.76–0.88]) and 0.8 (95% CI [0.74–0.87]), respectively. The positive predictive value and the negative predictive value of the stepwise model were 0.77 and 0.79, respectively. The validation cohort included 44 eligible patients and the AUC of the external validation model was 0.82 (95% CI [0.69–0.96]).

Conclusions

The use of a prediction model instead of an assessment based only on coagulation parameters could facilitate the identification of the patients with filter lifespan of ≥24 h when they accepted anticoagulation-free CRRT.

Machine learning for MRI radiomics: a study predicting tumor-infiltrating lymphocytes in patients with pancreatic ductal adenocarcinoma

OBJECTIVE

To develop and validate a machine learning classifier based on magnetic resonance imaging (MRI), for the preoperative prediction of tumor-infiltrating lymphocytes (TILs) in patients with pancreatic ductal adenocarcinoma (PDAC).

MATERIALS AND METHODS

In this retrospective study, 156 patients with PDAC underwent MR scan and surgical resection. The expression of CD4, CD8 and CD20 was detected and quantified using immunohistochemistry, and TILs score was achieved by Cox regression model. All patients were divided into TILs score-low and TILs score-high groups. The least absolute shrinkage and selection operator method and the extreme gradient boosting (XGBoost) were used to select the features and to construct a prediction model. The performance of the models was assessed using the training cohort (116 patients) and the validation cohort (40 patients), and decision curve analysis (DCA) was applied for clinical use.

RESULTS

The XGBoost prediction model showed good discrimination in the training (AUC 0.86; 95% CI 0.79-0.93) and validation sets (AUC 0.79; 95% CI 0.64-0.93). The sensitivity, specificity, and accuracy for the training set were 86.67%, 75.00%, and 0.81, respectively, whereas those for the validation set were 84.21%, 66.67%, and 0.75, respectively. Decision curve analysis indicated the clinical usefulness of the XGBoost classifier.

CONCLUSION

The model constructed by XGBoost could predict PDAC TILs and may aid clinical decision making for immune therapy.

A Clinical Prediction Tool for MRI in Emergency Department Patients with Spinal Infection

Introduction

Patients with pyogenic spinal Infection (PSI) are often not diagnosed at their initial presentation, and diagnostic delay is associated with increased morbidity and medical-legal risk. We derived a decision tool to estimate the risk of spinal infection and inform magnetic resonance imaging (MRI) decisions.

Methods

We conducted a two-part prospective observational cohort study that collected variables from spine pain patients over a six-year derivation phase. We fit a multivariable regression model with logistic coefficients rounded to the nearest integer and used them for variable weighting in the final risk score. This score, SIRCH (spine infection risk calculation heuristic), uses four clinical variables to predict PSI. We calculated the statistical performance, MRI utilization, and model fit in the derivation phase. In the second phase we used the same protocol but enrolled only confirmed cases of spinal infection to assess the sensitivity of our prediction tool.

Results

In the derivation phase, we evaluated 134 non-PSI and 40 PSI patients; median age in years was 55.5 (interquartile range [IQR] 38–70 and 51.5 (42–59), respectively. We identified four predictors for our risk score: historical risk factors; fever; progressive neurological deficit; and C-reactive protein (CRP) ≥ 50 milligrams per liter (mg/L). At a threshold SIRCH score of ≥ 3, the predictive model’s sensitivity, specificity, and positive predictive value were, respectively, as follows: 100% (95% confidence interval [CI], 100–100%); 56% (95% CI, 48–64%), and 40% (95% CI, 36–46%). The area under the receiver operator curve was 0.877 (95% CI, 0.829–0.925). The SIRCH score at a threshold of ≥ 3 would prompt significantly fewer MRIs compared to using an elevated CRP (only 99/174 MRIs compared to 144/174 MRIs, P <0.001). In the second phase (49 patient disease-only cohort), the sensitivities of the SIRCH score and CRP use (laboratory standard cut-off 3.5 mg/L) were 92% (95% CI, 84–98%), and 98% (95% CI, 94–100%), respectively.

Conclusion

The SIRCH score provides a sensitive estimate of spinal infection risk and prompts fewer MRIs than elevated CRP (cut-off 3.5 mg/L) or clinician suspicion.

Preselection of robust radiomic features does not improve outcome modelling in non-small cell lung cancer based on clinical routine FDG-PET imaging

Background

Radiomics is a promising tool for identifying imaging-based biomarkers. Radiomics-based models are often trained on single-institution datasets; however, multi-centre imaging datasets are preferred for external generalizability owing to the influence of inter-institutional scanning differences and acquisition settings. The study aim was to determine the value of preselection of robust radiomic features in routine clinical positron emission tomography (PET) images to predict clinical outcomes in locally advanced non-small cell lung cancer (NSCLC).

Methods

A total of 1404 primary tumour radiomic features were extracted from pre-treatment [¹⁸F]fluorodeoxyglucose (FDG)-PET scans of stage IIIA/N2 or IIIB NSCLC patients using a training cohort (n = 79; prospective Swiss multi-centre randomized phase III trial SAKK 16/00; 16 centres) and an internal validation cohort (n = 31; single centre). Robustness studies investigating delineation variation, attenuation correction and motion were performed (intraclass correlation coefficient threshold > 0.9). Two 12-/24-month event-free survival (EFS) and overall survival (OS) logistic regression models were trained using standardized imaging: (1) with robust features alone and (2) with all available features. Models were then validated using fivefold cross-validation, and validation on a separate single-centre dataset. Model performance was assessed using area under the receiver operating characteristic curve (AUC).

Results

Robustness studies identified 179 stable features (13%), with 25% stable features for 3D versus 4D acquisition, 31% for attenuation correction and 78% for delineation. Univariable analysis found no significant robust features predicting 12-/24-month EFS and 12-month OS (p value > 0.076). Prognostic models without robust preselection performed well for 12-month EFS in training (AUC = 0.73) and validation (AUC = 0.74). Patient stratification into two risk groups based on 12-month EFS was significant for training (p value = 0.02) and validation cohorts (p value = 0.03).

Conclusions

A PET-based radiomics model using a standardized, multi-centre dataset to predict EFS in locally advanced NSCLC was successfully established and validated with good performance. Prediction models with robust feature preselection were unsuccessful, indicating the need for a standardized imaging protocol.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13550-021-00809-3.

A nomogram for predicting pancreatic mucinous cystic neoplasm and serous cystic neoplasm

OBJECTIVES

To develop and validate a nomogram for the preoperative prediction of pancreatic serous cystic neoplasm (SCN) and mucinous cystic neoplasm (MCN) based on multidetector computed tomography (MDCT).

MATERIALS AND METHODS

In this retrospective study, the data of 227 patients with SCN and MCN were analyzed. Each patient underwent MDCT and surgical resection. A multivariable logistic regression model was developed using a training set consisting of 129 patients with SCN and 38 patients with MCN who were admitted between October 2012 and April 2019. The model was validated in 60 consecutive patients, 44 of whom had SCN and 16 of whom had MCN, admitted between May 2019 and April 2020. The regression model was adopted to establish a nomogram. Nomogram performance was determined by its discriminative ability and clinical utility.

RESULT

The multivariable logistic regression model included sex, size, location, shape, cyst characteristic, and cystic wall thickening. The individualized prediction nomogram showed good discrimination in the training sample (AUC 0.89; 95% CI 0.83-0.95) and in the validation sample (AUC 0.81; 95% CI 0.70-0.94). If the threshold probability is between 0.03 and 0.9, and > 0.93 in the prediction model, using the nomogram to predict SCN and MCN is more beneficial than the treat-all-patients as SCN scheme or the treat-all-patients as MCN scheme. The prediction model showed better discrimination than the radiologists' diagnosis (AUC = 0.68).

CONCLUSION

The nomogram could predict SCN and MCN preoperatively and may aid clinical decision-making.

Clinical Prediction of Iron Deficiency at Age 2 Years: A National Cross-sectional Study in France

OBJECTIVE

To assess the diagnostic accuracy of existing clinical criteria and to develop prediction tools for iron deficiency in 2-year-old children.

STUDY DESIGN

In a national cross-sectional study conducted in primary care pediatricians' practices throughout France, 2-year-old children were consecutively included (2016-2017). Multivariable logistic regression modeling and bootstrapping were used to develop several clinical models to predict iron deficiency (serum ferritin <12 μg/L). These models used the best criteria and combinations among the American Academy of Pediatrics' (AAP) criteria adapted to the European context (n = 10), then all potential predictors (n = 19). One model was then simplified into a simple prediction tool.

RESULTS

Among 568 included infants, 38 had iron deficiency (6.7%). In univariable analyses, no significant association with iron deficiency was observed for 8 of the 10 adapted AAP criteria. Three criteria (both parents born outside the European Union, low weight at 1 year old, and weaning to cow's milk without supplemental iron) were retained in the AAP model, which area under the receiver operating characteristic curve, sensitivity, and specificity were 0.62 (95% CI, 0.58-0.67), 30% (95% CI, 22%-39%), and 95% (95% CI, 92%-97%), respectively. Four criteria were retained in a newly derived simple prediction tool (≥1 criterion among the 3 previous plus duration of iron-rich formula consumption <12 months), which area under the receiver operating characteristic curve, sensitivity, and specificity were 0.72 (95% CI, 0.65-0.79), 63% (95% CI, 47%-80%), and 81% (95% CI, 70%-91%), respectively.

CONCLUSIONS

All prediction tools achieved acceptable diagnostic accuracy. The newly derived simple prediction tool offered potential ease of use.

TRIAL REGISTRATION

ClinicalTrials.gov NCT02484274.

Guidelines for Conducting Ethical Artificial Intelligence Research in Neurology: A Systematic Approach for Clinicians and Researchers

Pre-emptive recognition of the ethical implications of study design and algorithm choices in artificial intelligence (AI) research is an important but challenging process. AI applications have begun to transition from a promising future to clinical reality in neurology. As the clinical management of neurology is often concerned with discrete, often unpredictable, and highly consequential events linked to multimodal data streams over long timescales, forthcoming advances in AI have great potential to transform care for patients. However, critical ethical questions have been raised with implementation of the first AI applications in clinical practice. Clearly, AI will have far-reaching potential to promote, but also to endanger, ethical clinical practice. This article employs an anticipatory ethics approach to scrutinize how researchers in neurology can methodically identify ethical ramifications of design choices early in the research and development process, with a goal of pre-empting unintended consequences that may violate principles of ethical clinical care. First, we discuss the use of a systematic framework for researchers to identify ethical ramifications of various study design and algorithm choices. Second, using epilepsy as a paradigmatic example, anticipatory clinical scenarios that illustrate unintended ethical consequences are discussed, and failure points in each scenario evaluated. Third, we provide practical recommendations for understanding and addressing ethical ramifications early in methods development stages. Awareness of the ethical implications of study design and algorithm choices that may unintentionally enter AI is crucial to ensuring that incorporation of AI into neurology care leads to patient benefit rather than harm.

Towards individualised contraceptive counselling: clinical and reproductive factors associated with self-reported hormonal contraceptive-induced adverse mood symptoms

OBJECTIVE

The study aim was to establish which demographic, clinical, reproductive and psychiatric factors are associated with self-reported hormonal contraceptive (HC)-induced adverse mood symptoms.

STUDY DESIGN

We compiled baseline data from two Swedish studies: one cross-sectional study on combined oral contraceptive (COC)-induced adverse mood symptoms (n=118) and one randomised controlled trial on adverse mood symptoms on COC (n=184). Both included women eligible for COC use, aged over 18 years. All women answered a questionnaire on HC use and associated mood problems. The Mini-International Neuropsychiatric Interview (M.I.N.I.) was used to capture mood and anxiety disorders. Women who acknowledged HC-induced adverse mood symptoms, ongoing or previously (n=145), were compared with women without any such experience (n=157).

RESULTS

Compared with women without self-reported HC-induced adverse mood symptoms, women with these symptoms were younger at HC start (adjusted odds ratio (aOR) 0.83, 95% CI 0.72 to 0.95), had more often undergone induced abortion (OR 3.36, 95% CI 1.57 to 7.23), more often suffered from an ongoing minor depressive disorder (n=12 vs n=0) and had more often experienced any previous mental health problem (aOR 1.90, 95% CI 1.01 to 3.59).

CONCLUSIONS

In line with previous research, this study suggests that women with previous or ongoing mental health problems and women who are younger at HC start are more likely to experience HC-induced adverse mood symptoms. Former and current mental health should be addressed at contraceptive counselling, and ongoing mental health disorders should be adequately treated.

IMPLICATIONS

This study adds valuable knowledge for identification of women susceptible to HC-induced adverse mood symptoms. It should facilitate the assessment of whether or not a woman has an increased risk of such symptoms, and thus enable clinicians to adopt a more personalised approach to contraceptive counselling.

Derivation and validation of a predictive model for advanced colorectal neoplasia in asymptomatic adults

OBJECTIVE

Knowing risk for advanced colorectal neoplasia (AN) could help patients and providers choose among screening tests, improving screening efficiency and uptake. We created a risk prediction model for AN to help decide which test might be preferred, a use not considered for existing models.

DESIGN

Average-risk 50-to-80-year olds undergoing first-time screening colonoscopy were recruited from endoscopy units in Indiana. We measured sociodemographic and physical features, medical and family history and lifestyle factors and linked these to the most advanced finding. We derived a risk equation on two-thirds of the sample and assigned points to each variable to create a risk score. Scores with comparable risks were collapsed into risk categories. The model and score were tested on the remaining sample.

RESULTS

Among 3025 subjects in the derivation set (mean age 57.3 (6.5) years; 52% women), AN prevalence was 9.4%. The 13-variable model (c-statistic=0.77) produced three risk groups with AN risks of 1.5% (95% CI 0.72% to 2.74%), 7.06% (CI 5.89% to 8.38%) and 27.26% (CI 23.47% to 31.30%) in low-risk, intermediate-risk and high-risk groups (p value <0.001), containing 23%, 59% and 18% of subjects, respectively. In the validation set of 1475 subjects (AN prevalence of 8.4%), model performance was comparable (c-statistic=0.78), with AN risks of 2.73% (CI 1.25% to 5.11%), 5.57% (CI 4.12% to 7.34%) and 25.79% (CI 20.51% to 31.66%) in low-risk, intermediate-risk and high-risk subgroups, respectively (p<0.001), containing proportions of 23%, 59% and 18%.

CONCLUSION

Among average-risk persons, this model estimates AN risk with high discrimination, identifying a lower risk subgroup that may be screened non-invasively and a higher risk subgroup for which colonoscopy may be preferred. The model could help guide patient-provider discussions of screening options, may increase screening adherence and conserve colonoscopy resources.

Refining prediction of survival after TIPS with the novel Freiburg index of post-TIPS survival

BACKGROUND & AIMS

Transjugular intrahepatic portosystemic shunt (TIPS) implantation is an effective and safe treatment for complications of portal hypertension. Survival prediction is important in these patients as they constitute a high-risk population. Therefore, the aim of our study was to develop an alternative prognostic model for accurate survival prediction after planned TIPS implantation.

METHODS

A total of 1,871 patients with de novo TIPS implantation for ascites or secondary prophylaxis of variceal bleeding were recruited retrospectively. The study cohort was divided into a training set (80% of study patients; n = 1,496) and a validation set (20% of study patients; n = 375). Further, patients with early (preemptive) TIPS implantation due to variceal bleeding were included as another validation cohort (n = 290). Medical data and overall survival (OS) were assessed. A Cox regression model was used to create an alternative prediction model, which includes significant prognostic factors.

RESULTS

Age, bilirubin, albumin and creatinine were the most important prognostic factors. These parameters were included in a new score named the Freiburg index of post-TIPS survival (FIPS). The FIPS score was able to identify high-risk patients with a significantly reduced median survival of 5.0 (3.1-6.9) months after TIPS implantation in the training set. These results were confirmed in the validation set (median survival of 3.1 [0.9-5.3] months). The FIPS score showed better prognostic discrimination compared to the Child-Pugh, MELD, MELD-Na score and the bilirubin-platelet model. However, the FIPS score showed insufficient prognostic discrimination in patients with early TIPS implantation.

CONCLUSIONS

The FIPS score is superior to established scoring systems for the identification of high-risk patients with a worse prognosis following elective TIPS implantation.

LAY SUMMARY

Implantation of a transjugular intrahepatic portosystemic shunt (TIPS) is a safe and effective treatment for patients with cirrhosis and clinically significant portal hypertension. However, risk stratification is a major challenge in these patients as currently available scoring systems have major drawbacks. Age, bilirubin, albumin and creatinine were included in a new risk score which was named the Freiburg index of post-TIPS survival (FIPS). The FIPS score can identify patients at high risk and may guide clinical decision making.

Modeling-Based Decision Support System for Radical Prostatectomy Versus External Beam Radiotherapy for Prostate Cancer Incorporating an In Silico Clinical Trial and a Cost-Utility Study

Simple Summary

Low–intermediate prostate cancer has a number of viable treatment options, such as radical prostatectomy and radiotherapy, with similar survival outcomes but different treatment-related side effects. The aim of this study is to facilitate patient-specific treatment selection by developing a decision support system (DSS) that incorporates predictive models for cancer-free survival and treatment-related side effects. We challenged this DSS by validating it against randomized clinical trials and assessing the benefit through a cost–utility analysis. We aim to expand upon the applications of this DSS by using it as the basis for an in silico clinical trial for an underrepresented patient group. This modeling study shows that DSS-based treatment decisions will result in a clinically relevant increase in the patients’ quality of life and can be used for in silico trials.

Abstract

The aim of this study is to build a decision support system (DSS) to select radical prostatectomy (RP) or external beam radiotherapy (EBRT) for low- to intermediate-risk prostate cancer patients. We used an individual state-transition model based on predictive models for estimating tumor control and toxicity probabilities. We performed analyses on a synthetically generated dataset of 1000 patients with realistic clinical parameters, externally validated by comparison to randomized clinical trials, and set up an in silico clinical trial for elderly patients. We assessed the cost-effectiveness (CE) of the DSS for treatment selection by comparing it to randomized treatment allotment. Using the DSS, 47.8% of synthetic patients were selected for RP and 52.2% for EBRT. During validation, differences with the simulations of late toxicity and biochemical failure never exceeded 2%. The in silico trial showed that for elderly patients, toxicity has more influence on the decision than TCP, and the predicted QoL depends on the initial erectile function. The DSS is estimated to result in cost savings (EUR 323 (95% CI: EUR 213–433)) and more quality-adjusted life years (QALYs; 0.11 years, 95% CI: 0.00–0.22) than randomized treatment selection.

Machine Learning Algorithms Predict Prolonged Opioid Use in Opioid-Naïve Primary Hip Arthroscopy Patients

Introduction:

Excessive opioid use after orthopaedic surgery procedures remains a concern because it may result in increased morbidity and imposes a financial burden on the healthcare system. The purpose of this study was to develop machine learning algorithms to predict prolonged opioid use after hip arthroscopy in opioid-naïve patients.

Methods:

A registry of consecutive hip arthroscopy patients treated by a single fellowship-trained surgeon at one large academic and three community hospitals between January 2012 and January 2017 was queried. All patients were opioid-naïve and therefore had no history of opioid use before surgery. The primary outcome was prolonged postoperative opioid use, defined as patients who requested one or more opioid prescription refills postoperatively. Recursive feature elimination was used to identify the combination of variables that optimized model performance from an initial pool of 17 preoperative features. Five machine learning algorithms (stochastic gradient boosting, random forest, support vector machine, neural network, and elastic-net penalized logistic regression) were trained using 10-fold cross-validation five times and applied to an independent testing set of patients. These algorithms were assessed by calibration, discrimination, Brier score, and decision curve analysis.

Results:

A total of 775 patients were included, with 141 (18.2%) requesting and using one or more opioid refills after primary hip arthroscopy. The stochastic gradient boosting model achieved the best performance (c-statistic: 0.75, calibration intercept: −0.02, calibration slope: 0.88, and Brier score: 0.13). The five most important variables in predicting prolonged opioid use were the preoperative modified ones: Harris hip score, age, BMI, preoperative pain level, and worker's compensation status. The final algorithm was incorporated into an open-access web application available here: https://orthoapps.shinyapps.io/HPRG_OpioidUse/.

Conclusions:

Machine learning algorithms demonstrated good performance for predicting prolonged opioid use after hip arthroscopy in opioid-naïve patients. External validation of this algorithm is necessary to confirm the predictive ability and performance before use in clinical settings.

There is a paucity of economic evaluations of prediction methods of caries and periodontitis-A systematic review

Objectives

Direct cost for methods of prediction also named risk assessment in dentistry may be negligible compared with the cost of extensive constructions. On the other hand, as risk assessment is performed daily and for several patients in general dental practice, the costs may be considerable. The objective was to summarize evidence in studies of economic evaluation of prognostic prediction multivariable models and methods of caries and periodontitis and to identify knowledge gaps (PROSPERO registration number: CRD42020149763).

Material and methods

Four electronic databases (PubMed, Web of Science, The Cochrane Library, NHS Economic Evaluation Database) and reference lists of included studies were searched. Titles and abstracts were screened by two reviewers in parallel. Full‐text studies reporting resources used, costs and cost‐effectiveness of prediction models and methods were selected and critically appraised using a protocol based on items from the CHEERS checklist for economic evaluations and the CHARMS checklist for evaluation of prediction studies.

Results

From 38 selected studies, six studies on prediction fulfilled the eligibility criteria, four on caries and two on periodontitis. As the economic evaluations differed in method and perspective among the studies, the results could not be generalized. Our systematic review revealed methodological shortcomings regarding the description of predictive models and methods, and particularly of the economic evaluation.

Conclusions

The systematic review highlighted a paucity of economic evaluations regarding methods or multivariable models for prediction of caries and periodontitis. Our results indicate that what we currently practice using models and methods to predict caries and periodontitis lacks evidence regarding cost‐effectiveness.

Construction and validation of a nomogram for patients with skin cancer

Skin cancer is a common malignant tumor in human beings. At present, the construction of clinical prediction models mainly focuses on malignant melanoma and no researchers have constructed clinical prediction models for all kind of skin cancer to predict the prognosis of skin cancer. We used patient data collected from the surveillance, epidemiology, and end results program database to construct and validate our model for clinical prediction of skin cancer, hoping to provide a reference for clinical treatment of skin cancer.R software was used for univariate and multivariate Cox regression analysis of variables to screen out factors that have an impact on the survival of skin cancer patients. Then the prognostic model of skin cancer patients was constructed and the nomogram was drawn. Concordance Index (C-index), receiver operating characteristic (ROC) curve and calibration curve were used to evaluate the clinical prediction model.A total of 3180 skin cancer patients were included in this study. We constructed nomogram, a 3-year and 5-year clinical prediction model for skin cancer patients. We used C-index to evaluate the accuracy of nomogram model, and the result of C-index was 0.728, 95%CI (0.703-0.753). The nomogram model was evaluated by ROC curve. The area under the curve values of the ROC curve for 3-year survival rate and 5-year survival rate were 0.732 and 0.768 respectively. The model calibration diagram of the modeling group also shows that the model exhibits high accuracy.The nomogram model of postoperative survival of patients with skin cancer, based on the surveillance, epidemiology, and end results program database of patients with skin cancer, has shown good stability and accuracy in multi-method validation.

A Machine Learning Tool to Predict the Response to Neoadjuvant Chemotherapy in Patients with Locally Advanced Cervical Cancer

Despite several studies having identified factors associated with successful treatment outcomes in locally advanced cervical cancer, there is the lack of accurate predictive modeling for progression-free survival (PFS) in patients who undergo radical hysterectomy after neoadjuvant chemotherapy (NACT). Here we investigated whether machine learning (ML) may have the potential to provide a tool to predict neoadjuvant treatment response as PFS. In this retrospective observational study, we analyzed patients with locally advanced cervical cancer (FIGO stages IB2, IB3, IIA1, IIA2, IIB, and IIIC1) who were followed in a tertiary center from 2010 to 2018. Demographic and clinical characteristics were collected at either treatment baseline or at 24-month follow-up. Furthermore, we recorded data about magnetic resonance imaging (MRI) examinations and post-surgery histopathology. Proper feature selection was used to determine an attribute core set. Three different machine learning algorithms, namely Logistic Regression (LR), Random Forest (RFF), and K-nearest neighbors (KNN), were then trained and validated with 10-fold cross-validation to predict 24-month PFS. Our analysis included n. 92 patients. The attribute core set used to train machine learning algorithms included the presence/absence of fornix infiltration at pre-treatment MRI as well as of either parametrium invasion and lymph nodes involvement at post-surgery histopathology. RFF showed the best performance (accuracy 82.4%, precision 83.4%, recall 96.2%, area under receiver operating characteristic curve (AUROC) 0.82). We developed an accurate ML model to predict 24-month PFS.

Development and validation of prediction model to estimate 10-year risk of all-cause mortality using modern statistical learning methods: a large population-based cohort study and external validation

Background

In increasingly ageing populations, there is an emergent need to develop a robust prediction model for estimating an individual absolute risk for all-cause mortality, so that relevant assessments and interventions can be targeted appropriately. The objective of the study was to derive, evaluate and validate (internally and externally) a risk prediction model allowing rapid estimations of an absolute risk of all-cause mortality in the following 10 years.

Methods

For the model development, data came from English Longitudinal Study of Ageing study, which comprised 9154 population-representative individuals aged 50–75 years, 1240 (13.5%) of whom died during the 10-year follow-up. Internal validation was carried out using Harrell’s optimism-correction procedure; external validation was carried out using Health and Retirement Study (HRS), which is a nationally representative longitudinal survey of adults aged ≥50 years residing in the United States. Cox proportional hazards model with regularisation by the least absolute shrinkage and selection operator, where optimisation parameters were chosen based on repeated cross-validation, was employed for variable selection and model fitting. Measures of calibration, discrimination, sensitivity and specificity were determined in the development and validation cohorts.

Results

The model selected 13 prognostic factors of all-cause mortality encompassing information on demographic characteristics, health comorbidity, lifestyle and cognitive functioning. The internally validated model had good discriminatory ability (c-index=0.74), specificity (72.5%) and sensitivity (73.0%). Following external validation, the model’s prediction accuracy remained within a clinically acceptable range (c-index=0.69, calibration slope β=0.80, specificity=71.5% and sensitivity=70.6%). The main limitation of our model is twofold: 1) it may not be applicable to nursing home and other institutional populations, and 2) it was developed and validated in the cohorts with predominately white ethnicity.

Conclusions

A new prediction model that quantifies absolute risk of all-cause mortality in the following 10-years in the general population has been developed and externally validated. It has good prediction accuracy and is based on variables that are available in a variety of care and research settings. This model can facilitate identification of high risk for all-cause mortality older adults for further assessment or interventions.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-020-01204-7.

Development and Validation of Prognostic Nomograms Based on Gross Tumor Volume and Cervical Nodal Volume for Nasopharyngeal Carcinoma Patients With Concurrent Chemoradiotherapy

PURPOSE

Our study aimed to establish and validate prognostic nomograms based on gross tumor volume (GTV) and cervical nodal volume (CNV) for nasopharyngeal carcinoma (NPC) patients treated with two cycles of concurrent chemoradiotherapy (CCRT).

METHODS

From 2012 to 2015, 620 eligible patients who received radical treatment at the Cancer Hospital of Shantou University Medical College were recruited for a nomogram study. Variables were determined in a training set of 463 patients from 2012 to 2014 by X-tile analysis, univariate and multivariate Cox proportional hazard analyses, and the least absolute shrinkage and selection operator (LASSO). Another cohort of 157 patients in 2015 was validated with bootstrap resampling. The concordance index (C-index) and calibration curves were applied to assess its predictive discriminative and accuracy ability, while decision curve analysis (DCA), X-tile analysis and Kaplan-Meier curve for clinical application.

RESULTS

Independent prognostic variables for overall survival (OS) were age, GTV, CNV, cranial nerve, positive cervical lymph node laterality below the caudal border of cricoid cartilage (LNBC), and were selected for the nomogram. Optimal prognostic factors including Karnofsky performance status (KPS), age, GTV, CNV, LNBC were incorporated in the nomogram for progression-free survival (PFS). In the training set, the C-index of our nomograms for OS and PFS were 0.755 (95% CI, 0.704 to 0.807) and 0.698 (95% CI, 0.652 to 0.744). The calibration curve showed good agreement between nomogram-predicted and actual survival. DCA indicated that our nomograms were of clinical benefit.

CONCLUSION

Our nomograms are capable of effective prognostic prediction for patients with NPC.

CT-Radiomic Approach to Predict G1/2 Nonfunctional Pancreatic Neuroendocrine Tumor

RATIONALE AND OBJECTIVES

Tumor grading of nonfunctional pancreatic neuroendocrine tumors (NF-pNETs) determines the choice of clinical treatment and management. The pathological grade of pancreatic neuroendocrine tumors is usually assessed on postoperative specimens. The goal of our study is to establish a tumor grade (G) prediction model for preoperative G1/2 NF-pNETs using radiomics for multislice spiral CT image analysis.

MATERIALS AND METHODS

This retrospective study included a primary cohort of 59 patients and an independent validation cohort of 40 consecutive patients; their multislice spiral CT images were collected from October 2012 to October 2016 and October 2016 to June 2018, respectively. All 99 patients were diagnosed with clinicopathologically confirmed NF-pNETs. Most significant radiomic features were selected using the minimum redundancy and maximum relevance algorithm. Support vector machine classifier with a radial basis function-based predictive model was subsequently developed for clinical use.

RESULTS

A total of 585 radiomics features were extracted from every phase for each patient. Six of these radiomics features were identified as most discriminant features for G1 and G2 tumors and used to construct the tumor grade prediction model. The prediction model resulted in the area under the curve values of 0.968 (95% CI: 0.900-0.991) and 0.876 (95% CI: 0.700-0.963) for the training cohort and validation cohort, respectively. Sensitivity and specificity were 96.4% and 83.9%, and 90.9% and 88.9% for the training and validation cohorts, respectively. The decision curves indicated that if the threshold probability is above 0.1, using the rad-score in the current study on G1/2 NF-pNETs is more beneficial than the treat-all-patients scheme or the treat-none scheme.

CONCLUSION

Radiomics developed with a combination of nonenhanced and portal venous phases can achieve favorable predictive accuracy for histological grade for G1/G2 NF-pNETs.

Predicting onset of early- and late-treatment resistance in first-episode schizophrenia patients using advanced shrinkage statistical methods in a small sample

Evidence suggests there are two treatment-resistant schizophrenia subtypes (i.e. early treatment resistant (E-TR) and late-treatment resistant (L-TR)). We aimed to develop prediction models for estimating individual risk for these outcomes by employing advanced statistical shrinkage methods. 239 first-episode schizophrenia (FES) patients were followed-up for approximately 5 years after first presentation to psychiatric services; of these, n=56 (25.2%) were defined as E-TR and n=24 (12.6%) were defined as L-TR. Using known risk factors for poor schizophrenia outcomes, we developed prediction models for E-TR and L-TR using LASSO and RIDGE logistic regression models. Models' internal validation was performed employing Harrell's optimism-correction with repeated cross-validation; their predictive accuracy was assessed through discrimination and calibration. Both LASSO and RIDGE models had high discrimination, good calibration. While LASSO had moderate sensitivity for estimating an individual risk for E-TR and L-TR, sensitivity estimated for RIDGE model for these outcomes was extremely low, which was due to having a very large estimated optimism. Although it was possible to discriminate with sufficient accuracy who would meet criteria for E-TR and L-TR during the 5-year follow-up after first contact with mental health services for schizophrenia, further work is necessary to improve sensitivity for these models.

Risk score for predicting primary cesarean delivery in women with gestational diabetes mellitus

Background

Women with gestational diabetes mellitus (GDM) have a higher risk of cesarean delivery (CD) than glucose-tolerant women. The aim of this study was to develop and validate a risk score for predicting primary CD in women with GDM.

Methods

A risk score for predicting primary CD was developed using significant clinical features of 385 women who had a diagnosis of GDM and delivered at our institution between January 2011 and December 2014. The score was then tested for validity in another cohort of 448 individuals with GDM who delivered between January 2015 and December 2018.

Results

The risk score was developed using the features nulliparity, excess gestational weight gain, and insulin use. The scores that classified the pregnant women as low risk (0 points), intermediate risk (1–3 points), and high risk (≥ 4 points) were directly associated with the primary CD rates of the women in the development cohort: 14.7, 38.2 and 62.3%, respectively (P <  0.001). The model showed good calibration and acceptable discriminative power with a C statistic of 0.724 (95% confidence interval, 0.670–0.777). Similar results were observed in the validation cohort.

Conclusion

A risk score using the features nulliparity, excess gestational weight gain, and insulin use can estimate the risk for primary CD in women with GDM.

CT-Based Radiomics Score for Distinguishing Between Grade 1 and Grade 2 Nonfunctioning Pancreatic Neuroendocrine Tumors

OBJECTIVE. The objective of our study was to explore the relationship between a CT-based radiomics score and grade of nonfunctioning pancreatic neuroendocrine tumors (PNETs) and to evaluate the ability of a calculated CT radiomics score to distinguish between grade 1 and grade 2 nonfunctioning PNETs. MATERIALS AND METHODS. This retrospective study assessed 102 patients with surgically resected, pathologically confirmed nonfunctioning PNETs who underwent MDCT from January 2014 to December 2017. Radiomic methods were used to extract features from portal venous phase CT scans, and the least absolute shrinkage and selection operator (LASSO) method was used to select the features. Multivariate logistic regression models were used to analyze the association between the CT radiomics score and nonfunctioning PNET grades. The performance of the CT radiomics score was assessed on the basis of its discriminative ability and clinical usefulness. RESULTS. The CT radiomics score, which consisted of four selected features, was significantly associated with nonfunctioning PNET grades. Every 1-point increase in radiomics score was associated with a 57% increased risk of grade 2 disease. The score also showed high accuracy (AUC = 0.86 for all PNETs; AUC = 0.81 for PNETs ≤ 2 cm). The best cutoff point for maximal sensitivity and specificity was a CT radiomics score of -0.134. Decision curve analysis showed that the CT radiomics score is clinically useful. CONCLUSION. The CT radiomics score shows a significant association with the grade of nonfunctioning PNETs and provides a potentially valuable noninvasive tool for distinguishing between different grades of nonfunctioning PNET, especially among patients with tumors 2 cm or smaller.

Radiomics nomogram for the prediction of 2019 novel coronavirus pneumonia caused by SARS-CoV-2

Objectives

To develop and validate a radiomics model for predicting 2019 novel coronavirus (COVID-19) pneumonia.

Methods

For this retrospective study, a radiomics model was developed on the basis of a training set consisting of 136 patients with COVID-19 pneumonia and 103 patients with other types of viral pneumonia. Radiomics features were extracted from the lung parenchyma window. A radiomics signature was built on the basis of reproducible features, using the least absolute shrinkage and selection operator method (LASSO). Multivariable logistic regression model was adopted to establish a radiomics nomogram. Nomogram performance was determined by its discrimination, calibration, and clinical usefulness. The model was validated in 90 consecutive patients, of which 56 patients had COVID-19 pneumonia and 34 patients had other types of viral pneumonia.

Results

The radiomics signature, consisting of 3 selected features, was significantly associated with COVID-19 pneumonia (p < 0.05) in both training and validation sets. The multivariable logistic regression model included the radiomics signature and distribution; maximum lesion, hilar, and mediastinal lymph node enlargement; and pleural effusion. The individualized prediction nomogram showed good discrimination in the training sample (area under the receiver operating characteristic curve [AUC], 0.959; 95% confidence interval [CI], 0.933–0.985) and in the validation sample (AUC, 0.955; 95% CI, 0.899–0.995) and good calibration. The mixed model achieved better predictive efficacy than the clinical model. Decision curve analysis demonstrated that the radiomics nomogram was clinically useful.

Conclusions

The radiomics model derived has good performance for predicting COVID-19 pneumonia and may help in clinical decision-making.

Key Points

• A radiomics model showed good performance for prediction 2019 novel coronavirus pneumonia and favorable discrimination for other types of pneumonia on CT images.

• A central or peripheral distribution, a maximum lesion range > 10 cm, the involvement of all five lobes, hilar and mediastinal lymph node enlargement, and no pleural effusion is associated with an increased risk of 2019 novel coronavirus pneumonia.

• A radiomics model was superior to a clinical model in predicting 2019 novel coronavirus pneumonia.

Electronic supplementary material

The online version of this article (10.1007/s00330-020-07032-z) contains supplementary material, which is available to authorized users.

Noncontrast Radiomics Approach for Predicting Grades of Nonfunctional Pancreatic Neuroendocrine Tumors

BACKGROUND

Endoscopic ultrasound-guided fine-needle aspiration is associated with the accurate determination of tumor grade. However, because it is an invasive procedure there is a need to explore alternative noninvasive procedures.

PURPOSE

To develop and validate a noncontrast radiomics model for the preoperative prediction of nonfunctional pancreatic neuroendocrine tumor (NF-pNET) grade (G).

STUDY TYPE

Retrospective, single-center study.

SUBJECTS

Patients with pathologically confirmed PNETs (139) were included.

FIELD STRENGTH/SEQUENCE

3T/breath-hold single-shot fast-spin echo T₂ -weighted sequence and unenhanced and dynamic contrast-enhanced T₁ -weighted fat-suppressed sequences.

ASSESSMENT

Tumor features on contrast MR images were evaluated by three board-certified abdominal radiologists.

STATISTICAL TESTS

Multivariable logistic regression analysis was used to develop the clinical model. The least absolute shrinkage and selection operator method and linear discriminative analysis (LDA) were used to select the features and to construct a radiomics model. The performance of the models was assessed using the training cohort (97 patients) and the validation cohort (42 patients), and decision curve analysis (DCA) was applied for clinical use.

RESULTS

The clinical model included 14 imaging features, and the corresponding area under the curve (AUC) was 0.769 (95% confidence interval [CI], 0.675-0.863) in the training cohort and 0.729 (95% CI, 0.568-0.890) in the validation cohort. The LDA included 14 selected radiomics features that showed good discrimination-in the training cohort (AUC, 0.851; 95% CI, 0.758-0.916) and the validation cohort (AUC, 0.736; 95% CI, 0.518-0.874). In the decision curves, if the threshold probability was 0.17-0.84, using the radiomics score to distinguish NF-pNET G1 and G2/3, offered more benefit than did the use of a treat-all-patients or treat-none scheme.

DATA CONCLUSION

The developed radiomics model using noncontrast MRI could help differentiate G1 and G2/3 tumors, to make the clinical decision, and screen pNETs grade.

LEVEL OF EVIDENCE

4 TECHNICAL EFFICACY STAGE: 2 J. Magn. Reson. Imaging 2020;52:1124-1136.

Performance of CT-based radiomics in diagnosis of superior mesenteric vein resection margin in patients with pancreatic head cancer

OBJECTIVES

To accurately identify the relationship between a portal radiomics score (rad-score) and pathologic superior mesenteric vein (SMV) resection margin and to evaluate the diagnostic performance in patients with pancreatic head cancer.

MATERIALS AND METHODS

A total of 181 patients with postoperatively and pathologically confirmed pancreatic head cancer who underwent multislice computed tomography within one month of resection between January 2016 and December 2018 were retrospectively investigated. For each patient, 1029 radiomics features of the portal phase were extracted, which were reduced using the least absolute shrinkage and selection operator (LASSO) logistic regression algorithm. Multivariate logistic regression models were used to analyze the association between the portal rad-score and SMV resection margin.

RESULTS

Patients with negative (R0) and positive (R1) margins accounted for 70.17% (127) and 29.83% (54) of the cohort, respectively. The rad-score was significantly associated with the SMV resection margin status (p < 0.05). Multivariate analyses confirmed a significant and independent association between the portal rad-score and SMV resection margin (OR 4.62; 95% CI 2.19-9.76; p < 0.0001). The portal rad-score had high accuracy (area under the curve = 0.750). The best cut point based on maximizing the sum of sensitivity and specificity was - 0.741 (sensitivity = 64.8%; specificity = 74.0%; accuracy = 71.3%). Decision curve analysis indicated the clinical usefulness of radiomics score.

CONCLUSIONS

The portal rad-score is significantly associated with the pathologic SMV resection margin, and it can accurately and noninvasively predict the SMV resection margin in patients with pancreatic cancer.

Age-Specific Predictive Models of the Upper Quintile of Periodontal Attachment Loss

A practical method to identify people who are most affected by periodontitis in their age group is currently unavailable. We focused on individuals with mean clinical attachment loss (CAL) above the 80th percentile within each of 10 age groups (5-y intervals between 30 and 74 y as well as ≥75 y). We developed predictive models using combined data from 2 cohorts (2009 to 2010 and 2011 to 2012) from the NHANES (National Health and Nutrition Examination Survey; development cohort [DC], n = 6,757), and we carried out external validation using data from a third NHANES cohort (2013 to 2014; validation cohort [VC], n = 3,447). We used 1) age-specific logistic regression models with stepwise selection to identify significant demographic variables, habits, medical conditions, and selected clinical periodontal parameters (proportion of teeth with probing depth ≥4 mm at incisors and molars and with visible [≥2 mm] recession) and to calculate propensity scores (PSs); 2) Youden's J statistic to select optimum PS cutoffs to maximize diagnostic performance using receiver operating characteristic curves; and 3) bootstrap resampling with 1,000 replicates to validate the age-specific models and adjust the PS and optimal PS cutoffs for overfitting. The bootstrap-adjusted PSs were used as single predictors of mean CAL over the 80th percentile in the VC. The age-specific upper quintiles of mean CAL ranged between 1.63 and 3.24 mm in the DC and between 1.87 and 3.20 mm in the VC. The area under the curve of the models exceeded 0.85 in all age groups in the DC and 0.84 in the VC, indicating well-validated diagnostic performance. In the DC, sensitivity values ranged between 0.75 and 0.97 and exceeded 0.83 in 8 of 10 age groups. Corresponding values in the VC ranged between 0.56 and 0.89 and exceeded 0.68 in 8 of 10 age groups. We conclude that modeling that incorporates readily obtainable variables through a brief patient interview and an abbreviated periodontal examination accurately identifies individuals who are most affected by periodontitis in different ages.

Royston-Parmar flexible parametric survival model to predict the probability of keratoconus progression to corneal transplantation

PURPOSE

To assess a Royston-Parmar flexible parametric survival model to generate a personalised risk profile for keratoconus progression.

METHODS

We re-analysed a historic database of 2723 individuals with keratoconus. A Royston-Parmar survival model was fitted to predict the likelihood of the worse eye progressing to corneal transplantation. We used a backwards selection multivariable fractional polynomial procedure to assist with selection of covariates and identify appropriate transformation(s) to retain in the final model. Time-dependent receiver operating characteristic (ROC) curves from censored survival data using the Kaplan-Meier (KM) method were computed to visually assess how well the model identified eyes likely to progress.

RESULTS

In all, 5020 eyes from 2581 patients were available for model development. This included 2378 worst affected eyes, and 313 eyes that progressed to transplantation. The best fitting model [df = 1: Bayes information criterion (BIC) = 1573] included three variables, keratometry [hazard ratio (HR) 0.36: 95% confidence limits (CI) 0.32-0.42], age at baseline [HR 0.97: CI 0.95-0.99] and ethnicity [HR 3.92: CI 2.58-5.95]. Specificity at 1 year was 92.8% (CI 90.4-95.2%) with a corresponding sensitivity of 64.6% (CI 58.9-60.0%). These three prognostic factors account for 41.3% (CI 33.6 - 48.2%) of the variation among the survival curves.

CONCLUSION

Researchers should consider the Royston-Parmar model as an alternative to the Cox model. We illustrate the concepts and our results may lead to better tools that identify individuals at high risk of keratoconus progression.

Prediction models for determining the success of labor induction: A systematic review

INTRODUCTION

The purpose of this study was to systematically identify and compare clinical models using universally accessible clinical and demographic factors that were derived and/or validated to predict the success of labor induction with a view to making recommendations for practice.

MATERIAL AND METHODS

MEDLINE, Embase, www.clinicaltrials.gov, and PubMed (for non-MEDLINE and studies in-progress) were searched from inception to November 2017. Only studies that derived and/or validated clinical prediction models using variables obtained through antenatal history and digital cervical examination were included. Two reviewers independently screened titles and abstracts and extracted data from eligible studies into a standardized form. Extracted data included: participant characteristics, sample size, variables considered and included, endpoint definitions, study design and model performance. The Prediction Study Risk of Bias Assessment Tool (PROBAST) was used to appraise included studies. In view of clinical and methodologic heterogeneity between studies, only descriptive analysis was possible. The protocol was registered with the PROSPERO International prospective register of systematic reviews [CRD42017081548].

RESULTS

The search identified 16 studies describing 14 prediction models derived between 1966 and 2018. Models varied and demonstrated major limitations with regard to methodology, scope and performance. Of the derived models, six were internally validated and three were externally validated. Performance was most commonly measured using the area under the receiver operator characteristic curve, which ranged from 0.68 to 0.79, 0.67 to 0.77 and 0.61 to 0.73 for derived, internally validated and externally validated models, respectively. The risk-of-bias of included studies ranged from some studies fulfilling only 36% and some others fulfilling 86% of eligible PROBAST items.

CONCLUSIONS

No published model can be recommended for use at the bedside to determine the success of vaginal birth after labor induction. Based on the limitations of included models, a list of recommendations for improving model performance and utilization is provided, as well as measures for encouraging appropriate use of prediction models. The attitudes of women and care providers, and the clinical and resource implications must be explored prior to recommending the use of prediction models for determining the success of labor induction.

Forecasting future global food demand: A systematic review and meta-analysis of model complexity

Predicting future food demand is a critical step for formulating the agricultural, economic and conservation policies required to feed over 9 billion people by 2050 while doing minimal harm to the environment. However, published future food demand estimates range substantially, making it difficult to determine optimal policies. Here we present a systematic review of the food demand literature-including a meta-analysis of papers reporting average global food demand predictions-and test the effect of model complexity on predictions. We show that while estimates of future global kilocalorie demand have a broad range, they are not consistently dependent on model complexity or form. Indeed, time-series and simple income-based models often make similar predictions to integrated assessments (e.g., with expert opinions, future prices or climate influencing forecasts), despite having different underlying assumptions and mechanisms. However, reporting of model accuracy and uncertainty was uncommon, leading to difficulties in making evidence-based decisions about which forecasts to trust. We argue for improved model reporting and transparency to reduce this problem and improve the pace of development in this field.

Methods for the evaluation of biomarkers in patients with kidney and liver diseases: multicentre research programme including ELUCIDATE RCT

Background

Protein biomarkers with associations with the activity and outcomes of diseases are being identified by modern proteomic technologies. They may be simple, accessible, cheap and safe tests that can inform diagnosis, prognosis, treatment selection, monitoring of disease activity and therapy and may substitute for complex, invasive and expensive tests. However, their potential is not yet being realised.

Design and methods

The study consisted of three workstreams to create a framework for research: workstream 1, methodology – to define current practice and explore methodology innovations for biomarkers for monitoring disease; workstream 2, clinical translation – to create a framework of research practice, high-quality samples and related clinical data to evaluate the validity and clinical utility of protein biomarkers; and workstream 3, the ELF to Uncover Cirrhosis as an Indication for Diagnosis and Action for Treatable Event (ELUCIDATE) randomised controlled trial (RCT) – an exemplar RCT of an established test, the ADVIA Centaur® Enhanced Liver Fibrosis (ELF) test (Siemens Healthcare Diagnostics Ltd, Camberley, UK) [consisting of a panel of three markers – (1) serum hyaluronic acid, (2) amino-terminal propeptide of type III procollagen and (3) tissue inhibitor of metalloproteinase 1], for liver cirrhosis to determine its impact on diagnostic timing and the management of cirrhosis and the process of care and improving outcomes.

Results

The methodology workstream evaluated the quality of recommendations for using prostate-specific antigen to monitor patients, systematically reviewed RCTs of monitoring strategies and reviewed the monitoring biomarker literature and how monitoring can have an impact on outcomes. Simulation studies were conducted to evaluate monitoring and improve the merits of health care. The monitoring biomarker literature is modest and robust conclusions are infrequent. We recommend improvements in research practice. Patients strongly endorsed the need for robust and conclusive research in this area. The clinical translation workstream focused on analytical and clinical validity. Cohorts were established for renal cell carcinoma (RCC) and renal transplantation (RT), with samples and patient data from multiple centres, as a rapid-access resource to evaluate the validity of biomarkers. Candidate biomarkers for RCC and RT were identified from the literature and their quality was evaluated and selected biomarkers were prioritised. The duration of follow-up was a limitation but biomarkers were identified that may be taken forward for clinical utility. In the third workstream, the ELUCIDATE trial registered 1303 patients and randomised 878 patients out of a target of 1000. The trial started late and recruited slowly initially but ultimately recruited with good statistical power to answer the key questions. ELF monitoring altered the patient process of care and may show benefits from the early introduction of interventions with further follow-up. The ELUCIDATE trial was an ‘exemplar’ trial that has demonstrated the challenges of evaluating biomarker strategies in ‘end-to-end’ RCTs and will inform future study designs.

Conclusions

The limitations in the programme were principally that, during the collection and curation of the cohorts of patients with RCC and RT, the pace of discovery of new biomarkers in commercial and non-commercial research was slower than anticipated and so conclusive evaluations using the cohorts are few; however, access to the cohorts will be sustained for future new biomarkers. The ELUCIDATE trial was slow to start and recruit to, with a late surge of recruitment, and so final conclusions about the impact of the ELF test on long-term outcomes await further follow-up. The findings from the three workstreams were used to synthesise a strategy and framework for future biomarker evaluations incorporating innovations in study design, health economics and health informatics.

Trial registration

Current Controlled Trials ISRCTN74815110, UKCRN ID 9954 and UKCRN ID 11930.

Funding

This project was funded by the NIHR Programme Grants for Applied Research programme and will be published in full inProgramme Grants for Applied Research; Vol. 6, No. 3. See the NIHR Journals Library website for further project information.