Predicting intensive care need in women with preeclampsia using machine learning - a pilot study

BACKGROUND

Predicting severe preeclampsia with need for intensive care is challenging. To better predict high-risk pregnancies to prevent adverse outcomes such as eclampsia is still an unmet need worldwide. In this study we aimed to develop a prediction model for severe outcomes using routine biomarkers and clinical characteristics.

METHODS

We used machine learning models based on data from an intensive care cohort with severe preeclampsia (n=41) and a cohort of preeclampsia controls (n=40) with the objective to find patterns for severe disease not detectable with traditional logistic regression models.

RESULTS

The best model was generated by including the laboratory parameters aspartate aminotransferase (ASAT), uric acid and body mass index (BMI) with a cross-validation accuracy of 0.88 and an area under the curve (AUC) of 0.91. Our model was internally validated on a test-set where the accuracy was lower, 0.82, with an AUC of 0.85.

CONCLUSION

The clinical routine blood parameters ASAT and uric acid as well as BMI, were the parameters most indicative of severe disease. Aspartate aminotransferase reflects liver involvement, uric acid might be involved in several steps of the pathophysiologic process of preeclampsia, and obesity is a well-known risk factor for development of both severe and non-severe preeclampsia likely involving inflammatory pathways..[Figure: see text].

Making predictions under interventions: a case study from the PREDICT-CVD cohort in New Zealand primary care

Background

Most existing clinical prediction models do not allow predictions under interventions. Such predictions allow predicted risk under different proposed strategies to be compared and are therefore useful to support clinical decision making. We aimed to compare methodological approaches for predicting individual level cardiovascular risk under three interventions: smoking cessation, reducing blood pressure, and reducing cholesterol.

Methods

We used data from the PREDICT prospective cohort study in New Zealand to calculate cardiovascular risk in a primary care setting. We compared three strategies to estimate absolute risk under intervention: (a) conditioning on hypothetical interventions in non-causal models; (b) combining existing prediction models with causal effects estimated using observational causal inference methods; and (c) combining existing prediction models with causal effects reported in published literature.

Results

The median absolute cardiovascular risk among smokers was 3.9%; our approaches predicted that smoking cessation reduced this to a median between a non-causal estimate of 2.5% and a causal estimate of 2.8%, depending on estimation methods. For reducing blood pressure, the proposed approaches estimated a reduction of absolute risk from a median of 4.9% to a median between 3.2% and 4.5% (both derived from causal estimation). Reducing cholesterol was estimated to reduce median absolute risk from 3.1% to between 2.2% (non-causal estimate) and 2.8% (causal estimate).

Conclusions

Estimated absolute risk reductions based on non-causal methods were different to those based on causal methods, and there was substantial variation in estimates within the causal methods. Researchers wishing to estimate risk under intervention should be explicit about their causal modelling assumptions and conduct sensitivity analysis by considering a range of possible approaches.

Predictive performance of the sequential organ failure assessment score for in-hospital mortality in patients with end-stage kidney disease in intensive care units: A multicenter registry in Japan

INTRODUCTION

There is limited evidence regarding whether the performance of the Sequential Organ Failure Assessment (SOFA) score differs between patients with and without end-stage kidney disease (ESKD) in intensive care units (ICUs).

METHODS

We used a multicenter registry (Japanese Intensive care Patient Database) to enroll adult ICU patients between April 2018 and March 2021. We recalibrated the SOFA score using a logistic regression model and evaluated its predictive ability in both ESKD and non-ESKD groups. The primary outcome was in-hospital mortality.

RESULTS

128 134 patients were enrolled. The AUROC of the SOFA score was lower in the ESKD group than in the non-ESKD group [0.789 (95% CI, 0.774-0.804) vs. 0.846 (95% CI, 0.841-0.850)]. The calibration plot revealed good performance in both groups. However, it overestimated in-hospital mortality in ESKD groups.

CONCLUSION

The SOFA score demonstrated good predictive ability in patients with and without ESKD, but it overestimated the in-hospital mortality in ESKD patients.

Preoperative risk prediction models for acute kidney injury after noncardiac surgery: an independent external validation cohort study

BACKGROUND

Numerous models have been developed to predict acute kidney injury (AKI) after noncardiac surgery, yet there is a lack of independent validation and comparison among them.

METHODS

We conducted a systematic literature search to review published risk prediction models for AKI after noncardiac surgery. An independent external validation was performed using a retrospective surgical cohort at a large Chinese hospital from January 2019 to October 2022. The cohort included patients undergoing a wide range of noncardiac surgeries with perioperative creatinine measurements. Postoperative AKI was defined according to the Kidney Disease Improving Global Outcomes creatinine criteria. Model performance was assessed in terms of discrimination (area under the receiver operating characteristic curve, AUROC), calibration (calibration plot), and clinical utility (net benefit), before and after model recalibration through intercept and slope updates. A sensitivity analysis was conducted by including patients without postoperative creatinine measurements in the validation cohort and categorising them as non-AKI cases.

RESULTS

Nine prediction models were evaluated, each with varying clinical and methodological characteristics, including the types of surgical cohorts used for model development, AKI definitions, and predictors. In the validation cohort involving 13,186 patients, 650 (4.9%) developed AKI. Three models demonstrated fair discrimination (AUROC between 0.71 and 0.75); other models had poor or failed discrimination. All models exhibited some miscalibration; five of the nine models were well-calibrated after intercept and slope updates. Decision curve analysis indicated that the three models with fair discrimination consistently provided a positive net benefit after recalibration. The results were confirmed in the sensitivity analysis.

CONCLUSIONS

We identified three models with fair discrimination and potential clinical utility after recalibration for assessing the risk of acute kidney injury after noncardiac surgery.

External validation of a minimal-resource model to predict reduced estimated glomerular filtration rate in people with type 2 diabetes without diagnosis of chronic kidney disease in Mexico: a comparison between country-level and regional performance

Background

Patients with type 2 diabetes are at an increased risk of chronic kidney disease (CKD) hence it is recommended that they receive annual CKD screening. The huge burden of diabetes in Mexico and limited screening resource mean that CKD screening is underperformed. Consequently, patients often have a late diagnosis of CKD. A regional minimal-resource model to support risk-tailored CKD screening in patients with type 2 diabetes has been developed and globally validated. However, population heath and care services between countries within a region are expected to differ. The aim of this study was to evaluate the performance of the model within Mexico and compare this with the performance demonstrated within the Americas in the global validation.

Methods

We performed a retrospective observational study with data from primary care (Clinic Specialized in Diabetes Management in Mexico City), tertiary care (Instituto Nacional de Ciencias Médicas y Nutrición Salvador Zubirán) and the Mexican national survey of health and nutrition (ENSANUT-MC 2016). We applied the minimal-resource model across the datasets and evaluated model performance metrics, with the primary interest in the sensitivity and increase in the positive predictive value (PPV) compared to a screen-everyone approach.

Results

The model was evaluated on 2510 patients from Mexico (primary care: 1358, tertiary care: 735, ENSANUT-MC: 417). Across the Mexico data, the sensitivity was 0.730 (95% CI: 0.689 - 0.779) and the relative increase in PPV was 61.0% (95% CI: 52.1% - 70.8%). These were not statistically different to the regional performance metrics for the Americas (sensitivity: p=0.964; relative improvement: p=0.132), however considerable variability was observed across the data sources.

Conclusion

The minimal-resource model performs consistently in a representative Mexican population sample compared with the Americas regional performance. In primary care settings where screening is underperformed and access to laboratory testing is limited, the model can act as a risk-tailored CKD screening solution, directing screening resources to patients who are at highest risk.

Predicting COVID-19 Re-Positive Cases in Malnourished Older Adults: A Clinical Model Development and Validation

Purpose

Building and validating a clinical prediction model for novel coronavirus (COVID-19) re-positive cases in malnourished older adults.

Patients and Methods

Malnourished older adults from January to May 2023 were retrospectively collected from the Department of Geriatrics of the Affiliated Hospital of Chengdu University of Traditional Chinese Medicine. They were divided into a "non-re-positive" group and a "re-positive" group based on the number of COVID-19 infections, and into a training set and a validation set at a 7:3 ratio. The least absolute shrinkage and selection operator (LASSO) regression analysis was used to identify predictive factors for COVID-19 re-positivity in malnourished older adults, and a nomogram was constructed. Independent influencing factors were screened by multivariate logistic regression. The model's goodness-of-fit, discrimination, calibration, and clinical impact were assessed by Hosmer-Lemeshow test, area under the curve (AUC), calibration curve, decision curve analysis (DCA), and clinical impact curve analysis (CIC), respectively.

Results

We included 347 cases, 243 in the training set, and 104 in the validation set. We screened 10 variables as factors influencing the outcome. By multivariate logistic regression analysis, preliminary identified protective factors, risk factors, and independent influencing factors that affect the re-positive outcome. We constructed a clinical prediction model for COVID-19 re-positivity in malnourished older adults. The Hosmer-Lemeshow test yielded χ2 =5.916, P =0.657; the AUC was 0.881; when the threshold probability was >8%, using this model to predict whether malnourished older adults were re-positive for COVID-19 was more beneficial than implementing intervention programs for all patients; when the threshold was >80%, the positive estimated value was closer to the actual number of cases.

Conclusion

This model can help identify the risk of COVID-19 re-positivity in malnourished older adults early, facilitate early clinical decision-making and intervention, and have important implications for improving patient outcomes. We also expect more large-scale, multicenter studies to further validate, refine, and update this model.

[Study on mechanism of icariin-induced ferroptosis in HepG2 hepatoma carcinoma cells through PPARG/FABP4/GPX4 pathway]

The aim of this study was to explore the mechanism of icaritin-induced ferroptosis in hepatoma HepG2 cells. By bioinformatics screening, the target of icariin's intervention in liver cancer ferroptosis was selected, the protein-protein interaction(PPI) network was constructed, the related pathways were focused, the binding ability of icariin and target protein was evaluated by molecular docking, and the impact on patients' survival prognosis was predicted and the clinical prediction model was built. CCK-8, EdU, and clonal formation assays were used to detect cell viability and cell proliferation; colorimetric method and BODIPY 581/591 C1 fluorescent probe were used to detect the levels of Fe~(2+), MDA and GSH in cells, and the ability of icariin to induce HCC cell ferroptosis was evaluated; RT-qPCR and Western blot detection were used to verify the mRNA and protein levels of GPX4, xCT, PPARG, and FABP4 to determine the expression changes of these ferroptosis-related genes in response to icariin. Six intervention targets(AR, AURKA, PPARG, AKR1C3, ALB, NQO1) identified through bioinformatic analysis were used to establish a risk scoring system that aids in estimating the survival prognosis of HCC patients. In conjunction with patient age and TNM staging, a comprehensive Nomogram clinical prediction model was developed to forecast the 1-, 3-, and 5-year survival of HCC patients. Experimental results revealed that icariin effectively inhibited the activity and proliferation of HCC cells HepG2, significantly modulating levels of Fe~(2+), MDA, and lipid peroxidation ROS while reducing GSH levels, hence revealing its potential to induce ferroptosis in HCC cells. Icariin was found to diminish the expression of GPX4 and xCT(P<0.01), inducing ferroptosis in HCC cells, potentially in relation to inhibition of PPARG and FABP4(P<0.01). In summary, icariin induces ferroptosis in HCC cells via the PPARG/FABP4/GPX4 pathway, providing an experimental foundation for utilizing the traditional Chinese medicine icariin in the prevention or treatment of HCC.

Multivariable machine learning models for clinical prediction of subsequent hip fractures in older people using the Chinese population database

PURPOSE

This study aimed to develop and validate clinical prediction models using machine learning (ML) algorithms for reliable prediction of subsequent hip fractures in older individuals, who had previously sustained a first hip fracture, and facilitate early prevention and diagnosis, therefore effectively managing rapidly rising healthcare costs in China.

METHODS

Data were obtained from Grade A Tertiary hospitals for older patients (age ≥ 60 years) diagnosed with hip fractures in southwest China between 1 January 2009 and 1 April 2020. The database was built by collecting clinical and administrative data from outpatients and inpatients nationwide. Data were randomly split into training (80%) and testing datasets (20%), followed by six ML-based prediction models using 19 variables for hip fracture patients within 2 years of the first fracture.

RESULTS

A total of 40,237 patients with a median age of 66.0 years, who were admitted to acute-care hospitals for hip fractures, were randomly split into a training dataset (32,189 patients) and a testing dataset (8,048 patients). Our results indicated that three of our ML-based models delivered an excellent prediction of subsequent hip fracture outcomes (the area under the receiver operating characteristics curve: 0.92 (0.91-0.92), 0.92 (0·92-0·93), 0.92 (0·92-0·93)), outperforming previous prediction models based on claims and cohort data.

CONCLUSIONS

Our prediction models identify Chinese older people at high risk of subsequent hip fractures with specific baseline clinical and demographic variables such as length of hospital stay. These models might guide future targeted preventative treatments.

Development and validation of a clinical prediction model of fertilization failure during routine IVF cycles

Purpose

This study aims to create and validate a clinical model that predict the probability of fertilization failure in routine in-vitro fertilization (IVF) cycles.

Methods

This study employed a retrospective methodology, gathering data from 1770 couples that used reproductive center's of the Fourth Hospital of Hebei Medical University standard IVF fertilization between June 2015 and June 2023. 1062 were in the training set and 708 were in the validation set when it was randomly split into the training set and validation set in a 6:4 ratio. The study employed both univariate and multivariate logistic regression analysis to determine the factors those influence the failure of traditional in vitro fertilization. Based on the multiple regression model, a predictive model of traditional IVF fertilization failure was created. The calibration and decision curves were used to assess the effectiveness and therapeutic usefulness of this model.

Results

The following factors independently predicted the probability of an unsuccessful fertilization: infertility years, basal oestrogen, the rate of mature oocytes, oligoasthenozoospermia, sperm concentration, sperm vitality, percentage of abnormal morphological sperm, and percentage of progressive motility (PR%).The receiver operating characteristic curve's area under the curve (AUC) in the training set is 0.776 (95% CI: 0.740,0.812), while the validation set's AUC is 0.756 (95% CI: 0.708,0.805), indicating a rather high clinical prediction capacity.

Conclusion

Our generated nomogram has the ability to forecast the probability of fertilization failure in couples undergoing IVF, hence can assist clinical staff in making informed decisions.

Development and internal validation of a prediction model for long-term opioid use-an analysis of insurance claims data

ABSTRACT

In the United States, a public-health crisis of opioid overuse has been observed, and in Europe, prescriptions of opioids are strongly increasing over time. The objective was to develop and validate a multivariable prognostic model to be used at the beginning of an opioid prescription episode, aiming to identify individual patients at high risk for long-term opioid use based on routinely collected data. Predictors including demographics, comorbid diseases, comedication, morphine dose at episode initiation, and prescription practice were collected. The primary outcome was long-term opioid use, defined as opioid use of either >90 days duration and ≥10 claims or >120 days, independent of the number of claims. Traditional generalized linear statistical regression models and machine learning approaches were applied. The area under the curve, calibration plots, and the scaled Brier score assessed model performance. More than four hundred thousand opioid episodes were included. The final risk prediction model had an area under the curve of 0.927 (95% confidence interval 0.924-0.931) in the validation set, and this model had a scaled Brier score of 48.5%. Using a threshold of 10% predicted probability to identify patients at high risk, the overall accuracy of this risk prediction model was 81.6% (95% confidence interval 81.2% to 82.0%). Our study demonstrated that long-term opioid use can be predicted at the initiation of an opioid prescription episode, with satisfactory accuracy using data routinely collected at a large health insurance company. Traditional statistical methods resulted in higher discriminative ability and similarly good calibration as compared with machine learning approaches.

A simple and efficient clinical prediction scoring system to identify malignant pleural effusion

BACKGROUND

Early diagnosis of malignant pleural effusion (MPE) is of great significance. Current prediction models are not simple enough to be widely used in heavy clinical work.

OBJECTIVES

We aimed to develop a simple and efficient clinical prediction scoring system to distinguish MPE from benign pleural effusion (BPE).

DESIGN

This retrospective study involved patients with MPE or BPE who were admitted in West China Hospital from December 2010 to September 2016.

METHODS

Patients were divided into training, testing, and validation set. Prediction model was developed from training set and modified to a scoring system. The diagnostic efficacy and clinical benefits of the scoring system were estimated in all three sets.

RESULTS

Finally, 598 cases of MPE and 1094 cases of BPE were included. Serum neuron-specific enolase, serum cytokeratin 19 fragment (CYFRA21-1), pleural carcinoembryonic antigen (CEA), and ratio of pleural CEA to serum CEA were selected to establish the prediction models in training set, which were modified to the scoring system with scores of 6, 8, 10, and 9 points, respectively. Patients with scores >12 points have high MPE risk while ⩽12 points have low MPE risk. The scoring system has a high predictive value and good clinical benefits to differentiate MPE from BPE or lung-specific MPE from BPE.

CONCLUSION

This study developed a simple clinical prediction scoring system and was proven to have good clinical benefits, and it may help clinicians to separate MPE from BPE.

A model for predicting clinical prognosis based on brain metastasis-related genes in patients with breast cancer

Background

Brain metastasis (BM) is a clinically relevant cause of death in patients with breast cancer (BRCA). This study was designed to develop a clinical model capable of predicting BRCA patients' prognostic outcomes according to the expression of BM-related genes (BMRGs).

Methods

The public Gene Expression Omnibus (GEO) and The Cancer Genome Atlas (TCGA) databases served as data sources. BMRGs of BRCA were selected from previous literature. Differences among BRCA molecular subtypes were compared using R 'limma' package. The impact of BM-related differentially expressed genes (BM_DEGs) on BRCA patients' outcomes was explored with a risk score model, after which the relationship between these risk scores and immune cell infiltration was examined. Risk scores were also used to judge the predicted efficacy of immunotherapeutic interventions. The utility of risk scores in combination with clinicopathological characteristics was evaluated as a predictor of patient's survival through univariate and multivariate analyses.

Results

The R limma package was used to explore differential gene expression, after which 12 BM_DEGs were incorporated into a risk scoring model. The resultant risk scores were able to predict immunotherapeutic treatment efficacy. In addition, a nomogram incorporating risk scores, stage, and age was established. The nomogram was able to reliably predict the overall survival (OS) of BRCA patients, yielding predictive outcomes that aligned well with actual observations.

Conclusions

In summary, a predictive clinical model for BRCA patients was successfully established in this study, providing a valuable tool that may be particularly helpful for the assessment of patients facing a risk of BM development.

NephroCAGE-German-Canadian Consortium on AI for Improved Kidney Transplantation Outcome: Protocol for an Algorithm Development and Validation Study

BACKGROUND

Recent advances in hardware and software enabled the use of artificial intelligence (AI) algorithms for analysis of complex data in a wide range of daily-life use cases. We aim to explore the benefits of applying AI to a specific use case in transplant nephrology: risk prediction for severe posttransplant events. For the first time, we combine multinational real-world transplant data, which require specific legal and technical protection measures.

OBJECTIVE

The German-Canadian NephroCAGE consortium aims to develop and evaluate specific processes, software tools, and methods to (1) combine transplant data of more than 8000 cases over the past decades from leading transplant centers in Germany and Canada, (2) implement specific measures to protect sensitive transplant data, and (3) use multinational data as a foundation for developing high-quality prognostic AI models.

METHODS

To protect sensitive transplant data addressing the first and second objectives, we aim to implement a decentralized NephroCAGE federated learning infrastructure upon a private blockchain. Our NephroCAGE federated learning infrastructure enables a switch of paradigms: instead of pooling sensitive data into a central database for analysis, it enables the transfer of clinical prediction models (CPMs) to clinical sites for local data analyses. Thus, sensitive transplant data reside protected in their original sites while the comparable small algorithms are exchanged instead. For our third objective, we will compare the performance of selected AI algorithms, for example, random forest and extreme gradient boosting, as foundation for CPMs to predict severe short- and long-term posttransplant risks, for example, graft failure or mortality. The CPMs will be trained on donor and recipient data from retrospective cohorts of kidney transplant patients.

RESULTS

We have received initial funding for NephroCAGE in February 2021. All clinical partners have applied for and received ethics approval as of 2022. The process of exploration of clinical transplant database for variable extraction has started at all the centers in 2022. In total, 8120 patient records have been retrieved as of August 2023. The development and validation of CPMs is ongoing as of 2023.

CONCLUSIONS

For the first time, we will (1) combine kidney transplant data from nephrology centers in Germany and Canada, (2) implement federated learning as a foundation to use such real-world transplant data as a basis for the training of CPMs in a privacy-preserving way, and (3) develop a learning software system to investigate population specifics, for example, to understand population heterogeneity, treatment specificities, and individual impact on selected posttransplant outcomes.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/48892.

Construction of a clinical prediction model for the impact of acupuncture on pregnancy outcomes in poor ovarian response (POR) patients based on a patient registry research platform

OBJECTIVES

To construct a clinical prediction model for the impact of acupuncture on pregnancy outcomes in poor ovarian response (POR) patients, providing insights and methods for predicting pregnancy outcomes in POR patients undergoing acupuncture treatment.

METHODS

Clinical data of 268 POR patients (2 cases were eliminated) primarily treated with "thirteen needle acupuncture for Tiaojing Cuyun (regulating menstruation and promoting pregnancy)" was collected from the international patient registry platform of acupuncture moxibustion (IPRPAM) from September 19, 2017 to April 30, 2023, involving 24 clinical centers including Acupuncture-Moxibustion Hospital of China Academy of Chinese Medical Sciences. LASSO and univariate Cox regression were used to screen factors influencing pregnancy outcomes, and a multivariate Cox regression model was established based on the screening results. The best model was selected using the Akaike information criterion (AIC), and a nomogram for clinical pregnancy prediction was constructed. The prediction model was evaluated using receiver operating characteristic (ROC) curves and calibration curves, and internal validation was performed using the Bootstrap method.

RESULTS

(1) Age, level of anti-Müllerian hormone (AMH), and total treatment numbers of acupuncture were independent predictors of pregnancy outcomes in POR patients receiving acupuncture (P<0.05). (2) The AIC value of the best subset-Cox multivariate model (560.6) was the smallest, indicating it as the optimal model. (3) The areas under curve (AUCs) of the clinical prediction model after 6, 12, 24, and 36 months treatment were 0.627, 0.719, 0.770, and 0.766, respectively, and in the validation group, they were 0.620, 0.704, 0.759, and 0.765, indicating good discrimination and repeatability of the prediction model. (4) The calibration curve showed that the prediction curve of the clinical prediction model was close to the ideal model's prediction curve, indicating good calibration of the prediction model.

CONCLUSIONS

The clinical prediction model for the impact of acupuncture on pregnancy outcomes in POR patients based on the IPRPAM platform has good clinical application value and provides insights into predicting pregnancy outcomes in POR patients undergoing acupuncture treatment.

Assessing the Risk of Developing Delirium on Admission to Inpatient Rehabilitation: A Clinical Prediction Model

OBJECTIVES

To develop a clinical model to predict the risk of an individual patient developing delirium during inpatient rehabilitation, based on patient characteristics and clinical data available on admission.

DESIGN

Retrospective observational study based on electronic health record data.

SETTING AND PARTICIPANTS

We studied a previously validated data set of inpatients including incident delirium episodes during rehabilitation. These patients were admitted to ZURZACH Care, Rehaklinik Bad Zurzach, a Swiss inpatient rehabilitation clinic, between January 1, 2015, and December 31, 2018.

METHODS

We performed logistic regression analysis using backward and forward selection with alpha = 0.01 to remove any noninformative potential predictor. We subsequentially used the Akaike information criterion (AIC) to select the final model among the resulting "intermediate" models. Discrimination of the final prediction model was evaluated using the C-statistic.

RESULTS

Of the 20 candidate predictor variables, 6 were included in the final prediction model: a linear spline of age with 1 knot at 60 years and a linear spline of the functional independence measure (FIM), a measure of the functional degree of patients independency, with 1 knot at 64 points, diagnosis of disorders of fluid, electrolyte, and acid-base balance (E87), use of other analgesic and antipyretics (N02B), use of anti-parkinson drugs (N04B), and an anticholinergic burden score (ACB) of ≥3 points.

CONCLUSIONS AND IMPLICATIONS

Our clinical prediction model could, upon validation, identify patients at risk of incident delirium at admission to inpatient rehabilitation, and thus enable targeted prevention strategies.

The added value of text from Dutch general practitioner notes in predictive modeling

OBJECTIVE

This work aims to explore the value of Dutch unstructured data, in combination with structured data, for the development of prognostic prediction models in a general practitioner (GP) setting.

MATERIALS AND METHODS

We trained and validated prediction models for 4 common clinical prediction problems using various sparse text representations, common prediction algorithms, and observational GP electronic health record (EHR) data. We trained and validated 84 models internally and externally on data from different EHR systems.

RESULTS

On average, over all the different text representations and prediction algorithms, models only using text data performed better or similar to models using structured data alone in 2 prediction tasks. Additionally, in these 2 tasks, the combination of structured and text data outperformed models using structured or text data alone. No large performance differences were found between the different text representations and prediction algorithms.

DISCUSSION

Our findings indicate that the use of unstructured data alone can result in well-performing prediction models for some clinical prediction problems. Furthermore, the performance improvement achieved by combining structured and text data highlights the added value. Additionally, we demonstrate the significance of clinical natural language processing research in languages other than English and the possibility of validating text-based prediction models across various EHR systems.

CONCLUSION

Our study highlights the potential benefits of incorporating unstructured data in clinical prediction models in a GP setting. Although the added value of unstructured data may vary depending on the specific prediction task, our findings suggest that it has the potential to enhance patient care.

A Novel Clinical Tool to Detect Severe Obstructive Sleep Apnea

Purpose

Obstructive sleep apnea (OSA) is a disease with high morbidity and is associated with adverse health outcomes. Screening potential severe OSA patients will improve the quality of patient management and prognosis, while the accuracy and feasibility of existing screening tools are not so satisfactory. The purpose of this study is to develop and validate a well-feasible clinical predictive model for screening potential severe OSA patients.

Patients and Methods

We performed a retrospective cohort study including 1920 adults with overnight polysomnography among which 979 cases were diagnosed with severe OSA. Based on demography, symptoms, and hematological data, a multivariate logistic regression model was constructed and cross-validated and then a nomogram was developed to identify severe OSA. Moreover, we compared the performance of our model with the most commonly used screening tool, Stop-Bang Questionnaire (SBQ), among patients who completed the questionnaires.

Results

Severe OSA was associated with male, BMI≥ 28 kg/m2, high blood pressure, choke, sleepiness, apnea, white blood cell count ≥9.5×109/L, hemoglobin ≥175g/L, triglycerides ≥1.7 mmol/L. The AUC of the final model was 0.76 (95% CI: 0.74-0.78), with sensitivity and specificity under the optimal threshold selected by maximizing Youden Index of 73% and 66%. Among patients having the information of SBQ, the AUC of our model was statistically significantly greater than that of SBQ (0.78 vs 0.66, P = 0.002).

Conclusion

Based on common clinical examination of admission, we develop a novel model and a nomogram for identifying severe OSA from inpatient with suspected OSA, which provides physicians with a visual and easy-to-use tool for screening severe OSA.

External validation of models for predicting cumulative live birth over multiple complete cycles of IVF treatment

STUDY QUESTION

Can two prediction models developed using data from 1999 to 2009 accurately predict the cumulative probability of live birth per woman over multiple complete cycles of IVF in an updated UK cohort?

SUMMARY ANSWER

After being updated, the models were able to estimate individualized chances of cumulative live birth over multiple complete cycles of IVF with greater accuracy.

WHAT IS KNOWN ALREADY

The McLernon models were the first to predict cumulative live birth over multiple complete cycles of IVF. They were converted into an online calculator called OPIS (Outcome Prediction In Subfertility) which has 3000 users per month on average. A previous study externally validated the McLernon models using a Dutch prospective cohort containing data from 2011 to 2014. With changes in IVF practice over time, it is important that the McLernon models are externally validated on a more recent cohort of patients to ensure that predictions remain accurate.

STUDY DESIGN, SIZE, DURATION

A population-based cohort of 91 035 women undergoing IVF in the UK between January 2010 and December 2016 was used for external validation. Data on frozen embryo transfers associated with these complete IVF cycles conducted from 1 January 2017 to 31 December 2017 were also collected.

PARTICIPANTS/MATERIALS, SETTING, METHODS

Data on IVF treatments were obtained from the Human Fertilisation and Embryology Authority (HFEA). The predictive performances of the McLernon models were evaluated in terms of discrimination and calibration. Discrimination was assessed using the c-statistic and calibration was assessed using calibration-in-the-large, calibration slope, and calibration plots. Where any model demonstrated poor calibration in the validation cohort, the models were updated using intercept recalibration, logistic recalibration, or model revision to improve model performance.

MAIN RESULTS AND THE ROLE OF CHANCE

Following exclusions, 91 035 women who underwent 144 734 complete cycles were included. The validation cohort had a similar distribution age profile to women in the development cohort. Live birth rates over all complete cycles of IVF per woman were higher in the validation cohort. After calibration assessment, both models required updating. The coefficients of the pre-treatment model were revised, and the updated model showed reasonable discrimination (c-statistic: 0.67, 95% CI: 0.66 to 0.68). After logistic recalibration, the post-treatment model showed good discrimination (c-statistic: 0.75, 95% CI: 0.74 to 0.76). As an example, in the updated pre-treatment model, a 32-year-old woman with 2 years of primary infertility has a 42% chance of having a live birth in the first complete ICSI cycle and a 77% chance over three complete cycles. In a couple with 2 years of primary male factor infertility where a 30-year-old woman has 15 oocytes collected in the first cycle, a single fresh blastocyst embryo transferred in the first cycle and spare embryos cryopreserved, the estimated chance of live birth provided by the post-treatment model is 46% in the first complete ICSI cycle and 81% over three complete cycles.

LIMITATIONS, REASONS FOR CAUTION

Two predictors from the original models, duration of infertility and previous pregnancy, which were not available in the recent HFEA dataset, were imputed using data from the older cohort used to develop the models. The HFEA dataset does not contain some other potentially important predictors, e.g. BMI, ethnicity, race, smoking and alcohol intake in women, as well as measures of ovarian reserve such as antral follicle count.

WIDER IMPLICATIONS OF THE FINDINGS

Both updated models show improved predictive ability and provide estimates which are more reflective of current practice and patient case mix. The updated OPIS tool can be used by clinicians to help shape couples' expectations by informing them of their individualized chances of live birth over a sequence of multiple complete cycles of IVF.

STUDY FUNDING/COMPETING INTEREST(S)

This study was supported by an Elphinstone scholarship scheme at the University of Aberdeen and Aberdeen Fertility Centre, University of Aberdeen. S.B. has a commitment of research funding from Merck. D.J.M. and M.B.R. declare support for the present manuscript from Elphinstone scholarship scheme at the University of Aberdeen and Assisted Reproduction Unit at Aberdeen Fertility Centre, University of Aberdeen. D.J.M. declares grants received by University of Aberdeen from NHS Grampian, The Meikle Foundation, and Chief Scientist Office in the past 3 years. D.J.M. declares receiving an honorarium for lectures from Merck. D.J.M. is Associate Editor of Human Reproduction Open and Statistical Advisor for Reproductive BioMed Online. S.B. declares royalties from Cambridge University Press for a book. S.B. declares receiving an honorarium for lectures from Merck, Organon, Ferring, Obstetric and Gynaecological Society of Singapore, and Taiwanese Society for Reproductive Medicine. S.B. has received support from Merck, ESHRE, and Ferring for attending meetings as speaker and is on the METAFOR and CAPRE Trials Data Monitoring Committee.

TRIAL REGISTRATION NUMBER

N/A.

Prediction of early functional outcomes in patients after robotic-assisted total knee arthroplasty: a nomogram prediction model

BACKGROUND

Robotic-assisted total knee arthroplasty (RA-TKA) is becoming more and more popular as a treatment option for advanced knee diseases due to its potential to reduce operator-induced errors. However, the development of accurate prediction models for postoperative outcomes is challenging. This study aimed to develop a nomogram model to predict the likelihood of achieving a beneficial functional outcome. The beneficial outcome is defined as a postoperative improvement of the functional Knee Society Score (fKSS) of more than 10 points, 3 months after RA-TKA by early collection and analysis of possible predictors.

METHODS

This is a retrospective study on 171 patients who underwent unilateral RA-TKA at our hospital. The collected data included demographic information, preoperative imaging data, surgical data, and preoperative and postoperative scale scores. Participants were randomly divided into a training set ( N =120) and a test set ( N =51). Univariate and multivariate logistic regression analyses were employed to screen for relevant factors. Variance inflation factor was used to investigate for variable collinearity. The accuracy and stability of the models were evaluated using calibration curves with the Hosmer-Lemeshow goodness-of-fit test, consistency index and receiver operating characteristic curves.

RESULTS

Predictors of the nomogram included preoperative hip-knee-ankle angle deviation, preoperative 10-cm Visual Analogue Scale score, preoperative fKSS score and preoperative range of motion. Collinearity analysis with demonstrated no collinearity among the variables. The consistency index values for the training and test sets were 0.908 and 0.902, respectively. Finally, the area under the receiver operating characteristic curve was 0.908 (95% CI 0.846-0.971) in the training set and 0.902 (95% CI 0.806-0.998) in the test set.

CONCLUSION

A nomogram model was designed hereby aiming to predict the functional outcome 3 months after RA-TKA in patients. Rigorous validation showed that the model is robust and reliable. The identified key predictors include preoperative hip-knee-ankle angle deviation, preoperative visual analogue scale score, preoperative fKSS score, and preoperative range of motion. These findings have major implications for improving therapeutic interventions and informing clinical decision-making in patients undergoing RA-TKA.

Dynamic prediction modeling of postoperative mortality among patients undergoing surgical aortic valve replacement in a statewide cohort over a 12-year period

Objective

Clinical prediction models for surgical aortic valve replacement mortality, are valuable decision tools but are often limited in their ability to account for changes in medical practice, patient selection, and the risk of outcomes over time. Recent research has identified methods to update models as new data accrue, but their effect on model performance has not been rigorously tested.

Methods

The study population included 44,546 adults who underwent an isolated surgical aortic valve replacement from January 1, 1999, to December 31, 2018, statewide in Pennsylvania. After chronologically splitting the data into training and validation sets, we compared calibration, discrimination, and accuracy measures amongst a nonupdating model to 2 methods of model updating: calibration regression and the novel dynamic logistic state space model.

Results

The risk of mortality decreased significantly during the validation period (P < .01) and the nonupdating model demonstrated poor calibration and reduced accuracy over time. Both updating models maintained better calibration (Hosmer-Lemeshow χ2 statistic) than the nonupdating model: nonupdating (156.5), calibration regression (4.9), and dynamic logistic state space model (8.0). Overall accuracy (Brier score) was consistently better across both updating models: dynamic logistic state space model (0.0252), calibration regression (0.0253), and nonupdating (0.0256). Discrimination improved with the dynamic logistic state space model (area under the curve, 0.696) compared with the nonupdating model (area under the curve, 0.685) and calibration regression method (area under the curve, 0.687).

Conclusions

Dynamic model updating can improve model accuracy, discrimination, and calibration. The decision as to which method to use may depend on which measure is most important in each clinical context. Because competing therapies have emerged for valve replacement models, updating may guide clinical decision making.

Optimizing individualized management of patients with ulcerative colitis: Identification of risk factors predicting ulcerative colitis-associated neoplasia

The risk of developing colorectal neoplasia in patients with ulcerative colitis (UC) is increased. The purpose of this study is to analyze the risk factors of UC-associated neoplasia (UCAN) in UC patients and establish a clinical prediction model. 828 UC patients were included in this retrospective study. 602 patients were in discovery cohort and 226 patients were in validation cohort (internal validation cohort/external validation cohort: 120/106). Clinical and endoscopic data were collected. The discovery cohort was divided into UC group and UCAN group for univariate and multivariate binary logistic analyses. The UCAN clinical prediction model was established and verified. In the univariate analysis, 7 risk factors were related to UCAN. Multivariate logistic regression analysis showed that age at diagnosis of UC (OR: 1.018, 95% CI: 1.003-1.033), Ulcerative Colitis Endoscopic Index of Severity (UCEIS) score (OR: 1.823, 95% CI: 1.562-2.128), and size of polyps (size1: OR: 6.297, 95% CI: 3.669-10.809; size2: OR: 12.014, 95% CI: 6.327-22.814) were independent risk factors of UCAN. A mathematical equation was established. The area under the ROC curve (AUC) of this model was calculated to be 0.845 (95%CI: 0.809-0.881). The sensitivity was 0.884 and the specificity was 0.688. The AUC of internal validation cohort was 0.901 (95%CI: 0.815, 0.988), sensitivity was 75.0% and specificity was 92.6%. The AUC of external validation cohort was 0.842 (95%CI: 0.709, 0.976), sensitivity was 62.5% and specificity was 93.9%. This prediction model is simple, practical, and effective for predicting the risk of UCAN, which is beneficial to the individualized management of patients with UC.

Enhancing postpartum hemorrhage prediction in pernicious placenta previa: a comparative study of magnetic resonance imaging and ultrasound nomogram

Objective: To explore the risk factors of postpartum hemorrhage (PPH) in patients with pernicious placenta previa (PPP) and to develop and validate a clinical and imaging-based predictive model. Methods: A retrospective analysis was conducted on patients diagnosed surgically and pathologically with PPP between January 2018 and June 2022. All patients underwent PPP magnetic resonance imaging (MRI) and ultrasound scoring in the second trimester and before delivery, and were categorized into two groups according to PPH occurrence. The total imaging score and sub-item prediction models of the MRI risk score/ultrasound score were used to construct Models A and B/Models C and D. Models E and F were the total scores of the MRI combined with the ultrasound risk and sub-item prediction model scores. Model G was based on the subscores of MRI and ultrasound with the introduction of clinical data. Univariate logistic regression analysis and the logical least absolute shrinkage and selection operator (LASSO) model were used to construct models. The receiver operating characteristic curve andision curve analysis (DCA) were drawn, and the model with the strongest predictive ability and the best clinical effect was selected to construct a nomogram. Internal sampling was used to verify the prediction model's consistency. Results: 158 patients were included and the predictive power and clinical benefit of Models B and D were better than those of Models A and C. The results of the area under the curve of Models B, D, E, F, and G showed that Model G was the best, which could reach 0.93. Compared with Model F, age, vaginal hemorrhage during pregnancy, and amniotic fluid volume were independent risk factors for PPH in patients with PPP (p < 0.05). We plotted the DCA of Models B, D, E, F, and G, which showed that Model G had better clinical benefits and that the slope of the calibration curve of Model G was approximately 45°. Conclusion: LASSO regression nomogram based on clinical risk factors and multiple conventional ultrasound plus MRI signs has a certain guiding significance for the personalized prediction of PPH in patients with PPP before delivery.

Prediction of subsequent vertebral compression fractures after thoracolumbar kyphoplasty: a multicenter retrospective analysis

OBJECTIVE

Second fractures at the cemented vertebrae (SFCV) are often seen after percutaneous kyphoplasty, especially at the thoracolumbar junction. Our study aimed to develop and validate a preoperative clinical prediction model for predicting SFCV.

METHODS

A cohort of 224 patients with single-level thoracolumbar osteoporotic vertebral fractures (T11-L2) from 3 medical centers was analyzed between January 2017 and June 2020 to derive a preoperative clinical prediction model for SFCV. Backward-stepwise selection was used to select preoperative predictors. We assigned a score to each selected variable and developed the SFCV scoring system. Internal validation and calibration were conducted for the SFCV score.

RESULTS

Among the 224 patients included, 58 had postoperative SFCV (25.9%). The following preoperative measures on multivariable analysis were summarized in the 5-point SFCV score: bone mineral density (≤-3.05), serum 25-hydroxy vitamin D3 (≤17.55 ng/mL), standardized signal intensity of fractured vertebra on T1-weighted images (≤59.52%), C7-S1 sagittal vertical axis (≥3.25 cm), and intravertebral cleft. Internal validation showed a corrected area under the curve of 0.794. A cutoff of ≤1 point was chosen to classify a low risk of SFCV, for which only 6 of 100 patients (6%) had SFCV. A cutoff of ≥4 points was chosen to classify a high risk of SFCV, for which 28 of 41 (68.3%) had SFCV.

CONCLUSION

The SFCV score was found to be a simple preoperative method for identification of patients at low and high risk of postoperative SFCV. This model could be applied to individual patients and aid in the decision-making before percutaneous kyphoplasty.

Validation of a Diagnostic Model to Differentiate Multiple Myeloma from Bone Metastasis

Purpose

A diagnostic model to differentiate multiple myeloma (MM) from bone metastasis (BM) in patients with destructive bone lesions (MM-BM DDx) was developed to promote timely and appropriate referral of patients with MM to hematologists. External validation has never been conducted. This study aims to externally validate the performance of the MM-BM DDx model.

Patients and Methods

This multi-center external validation study was conducted using retrospective data of patients over 45 years old diagnosed with MM or BM at six university-affiliated hospitals in Thailand from 2016 to 2022. The MM-BM DDx development dataset, including patients from 2012 to 2015, was utilized during external validation. Diagnostic indicators for MM included in the MM-BM DDx model are serum creatinine, serum globulin, and serum alkaline phosphatase (ALP). MM and BM diagnosis was based on the documented International Classification of Diseases 10th Revision codes. Model performance was evaluated in terms of discrimination, calibration, and accuracy.

Results

A total of 3018 patients were included in the validation dataset (586 with MM and 2432 with BM). Clinical characteristics were similar between the validation and development datasets. The MM-BM DDx model's predictions showed an AUC of 0.89 (95% CI, 0.87, 0.90). The predicted probabilities of MM from the model increased concordantly with the observed proportion of MM within the validation dataset. The estimated sensitivity, specificity, and LR for each odds class in the validation dataset were similar to those of the development dataset.

Conclusion

The discriminative ability and calibration of the MM-BM DDx model were found to be preserved during external validation. These findings provide support for the practical use of the MM-BM DDx model to assist clinicians in identifying patients with destructive bone lesions who are likely to have MM and enable them to arrange timely referrals for further evaluation by hematologists.

Derivation and External Validation of a Clinical Prediction Model for Viral Diarrhea Etiology in Bangladesh

Background

Antibiotics are commonly overused for diarrheal illness in many low- and middle-income countries, partly due to a lack of diagnostics to identify viral cases, in which antibiotics are not beneficial. This study aimed to develop clinical prediction models to predict risk of viral-only diarrhea across all ages, using routinely collected demographic and clinical variables.

Methods

We used a derivation dataset from 10 hospitals across Bangladesh and a separate validation dataset from the icddr,b Dhaka Hospital. The primary outcome was viral-only etiology determined by stool quantitative polymerase chain reaction. Multivariable logistic regression models were fit and externally validated; discrimination was quantified using area under the receiver operating characteristic curve (AUC) and calibration assessed using calibration plots.

Results

Viral-only diarrhea was common in all age groups (<1 year, 41.4%; 18-55 years, 17.7%). A forward stepwise model had AUC of 0.82 (95% confidence interval [CI], .80-.84) while a simplified model with age, abdominal pain, and bloody stool had AUC of 0.81 (95% CI, .78-.82). In external validation, the models performed adequately although less robustly (AUC, 0.72 [95% CI, .70-.74]).

Conclusions

Prediction models consisting of 3 routinely collected variables can accurately predict viral-only diarrhea in patients of all ages in Bangladesh and may help support efforts to reduce inappropriate antibiotic use.

Developing prediction models to estimate the risk of two survival outcomes both occurring: A comparison of techniques

INTRODUCTION

This study considers the prediction of the time until two survival outcomes have both occurred. We compared a variety of analytical methods motivated by a typical clinical problem of multimorbidity prognosis.

METHODS

We considered five methods: product (multiply marginal risks), dual-outcome (directly model the time until both events occur), multistate models (msm), and a range of copula and frailty models. We assessed calibration and discrimination under a variety of simulated data scenarios, varying outcome prevalence, and the amount of residual correlation. The simulation focused on model misspecification and statistical power. Using data from the Clinical Practice Research Datalink, we compared model performance when predicting the risk of cardiovascular disease and type 2 diabetes both occurring.

RESULTS

Discrimination was similar for all methods. The product method was poorly calibrated in the presence of residual correlation. The msm and dual-outcome models were the most robust to model misspecification but suffered a drop in performance at small sample sizes due to overfitting, which the copula and frailty model were less susceptible to. The copula and frailty model's performance were highly dependent on the underlying data structure. In the clinical example, the product method was poorly calibrated when adjusting for 8 major cardiovascular risk factors.

DISCUSSION

We recommend the dual-outcome method for predicting the risk of two survival outcomes both occurring. It was the most robust to model misspecification, although was also the most prone to overfitting. The clinical example motivates the use of the methods considered in this study.

Early-onset locally advanced rectal cancer characteristics, a practical nomogram and risk stratification system: a population-based study

Background

The purpose of this study is to construct a novel and practical nomogram and risk stratification system to accurately predict cancer-specific survival (CSS) of early-onset locally advanced rectal cancer (EO-LARC) patients.

Methods

A total of 2440 patients diagnosed with EO-LARC between 2010 and 2019 were screened from the Surveillance, Epidemiology, and End Results (SEER) database. The pool of potentially eligible patients was randomly divided into two groups: a training cohort (N=1708) and a validation cohort (N=732). The nomogram was developed and calibrated using various methods, including the coherence index (C-index), receiver operating characteristic curve (ROC), calibration curves, and decision curves (DCA). A new risk classification system was established based on the nomogram. To compare the performance of this nomogram to that of the American Joint Committee on Cancer (AJCC) staging system, DCA, net reclassification index (NRI), and integrated discrimination improvement (IDI) were employed.

Result

Seven variables were included in the model. The area under the ROC curve (AUC) for the training cohort was 0.766, 0.736, and 0.731 at 3, 6, and 9 years, respectively. Calibration plots displayed good consistency between actual observations and the nomogram's predictions. The DCA curve further demonstrated the validity of the nomination form in clinical practice. Based on the scores of the nomogram, all patients were divided into a low-risk group, a middle-risk group, and a high-risk group. NRI for the 3-, 6-, and 9-year CSS(training cohort: 0.48, 0.45, 0.52; validation cohort: 0.42, 0.37, 0.37), IDI for the 3-, 6-, and 9-year CSS (training cohort: 0.09, 0.10, 0.11; validation cohort: 0.07, 0.08, 0.08). The Kaplan-Meier curve revealed that the new risk classification system possesses a more extraordinary ability to identify patients in different risk groups than the AJCC staging.

Conclusion

A practical prognostic nomogram and novel risk classification system have been developed to efficiently predict the prognosis of EO-LARC. These tools can serve as a guide to individualize patient treatment and improve clinical decision-making.

Prediction of postoperative pancreatic fistula and pancreatitis after pancreatoduodenectomy or distal pancreatectomy: A review

BACKGROUND AND OBJECTIVE

Postoperative pancreatic fistula (POPF) is the leading cause of morbidity and early mortality in patients undergoing pancreatic resection. In addition, recent studies have identified postoperative acute pancreatitis (POAP) as an independent contributor to morbidity. Most perioperative mitigation strategies experimented for POPF have been shown to be in vain with no consensus on the best perioperative management. Clinical prediction models have been developed with the hope of identifying high POPF risk patients with the leading idea of finding subpopulations possibly benefiting from pre-existing or novel mitigation strategies. The aim of this review was to map out the existing prediction modeling studies to better understand the current stage of POPF prediction modeling, and the methodology behind them.

METHODS

A narrative review of the existing POPF prediction model studies was performed. Studies published before September 2022 were included.

RESULTS

While the number of POPF prediction models for pancreatoduodenectomy has increased, none of the currently existing models stand out from the crowd. For distal pancreatectomy, two unique POPF prediction models exist, but due to their freshness, no further external validation or adoption in clinics or research has been reported. There seems to be a lack of adherence to correct methodology or reporting guidelines in most of the studies, which has rendered external validity-if assessed-low. Few of the most recent studies have demonstrated preoperative assessment of pancreatic aspects from computed tomography (CT) scans to provide relatively strong predictors of POPF.

CONCLUSIONS

Main goal for the future would be to reach a consensus on the most important POPF predictors and prediction model. At their current state, few models have demonstrated adequate transportability and generalizability to be up to the task. Better understanding of POPF pathophysiology and the possible driving force of acute inflammation and POAP might be required before such a prediction model can be accessed.

Development of machine learning and multivariable models for predicting blood transfusion in head and neck microvascular reconstruction for risk-stratified patient blood management

BACKGROUND

Although blood transfusions have adverse consequences for microvascular head and neck reconstruction, they are frequently administered. Pre-identifying patients would allow risk-stratified patient blood management.

METHODS

Development of machine learning (ML) and logistic regression (LR) models based on retrospective inclusion of 657 patients from 2011 to 2021. Internal validation and comparison with models from the literature by external validation. Development of a web application and a score chart.

RESULTS

Our models achieved an area under the receiver operating characteristic curve (ROC-AUC) of up to 0.825, significantly outperforming LR models from the literature. Preoperative hemoglobin, blood volume, duration of surgery and flap type/size were strong predictors.

CONCLUSIONS

The use of additional variables improves the prediction for blood transfusion, while models seems to have good generalizability due to surgical standardization and underlying physiological mechanism. The ML models developed showed comparable predictive performance to an LR model. However, ML models face legal hurdles, whereas score charts based on LR could be used after further validation.

Area under the curve-optimized synthesis of prediction models from a meta-analytical perspective

The number of clinical prediction models sharing the same prediction task has increased in the medical literature. However, evidence synthesis methodologies that use the results of these prediction models have not been sufficiently studied, particularly in the context of meta-analysis settings where only summary statistics are available. In particular, we consider the following situation: we want to predict an outcome Y, that is not included in our current data, while the covariate data are fully available. In addition, the summary statistics from prior studies, which share the same prediction task (i.e., the prediction of Y), are available. This study introduces a new method for synthesizing the summary results of binary prediction models reported in the prior studies using a linear predictor under a distributional assumption between the current and prior studies. The method provides an integrated predictor combining all predictors reported in the prior studies with weights. The vector of the weights is designed to achieve the hypothetical improvement of area under the receiver operating characteristic curve (AUC) on the current available data under a practical situation where there are different sets of covariates in the prior studies. We observe a counterintuitive aspect in typical situations where a part of weight components in the proposed method becomes negative. It implies that flipping the sign of the prediction results reported in each individual study would improve the overall prediction performance. Finally, numerical and real-world data analysis were conducted and showed that our method outperformed conventional methods in terms of AUC.

The Value of Handgrip Strength and Self-Rated Squat Ability in Predicting Mild Cognitive Impairment: Development and Validation of a Prediction Model

Early identification of individuals with mild cognitive impairment (MCI) is essential to combat worldwide dementia threats. Physical function indicators might be low-cost early markers for cognitive decline. To establish an early identification tool for MCI by combining physical function indicators (upper and lower limb function) via a clinical prediction modeling strategy. A total of 5393 participants aged 60 or older were included in the model. The variables selected for the model included sociodemographic characteristics, behavioral factors, mental status and chronic conditions, upper limb function (handgrip strength), and lower limb function (self-rated squat ability). Two models were developed to test the predictive value of handgrip strength (Model 1) or self-rated squat ability (Model 2) separately, and Model 3 was developed by combining handgrip strength and self-rated squat ability. The 3 models all yielded good discrimination performance (area under the curve values ranged from 0.719 to 0.732). The estimated net reclassification improvement values were 0.3279 and 0.1862 in Model 3 when comparing Model 3 to Model 1 and Model 2, respectively. The integrated discrimination improvement values were estimated as 0.0139 and 0.0128 when comparing Model 3 with Model 1 and Model 2, respectively. The model that contains both upper and lower limb function has better performance in predicting MCI. The final prediction model is expected to assist health workers in early identification of MCI, thus supporting early interventions to reduce future risk of AD, particularly in socioeconomically deprived communities.

Construction and Validation of a Nomogram Clinical Prediction Model for Predicting Osteoporosis in an Asymptomatic Elderly Population in Beijing

BACKGROUND

Based on the high prevalence and occult-onset of osteoporosis, the development of novel early screening tools was imminent. Therefore, this study attempted to construct a nomogram clinical prediction model for predicting osteoporosis.

METHODS

Asymptomatic elderly residents in the training (n = 438) and validation groups (n = 146) were recruited. BMD examinations were performed and clinical data were collected for the participants. Logistic regression analyses were performed. A logistic nomogram clinical prediction model and an online dynamic nomogram clinical prediction model were constructed. The nomogram model was validated by means of ROC curves, calibration curves, DCA curves, and clinical impact curves.

RESULTS

The nomogram clinical prediction model constructed based on gender, education level, and body weight was well generalized and had moderate predictive value (AUC > 0.7), better calibration, and better clinical benefit. An online dynamic nomogram was constructed.

CONCLUSIONS

The nomogram clinical prediction model was easy to generalize, and could help family physicians and primary community healthcare institutions to better screen for osteoporosis in the general elderly population and achieve early detection and diagnosis of the disease.

Development and validation of a risk prediction model for the recurrence of foot ulcer in type 2 diabetes in China: A longitudinal cohort study based on a systematic review and meta-analysis

AIMS

To develop and validate a risk prediction model for Chinese patients with type 2 diabetes with the recurrence of diabetic foot ulcers (DFUs) based on a systematic review and meta-analysis.

METHODS

A prospective analysis was performed with 1333 participants and followed up for 60 months. Three models were analysed using a derived cohort. The risk factors were screened using meta-analysis and logistic regression, and the missing variables were interpolated by multiple imputation. The internal validation was performed using the bootstrap procedure, and the validation cohort was applied to the external validation. The performance of the model was evaluated in the area under the discrimination Receiver Operating Characteristic Curve (ROC). Calibration and discrimination methods were used for the validation cohort. The variables were selected according to their clinical and statistical importance to construct the nomograms.

RESULTS

Three models were developed and validated. Model 1 included seven social and clinical indicators like sex, diabetes mellitus duration, previous DFU, location of ulcer, smoking, history of amputation, and foot deformity. Model 2 included four more indicators besides those in Model 1, which were statin agents used, antiplatelet agents used, systolic blood pressure, and body mass index. Model 3 added further laboratory indicators to Model 2, such as LDL-C, HbA1C, fibrinogen, and blood urea nitrogen. In the derivation cohort, 20.1% (206/1027) participants with DFU recurred as compared to the validation cohort, which was 38.2% (117/306). The areas under the curve in the derivation cohort for Models 1-3 were 0.781 (0.744-0.817), 0.843 (0.813-0.873), and 0.899 (0.876-0.922), respectively. The Youden indexes for Models 1-3 were 0.430, 0.559, and 0.653, respectively. Model 3 showed the highest sensitivity and specificity. All models performed well for both discrimination and calibration.

CONCLUSIONS

Models 1-2 were non-invasive, which indicate their role in general screening for patients at a high risk of recurrence of DFU. However, Model 3 offers a more specific screening due to its best performance in predicting the risk of DFU recurrence amongst the three models.

Survival nomogram for osteosarcoma patients: SEER data retrospective analysis with external validation

This study aimed to develop a nomogram based on the clinicopathological factors affecting the prognosis of osteosarcoma patients to help clinicians predict the overall survival of osteosarcoma patients. A total of 1362 patients diagnosed with osteosarcoma were enrolled in this study, among which, 1081 cases were enrolled from the SEER (Surveillance, Epidemiology, and End Results) database as training group, while 281 patients from two Clinical Medicine Center database were used in validation group. Univariate and multivariate Cox analyses were performed to identify the independent prognostic factors for overall survival. Nomogram predicting the 3- and 5-year overall survival probability was constructed and validated. Multiple validation methods, including calibration plots, consistency indices (C-index), and area under the receiver operating characteristic curve (AUC) were used to validate the accuracy and the reliability of the prediction models. Decision curve analysis (DCA) was conducted to validate the clinical application of the prediction model. Furthermore, all patients were divided into low- and high-risk groups based on their nomogram scores. Kaplan-Meier (KM) curves were applied to compare the difference in survival between the two groups. Predictors in the prediction model included age, sex, tumor size, primary site, grade, M stage, and surgery. Our results showed that the model displayed good prediction ability, and the calibration plots demonstrated great power both in the training and the validation groups. In the training group, C-index was 0.80, and the 3- and 5-year AUCs of the nomogram were 0.82 and 0.81, respectively. In the validation group, C-index was 0.79, and the 3- and 5-year AUCs of the nomogram were 0.85 and 0.83, respectively. Furthermore, DCA data indicated the potential clinical application of this model. Therefore, our prediction model could help clinicians evaluate prognoses, identify high-risk individuals, and provide individualized treatment recommendation for patients with osteosarcoma.

A Simplified Risk Score to Predict In-Hospital Newly-Diagnosed Atrial Fibrillation in Acute Ischemic Stroke Patients

Purpose

Atrial fibrillation (AF) is a significant cause of stroke, and newly diagnosed AF (NDAF) is typically detected in the early period of stroke onset. We aimed to identify the factors associated with in-hospital NDAF in acute ischemic stroke patients and developed a simplified clinical prediction model.

Methods

Patients with cryptogenic stroke aged 18 years or older who were admitted between January 2017 and December 2021 were recruited. NDAF was determined by inpatient cardiac telemetry. Univariable and multivariable regression analyses were used to evaluate the factors associated with in-hospital NDAF. The predictive model was developed using regression coefficients.

Results

The study enrolled 244 eligible participants, of which 52 NDAFs were documented (21.31%), and the median time to detection was two days (1-3.5). After multivariable regression analysis, parameters significantly associated with in-hospital NDAF were elderly (>75 years) (adjusted Odds ratio, 2.99; 95% confident interval, 1.51-5.91; P = 0.002), female sex (2.08; 1.04-4.14; P = 0.04), higher admission national institute of health stroke scale (1.04; 1.00-1.09; P = 0.05), and presence of hyperdense middle cerebral artery sign (2.33; 1.13-4.79; P = 0.02). The area under the receiver operating characteristic curve resulted in 0.74 (95% CI 0.65-0.80), and the cut-point of 2 showed 87% sensitivity and 42% specificity.

Conclusion

The validated and simplified risk scores for predicting in-hospital NDAF primarily rely on simplified parameters and high sensitivity. It might be used as a screening tool for in-hospital NDAF in stroke patients who initially presumed cryptogenic stroke.

Risk factor analysis and construction of prediction models of gallbladder carcinoma in patients with gallstones

Background

Gallbladder carcinoma (GBC) is a biliary tract tumor with a high mortality rate. The objectives of this study were to explore the risk factors of GBC in patients with gallstones and to establish effective screening indicators.

Methods

A total of 588 patients from medical centers in two different regions of China were included in this study and defined as the internal test samples and the external validation samples, respectively. We retrospectively reviewed the differences in clinicopathologic data of the internal test samples to find the independent risk factors that affect the occurrence of GBC. Then, we constructed three different combined predictive factors (CPFs) through the weighting method, integral system, and nomogram, respectively, and named them CPF-A, CPF-B, and CPF-C sequentially. Furthermore, we evaluated these indicators through calibration and DCA curves. The ROC curve was used to analyze their diagnostic efficiency. Finally, their diagnostic capabilities were validated in the external validation samples.

Results

In the internal test samples, the results showed that five factors, namely, age (RR = 3.077, 95% CI: 1.731-5.496), size of gallstones (RR = 13.732, 95% CI: 5.937-31.762), course of gallstones (RR = 2.438, 95% CI: 1.350-4.403), CEA (RR = 9.464, 95% CI: 3.394-26.392), and CA199 (RR = 9.605, 95% CI: 4.512-20.446), were independent risk factors for GBC in patients with gallstones. Then, we established three predictive indicators: CPF-A, CPF-B, and CPF-C. These models were further validated using bootstrapping with 1,000 repetitions. Calibration and decision curve analysis showed that the three models fit well. Meanwhile, multivariate analysis showed that CPF-B and CPF-C were independent risk factors for GBC in patients with gallstones. In addition, the validation results of the external validation samples are essentially consistent with the internal test samples.

Conclusion

Age (≤58.5 vs. >58.5 years), size of gallstones (≤1.95 vs. >1.95cm), course of gallstones (≤10 vs. >10 years), CEA (≤5 vs. >5 ng/ml), and CA199 (≤37 vs. >37 U/ml) are independent risk factors for GBC in patients with gallstones. When positive indicators were ≥2 among the five independent risk factors or the score of the nomogram was >82.64, the risk of GBC was high in gallstone patients.

Reduced Skeletal Muscle Mass Is Associated with an Increased Risk of Asthma Control and Exacerbation

BACKGROUND

Skeletal muscle mass (SMM) has been suggested to be associated with multiple health-related outcomes. However, the potential influence of SMM on asthma has not been largely explored.

OBJECTIVE

To study the association between SMM and clinical features of asthma, including asthma control and exacerbation, and to construct a model based on SMM to predict the risk of asthma exacerbation (AEx).

METHODS

In this prospective cohort study, we consecutively recruited patients with asthma (n = 334), classified as the SMM ^Normal group (n = 223), SMM ^Low group (n = 88), and SMM ^High group (n = 23). We investigated the association between SMM and clinical asthma characteristics and explored the association between SMM and asthma control and AEx within a 12-month follow-up period. Based on SMM, an exacerbation prediction model was developed, and the overall performance was externally validated in an independent cohort (n = 157).

RESULTS

Compared with the SMM ^Normal group, SMM ^Low group exhibited more airway obstruction and worse asthma control, while SMM ^High group had a reduced eosinophil percentage in induced sputum. Furthermore, SMM ^Low group was at a significantly increased risk of moderate-to-severe exacerbation compared with the SMM ^Normal group (relative risk _adjusted 2.02 [95% confidence interval (CI), 1.35-2.68]; p = 0.002). In addition, a model involving SMM was developed which predicted AEx (area under the curve: 0.750, 95% CI: 0.691-0.810).

CONCLUSIONS

Low SMM was an independent risk factor for future AEx. Furthermore, a model involving SMM for predicting the risk of AEx in patients with asthma indicated that assessment of SMM has potential clinical implications for asthma management.

Developing Clinical Prediction Models Using Primary Care Electronic Health Record Data: The Impact of Data Preparation Choices on Model Performance

Objective

To quantify prediction model performance in relation to data preparation choices when using electronic health records (EHR).

Study Design and Setting

Cox proportional hazards models were developed for predicting the first-ever main adverse cardiovascular events using Dutch primary care EHR data. The reference model was based on a 1-year run-in period, cardiovascular events were defined based on both EHR diagnosis and medication codes, and missing values were multiply imputed. We compared data preparation choices based on (i) length of the run-in period (2- or 3-year run-in); (ii) outcome definition (EHR diagnosis codes or medication codes only); and (iii) methods addressing missing values (mean imputation or complete case analysis) by making variations on the derivation set and testing their impact in a validation set.

Results

We included 89,491 patients in whom 6,736 first-ever main adverse cardiovascular events occurred during a median follow-up of 8 years. Outcome definition based only on diagnosis codes led to a systematic underestimation of risk (calibration curve intercept: 0.84; 95% CI: 0.83-0.84), while complete case analysis led to overestimation (calibration curve intercept: -0.52; 95% CI: -0.53 to -0.51). Differences in the length of the run-in period showed no relevant impact on calibration and discrimination.

Conclusion

Data preparation choices regarding outcome definition or methods to address missing values can have a substantial impact on the calibration of predictions, hampering reliable clinical decision support. This study further illustrates the urgency of transparent reporting of modeling choices in an EHR data setting.

Comparison of the First and Second Waves of the Coronavirus Disease 2019 Pandemic in Children and Adolescents in a Middle-Income Country: Clinical Impact Associated with Severe Acute Respiratory Syndrome Coronavirus 2 Gamma Lineage

OBJECTIVE

To evaluate the severity and clinical outcomes of the SARS-CoV-2 gamma variant in children and adolescents hospitalized with COVID-19 in Brazil.

STUDY DESIGN

In this observational retrospective cohort study, we performed an analysis of all 21 591 hospitalized patients aged <20 years with confirmed SARS-CoV-2 infection registered in a national database in Brazil. The cohort was divided into 2 groups according to the predominance of SARS-CoV-2 lineages (WAVE1, n = 11 574; WAVE2, n = 10 017). The characteristics of interest were age, sex, geographic region, ethnicity, clinical presentation, and comorbidities. The primary outcome was time to death, which was evaluated by competing-risks analysis, using cumulative incidence functions. A predictive Fine and Gray competing-risks model was developed based on the WAVE1 cohort with temporal validation in the WAVE2 cohort.

RESULTS

Compared with children and adolescents admitted during the first wave, those admitted during the second wave had significantly more hypoxemia (52.5% vs 41.1%; P < .0001) and intensive care unit admissions (28.3% vs 24.9%; P < .0001) and needed more noninvasive ventilatory support (37.3% vs 31.6%; P < .0001). In-hospital deaths and death rates were 896 (7.7%) in the first wave and 765 (7.6%) in the second wave (P = .07). The prediction model of death included age, ethnicity, region, respiratory symptoms, and comorbidities. In the validation set (WAVE2), the C statistic was 0.750 (95% CI, 0.741-0.758; P < .0001).

CONCLUSIONS

This large national study found a more severe spectrum of risk for pediatric patients with COVID-19 caused by the gamma variant. However, there was no difference regarding the probability of death between the waves.

Trends in the conduct and reporting of clinical prediction model development and validation: a systematic review

Objectives

This systematic review aims to provide further insights into the conduct and reporting of clinical prediction model development and validation over time. We focus on assessing the reporting of information necessary to enable external validation by other investigators.

Materials and Methods

We searched Embase, Medline, Web-of-Science, Cochrane Library, and Google Scholar to identify studies that developed 1 or more multivariable prognostic prediction models using electronic health record (EHR) data published in the period 2009–2019.

Results

We identified 422 studies that developed a total of 579 clinical prediction models using EHR data. We observed a steep increase over the years in the number of developed models. The percentage of models externally validated in the same paper remained at around 10%. Throughout 2009–2019, for both the target population and the outcome definitions, code lists were provided for less than 20% of the models. For about half of the models that were developed using regression analysis, the final model was not completely presented.

Discussion

Overall, we observed limited improvement over time in the conduct and reporting of clinical prediction model development and validation. In particular, the prediction problem definition was often not clearly reported, and the final model was often not completely presented.

Conclusion

Improvement in the reporting of information necessary to enable external validation by other investigators is still urgently needed to increase clinical adoption of developed models.

A clinical prediction model to assess risk for pancreatic cancer among patients with prediabetes

BACKGROUND

Early detection of pancreatic ductal adenocarcinoma (PDA) may improve survival. We previously developed a clinical prediction model among patients with new-onset diabetes to help identify PDAs 6 months prior to the clinical diagnosis of the cancer. We developed and internally validated a new model to predict PDA risk among those newly diagnosed with impaired fasting glucose (IFG).

METHODS

We conducted a retrospective cohort study in The Health Improvement Network (THIN) (1995-2013) from the UK. Eligible study patients had newly diagnosed IFG during follow-up in THIN. The outcome was incident PDA diagnosed within 3 years of IFG diagnosis. Candidate predictors were factors associated with PDA, glucose metabolism or both.

RESULTS

Among the 138 232 eligible patients with initial IFG diagnosis, 245 (0.2%) were diagnosed with PDA within 3 years. The median time from IFG diagnosis to clinical PDA diagnosis was 326 days (IQR 120-588). The final prediction model included age, BMI, proton pump inhibitor use, total cholesterol, low-density lipoprotein, alanine aminotransferase and alkaline phosphatase. The model achieved good discrimination [area under the curve 0.71 (95% CI, 0.67-0.75)] and calibration (Hosmer and Lemeshow goodness-of-fit test P > 0.05 in 17 of the 20 imputed data sets) with optimism of 0.0012662 (95% CI, -0.00932 to 0.0108771).

CONCLUSIONS

We developed and internally validated a sequential PDA prediction model based on clinical information routinely available at the initial appearance of IFG. If externally validated, this model could significantly extend our ability to detect PDAs at an earlier stage.

Development and Validation of a Nomogram for Predicting Obstructive Sleep Apnea in Patients with Pulmonary Arterial Hypertension

PURPOSE

Patients with pulmonary arterial hypertension (PAH) are at high risk for obstructive sleep apnea (OSA), which may adversely affect pulmonary hemodynamics and long-term prognosis. However, there is no clinical prediction model to evaluate the probability of OSA among patients with PAH. Our study aimed to develop and validate a nomogram for predicting OSA in the setting of PAH.

PATIENTS AND METHODS

From May 2020 to November 2021, we retrospectively analyzed the medical records of 258 patients diagnosed with PAH via right-heart catheterization. All participants underwent overnight cardiorespiratory polygraphy for OSA assessment. General clinical materials and biochemical measurements were collected and compared between PAH patients with or without OSA. Lasso regression was performed to screen potential predictors. Multivariable logistic regression analysis was conducted to establish the nomogram. Concordance index, calibration curve, and decision curve analysis were used to determine the discrimination, calibration, and clinical usefulness of the nomogram.

RESULTS

OSA was present in 26.7% of the PAH patients, and the prevalence did not differ significantly between male (29.7%) and female (24.3%) patients. Six variables were selected to construct the nomogram, including age, body mass index, hypertension, uric acid, glycated hemoglobin, and interleukin-6 levels. Based on receiver operating characteristic analysis, the nomogram demonstrated favorable discrimination accuracy with an area under the curve (AUC) of 0.760 for predicting OSA, exhibiting a better predictive value in contrast to ESS (AUC = 0.528) (P < 0.001). Decision curve analysis and clinical impact curve analysis also indicated the clinical utility of the nomogram.

CONCLUSION

By establishing a comprehensive and practical nomogram, we were able to predict the presence of OSA in patients with PAH, which may facilitate the early identification of patients that benefit from further diagnostic confirmation and intervention.

Predicting Violent Reoffending in Individuals Released From Prison in a Lower-Middle-Income Country: A Validation of OxRec in Tajikistan

BACKGROUND

Although around 70% of the world's prison population live in low- and middle-income countries (LMICs), risk assessment tools for criminal recidivism have been developed and validated in high-income countries (HICs). Validating such tools in LMIC settings is important for the risk management of people released from prison, development of evidence-based intervention programmes, and effective allocation of limited resources.

METHODS

We aimed to externally validate a scalable risk assessment tool, the Oxford Risk of Recidivism (OxRec) tool, which was developed in Sweden, using data from a cohort of people released from prisons in Tajikistan. Data were collected from interviews (for predictors) and criminal records (for some predictors and main outcomes). Individuals were first interviewed in prison and then followed up over a 1-year period for post-release violent reoffending outcomes. We assessed the predictive performance of OxRec by testing discrimination (area under the receiver operating characteristic curve; AUC) and calibration (calibration statistics and plots). In addition, we calculated sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) for different predetermined risk thresholds.

RESULTS

The cohort included 970 individuals released from prison. During the 12-month follow-up, 144 (15%) were reincarcerated for violent crimes. The original model performed well. The discriminative ability of OxRec Tajikistan was good (AUC = 0.70; 95% CI 0.66-0.75). The calibration plot suggested an underestimation of observed risk probabilities. However, after recalibration, model performance was improved (Brier score = 0.12; calibration in the large was 1.09). At a selected risk threshold of 15%, the tool had a sensitivity of 60%, specificity of 65%, PPV 23% and NPV 90%. In addition, OxRec was feasible to use, despite challenges to risk prediction in LMICs.

CONCLUSION

In an external validation in a LMIC, the OxRec tool demonstrated good performance in multiple measures. OxRec could be used in Tajikistan to help prioritize interventions for people who are at high-risk of violent reoffending after incarceration and screen out others who are at lower risk of violent reoffending. The use of validated risk assessment tools in LMICs could improve risk stratification and inform the development of future interventions tailored at modifiable risk factors for recidivism, such as substance use and mental health problems.

The role of endometrial thickness in predicting ectopic pregnancy after in vitro fertilization and the establishment of a prediction model

OBJECTIVE

To explore the risk factors of ectopic pregnancy after in vitro fertilization.

METHODS

This retrospective cohort study was conducted at the Reproductive Medical Center of the Third Affiliated Hospital of Zhengzhou University from January 2016 to April 2020. Univariate and multivariate analysis were used to analyze the related factors affecting the occurrence of ectopic pregnancy (EP) and to construct a nomographic prediction model for the incidence of ectopic pregnancy.

RESULTS

A total of 12,766 cycles of 10109 patients were included, comprising 214 cases of EP and 12,552 cases of intrauterine pregnancy (IUP). Multivariate logistic regression analysis showed that the tubal factor was associated with a 2-fold increased risk for EP (aOR = 2.72, 95% CI: 1.69-4.39, P < 0.0001). A stratified analysis showed that women with an endometrial thickness (EMT) between 7.6 to 12.1mm (aOR = 0.57, 95%CI: 0.36-0.90, P = 0.0153) and >12.1mm (aOR = 0.42, 95%CI: 0.24-0.74, P = 0.0026) had a significant reduction of the risk of EP compared to women with an EMT of <7.6mm. Compared to cleavage stage transfer, blastocyst transfer can reduce the risk of ectopic pregnancy (aOR = 0.36, 95%CI: 0.26-0.50, P < 0.0001). The saturation model (full mode) establishes a nomographic prediction model with an AUC = 0.68 and a sensitivity and specificity of 0.67and 0.64, respectively. The nomination model was internally verified by self-sampling method (bootstrap sampling resampling times = 500). The resulting AUC = 0.68 (sensitivity: 0.65; specificity: 0.65) showed that the model was relatively stable.

CONCLUSIONS

Our findings indicate that EMT is inversely proportional to the risk of EP. Embryo stage, number of embryos transferred were also significantly associated with EP rate. A simple nomogram for the predicting the risk of EP was established in order to reduce the occurrence of EP.

Prediction of Postoperative Delirium in Geriatric Hip Fracture Patients: A Clinical Prediction Model Using Machine Learning Algorithms

Introduction

Postoperative delirium in geriatric hip fracture patients adversely affects clinical and functional outcomes and increases costs. A preoperative prediction tool to identify high-risk patients may facilitate optimal use of preventive interventions. The purpose of this study was to develop a clinical prediction model using machine learning algorithms for preoperative prediction of postoperative delirium in geriatric hip fracture patients.

Materials & Methods

Geriatric patients undergoing operative hip fracture fixation were queried in the American College of Surgeons National Surgical Quality Improvement Program database (ACS NSQIP) from 2016 through 2019. A total of 28 207 patients were included, of which 8030 (28.5%) developed a postoperative delirium. First, the dataset was randomly split 80:20 into a training and testing subset. Then, a random forest (RF) algorithm was used to identify the variables predictive for a postoperative delirium. The machine learning-model was developed on the training set and the performance was assessed in the testing set. Performance was assessed by discrimination (c-statistic), calibration (slope and intercept), overall performance (Brier-score), and decision curve analysis.

Results

The included variables identified using RF algorithms were (1) age, (2) ASA class, (3) functional status, (4) preoperative dementia, (5) preoperative delirium, and (6) preoperative need for mobility-aid. The clinical prediction model reached good discrimination (c-statistic = .79), almost perfect calibration (intercept = −.01, slope = 1.02), and excellent overall model performance (Brier score = .15). The clinical prediction model was deployed as an open-access web-application: https://sorg-apps.shinyapps.io/hipfxdelirium/.

Discussion & Conclusions

We developed a clinical prediction model that shows promise in estimating the risk of postoperative delirium in geriatric hip fracture patients. The clinical prediction model can play a beneficial role in decision-making for preventative measures for patients at risk of developing a delirium. If found to be externally valid, clinicians might use the available web-based application to help incorporate the model into clinical practice to aid decision-making and optimize preoperative prevention efforts.

Predicting 30-day return hospital admissions in patients with COVID-19 discharged from the emergency department: A national retrospective cohort study

OBJECTIVES

Identification of patients with coronavirus disease 2019 (COVID-19) at risk for deterioration after discharge from the emergency department (ED) remains a clinical challenge. Our objective was to develop a prediction model that identifies patients with COVID-19 at risk for return and hospital admission within 30 days of ED discharge.

METHODS

We performed a retrospective cohort study of discharged adult ED patients (n = 7529) with SARS-CoV-2 infection from 116 unique hospitals contributing to the National Registry of Suspected COVID-19 in Emergency Care. The primary outcome was return hospital admission within 30 days. Models were developed using classification and regression tree (CART), gradient boosted machine (GBM), random forest (RF), and least absolute shrinkage and selection (LASSO) approaches.

RESULTS

Among patients with COVID-19 discharged from the ED on their index encounter, 571 (7.6%) returned for hospital admission within 30 days. The machine-learning (ML) models (GBM, RF, and LASSO) performed similarly. The RF model yielded a test area under the receiver operating characteristic curve of 0.74 (95% confidence interval [CI], 0.71-0.78), with a sensitivity of 0.46 (95% CI, 0.39-0.54) and a specificity of 0.84 (95% CI, 0.82-0.85). Predictive variables, including lowest oxygen saturation, temperature, or history of hypertension, diabetes, hyperlipidemia, or obesity, were common to all ML models.

CONCLUSIONS

A predictive model identifying adult ED patients with COVID-19 at risk for return for return hospital admission within 30 days is feasible. Ensemble/boot-strapped classification methods (eg, GBM, RF, and LASSO) outperform the single-tree CART method. Future efforts may focus on the application of ML models in the hospital setting to optimize the allocation of follow-up resources.

Application of Extreme Learning Machine in the Survival Analysis of Chronic Heart Failure Patients With High Percentage of Censored Survival Time

Objective: To explore the application of the Cox model based on extreme learning machine in the survival analysis of patients with chronic heart failure.

Methods: The medical records of 5,279 inpatients diagnosed with chronic heart failure in two grade 3 and first-class hospitals in Taiyuan from 2014 to 2019 were collected; with death as the outcome and after the feature selection, the Lasso Cox, random survival forest (RSF), and the Cox model based on extreme learning machine (ELM Cox) were constructed for survival analysis and prediction; the prediction performance of the three models was explored based on simulated data with three censoring ratios of 25, 50, and 75%.

Results: Simulation results showed that the prediction performance of the three models decreased with increasing censoring proportion, and the ELM Cox model performed best overall; the ELM Cox model constructed with 21 highly influential survival predictors screened from actual chronic heart failure data showed the best performance with C-index and Integrated Brier Score (IBS) of 0.775(0.755, 0.802) and 0.166(0.150, 0.182), respectively.

Conclusion: The ELM Cox model showed good discrimination performance in the survival analysis of patients with chronic heart failure; it performs consistently for data with a high proportion of censored survival time; therefore, the model could help physicians identify patients at high risk of poor prognosis and target therapeutic measures to patients as early as possible.

Predicting Survived Events in Nontraumatic Out-of-Hospital Cardiac Arrest: A Comparison Study on Machine Learning and Regression Models

BACKGROUND

Prediction of early outcomes of nontraumatic out-of-hospital cardiac arrest (OHCA) by emergency physicians is inaccurate.

OBJECTIVE

Our aim was to develop and validate practical machine learning (ML)-based models to predict early outcomes of nontraumatic OHCA for use in the emergency department (ED). We compared their discrimination and calibration performances with the traditional logistic regression (LR) approach.

METHODS

Between October 1, 2017 and March 31, 2020, prehospital resuscitation was performed on 17,166 OHCA patients. There were 8157 patients 18 years or older with nontraumatic OHCA who received continued resuscitation in the ED included for analysis. Eleven demographic and resuscitation predictor variables were extracted to predict survived events, defined as any sustained return of spontaneous circulation until in-hospital transfer of care. Prediction models based on random forest (RF), multilayer perceptron (MLP), and LR were created with hyperparameter optimization. Model performances on internal and external validation were compared using discrimination and calibration statistics.

RESULTS

The three models showed similar discrimination performances with c-statistics values of 0.712 (95% confidence interval [CI] 0.711-0.713) for LR, 0.714 (95% CI 0.712-0.717) for RF, and 0.712 (95% CI 0.710-0.713) for MLP models on external validation. For calibration, MLP model had a better performance (slope of calibration regression line = 1.10, intercept = -0.09) than LR (slope = 1.17, intercept = -0.11) and RF (slope = 1.16, intercept= -0.10).

CONCLUSIONS

Two practical ML-based and one regression-based clinical prediction models of nontraumatic OHCA for survived events were developed and validated. The ML-based models did not outperform LR in discrimination, but the MLP model showed a better calibration performance.

The Post-Stroke Depression Risk Scale (PoStDeRiS): Development of an Acute-Phase Prediction Model for Depression 6 Months After Stroke

BACKGROUND

Depression after stroke is common but often undertreated as increasing depression prevalence and decreasing health care contacts diverge after the event.

OBJECTIVE

To develop an acute-phase prediction scale for prognosis of depression 6 months after stroke.

METHODS

Participants (N = 226) were consecutively recruited and assessed within the first week after ischemic stroke for history of depression, stroke severity (National Institutes of Health Stroke Scale), and functional independence (Barthel Index). Early depressive symptoms were self-reported via the Patient Health Questionnaire-2 and external-rated by nurses via the Signs of Depression Scale. Six months later, 183 participants were assessed for Diagnostic and Statistical Manual of Mental Disorders, 5th edition diagnosis of depression. Significant predictors of depression were identified in multivariate logistic regression analysis and their coefficients transformed into a risk scale. Measurement precision was identified using receiver operating characteristic curve analysis.

RESULTS

Depression was diagnosed in 32 (17.5%) participants 6 months after stroke. History of depression, the Barthel Index, and the Patient Health Questionnaire-2 were significant predictors of depression. Transformation of the coefficients yielded the Post-Stroke Depression Risk Scale that demonstrated good discrimination (area under the receiver operating characteristic curve = 0.84; 95% confidence interval = 0.78/0.90). The optimum cutoff showed a sensitivity of 0.81, a specificity of 0.72, a positive predictive value of 0.38, and a negative predictive value of 0.95.

CONCLUSIONS

The Post-Stroke Depression Risk Scale accurately identifies people in the acute phase with low risk of depression 6 months later, which saves expendable psychiatric interviews in stroke patients. While the sensitivity indicates that recognition of people with later depression is adequate, positive results in the acute phase show low predictivity. Clinical and methodological reasons for these results as well as implications for future research to increase case-finding ability are discussed.