Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration

External Validation of the International Prognosis Prediction Model of IgA Nephropathy

BACKGROUND

The International IgA Nephropathy (IgAN) Network developed and validated two prognostic prediction models for IgAN, one incorporating a race parameter. These models could anticipate the risk of a 50% reduction in estimated glomerular filtration rate (eGFR) or progression to end-stage renal disease (ESRD) subsequent to an IgAN diagnosis via renal biopsy. This investigation aimed to validate the International IgA Nephropathy Prediction Tool (IIgANPT) within a contemporary Chinese cohort.

METHODS

Within this study,185 patients diagnosed with IgAN via renal biopsy at the Center for Kidney Disease, Second Affiliated Hospital of Nanjing Medical University, between January 2012 and December 2021, were encompassed. Each patient's risk of progression was assessed utilizing the IIgANPT formula. The primary outcome, a 50% decline in eGFR or progression to ESRD, was examined. Two predictive models, one inclusive and the other exclusive of a race parameter, underwent evaluation via receiver-operating characteristic (ROC) curves, subgroup survival analyses, calibration plots, and decision curve analyses.

RESULTS

The median follow-up duration within our cohort spanned 5.1 years, during which 18 patients encountered the primary outcome. The subgroup survival curves exhibited distinct separations, and the comparison of clinical and histological characteristics among the risk subgroups revealed significant differences. Both models demonstrated outstanding discrimination, evidenced by the areas under the ROC curve at five years: 0.882 and 0.878. Whether incorporating the race parameter or not, both prediction models exhibited acceptable calibration. Decision curve analysis affirmed the favorable clinical utility of both models.

CONCLUSIONS

Both prognostic risk evaluation models for IgAN exhibited remarkable discrimination, sound calibration, and acceptable clinical utility.

External validation of the prediction model of intradialytic hypotension: a multicenter prospective cohort study

OBJECTIVE

Intradialytic hypotension (IDH) is a common and serious complication in patients with Maintenance Hemodialysis (MHD). The purpose of this study is to externally verify three IDH risk prediction models recently developed by Ma et al. and recalibrate, update and present the optimal model to improve the accuracy and applicability of the model in clinical environment.

METHODS

A multicenter prospective cohort study of patients from 11 hemodialysis centers in Sichuan Province, China, was conducted using convenience sampling from March 2022 to July 2022, with a follow-up period of 1 month. Model performance was assessed by: (1) Discrimination: Evaluated through the computation of the Area Under Curve (AUC) and its corresponding 95% confidence intervals. (2) Calibration: scrutinized through visual inspection of the calibration plot and utilization of the Brier score. (3) The incremental value of risk prediction and the utility of updating the model were gauged using NRI (Net Reclassification Improvement) and IDI (Integrated Discrimination Improvement). Decision Curve Analysis (DCA) was employed to evaluate the clinical benefit of updating the model.

RESULTS

The final cohort comprised 2235 individuals undergoing maintenance hemodialysis, exhibiting a 14.6% occurrence rate of IDH. The externally validated Area Under the Curve (AUC) values for the three original prediction models were 0.746 (95% CI: 0.718 to 0.775), 0.709 (95% CI: 0.679 to 0.739), and 0.735 (95% CI: 0.706 to 0.764) respectively. Conversely, the AUC value for the recalibrated and updated columnar plot model reached 0.817 (95% CI: 0.791 to 0.842), accompanied by a Brier score of 0.081. Furthermore, Decision Curve Analysis (DCA) exhibited a net benefit within the threshold probability range of 15.2% to 87.1%.

CONCLUSION

Externally validated, recalibrated, updated, and presented IDH prediction models may serve as a valuable instrument for evaluating IDH risk in clinical practice. Furthermore, they hold the potential to guide clinical providers in discerning individuals at risk and facilitating judicious clinical intervention decisions.

A Nomogram Based on Conventional Ultrasound Radiomics for Differentiating Between Radial Scar and Invasive Ductal Carcinoma of the Breast

ABSTRACT

We aimed to develop and validate a nomogram based on conventional ultrasound (CUS) radiomics model to differentiate radial scar (RS) from invasive ductal carcinoma (IDC) of the breast. In total, 208 patients with histopathologically diagnosed RS or IDC of the breast were enrolled. They were randomly divided in a 7:3 ratio into a training cohort (n = 145) and a validation cohort (n = 63). Overall, 1316 radiomics features were extracted from CUS images. Then a radiomics score was constructed by filtering unstable features and using the maximum relevance minimum redundancy algorithm and the least absolute shrinkage and selection operator logistic regression algorithm. Two models were developed using data from the training cohort: one using clinical and CUS characteristics (Clin + CUS model) and one using clinical information, CUS characteristics, and the radiomics score (radiomics model). The usefulness of nomogram was assessed based on their differentiating ability and clinical utility. Nine features from CUS images were used to build the radiomics score. The radiomics nomogram showed a favorable predictive value for differentiating RS from IDC, with areas under the curve of 0.953 and 0.922 for the training and validation cohorts, respectively. Decision curve analysis indicated that this model outperformed the Clin + CUS model and the radiomics score in terms of clinical usefulness. The results of this study may provide a novel method for noninvasively distinguish RS from IDC.

Development, evaluation and validation of machine learning models to predict hospitalizations of patients with coronary artery disease within the next 12 months

BACKGROUND

Improved survival of patients after acute coronary syndromes, population growth, and overall life expectancy rise have led to a significant increase in the proportion of patients with stable coronary artery disease (CAD), creating a significant load on the entire healthcare system. The disease often progresses with the development of many complications while significantly increasing the likelihood of hospitalization. Developing and applying a machine learning model for predicting hospitalizations of patients with CAD to an inpatient medical facility will allow for close monitoring of high-risk patients, early preventive interventions, and optimized medical care.

AIMS

Development and external validation of personalized models for predicting the preventable hospitalizations of patients with stable CAD and its complications using ML algorithms and data of real-world clinical practice.

METHODS

135,873 depersonalized electronic health records of 49,103 patients with stable CAD were included in the study. Anthropometric measurements, physical examination results, laboratory, instrumental, anamnestic, and socio-demographic data, widely used in routine medical practice, were considered as potential predictors, a total of 73 features. Logistic regression, decision tree-based methods including gradient boosting (AdaBoost, LightGBM, XGBoost, CatBoost) and bagging (RandomForest and ExtraTrees), discriminant analysis (LinearDiscriminant, QuadraticDiscriminant), and naive Bayes classifier were compared. External validation was performed on the data of a separate region.

RESULTS

The best results and stability to external validation data were shown by the CatBoost model with an AUC of 0.875 (95% CI 0.865-0.885) for the internal testing and 0.872 (95% CI 0.856-0.886) for the external validation. The best model showed good performance evaluated through AUROC, Brier score and standardized net benefit (for the target NPV threshold) for the validation dataset that was only slightly similar to the train data.

CONCLUSION

The metrics of the best model were superior to previously published studies. The results of external validation demonstrated the relative stability of the model to new data from another region that confirms the possibility of the model's application in real clinical practice.

MAATrica: a measure for assessing consistency and methods in medicinal and nutraceutical chemistry papers

The growing number of scientific papers and document sources underscores the need for methods capable of evaluating the quality of publications. Researchers who are looking for relevant papers for their studies need ways to assess the scientific value of these documents. One approach involves using semantic search engines that can automatically extract important knowledge from the growing body of text. In this study, we introduce a new metric called "MAATrica," which serves as the foundation for an innovative method designed to evaluate research papers. MAATrica offers a new way to analyze and categorize text, focusing on the consistency of research documents in the life sciences, particularly in the fields of medicinal and nutraceutical chemistry. This method utilizes semantic descriptions to cover in silico experiments, as well as in vitro and in vivo essays. Created to aid in evaluation processes like peer review, MAATrica uses toolkits and semantic applications to build the proposed measure, identify scientific entities, and gather information. We have applied MAATrica to roughly 90,000 papers and present our findings here.

An explainable machine learning framework for predicting the risk of buprenorphine treatment discontinuation for opioid use disorder among commercially insured individuals

OBJECTIVES

Buprenorphine is an effective evidence-based medication for opioid use disorder (OUD). Yet premature discontinuation undermines treatment effectiveness, increasing the risk of mortality and overdose. We developed and evaluated a machine learning (ML) framework for predicting buprenorphine care discontinuity within 12 months following treatment initiation.

METHODS

This retrospective study used United States (US) 2018-2021 MarketScan commercial claims data of insured individuals aged 18-64 who initiated buprenorphine between July 2018 and December 2020 with no buprenorphine prescriptions in the previous six months. We measured buprenorphine prescription discontinuation gaps of ≥30 days within 12 months of initiating treatment. We developed predictive models employing logistic regression, decision tree classifier, random forest, extreme gradient boosting, Adaboost, and random forest-extreme gradient boosting ensemble. We applied recursive feature elimination with cross-validation to reduce dimensionality and identify the most predictive features while maintaining model robustness. For model validation, we used several statistics to evaluate performance, such as C-statistics and precision-recall curves. We focused on two distinct treatment stages: at the time of treatment initiation and one and three months after treatment initiation. We employed SHapley Additive exPlanations (SHAP) analysis that helped us explain the contributions of different features in predicting buprenorphine discontinuation. We stratified patients into risk subgroups based on their predicted likelihood of treatment discontinuation, dividing them into decile subgroups. Additionally, we used a calibration plot to analyze the reliability of the models.

RESULTS

A total of 30,373 patients initiated buprenorphine and 14.98% (4551) discontinued treatment. C-statistic varied between 0.56 and 0.76 for the first-stage models including patient-level demographic and clinical variables. Inclusion of proportion of days covered (PDC) measured after one month and three months following treatment initiation significantly increased the models' discriminative power (C-statistics: 0.60 to 0.82). Random forest (C-statistics: 0.76, 0.79 and 0.82 with baseline predictors, one-month PDC and three-months PDC, respectively) outperformed other ML models in discriminative performance in all stages (C-statistics: 0.56 to 0.77). Most influential risk factors of discontinuation included early stage medication adherence, age, and initial days of supply.

CONCLUSION

ML algorithms demonstrated a good discriminative power in identifying patients at higher risk of buprenorphine care discontinuity. The proposed framework may help healthcare providers optimize treatment strategies and deliver targeted interventions to improve buprenorphine care continuity.

Comparison of Simplified Palliative Prognostic Index and Palliative Performance Scale in Patients with Advanced Cancer in a Home Palliative Care Setting

Objective: The Palliative Performance Scale (PPS) has been reported to be as accurate as Palliative Prognostic Index (PPI). PPS is a component of the simplified PPI (sPPI). It is unknown whether PPS is as accurate as sPPI. This study aimed to compare the prognostic performance of the PPS and sPPI in patients with advanced cancer in a home palliative care setting in South Korea. Methods: This was a secondary analysis of a prospective cohort study that included Korean patients with advanced cancer who received home-based palliative care. We used the medical records maintained by specialized palliative care nurses. We computed the prognostic performance of PPS and sPPI using the area under the receiver operating characteristic curve (AUROC) and calibration plots for the 3- and 6-week survival. Results: A total of 80 patients were included, with a median overall survival of 47.0 days. The AUROCs of PPS were 0.71 and 0.69 at the 3- and 6-week survival predictions, respectively. The AUROCs of sPPI were 0.87 and 0.73 at the 3- and 6-week survival predictions, respectively. The calibration plot demonstrated satisfactory agreement across all score ranges for both the PPS and sPPI. Conclusions: This study showed that the sPPI assessed by nurses was more accurate than the PPS in a home palliative care setting in predicting the 3-week survival in patients with advanced cancer. The PPS can be used for a quick assessment.

Computational simulation of the potential improvement in clinical outcomes of cardiovascular diseases with the use of a personalized predictive medicine approach

IMPORTANCE AND OBJECTIVES

The current medical paradigm of evidence-based medicine relies on clinical guidelines derived from randomized clinical trials (RCTs), but these guidelines often overlook individual variations in treatment effects. Approaches have been proposed to develop models predicting the effects of individualized management, such as predictive allocation, individualizing treatment allocation. It is currently unknown whether widespread implementation of predictive allocation could result in better population-level outcomes over guideline-based therapy. We sought to simulate the potential effect of predictive allocation using data from previously conducted RCTs.

METHODS AND RESULTS

Data from 3 RCTs (positive trial, negative trial, trial stopped for futility) in pediatric cardiology were used in a computational simulation study to quantify the potential benefits of a personalized approach based on predictive allocation. Outcomes were compared when using a universal approach vs predictive allocation where each patient was allocated to the treatment associated with the lowest predicted probability of negative outcome. Compared to results from RCTs, predictive allocation yielded absolute risk reductions of 13.8% (95% confidence interval [CI] -1.9 to 29.5), 13.9% (95% CI 4.5-23.2), and 15.6% (95% CI 1.5-29.6), respectively, corresponding to a number needed to treat of 7.3, 7.2, and 6.4. The net benefit of predictive allocation was directly proportional to the performance of the prediction models and disappeared as model performance degraded below an area under the curve of 0.55.

DISCUSSION

These findings highlight that predictive allocation could result in improved group-level outcomes, particularly when highly predictive models are available. These findings will need to be confirmed in simulations of other trials with varying conditions and eventually in RCTs of predictive vs guideline-based treatment allocation.

Comparing penalization methods for linear models on large observational health data

OBJECTIVE

This study evaluates regularization variants in logistic regression (L1, L2, ElasticNet, Adaptive L1, Adaptive ElasticNet, Broken adaptive ridge [BAR], and Iterative hard thresholding [IHT]) for discrimination and calibration performance, focusing on both internal and external validation.

MATERIALS AND METHODS

We use data from 5 US claims and electronic health record databases and develop models for various outcomes in a major depressive disorder patient population. We externally validate all models in the other databases. We use a train-test split of 75%/25% and evaluate performance with discrimination and calibration. Statistical analysis for difference in performance uses Friedman's test and critical difference diagrams.

RESULTS

Of the 840 models we develop, L1 and ElasticNet emerge as superior in both internal and external discrimination, with a notable AUC difference. BAR and IHT show the best internal calibration, without a clear external calibration leader. ElasticNet typically has larger model sizes than L1. Methods like IHT and BAR, while slightly less discriminative, significantly reduce model complexity.

CONCLUSION

L1 and ElasticNet offer the best discriminative performance in logistic regression for healthcare predictions, maintaining robustness across validations. For simpler, more interpretable models, L0-based methods (IHT and BAR) are advantageous, providing greater parsimony and calibration with fewer features. This study aids in selecting suitable regularization techniques for healthcare prediction models, balancing performance, complexity, and interpretability.

Predictive modeling of ambulatory outcomes after spinal cord injury using machine learning

STUDY DESIGN

Retrospective multi-site cohort study.

OBJECTIVES

To develop an accurate machine learning predictive model using predictor variables from the acute rehabilitation period to determine ambulatory status in spinal cord injury (SCI) one year post injury.

SETTING

Model SCI System (SCIMS) database between January 2000 and May 2019.

METHODS

Retrospective cohort study using data that were previously collected as part of the SCI Model System (SCIMS) database. A total of 4523 patients were analyzed comparing traditional models (van Middendorp and Hicks) compared to machine learning algorithms including Elastic Net Penalized Logistic Regression (ENPLR), Gradient Boosted Machine (GBM), and Artificial Neural Networks (ANN).

RESULTS

Compared with GBM and ANN, ENPLR was determined to be the preferred model based on predictive accuracy metrics, calibration, and variable selection. The primary metric to judge discrimination was the area under the receiver operating characteristic curve (AUC). When compared to the van Middendorp all patients (0.916), ASIA A and D (0.951) and ASIA B and C (0.775) and Hicks all patients (0.89), ASIA A and D (0.934) and ASIA B and C (0.775), ENPLR demonstrated improved AUC for all patients (0.931), ASIA A and D (0.965) ASIA B and C (0.803).

CONCLUSIONS

Utilizing artificial intelligence and machine learning methods are feasible for accurately classifying outcomes in SCI and may provide improved sensitivity in identifying which individuals are less likely to ambulate and may benefit from augmentative strategies, such as neuromodulation. Future directions should include the use of additional variables to further refine these models.

Systematic Review and Meta-Analysis of Prehospital Machine Learning Scores as Screening Tools for Early Detection of Large Vessel Occlusion in Patients With Suspected Stroke

BACKGROUND

Enhanced detection of large vessel occlusion (LVO) through machine learning (ML) for acute ischemic stroke appears promising. This systematic review explored the capabilities of ML models compared with prehospital stroke scales for LVO prediction.

METHODS AND RESULTS

Six bibliographic databases were searched from inception until October 10, 2023. Meta-analyses pooled the model performance using area under the curve (AUC), sensitivity, specificity, and summary receiver operating characteristic curve. Of 1544 studies screened, 8 retrospective studies were eligible, including 32 prehospital stroke scales and 21 ML models. Of the 9 prehospital scales meta-analyzed, the Rapid Arterial Occlusion Evaluation had the highest pooled AUC (0.82 [95% CI, 0.79-0.84]). Support Vector Machine achieved the highest AUC of 9 ML models included (pooled AUC, 0.89 [95% CI, 0.88-0.89]). Six prehospital stroke scales and 10 ML models were eligible for summary receiver operating characteristic analysis. Pooled sensitivity and specificity for any prehospital stroke scale were 0.72 (95% CI, 0.68-0.75) and 0.77 (95% CI, 0.72-0.81), respectively; summary receiver operating characteristic curve AUC was 0.80 (95% CI, 0.76-0.83). Pooled sensitivity for any ML model for LVO was 0.73 (95% CI, 0.64-0.79), specificity was 0.85 (95% CI, 0.80-0.89), and summary receiver operating characteristic curve AUC was 0.87 (95% CI, 0.83-0.89).

CONCLUSIONS

Both prehospital stroke scales and ML models demonstrated varying accuracies in predicting LVO. Despite ML potential for improved LVO detection in the prehospital setting, application remains limited by the absence of prospective external validation, limited sample sizes, and lack of real-world performance data in a prehospital setting.

Development of a predictive nomogram for in-hospital death risk in multimorbid patients with hepatocellular carcinoma undergoing Palliative Locoregional Therapy

Patients diagnosed with hepatocellular carcinoma (HCC) often present with multimorbidity, significantly contributing to adverse outcomes, particularly in-hospital mortality. This study aimed to develop a predictive nomogram to assess the impact of comorbidities on in-hospital mortality risk in HCC patients undergoing palliative locoregional therapy. We retrospectively analyzed data from 345 hospitalized HCC patients who underwent palliative locoregional therapy between January 2015 and December 2022. The nomogram was constructed using independent risk factors such as length of stay (LOS), hepatitis B virus (HBV) infection, hypertension, chronic obstructive pulmonary disease (COPD), anemia, thrombocytopenia, liver cirrhosis, hepatic encephalopathy (HE), N stage, and microvascular invasion. The model demonstrated high predictive accuracy with an AUC of 0.908 (95% CI: 0.859-0.956) for the overall dataset, 0.926 (95% CI: 0.883-0.968) for the training set, and 0.862 (95% CI: 0.728-0.994) for the validation set. Calibration curves indicated a strong correlation between predicted and observed outcomes, validated by statistical tests. Decision curve analysis (DCA) and clinical impact curves (CIC) confirmed the model's clinical utility in predicting in-hospital mortality. This nomogram offers a practical tool for personalized risk assessment in HCC patients undergoing palliative locoregional therapy, facilitating informed clinical decision-making and improving patient management.

Machine learning evaluation of immune infiltrate through digital tumour score allows prediction of survival outcome in a pooled analysis of three international stage III colon cancer cohorts

BACKGROUND

T-cell immune infiltrates are robust prognostic variables in localised colon cancer. Evaluation of prognosis using artificial intelligence is an emerging field. We evaluated whether machine learning analysis improved prediction of patient outcome in comparison with analysis of T cell infiltrate only or in association with clinical variables.

METHODS

We used data from two phase III clinical trials (Prodige-13 and PETACC08) and one retrospective Italian cohort (HARMONY). Cohorts were split into training (N = 692), internal validation (N = 297) and external validation (N = 672) sets. Tumour slides were stained with CD3mAb. CD3 Machine Learning (CD3ML) score was computed using graphical parameters within the tumour tiles obtained from CD3 slides. CD3 infiltrates in tumour core and invasive margin were automatically detected. Associations of CD3 infiltrates and CD3ML with 5-year Disease-Free Survival (DFS) were examined using univariate and multivariable survival models by Cox regression.

FINDINGS

CD3 density both in the invasive margin and the tumour core were significantly associated with DFS in the different sets. Similarly, CD3ML score was significantly associated with DFS in all sets. CD3 assessment did not provide added value on top of CD3ML assessment (Likelihood Ratio Test (LRT), p = 0.13). In contrast, CD3ML improved prediction of DFS when combined with a clinical risk stage (LRT, p = 0.001). Stratified by clinical risk score (High or Low), patients with low CD3ML score had better DFS.

INTERPRETATION

In all tested sets, machine learning analysis of tumour cells improved prediction of prognosis compared to clinical parameters. Adding tumour-infiltrating lymphocytes assessment did not improve prognostic determination.

FUNDING

This research received no external funding.

Development of a diagnostic nomogram for alpha-fetoprotein-negative hepatocellular carcinoma based on serological biomarkers

BACKGROUND

Hepatocellular carcinoma (HCC) is the third leading cause of cancer-related death worldwide. Serum biomarkers play an important role in the early diagnosis and prognosis of HCC. Because a certain percentage of HCC patients are negative for alpha-fetoprotein (AFP), the diagnosis of AFP-negative HCC is essential to improve the detection rate of HCC.

AIM

To establish an effective model for diagnosing AFP-negative HCC based on serum tumour biomarkers.

METHODS

A total of 180 HCC patients were enrolled in this study. The expression levels of GP73, des-γ-carboxyprothrombin (DCP), CK18-M65, and CK18-M30 were detected by a fully automated chemiluminescence analyser. The variables were selected by logistic regression analysis. Several models were constructed using stepwise backward logistic regression. The performance of the models was compared using the C statistic, integrated discrimination improvement, net reclassification improvement, and calibration curves. The clinical utility of the nomogram was assessed using decision curve analysis (DCA).

RESULTS

The results showed that the expression levels of GP73, DCP, CK18-M65, and CK18-M30 were significantly greater in AFP-negative HCC patients than in healthy controls (P < 0.001). Multivariate logistic regression analysis revealed that GP73, DCP, and CK18-M65 were independent factors for diagnosing AFP-negative HCC. By comparing the diagnostic performance of multiple models, we included GP73 and CK18-M65 as the model variables, and the model had good discrimination ability (area under the curve = 0.946) and good goodness of fit. The DCA curves indicated the good clinical utility of the nomogram.

CONCLUSION

Our study identified GP73 and CK18-M65 as serum biomarkers with certain application value in the diagnosis of AFP-negative HCC. The diagnostic nomogram based on CK18-M65 combined with GP73 demonstrated good performance and effectively identified high-risk groups of patients with HCC.

Development and validation of a prediction score to assess the risk of depression in primary care

BACKGROUND

Major depression is the most frequent psychiatric disorder and primary care is a crucial setting for its early recognition. This study aimed to develop and validate the DEP-HScore as a tool to predict depression risk in primary care and increase awareness and investigation of this condition among General Practitioners (GPs).

METHODS

The DEP-HScore was developed using data from the Italian Health Search Database (HSD). A cohort of 903,748 patients aged 18 years or older was selected and followed until the occurrence of depression, death or end of data availability (December 2019). Demographics, somatic signs/symptoms and psychiatric/medical comorbidities were entered in a multivariate Cox regression to predict the occurrence of depression. The coefficients formed the DEP-HScore for individual patients. Explained variance (pseudo-R2), discrimination (AUC) and calibration (slope estimating predicted-observed risk relationship) assessed the prediction accuracy.

RESULTS

The DEP-HScore explained 18.1 % of the variation in occurrence of depression and the discrimination value was equal to 67 %. With an event horizon of three months, the slope and intercept were not significantly different from the ideal calibration.

LIMITATIONS

The DEP-HScore has not been tested in other settings. Furthermore, the model was characterized by limited calibration performance when the risk of depression was estimated at the 1-year follow-up.

CONCLUSIONS

The DEP-HScore is reliable tool that could be implemented in primary care settings to evaluate the risk of depression, thus enabling prompt and suitable investigations to verify the presence of this condition.

Development of a diagnostic nomogram for alpha-fetoprotein-negative hepatocellular carcinoma based on serological biomarkers

BACKGROUND

Hepatocellular carcinoma (HCC) is the third leading cause of cancer-related death worldwide. Serum biomarkers play an important role in the early diagnosis and prognosis of HCC. Because a certain percentage of HCC patients are negative for alpha-fetoprotein (AFP), the diagnosis of AFP-negative HCC is essential to improve the detection rate of HCC.

AIM

To establish an effective model for diagnosing AFP-negative HCC based on serum tumour biomarkers.

METHODS

A total of 180 HCC patients were enrolled in this study. The expression levels of GP73, des-γ-carboxyprothrombin (DCP), CK18-M65, and CK18-M30 were detected by a fully automated chemiluminescence analyser. The variables were selected by logistic regression analysis. Several models were constructed using stepwise backward logistic regression. The performance of the models was compared using the C statistic, integrated discrimination improvement, net reclassification improvement, and calibration curves. The clinical utility of the nomogram was assessed using decision curve analysis (DCA).

RESULTS

The results showed that the expression levels of GP73, DCP, CK18-M65, and CK18-M30 were significantly greater in AFP-negative HCC patients than in healthy controls (P < 0.001). Multivariate logistic regression analysis revealed that GP73, DCP, and CK18-M65 were independent factors for diagnosing AFP-negative HCC. By comparing the diagnostic performance of multiple models, we included GP73 and CK18-M65 as the model variables, and the model had good discrimination ability (area under the curve = 0.946) and good goodness of fit. The DCA curves indicated the good clinical utility of the nomogram.

CONCLUSION

Our study identified GP73 and CK18-M65 as serum biomarkers with certain application value in the diagnosis of AFP-negative HCC. The diagnostic nomogram based on CK18-M65 combined with GP73 demonstrated good performance and effectively identified high-risk groups of patients with HCC.

Equations to estimate human milk intake in infants aged 7 to 10 months: prediction models from a cross-sectional study

BACKGROUND

Obtaining valid estimates of nutrient intake in infants is currently limited by the difficulties of accurately measuring human milk intake. Current methods are either unsuitable for large-scale studies (i.e., the gold standard dose-to-mother stable isotope technique) or use set amounts, regardless of known variability in individual intake.

OBJECTIVES

This cross-sectional study aimed to develop equations to predict human milk intake using simple measures and to carry out external validation of existing methods against the gold standard technique.

METHODS

Data on human milk intake were obtained using the dose-to-mother stable isotope technique in 157 infants aged 7-10 mo and their mothers. Predictive equations were developed using questionnaire and anthropometric data (Model 1) and additional dietary data (Model 2) using lasso regression. Bland-Altman plots and intraclass correlation coefficients (ICC) also assessed the validity of existing methods (FITS and ALSPAC studies).

RESULTS

The strongest univariate predictors of human milk intake in infants of 8.3 mo on average (46% female) were infant age, infant body mass index (BMI), number of breastfeeds a day, infant formula consumption, and energy from complementary food intake. Mean [95% confidence interval (CI)] differences in predicted versus measured human milk intake [mean (SD): 762 (257) mL/day] were 0.0 mL/day (-26, 26) for Model 1 (ICC 0.74) and 0.5 mL/day (-21, 22) for Model 2 (ICC 0.83). Corresponding differences were -197 mL/day (-233, -161; ICC 0.32) and -175 mL/day (-216, -134; ICC 0.41) for the methods used by FITS and ALSPAC, respectively.

CONCLUSIONS

The Human Milk Intake Level Calculation provides substantial improvements on existing methods to estimate human milk intake in infants aged 7-10 mo, while utilizing data commonly collected in nutrition surveys. Although further validation in an external sample is recommended, these equations can be used to estimate human milk intake at this age with some confidence. This clinical trial was registered at http://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=379436) as ACTRN12620000459921.

Development and validation of an interpretable model for predicting sepsis mortality across care settings

There are numerous prognostic predictive models for evaluating mortality risk, but current scoring models might not fully cater to sepsis patients' needs. This study developed and validated a new model for sepsis patients that is suitable for any care setting and accurately forecasts 28-day mortality. The derivation dataset, gathered from 20 hospitals between September 2019 and December 2021, contrasted with the validation dataset, collected from 15 hospitals from January 2022 to December 2022. In this study, 7436 patients were classified as members of the derivation dataset, and 2284 patients were classified as members of the validation dataset. The point system model emerged as the optimal model among the tested predictive models for foreseeing sepsis mortality. For community-acquired sepsis, the model's performance was satisfactory (derivation dataset AUC: 0.779, 95% CI 0.765-0.792; validation dataset AUC: 0.787, 95% CI 0.765-0.810). Similarly, for hospital-acquired sepsis, it performed well (derivation dataset AUC: 0.768, 95% CI 0.748-0.788; validation dataset AUC: 0.729, 95% CI 0.687-0.770). The calculator, accessible at https://avonlea76.shinyapps.io/shiny_app_up/ , is user-friendly and compatible. The new predictive model of sepsis mortality is user-friendly and satisfactorily forecasts 28-day mortality. Its versatility lies in its applicability to all patients, encompassing both community-acquired and hospital-acquired sepsis.

Association of weight-adjusted waist index with all-cause mortality among non-Asian individuals: a national population-based cohort study

INTRODUCTION

The Weight-Adjusted Waist Index (WWI) is a new indicator of obesity that is associated with all-cause mortality in Asian populations. Our study aimed to investigate the linear and non-linear associations between WWI and all-cause mortality in non-Asian populations in the United States, and whether WWI was superior to traditional obesity indicators as a predictor of all-cause mortality.

METHODS

We conducted a cohort study using data from the 2011-2018 National Health and Nutrition Examination Survey (NHANES), involving 18,592 participants. We utilized Cox proportional hazard models to assess the association between WWI, BMI, WC, and the risk of all-cause mortality, and performed subgroup analyses and interaction tests. We also employed a receiver operating characteristics (ROC) curve study to evaluate the effectiveness of WWI, BMI, and WC in predicting all-cause mortality.

RESULTS

After adjusting for confounders, WWI, BMI, and WC were positively associated with all-cause mortality. The performance of WWI, BMI, and WC in predicting all-cause mortality yielded AUCs of 0.697, 0.524, and 0.562, respectively. The data also revealed a U-shaped relationship between WWI and all-cause mortality. Race and cancer modified the relationship between WWI and all-cause mortality, with the relationship being negatively correlated in African Americans and cancer patients.

CONCLUSIONS

In non-Asian populations in the United States, there is a U-shaped relationship between WWI and all-cause mortality, and WWI outperforms BMI and WC as a predictor of all-cause mortality. These findings may contribute to a better understanding and prediction of the relationship between obesity and mortality, and provide support for effective obesity management strategies.

Development and validation of a LASSO prediction model for cisplatin induced nephrotoxicity: a case-control study in China

BACKGROUND

Early identification of high-risk individuals with cisplatin-induced nephrotoxicity (CIN) is crucial for avoiding CIN and improving prognosis. In this study, we developed and validated a CIN prediction model based on general clinical data, laboratory indications, and genetic features of lung cancer patients before chemotherapy.

METHODS

We retrospectively included 696 lung cancer patients using platinum chemotherapy regimens from June 2019 to June 2021 as the traing set to construct a predictive model using Absolute shrinkage and selection operator (LASSO) regression, cross validation, and Akaike's information criterion (AIC) to select important variables. We prospectively selected 283 independent lung cancer patients from July 2021 to December 2022 as the test set to evaluate the model's performance.

RESULTS

The prediction model showed good discrimination and calibration, with AUCs of 0.9217 and 0.8288, sensitivity of 79.89% and 45.07%, specificity of 94.48% and 94.81%, in the training and test sets respectively. Clinical decision curve analysis suggested that the model has value for clinical use when the risk threshold ranges between 0.1 and 0.9. Precision-Recall (PR) curve shown in recall interval from 0.5 to 0.75: precision gradually declines with increasing Recall, up to 0.9.

CONCLUSIONS

Predictive models based on laboratory and demographic variables can serve as a beneficial complementary tool for identifying high-risk populations with CIN.

Data set terminology of deep learning in medicine: a historical review and recommendation

Medicine and deep learning-based artificial intelligence (AI) engineering represent two distinct fields each with decades of published history. The current rapid convergence of deep learning and medicine has led to significant advancements, yet it has also introduced ambiguity regarding data set terms common to both fields, potentially leading to miscommunication and methodological discrepancies. This narrative review aims to give historical context for these terms, accentuate the importance of clarity when these terms are used in medical deep learning contexts, and offer solutions to mitigate misunderstandings by readers from either field. Through an examination of historical documents, including articles, writing guidelines, and textbooks, this review traces the divergent evolution of terms for data sets and their impact. Initially, the discordant interpretations of the word 'validation' in medical and AI contexts are explored. We then show that in the medical field as well, terms traditionally used in the deep learning domain are becoming more common, with the data for creating models referred to as the 'training set', the data for tuning of parameters referred to as the 'validation (or tuning) set', and the data for the evaluation of models as the 'test set'. Additionally, the test sets used for model evaluation are classified into internal (random splitting, cross-validation, and leave-one-out) sets and external (temporal and geographic) sets. This review then identifies often misunderstood terms and proposes pragmatic solutions to mitigate terminological confusion in the field of deep learning in medicine. We support the accurate and standardized description of these data sets and the explicit definition of data set splitting terminologies in each publication. These are crucial methods for demonstrating the robustness and generalizability of deep learning applications in medicine. This review aspires to enhance the precision of communication, thereby fostering more effective and transparent research methodologies in this interdisciplinary field.

Accurate prediction of neurologic changes in critically ill infants using pose AI

Infant alertness and neurologic changes can reflect life-threatening pathology but are assessed by exam, which can be intermittent and subjective. Reliable, continuous methods are needed. We hypothesized that our computer vision method to track movement, pose AI, could predict neurologic changes in the neonatal intensive care unit (NICU). We collected 4,705 hours of video linked to electroencephalograms (EEG) from 115 infants. We trained a deep learning pose algorithm that accurately predicted anatomic landmarks in three evaluation sets (ROC-AUCs 0.83-0.94), showing feasibility of applying pose AI in an ICU. We then trained classifiers on landmarks from pose AI and observed high performance for sedation (ROC-AUCs 0.87-0.91) and cerebral dysfunction (ROC-AUCs 0.76-0.91), demonstrating that an EEG diagnosis can be predicted from video data alone. Taken together, deep learning with pose AI may offer a scalable, minimally invasive method for neuro-telemetry in the NICU.

Prediction models for amputation after diabetic foot: systematic review and critical appraisal

BACKGROUND

Numerous studies have developed or validated prediction models aimed at estimating the likelihood of amputation in diabetic foot (DF) patients. However, the quality and applicability of these models in clinical practice and future research remain uncertain. This study conducts a systematic review and assessment of the risk of bias and applicability of amputation prediction models among individuals with DF.

METHODS

A comprehensive search was conducted across multiple databases, including PubMed, Web of Science, EBSCO CINAHL Plus, Embase, Cochrane Library, China National Knowledge Infrastructure (CNKI), Wanfang, Chinese Biomedical Literature Database (CBM), and Weipu (VIP) from their inception to December 24, 2023. Two investigators independently screened the literature and extracted data using the checklist for critical appraisal and data extraction for systematic reviews of prediction modeling studies. The Prediction Model Risk of Bias Assessment Tool (PROBAST) checklist was employed to evaluate both the risk of bias and applicability.

RESULTS

A total of 20 studies were included in this analysis, comprising 17 development studies and three validation studies, encompassing 20 prediction models and 11 classification systems. The incidence of amputation in patients with DF ranged from 5.9 to 58.5%. Machine learning-based methods were employed in more than half of the studies. The reported area under the curve (AUC) varied from 0.560 to 0.939. Independent predictors consistently identified by multivariate models included age, gender, HbA1c, hemoglobin, white blood cell count, low-density lipoprotein cholesterol, diabetes duration, and Wagner's Classification. All studies were found to exhibit a high risk of bias, primarily attributed to inadequate handling of outcome events and missing data, lack of model performance assessment, and overfitting.

CONCLUSIONS

The assessment using PROBAST revealed a notable risk of bias in the existing prediction models for amputation in patients with DF. It is imperative for future studies to concentrate on enhancing the robustness of current prediction models or constructing new models with stringent methodologies.

Brain metastasis risk prediction model in females with hormone receptor-positive breast cancer

BACKGROUND

Breast cancer is a leading cause of cancer-related deaths in females, and the hormone receptor-positive subtype is the most frequent. Breast cancer is a common source of brain metastases; therefore, we aimed to generate a brain metastases prediction model in females with hormone receptor-positive breast cancer.

METHODS

The primary cohort included 3,682 females with hormone receptor-positive breast cancer treated at a single center from May 2009 to May 2020. Patients were randomly divided into a training dataset (n = 2,455) and a validation dataset (n = 1,227). In the training dataset, simple logistic regression analyses were used to measure associations between variables and the diagnosis of brain metastases and to build multivariable models. The model with better calibration and discrimination capacity was tested in the validation dataset to measure its predictive performance.

RESULTS

The variables incorporated in the model included age, tumor size, axillary lymph node status, clinical stage at diagnosis, HER2 expression, Ki-67 proliferation index, and the modified Scarff-Bloom-Richardson grade. The area under the curve was 0.81 (95 % CI 0.75-0.86), p < 0.001 in the validation dataset. The study presents a guide for the clinical use of the model.

CONCLUSION

A brain metastases prediction model in females with hormone receptor-positive breast cancer helps assess the individual risk of brain metastases.

Development of machine learning models to predict perioperative blood transfusion in hip surgery

BACKGROUND

Allogeneic Blood transfusion is common in hip surgery but is associated with increased morbidity. Accurate prediction of transfusion risk is necessary for minimizing blood product waste and preoperative decision-making. The study aimed to develop machine learning models for predicting perioperative blood transfusion in hip surgery and identify significant risk factors.

METHODS

Data of patients undergoing hip surgery between January 2013 and October 2021 in the Peking Union Medical College Hospital were collected to train and test predictive models. The primary outcome was perioperative red blood cell (RBC) transfusion within 72 h of surgery. Fourteen machine learning algorithms were established to predict blood transfusion risk incorporating patient demographic characteristics, preoperative laboratory tests, and surgical information. Discrimination, calibration, and decision curve analysis were used to evaluate machine learning models. SHapley Additive exPlanations (SHAP) was performed to interpret models.

RESULTS

In this study, 2431 hip surgeries were included. The Ridge Classifier performed the best with an AUC = 0.85 (95% CI, 0.81 to 0.88) and a Brier score = 0.21. Patient-related risk factors included lower preoperative hemoglobin, American Society of Anesthesiologists (ASA) Physical Status > 2, anemia, lower preoperative fibrinogen, and lower preoperative albumin. Surgery-related risk factors included longer operation time, total hip arthroplasty, and autotransfusion.

CONCLUSIONS

The machine learning model developed in this study achieved high predictive performance using available variables for perioperative blood transfusion in hip surgery. The predictors identified could be helpful for risk stratification, preoperative optimization, and outcomes improvement.

Radiomics workflow definition & challenges - German priority program 2177 consensus statement on clinically applied radiomics

OBJECTIVES

Achieving a consensus on a definition for different aspects of radiomics workflows to support their translation into clinical usage. Furthermore, to assess the perspective of experts on important challenges for a successful clinical workflow implementation.

MATERIALS AND METHODS

The consensus was achieved by a multi-stage process. Stage 1 comprised a definition screening, a retrospective analysis with semantic mapping of terms found in 22 workflow definitions, and the compilation of an initial baseline definition. Stages 2 and 3 consisted of a Delphi process with over 45 experts hailing from sites participating in the German Research Foundation (DFG) Priority Program 2177. Stage 2 aimed to achieve a broad consensus for a definition proposal, while stage 3 identified the importance of translational challenges.

RESULTS

Workflow definitions from 22 publications (published 2012-2020) were analyzed. Sixty-nine definition terms were extracted, mapped, and semantic ambiguities (e.g., homonymous and synonymous terms) were identified and resolved. The consensus definition was developed via a Delphi process. The final definition comprising seven phases and 37 aspects reached a high overall consensus (> 89% of experts "agree" or "strongly agree"). Two aspects reached no strong consensus. In addition, the Delphi process identified and characterized from the participating experts' perspective the ten most important challenges in radiomics workflows.

CONCLUSION

To overcome semantic inconsistencies between existing definitions and offer a well-defined, broad, referenceable terminology, a consensus workflow definition for radiomics-based setups and a terms mapping to existing literature was compiled. Moreover, the most relevant challenges towards clinical application were characterized.

CRITICAL RELEVANCE STATEMENT

Lack of standardization represents one major obstacle to successful clinical translation of radiomics. Here, we report a consensus workflow definition on different aspects of radiomics studies and highlight important challenges to advance the clinical adoption of radiomics.

KEY POINTS

Published radiomics workflow terminologies are inconsistent, hindering standardization and translation. A consensus radiomics workflow definition proposal with high agreement was developed. Publicly available result resources for further exploitation by the scientific community.

LORIS robustly predicts patient outcomes with immune checkpoint blockade therapy using common clinical, pathologic and genomic features

Despite the revolutionary impact of immune checkpoint blockade (ICB) in cancer treatment, accurately predicting patient responses remains challenging. Here, we analyzed a large dataset of 2,881 ICB-treated and 841 non-ICB-treated patients across 18 solid tumor types, encompassing a wide range of clinical, pathologic and genomic features. We developed a clinical score called LORIS (logistic regression-based immunotherapy-response score) using a six-feature logistic regression model. LORIS outperforms previous signatures in predicting ICB response and identifying responsive patients even with low tumor mutational burden or programmed cell death 1 ligand 1 expression. LORIS consistently predicts patient objective response and short-term and long-term survival across most cancer types. Moreover, LORIS showcases a near-monotonic relationship with ICB response probability and patient survival, enabling precise patient stratification. As an accurate, interpretable method using a few readily measurable features, LORIS may help improve clinical decision-making in precision medicine to maximize patient benefit. LORIS is available as an online tool at https://loris.ccr.cancer.gov/ .

Machine learning improves prediction of postoperative outcomes after gastrointestinal surgery: a systematic review and meta-analysis

BACKGROUND

Machine learning (ML) approaches have become increasingly popular in predicting surgical outcomes. However, it is unknown whether they are superior to traditional statistical methods such as logistic regression (LR). This study aimed to perform a systematic review and meta-analysis to compare the performance of ML vs LR models in predicting postoperative outcomes for patients undergoing gastrointestinal (GI) surgery.

METHODS

A systematic search of Embase, MEDLINE, Cochrane, Web of Science, and Google Scholar was performed through December 2022. The primary outcome was the discriminatory performance of ML vs LR models as measured by the area under the receiver operating characteristic curve (AUC). A meta-analysis was then performed using a random effects model.

RESULTS

A total of 62 LR models and 143 ML models were included across 38 studies. On average, the best-performing ML models had a significantly higher AUC than the LR models (ΔAUC, 0.07; 95% CI, 0.04-0.09; P < .001). Similarly, on average, the best-performing ML models had a significantly higher logit (AUC) than the LR models (Δlogit [AUC], 0.41; 95% CI, 0.23-0.58; P < .001). Approximately half of studies (44%) were found to have a low risk of bias. Upon a subset analysis of only low-risk studies, the difference in logit (AUC) remained significant (ML vs LR, Δlogit [AUC], 0.40; 95% CI, 0.14-0.66; P = .009).

CONCLUSION

We found a significant improvement in discriminatory ability when using ML over LR algorithms in predicting postoperative outcomes for patients undergoing GI surgery. Subsequent efforts should establish standardized protocols for both developing and reporting studies using ML models and explore the practical implementation of these models.

Development and external validation of preoperative clinical prediction models for postoperative outcomes including preoperative aerobic fitness in patients approaching elective colorectal cancer surgery

INTRODUCTION

Preoperative aerobic fitness is associated with postoperative outcomes after elective colorectal cancer (CRC) surgery. This study aimed to develop and externally validate two clinical prediction models incorporating a practical test to assess preoperative aerobic fitness to distinguish between patients with and without an increased risk for 1) postoperative complications and 2) a prolonged time to in-hospital recovery of physical functioning after elective colorectal cancer (CRC) surgery.

MATERIALS AND METHODS

Models were developed using prospective data from 256 patients and externally validated using prospective data of 291 patients. Postoperative complications were classified according to Clavien-Dindo. The modified Iowa level of assistance scale (mILAS) was used to determine time to postoperative in-hospital physical recovery. Aerobic fitness, age, sex, body mass index, American Society of Anesthesiologists (ASA) classification, neoadjuvant treatment, surgical approach, tumour location, and preoperative haemoglobin level were potential predictors. Areas under the curve (AUC), calibration plots, and Hosmer-Lemeshow tests evaluated predictive performance.

RESULTS

Aerobic fitness, sex, age, ASA, tumour location, and surgical approach were included in the final models. External validation of the model for complications and postoperative recovery presented moderate to fair discrimination (AUC 0.666 (0.598-0.733) and 0.722 (0.651-0.794), respectively) and good calibration. High sensitivity and high negative predictive values were observed in the lower predicted risk categories (<40 %).

CONCLUSION

Both models identify patients with and without an increased risk of complications or a prolonged time to in-hospital physical recovery. They might be used for improving patient-tailored preoperative risk assessment and targeted and cost-effective application of prehabilitation interventions.

A brief step-by-step guide on conducting a systematic review and meta-analysis of prognostic model studies

Prognostic models provide an avenue to predict the risk of individual patients and support shared-decision making. Many prognostic models are published annually, and systematic reviews provide an avenue to collate the existing evidence behind prognostic models to determine whether a model demonstrates adequate predictive performance and is ready for real-world use. This article provides a brief step-by-step guide on how to conduct a systematic review and meta-analysis of prognostic model studies and how these reviews differ from systematic reviews of therapy and diagnosis.

Radiomics for the identification of extraprostatic extension with prostate MRI: a systematic review and meta-analysis

OBJECTIVES

Extraprostatic extension (EPE) of prostate cancer (PCa) is predicted using clinical nomograms. Incorporating MRI could represent a leap forward, although poor sensitivity and standardization represent unsolved issues. MRI radiomics has been proposed for EPE prediction. The aim of the study was to systematically review the literature and perform a meta-analysis of MRI-based radiomics approaches for EPE prediction.

MATERIALS AND METHODS

Multiple databases were systematically searched for radiomics studies on EPE detection up to June 2022. Methodological quality was appraised according to Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) tool and radiomics quality score (RQS). The area under the receiver operating characteristic curves (AUC) was pooled to estimate predictive accuracy. A random-effects model estimated overall effect size. Statistical heterogeneity was assessed with I2 value. Publication bias was evaluated with a funnel plot. Subgroup analyses were performed to explore heterogeneity.

RESULTS

Thirteen studies were included, showing limitations in study design and methodological quality (median RQS 10/36), with high statistical heterogeneity. Pooled AUC for EPE identification was 0.80. In subgroup analysis, test-set and cross-validation-based studies had pooled AUC of 0.85 and 0.89 respectively. Pooled AUC was 0.72 for deep learning (DL)-based and 0.82 for handcrafted radiomics studies and 0.79 and 0.83 for studies with multiple and single scanner data, respectively. Finally, models with the best predictive performance obtained using radiomics features showed pooled AUC of 0.82, while those including clinical data of 0.76.

CONCLUSION

MRI radiomics-powered models to identify EPE in PCa showed a promising predictive performance overall. However, methodologically robust, clinically driven research evaluating their diagnostic and therapeutic impact is still needed.

CLINICAL RELEVANCE STATEMENT

Radiomics might improve the management of prostate cancer patients increasing the value of MRI in the assessment of extraprostatic extension. However, it is imperative that forthcoming research prioritizes confirmation studies and a stronger clinical orientation to solidify these advancements.

KEY POINTS

• MRI radiomics deserves attention as a tool to overcome the limitations of MRI in prostate cancer local staging. • Pooled AUC was 0.80 for the 13 included studies, with high heterogeneity (84.7%, p < .001), methodological issues, and poor clinical orientation. • Methodologically robust radiomics research needs to focus on increasing MRI sensitivity and bringing added value to clinical nomograms at patient level.

External validation of the CholeS conversion from laparoscopic to open cholecystectomy (CLOC) risk score in Aotearoa New Zealand: a validation study

BACKGROUND

Conversion of laparoscopic cholecystectomy to open is uncommon, but is associated with longer hospital stay and recovery. Prognosticating conversion may aid service planning and provision. We therefore aimed to assess the external validity of the largest risk score for operative conversion.

METHODS

CHOLENZ was a multicentre, prospective, national cohort study of cholecystectomy for benign biliary disease conducted by STRATA, a trainee-led collaborative network. Data were collected from patients undergoing cholecystectomy in New Zealand hospitals between 1 August and 30 October 2021 with 30-day follow-up. The Conversion from Laparoscopic to Open Cholecystectomy (CLOC) score from the CholeS study was assessed for external validity by interrogating its accuracy and calibration in the CHOLENZ dataset.

RESULTS

Of 1162 cholecystectomies started laparoscopically, 20 (1.7%) were converted to open in the CHOLENZ dataset. The CLOC score predicted 2.9% (IQR 1.3%-8.1%) would be converted. Area under the curve was 0.65 (95% 0.51-0.79) and calibration was acceptable with a Hosmer-Lemeshow p value of 0.45; with evidence of tendency to overestimate with interrogation of calibration across a continuous risk profile (intercept 1.27, slope 0.4). Sensitivity analysis with imputed data improved accuracy. Recalibration with the addition of body mass index, and preoperative bilirubin also improved accuracy to 0.86 (95% CI 0.78-0.95).

CONCLUSIONS

The CLOC score in its original form is not generalisable to the Aotearoa New Zealand setting and is therefore not suitable for clinical use in our local setting.

Predicting peritoneal recurrence after radical gastrectomy for gastric cancer: Validation of a prediction model (PERI-Gastric 1 and PERI-Gastric 2) on a Korean database

BACKGROUND

Peritoneal recurrence is a significant cause of treatment failure after radical gastrectomy for gastric cancer. The prediction of metachronous peritoneal recurrence would have a significantly impact risk stratification and tailored treatment planning. This study aimed to externally validate the previously established PERI-Gastric 1 and 2 models to assess their generalizability in an independent population.

METHODS

Retrospective external validation was conducted on a cohort of 8564 patients who underwent elective gastrectomy for stage Ib-IIIc gastric cancer between 1998 and 2018 at the Yonsei Cancer Center. Discrimination was tested using the area under the receiver operating characteristic curves (AUROC). Accuracy was tested by plotting observations against the predicted risk of peritoneal recurrence and analyzing the resulting calibration plots. Clinical usefulness was tested with a decision curve analysis.

RESULTS

In the validation cohort, PERI-Gastric 1 and PERI-Gastric 2 exhibited an AUROC of 0.766 (95 % C.I. 0.752-0.778) and 0.767 (95 % C.I. 0.755-0.780), a calibration-in-the-large of 0.935 and 0.700, a calibration belt with a 95 % C.I. over the bisector in the risk range of 24%-33 % and 35%-47 %. The decision curve analysis revealed a positive net benefit in the risk range of 10%-42 % and 15%-45 %, respectively.

CONCLUSIONS

This study presents the external validation of the PERI-Gastric 1 and 2 scores in an Eastern population. The models demonstrated fair discrimination and satisfactory calibration for predicting the risk of peritoneal recurrence after radical gastrectomy, even in Eastern patients. PERI-Gastric 1 and 2 scores could also be applied to predict the risk of metachronous peritoneal recurrence in Eastern populations.

Development and validation of a prediction score to assess the risk of incurring in COPD-related exacerbations: a population-based study in primary care

BACKGROUND

Chronic obstructive pulmonary disease (COPD) is the fourth most important cause of death in high-income countries. Inappropriate use of COPD inhaled therapy, including the low adherence (only 10 %-40 % of patients reporting an adequate compliance) may shrink or even nullify the proven benefits of these medications. As such, an accurate prediction algorithm to assess at national level the risk of COPD exacerbation might be relevant for general practictioners (GPs) to improve patient's therapy.

METHODS

We formed a cohort of patients aged 45 years or older being diagnosed with COPD in the period between January 2013 to December 2021. Each patient was followed until occurrence of COPD exacerbation up to the end of 2021. Sixteen determinants were adopted to assemble the CopdEX(CEX)-Health Search(HS)core, which was therefore developed and validated through the related two sub-cohorts.

RESULTS

We idenfied 63763 patients aged 45 years or older being diagnosed with COPD (mean age: 67.8 (SD:11.7); 57.7 % males).When the risk of COPD exacerbation was estimated via CEX-HScore, its predicted value was equal to 14.22 % over a 6-month event horizon. Discrimination accuracy and explained variation were equal to 66 % (95 % CI: 65-67 %) and 10 % (95 % CI: 9-11 %), respectively. The calibration slope did not significantly differ from the unit (p = 0.514).

CONCLUSIONS

The CEX-HScore was featured by fair accuracy for prediction of COPD-related exacerbations over a 6-month follow-up. Such a tool might therefore support GPs to enhance COPD patients' care, and improve their outcomes by facilitating personalized approaches through a score-based decision support system.

Development of Machine Learning Models for the Identification of Elevated Ketone Bodies During Hyperglycemia in Patients with Type 1 Diabetes

Aims: Diabetic ketoacidosis (DKA) is a serious life-threatening condition caused by a lack of insulin, which leads to elevated plasma glucose and metabolic acidosis. Early identification of developing DKA is important to start treatment and minimize complications and risk of death. The aim of the present study is to develop and test prediction model(s) that gives an alarm about their risk of developing elevated ketone bodies during hyperglycemia. Methods: We analyzed data from 138 type 1 diabetes patients with measurements of ketone bodies and continuous glucose monitoring (CGM) data from over 30,000 days of wear time. We utilized a supervised binary classification machine learning approach to identify elevated levels of ketone bodies (≥0.6 mmol/L). Data material was randomly divided at patient level in 70%/30% (training/test) dataset. Logistic regression (LR) and random forest (RF) classifier were compared. Results: Among included patients, 913 ketone samples were eligible for modeling, including 273 event samples with ketone levels ≥0.6 mmol/L. An area under the receiver operating characteristic curve from the RF classifier was 0.836 (confidence interval [CI] 90%, 0.783-0.886) and 0.710 (CI 90%, 0.646-0.77) for the LR classifier. Conclusions: The novel approach for identifying elevated ketone levels in patients with type 1 diabetes utilized in this study indicates that CGM could be a valuable resource for the early prediction of patients at risk of developing DKA. Future studies are needed to validate the results.

SPIN-PM: a consensus framework to evaluate the presence of spin in studies on prediction models

OBJECTIVES

To develop a framework to identify and evaluate spin practices and its facilitators in studies on clinical prediction model regardless of the modeling technique.

STUDY DESIGN AND SETTING

We followed a three-phase consensus process: (1) premeeting literature review to generate items to be included; (2) a series of structured meetings to provide comments discussed and exchanged viewpoints on items to be included with a panel of experienced researchers; and (3) postmeeting review on final list of items and examples to be included. Through this iterative consensus process, a framework was derived after all panel's researchers agreed.

RESULTS

This consensus process involved a panel of eight researchers and resulted in SPIN-Prediction Models which consists of two categories of spin (misleading interpretation and misleading transportability), and within these categories, two forms of spin (spin practices and facilitators of spin). We provide criteria and examples.

CONCLUSION

We proposed this guidance aiming to facilitate not only the accurate reporting but also an accurate interpretation and extrapolation of clinical prediction models which will likely improve the reporting quality of subsequent research, as well as reduce research waste.

Development and Validation of an 18-Gene Urine Test for High-Grade Prostate Cancer

Importance

Benefits of prostate cancer (PCa) screening with prostate-specific antigen (PSA) alone are largely offset by excess negative biopsies and overdetection of indolent cancers resulting from the poor specificity of PSA for high-grade PCa (ie, grade group [GG] 2 or greater).

Objective

To develop a multiplex urinary panel for high-grade PCa and validate its external performance relative to current guideline-endorsed biomarkers.

Design, Setting, and Participants

RNA sequencing analysis of 58 724 genes identified 54 markers of PCa, including 17 markers uniquely overexpressed by high-grade cancers. Gene expression and clinical factors were modeled in a new urinary test for high-grade PCa (MyProstateScore 2.0 [MPS2]). Optimal models were developed in parallel without prostate volume (MPS2) and with prostate volume (MPS2+). The locked models underwent blinded external validation in a prospective National Cancer Institute trial cohort. Data were collected from January 2008 to December 2020, and data were analyzed from November 2022 to November 2023.

Exposure

Protocolized blood and urine collection and transrectal ultrasound-guided systematic prostate biopsy.

Main Outcomes and Measures

Multiple biomarker tests were assessed in the validation cohort, including serum PSA alone, the Prostate Cancer Prevention Trial risk calculator, and the Prostate Health Index (PHI) as well as derived multiplex 2-gene and 3-gene models, the original 2-gene MPS test, and the 18-gene MPS2 models. Under a testing approach with 95% sensitivity for PCa of GG 2 or greater, measures of diagnostic accuracy and clinical consequences of testing were calculated. Cancers of GG 3 or greater were assessed secondarily.

Results

Of 761 men included in the development cohort, the median (IQR) age was 63 (58-68) years, and the median (IQR) PSA level was 5.6 (4.6-7.2) ng/mL; of 743 men included in the validation cohort, the median (IQR) age was 62 (57-68) years, and the median (IQR) PSA level was 5.6 (4.1-8.0) ng/mL. In the validation cohort, 151 (20.3%) had high-grade PCa on biopsy. Area under the receiver operating characteristic curve values were 0.60 using PSA alone, 0.66 using the risk calculator, 0.77 using PHI, 0.76 using the derived multiplex 2-gene model, 0.72 using the derived multiplex 3-gene model, and 0.74 using the original MPS model compared with 0.81 using the MPS2 model and 0.82 using the MPS2+ model. At 95% sensitivity, the MPS2 model would have reduced unnecessary biopsies performed in the initial biopsy population (range for other tests, 15% to 30%; range for MPS2, 35% to 42%) and repeat biopsy population (range for other tests, 9% to 21%; range for MPS2, 46% to 51%). Across pertinent subgroups, the MPS2 models had negative predictive values of 95% to 99% for cancers of GG 2 or greater and of 99% for cancers of GG 3 or greater.

Conclusions and Relevance

In this study, a new 18-gene PCa test had higher diagnostic accuracy for high-grade PCa relative to existing biomarker tests. Clinically, use of this test would have meaningfully reduced unnecessary biopsies performed while maintaining highly sensitive detection of high-grade cancers. These data support use of this new PCa biomarker test in patients with elevated PSA levels to reduce the potential harms of PCa screening while preserving its long-term benefits.

AB2CO risk score for in-hospital mortality of COVID-19 patients admitted to intensive care units

PURPOSE

To develop a mortality risk score for COVID-19 patients admitted to intensive care units (ICU), and to compare it with other existing scores.

MATERIALS AND METHODS

This retrospective observational study included consecutive adult patients with laboratory-confirmed COVID-19 admitted to ICUs of 18 hospitals from nine Brazilian cities, from September 2021 to July 2022. Potential predictors were selected based on the literature review. Generalized Additive Models were used to examine outcomes and predictors. LASSO regression was used to derive the mortality score.

RESULTS

From 558 patients, median age was 69 years (IQR 58-78), 56.3 % were men, 19.7 % required mechanical ventilation (MV), and 44.8 % died. The final model comprised six variables: age, pO2/FiO2, respiratory function (respiratory rate or if in MV), chronic obstructive pulmonary disease, and obesity. The AB2CO had an AUROC of 0.781 (95 % CI 0.744 to 0.819), good overall performance (Brier score = 0.191) and an excellent calibration (slope = 1.063, intercept = 0.015, p-value = 0.834). The model was compared with other scores and displayed better discrimination ability than the majority of them.

CONCLUSIONS

The AB2CO score is a fast and easy tool to be used upon ICU admission.

An advanced nomogram model using deep learning radiomics and clinical data for predicting occult lymph node metastasis in lung adenocarcinoma

PURPOSE

To evaluate the effectiveness of deep learning radiomics nomogram in distinguishing the occult lymph node metastasis (OLNM) status in clinical stage IA lung adenocarcinoma.

METHODS

A cohort of 473 cases of lung adenocarcinomas from two hospitals was included, with 404 cases allocated to the training cohort and 69 cases to the testing cohort. Clinical characteristics and semantic features were collected, and radiomics features were extracted from the computed tomography (CT) images. Additionally, deep transfer learning (DTL) features were generated using RseNet50. Predictive models were developed using the logistic regression (LR) machine learning algorithm. Moreover, gene analysis was conducted on RNA sequencing data from 14 patients to explore the underlying biological basis of deep learning radiomics scores.

RESULT

The training and testing cohorts achieved AUC values of 0.826 and 0.775 for the clinical model, 0.865 and 0.801 for the radiomics model, 0.927 and 0.885 for the DTL-radiomics model, and 0.928 and 0.898 for the nomogram model. The nomogram model demonstrated superiority over the clinical model. The decision curve analysis (DCA) revealed a net benefit in predicting OLNM for all models. The investigation into the biological basis of deep learning radiomics scores identified an association between high scores and pathways related to tumor proliferation and immune cell infiltration in the microenvironment.

CONCLUSIONS

The nomogram model, incorporating clinical-semantic features, radiomics, and DTL features, exhibited promising performance in predicting OLNM. It has the potential to provide valuable information for non-invasive lymph node staging and individualized therapeutic approaches.

Development and Validation of Staging Systems for AA Amyloidosis

Key Points

Patients with AA amyloidosis and age ≥65 years, eGFR <45 ml/min per 1.73 m2, and N -terminal type-B natriuretic peptide >1000 ng/L and/or type-B natriuretic peptide >130 ng/L at diagnosis have poorer survival. Proteinuria >3.0 g/24 hours and eGFR <35 ml/min per 1.73 m2 identify patients at high risk of progression to end-stage kidney failure. Prognostic stratification in AA amyloidosis can be easily made by staging systems, similarly to AL and transthyretin amyloidosis.

Background

The kidney is involved in almost 100% of cases of AA amyloidosis, a rare disease caused by persistent inflammation with long overall survival but frequent progression to kidney failure. Identification of patients with advanced disease at diagnosis is difficult, given the absence of validated staging systems.

Methods

Patients with newly diagnosed AA amyloidosis from the Pavia (n =233, testing cohort) and Heidelberg (n =243, validation cohort) centers were included in this study. Cutoffs of continuous variables were determined by receiver operating characteristic analysis predicting death or dialysis at 24 months. Prognostic factors included in staging systems were identified by multivariable models in the testing cohort.

Results

Age ≥65 years, eGFR <45 ml/min per 1.73 m2, and elevated natriuretic peptides (type-B natriuretic peptide >130 ng/L and/or N -terminal type-B natriuretic peptide >1000 ng/L) were associated with overall survival and included in the staging system (all with simplified coefficients 1). Mean 36-month overall survival was lower with higher staging system scores (score 0–1: 92%; score 2: 72%; score 3: 32%). These results were confirmed in the validation cohort. For kidney failure, variables selected to enter in the staging system model were proteinuria >3 g/24 hour and eGFR <35 ml/min per 1.73 m2 (both with simplified coefficients 1). The 36-month cumulative incidence of kidney failure was higher with higher staging system scores (score 0: 0%; score 1: 24%; score 2: 51%). Again, similar results were obtained in validation cohort.

Conclusions

We identified and validated biomarker-based staging systems for overall survival and kidney failure in AA amyloidosis.

Temporal validation of a Generalized Additive2 Model (GA2M) to assess the risk of Chronic Kidney Disease (CKD)

OBJECTIVE

To assess the temporal validity of a model predicting the risk of Chronic Kidney Disease (CKD) using Generalized Additive2 Models (GA2M).

MATERIALS

We adopted the Italian Health Search Database (HSD) with which the original algorithm was developed and validated by comparing different machine learnings models.

METHODS

We selected all patients aged >=15 being active in HSD in 2019. They were followed up until December 2022 so being updated with three years of data collection. Those with prior diagnosis of CKD were excluded. A GA2M-based algorithm for CKD prediction was applied to this cohort in order to compare observed and predicted risk. Area Under Curve (AUC) and Average Precision (AP) were calculated.

RESULTS

We obtained an AUC and AP equal to 88% and 30%, respectively.

DISCUSSION

The prediction accuracy of the algorithm was largely consistent with that obtained in our prior work which was based on a different time-window for data collection. We therefore underlined and demonstrated the relevance of temporal validation for this prediction tool.

CONCLUSION

The GA2M confirmed its high accuracy in prediction of CKD. As such, the respective patient- and population-based informatic tools might be implemented in primary care.

Predicting the risk of pancreatic cancer in women with new-onset diabetes mellitus

BACKGROUND AND AIM

People with new-onset diabetes mellitus (diabetes) could be a possible target population for pancreatic cancer surveillance. However, distinguishing diabetes caused by pancreatic cancer from type 2 diabetes remains challenging. We aimed to develop and validate a model to predict pancreatic cancer among women with new-onset diabetes.

METHODS

We conducted a retrospective cohort study among Australian women newly diagnosed with diabetes, using first prescription of anti-diabetic medications, sourced from administrative data, as a surrogate for the diagnosis of diabetes. The outcome was a diagnosis of pancreatic cancer within 3 years of diabetes diagnosis. We used prescription medications, severity of diabetes (i.e., change/addition of medication within 2 months after first medication), and age at diabetes diagnosis as potential predictors of pancreatic cancer.

RESULTS

Among 99 687 women aged ≥ 50 years with new-onset diabetes, 602 (0.6%) were diagnosed with pancreatic cancer within 3 years. The area under the receiver operating curve for the risk prediction model was 0.73. Age and diabetes severity were the two most influential predictors followed by beta-blockers, acid disorder drugs, and lipid-modifying agents. Using a risk threshold of 50%, sensitivity and specificity were 69% and the positive predictive value (PPV) was 1.3%.

CONCLUSIONS

Our model doubled the PPV of pancreatic cancer in women with new-onset diabetes from 0.6% to 1.3%. Age and rapid progression of diabetes were important risk factors, and pancreatic cancer occurred more commonly in women without typical risk factors for type 2 diabetes. This model could prove valuable as an initial screening tool, especially as new biomarkers emerge.

Gupta Perioperative Risk for Myocardial Infarction or Cardiac Arrest score is a long-term cardiovascular risk predictor after aortoiliac revascularization

INTRODUCTION

Gupta Perioperative Risk for Myocardial Infarction or Cardiac Arrest (MICA) is a validated self-explanatory score applied in cardiac or non-cardiac surgeries. This study aims to assess the predictive value of the MICA score for cardiovascular events after aortoiliac revascularization.

METHODS

This prospective cohort underwent elective aortoiliac revascularization between 2013 and 2021. Patients' demographic, clinical characteristics and outcomes were registered. The patients were divided into two groups according to the MICA score using optimal binning. Survival analysis to test for time-dependent variables and multivariate Cox regression analysis for independent predictors were performed.

RESULTS

This study included 130 patients with a median follow-up of 55 months. Preoperative MICA score was ≥ 6.5 in 41 patients. MICA ≥ 6.5 presented a statistically significant association with long-term occurrence of acute heart failure (HR=1.695, 95% CI 1.208-2.379, p=0.002), major adverse cardiovascular events (HR=1.222, 95% CI 1.086-1.376, p<0.001) and all-cause mortality (HR=1.256, 95% CI 1.107-1.425, p<0.001). Multivariable Cox regression confirmed MICA as a significant independent predictor of long-term major adverse cardiovascular events (aHR=1.145 95%CI 1.010-1.298, p=0.034) and all-cause mortality (aHR=1.172 95%CI 1.026-1.339, p=0.020).

CONCLUSION

The MICA score is a quick, easy-to-obtain, predictive tool in identifying patients with a higher risk of post-aortoiliac revascularization cardiovascular events, such as acute heart failure, major adverse cardiovascular events and all-cause mortality. Additional research for validation of the MICA score in the context of aortoiliac revascularization and specific interventions are necessary.

Predictive models of radiographic progression and pain progression in patients with knee osteoarthritis: data from the FNIH OA biomarkers consortium project

OBJECTIVES

The progression of knee osteoarthritis (OA) can be defined as either radiographic progression or pain progression. This study aimed to construct models to predict radiographic progression and pain progression in patients with knee OA.

METHODS

We retrieved data from the FNIH OA Biomarkers Consortium project, a nested case-control study. A total of 600 subjects with mild to moderate OA (Kellgren-Lawrence grade of 1, 2, or 3) in one target knee were enrolled. The patients were classified as radiographic progressors (n = 297), non-radiographic progressors (n = 303), pain progressors (n = 297), or non-pain progressors (n = 303) according to the change in the minimum joint space width of the medial compartment and the WOMAC pain score during the follow-up period of 24-48 months. Initially, 376 variables concerning demographics, clinical questionnaires, imaging measurements, and biochemical markers were included. We developed predictive models based on multivariate logistic regression analysis and visualized the models with nomograms. We also tested whether adding changes in predictors from baseline to 24 months would improve the predictive efficacy of the models.

RESULTS

The predictive models of radiographic progression and pain progression consisted of 8 and 10 variables, respectively, with area under curve (AUC) values of 0.77 and 0.76, respectively. Incorporating the change in the WOMAC pain score from baseline to 24 months into the pain progression predictive model significantly improved the predictive effectiveness (AUC = 0.86).

CONCLUSIONS

We identified risk factors for imaging progression and pain progression in patients with knee OA over a 2- to 4-year period, and provided effective predictive models, which could help identify patients at high risk of progression.

Community-acquired pneumonia: use of clinical characteristics of acutely admitted patients for the development of a diagnostic model - a cross-sectional multicentre study

OBJECTIVES

This study aimed to describe the clinical characteristics of adults with suspected acute community-acquired pneumonia (CAP) on hospitalisation, evaluate their prediction performance for CAP and compare the performance of the model to the initial assessment of the physician.

DESIGN

Cross-sectional, multicentre study.

SETTING

The data originated from the INfectious DisEases in Emergency Departments study and were collected prospectively from patient interviews and medical records. The study included four Danish medical emergency departments (EDs) and was conducted between 1 March 2021 and 28 February 2022.

PARTICIPANTS

A total of 954 patients admitted with suspected infection were included in the study.

PRIMARY AND SECONDARY OUTCOME

The primary outcome was CAP diagnosis assessed by an expert panel.

RESULTS

According to expert evaluation, CAP had a 28% prevalence. 13 diagnostic predictors were identified using least absolute shrinkage and selection operator regression to build the prediction model: dyspnoea, expectoration, cough, common cold, malaise, chest pain, respiratory rate (>20 breaths/min), oxygen saturation (<96%), abnormal chest auscultation, leucocytes (<3.5×109/L or >8.8×109/L) and neutrophils (>7.5×109/L). C reactive protein (<20 mg/L) and having no previous event of CAP contributed negatively to the final model. The predictors yielded good prediction performance for CAP with an area under the receiver-operator characteristic curve (AUC) of 0.85 (CI 0.77 to 0.92). However, the initial diagnosis made by the ED physician performed better, with an AUC of 0.86 (CI 84% to 89%).

CONCLUSION

Typical respiratory symptoms combined with abnormal vital signs and elevated infection biomarkers were predictors for CAP on admission to an ED. The clinical value of the prediction model is questionable in our setting as it does not outperform the clinician's assessment. Further studies that add novel diagnostic tools and use imaging or serological markers are needed to improve a model that would help diagnose CAP in an ED setting more accurately.

TRIAL REGISTRATION NUMBER

NCT04681963.

Deep learning for risk stratification of thymoma pathological subtypes based on preoperative CT images

OBJECTIVES

This study aims to develop an innovative, deep model for thymoma risk stratification using preoperative CT images. Current algorithms predominantly focus on radiomic features or 2D deep features and require manual tumor segmentation by radiologists, limiting their practical applicability.

METHODS

The deep model was trained and tested on a dataset comprising CT images from 147 patients (82 female; mean age, 54 years ± 10) who underwent surgical resection and received subsequent pathological confirmation. The eligible participants were divided into a training cohort (117 patients) and a testing cohort (30 patients) based on the CT scan time. The model consists of two stages: 3D tumor segmentation and risk stratification. The radiomic model and deep model (2D) were constructed for comparative analysis. Model performance was evaluated through dice coefficient, area under the curve (AUC), and accuracy.

RESULTS

In both the training and testing cohorts, the deep model demonstrated better performance in differentiating thymoma risk, boasting AUCs of 0.998 and 0.893 respectively. This was compared to the radiomic model (AUCs of 0.773 and 0.769) and deep model (2D) (AUCs of 0.981 and 0.760). Notably, the deep model was capable of simultaneously identifying lesions, segmenting the region of interest (ROI), and differentiating the risk of thymoma on arterial phase CT images. Its diagnostic prowess outperformed that of the baseline model.

CONCLUSIONS

The deep model has the potential to serve as an innovative decision-making tool, assisting on clinical prognosis evaluation and the discernment of suitable treatments for different thymoma pathological subtypes.

KEY POINTS

• This study incorporated both tumor segmentation and risk stratification. • The deep model, using clinical and 3D deep features, effectively predicted thymoma risk. • The deep model improved AUCs by 16.1pt and 17.5pt compared to radiomic model and deep model (2D) respectively.

Telephone triage of chest pain in out-of-hours primary care: external validation of a symptom-based prediction rule to rule out acute coronary syndromes

INTRODUCTION

Telephone triage is pivotal for evaluating the urgency of patient care, and in the Netherlands, the Netherlands Triage Standard (NTS) demonstrates moderate discrimination for chest pain. To address this, the Safety First Prediction Rule (SFPR) was developed to improve the safety of ruling out acute coronary syndrome (ACS) during telephone triage.

METHODS

We conducted an external validation of the SFPR using data from the TRACE study, a retrospective cohort study in out-of-hours primary care. We evaluated the diagnostic accuracy assessment for ACS, major adverse cardiovascular events (MACE), and major events within 6 weeks. Moreover, we compared its performance with that of the NTS algorithm.

RESULTS

Among 1404 included patients (57.3% female, 6.8% ACS, 8.6% MACE), the SFPR demonstrated good discrimination for ACS (C-statistic: 0.79; 95%-CI: 0.75-0.83) and MACE (C-statistic: 0.79; 95%-CI: 0.0.76-0.82). Calibration was satisfactory, with overestimation observed in high-risk patients for ACS. The SFPR (risk threshold 2.5%) trended toward higher sensitivity (95.8% vs. 86.3%) and negative predictive value (99.3% vs. 97.6%) with a lower negative likelihood ratio (0.10 vs. 0.34) than the NTS algorithm.

CONCLUSION

The SFPR proved robust for risk stratification in patients with acute chest pain seeking out-of-hours primary care in the Netherlands. Further prospective validation and implementation are warranted to refine and establish the rule's clinical utility.

Sample size determination for prediction models via learning-type curves

This article is concerned with sample size determination methodology for prediction models. We propose to combine the individual calculations via learning-type curves. We suggest two distinct ways of doing so, a deterministic skeleton of a learning curve and a Gaussian process centered upon its deterministic counterpart. We employ several learning algorithms for modeling the primary endpoint and distinct measures for trial efficacy. We find that the performance may vary with the sample size, but borrowing information across sample size universally improves the performance of such calculations. The Gaussian process-based learning curve appears more robust and statistically efficient, while computational efficiency is comparable. We suggest that anchoring against historical evidence when extrapolating sample sizes should be adopted when such data are available. The methods are illustrated on binary and survival endpoints.

Prognosis prediction models for post-stroke depression: a protocol for systematic review, meta-analysis, and critical appraisal

INTRODUCTION

Post-stroke depression (PSD) is a prevalent complication that has been shown to have a negative impact on rehabilitation outcomes and quality of life and poses a significant risk for suicidal intention. However, models for discriminating and predicting PSD in stroke survivors for effective secondary prevention strategies are inadequate as the pathogenesis of PSD remains unknown. Prognostic prediction models that exhibit greater rule-in capacity have the potential to mitigate the issue of underdiagnosis and undertreatment of PSD. Thus, the planned study aims to systematically review and critically evaluate published studies on prognostic prediction models for PSD.

METHODS AND ANALYSIS

A systematic literature search will be conducted in PubMed and Embase through Ovid. Two reviewers will complete study screening, data extraction, and quality assessment utilizing appropriate tools. Qualitative data on the characteristics of the included studies, methodological quality, and the appraisal of the clinical applicability of models will be summarized in the form of narrative comments and tables or figures. The predictive performance of the same model involving multiple studies will be synthesized with a random effects meta-analysis model or meta-regression, taking into account heterogeneity.

ETHICS AND DISSEMINATION

Ethical approval is considered not applicable for this systematic review. Findings will be shared through dissemination at academic conferences and/or publication in peer-reviewed academic journals.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42023388548.