A discussion of calibration techniques for evaluating binary and categorical predictive models

An explainable machine learning framework for predicting the risk of buprenorphine treatment discontinuation for opioid use disorder among commercially insured individuals

OBJECTIVES

Buprenorphine is an effective evidence-based medication for opioid use disorder (OUD). Yet premature discontinuation undermines treatment effectiveness, increasing the risk of mortality and overdose. We developed and evaluated a machine learning (ML) framework for predicting buprenorphine care discontinuity within 12 months following treatment initiation.

METHODS

This retrospective study used United States (US) 2018-2021 MarketScan commercial claims data of insured individuals aged 18-64 who initiated buprenorphine between July 2018 and December 2020 with no buprenorphine prescriptions in the previous six months. We measured buprenorphine prescription discontinuation gaps of ≥30 days within 12 months of initiating treatment. We developed predictive models employing logistic regression, decision tree classifier, random forest, extreme gradient boosting, Adaboost, and random forest-extreme gradient boosting ensemble. We applied recursive feature elimination with cross-validation to reduce dimensionality and identify the most predictive features while maintaining model robustness. For model validation, we used several statistics to evaluate performance, such as C-statistics and precision-recall curves. We focused on two distinct treatment stages: at the time of treatment initiation and one and three months after treatment initiation. We employed SHapley Additive exPlanations (SHAP) analysis that helped us explain the contributions of different features in predicting buprenorphine discontinuation. We stratified patients into risk subgroups based on their predicted likelihood of treatment discontinuation, dividing them into decile subgroups. Additionally, we used a calibration plot to analyze the reliability of the models.

RESULTS

A total of 30,373 patients initiated buprenorphine and 14.98% (4551) discontinued treatment. C-statistic varied between 0.56 and 0.76 for the first-stage models including patient-level demographic and clinical variables. Inclusion of proportion of days covered (PDC) measured after one month and three months following treatment initiation significantly increased the models' discriminative power (C-statistics: 0.60 to 0.82). Random forest (C-statistics: 0.76, 0.79 and 0.82 with baseline predictors, one-month PDC and three-months PDC, respectively) outperformed other ML models in discriminative performance in all stages (C-statistics: 0.56 to 0.77). Most influential risk factors of discontinuation included early stage medication adherence, age, and initial days of supply.

CONCLUSION

ML algorithms demonstrated a good discriminative power in identifying patients at higher risk of buprenorphine care discontinuity. The proposed framework may help healthcare providers optimize treatment strategies and deliver targeted interventions to improve buprenorphine care continuity.

To establish a prognostic model of epidermal growth factor receptor mutated non-small cell lung cancer patients based on Least Absolute Shrinkage and Selection Operator regression

BACKGROUND

There is currently a shortage of effective diagnostic tools that are used for identifying long-term survival among non-small cell lung cancer (NSCLC) patients with epidermal growth factor receptor (EGFR) mutations. This research utilized the development of a prognostic model to assist clinicians in forecasting the survival over 24 months.

METHODS

In Phase III and IV those patients who were diagnosed with EGFR mutation from January 2018 to June 2022 were enrolled into the lung cancer group of Thoracic Surgery Department of Hebei Provincial People's Hospital. Long-run survival was stated as survival for 24 months after being diagnosed. A multivariate prognostic pattern was constructed by means of internal validation and binary logistic regression by bootstrapping. One nomogram was created with a view to boosting the explanation and applicability of the pattern.

RESULTS

A total of 603 patients with EGFR mutation were registered. Elements linked to the whole survival beyond 24 months were age (OR 6.15); female (OR 1.79); functional status (ECOG 0-1) (OR 5.26); Exon 20 insertion mutation deletion (OR 2.08); No central nervous system metastasis (OR 2.66), targeted therapy (OR 0.43); Immunotherapy (OR 0.24). The model has good internal validation.

CONCLUSION

Seven pretreatment clinicopathological variables predicted survival over 24 months. That pattern owns a great discriminative capability. It is hypothesized that this pattern is capable of assisting in selecting the optimal treatment sequence for NSCLC patients with EGFR mutations.

Development of a machine learning model for predicting pneumothorax risk in coaxial core needle biopsy (≤3 cm)

PURPOSE

The aim is to devise a machine learning algorithm exploiting preoperative clinical data to forecast the hazard of pneumothorax post-coaxial needle lung biopsy (CCNB), thereby informing clinical decision-making and enhancing perioperative care.

METHOD

This retrospective analysis aggregated clinical and imaging data from patients with lung nodules (≤3 cm) biopsies. Variable selection was done using univariate analysis and LASSO regression, with the dataset subsequently divided into training (80 %) and validation (20 %) subsets. Various machine learning (ML) classifiers were employed in a consolidated approach to ascertain the paramount model, which was followed by individualized risk profiling showcased through Shapley Additive eXplanations (SHAP).

RESULTS

Out of the 325 patients included in the study, 19.6% (64/325) experienced postoperative pneumothorax. High-risk factors determined were Cancer, Lesion_type, GOLD, Size, and Depth. The Gaussian Naive Bayes (GNB) classifier demonstrated superior prediction with an Area Under the Curve (AUC) of 0.82 (95% CI 0.71-0.94), complemented by an accuracy rate of 0.8, sensitivity of 0.71, specificity of 0.84, and an F1 score of 0.61 in the test cohort.

CONCLUSION

The formulated prognostic algorithm exhibited commendable efficacy in preoperatively prognosticating CCNB-induced pneumothorax, harboring the potential to refine personalized risk appraisals, steer clinical judgment, and ameliorate perioperative patient stewardship.

Development of an immunoinflammatory indicator-related dynamic nomogram based on machine learning for the prediction of intravenous immunoglobulin-resistant Kawasaki disease patients

BACKGROUND

Approximately 10-20% of Kawasaki disease (KD) patients suffer from intravenous immunoglobulin (IVIG) resistance, placing them at higher risk of developing coronary artery aneurysms. Therefore, we aimed to construct an IVIG resistance prediction tool for children with KD in Shanghai, China.

METHODS

Retrospective analysis was conducted on data from 1271 patients diagnosed with KD and the patients were randomly divided into a training set and a validation set in a 2:1 ratio. Machine learning algorithms were employed to identify important predictors associated with IVIG resistance and to build a predictive model. The best-performing model was used to construct a dynamic nomogram. Moreover, receiver operating characteristic curves, calibration plots, and decision-curve analysis were utilized to measure the discriminatory power, accuracy, and clinical utility of the nomogram.

RESULTS

Six variables were identified as important predictors, including C-reactive protein, neutrophil ratio, procalcitonin, CD3 ratio, CD19 count, and IgM level. A dynamic nomogram constructed with these factors was available at https://hktk.shinyapps.io/dynnomapp/. The nomogram demonstrated good diagnostic performance in the training and validation sets (area under the receiver operating characteristic curve = 0.816 and 0.800, respectively). Moreover, the calibration curves and decision curves analysis indicated that the nomogram showed good consistency between predicted and actual outcomes and had good clinical benefits.

CONCLUSION

A web-based dynamic nomogram for IVIG resistance was constructed with good predictive performance, which can be used as a practical approach for early screening to assist physicians in personalizing the treatment of KD patients in Shanghai.

Predicting short-term thromboembolic risk following Roux-en-Y gastric bypass using supervised machine learning

BACKGROUND

Roux-en-Y gastric bypass (RYGB) is a widely recognized bariatric procedure that is particularly beneficial for patients with class III obesity. It aids in significant weight loss and improves obesity-related medical conditions. Despite its effectiveness, postoperative care still has challenges. Clinical evidence shows that venous thromboembolism (VTE) is a leading cause of 30-d morbidity and mortality after RYGB. Therefore, a clear unmet need exists for a tailored risk assessment tool for VTE in RYGB candidates.

AIM

To develop and internally validate a scoring system determining the individualized risk of 30-d VTE in patients undergoing RYGB.

METHODS

Using the 2016-2021 Metabolic and Bariatric Surgery Accreditation Quality Improvement Program, data from 6526 patients (body mass index ≥ 40 kg/m2) who underwent RYGB were analyzed. A backward elimination multivariate analysis identified predictors of VTE characterized by pulmonary embolism and/or deep venous thrombosis within 30 d of RYGB. The resultant risk scores were derived from the coefficients of statistically significant variables. The performance of the model was evaluated using receiver operating curves through 5-fold cross-validation.

RESULTS

Of the 26 initial variables, six predictors were identified. These included a history of chronic obstructive pulmonary disease with a regression coefficient (Coef) of 2.54 (P < 0.001), length of stay (Coef 0.08, P < 0.001), prior deep venous thrombosis (Coef 1.61, P < 0.001), hemoglobin A1c > 7% (Coef 1.19, P < 0.001), venous stasis history (Coef 1.43, P < 0.001), and preoperative anticoagulation use (Coef 1.24, P < 0.001). These variables were weighted according to their regression coefficients in an algorithm that was generated for the model predicting 30-d VTE risk post-RYGB. The risk model's area under the curve (AUC) was 0.79 [95% confidence interval (CI): 0.63-0.81], showing good discriminatory power, achieving a sensitivity of 0.60 and a specificity of 0.91. Without training, the same model performed satisfactorily in patients with laparoscopic sleeve gastrectomy with an AUC of 0.63 (95%CI: 0.62-0.64) and endoscopic sleeve gastroplasty with an AUC of 0.76 (95%CI: 0.75-0.78).

CONCLUSION

This simple risk model uses only six variables to assist clinicians in the preoperative risk stratification of RYGB patients, offering insights into factors that heighten the risk of VTE events.

Prophet time series modeling of waste disposal rates in four North American cities

In this study, three different univariate municipal solid waste (MSW) disposal rate forecast models (SARIMA, Holt-Winters, Prophet) were examined using different testing periods in four North American cities with different socioeconomic conditions. A review of the literature suggests that the selected models are able to handle seasonality in a time series; however, their ability to handle outliers is not well understood. The Prophet model generally outperformed the Holt-Winters model and the SARIMA model. The MAPE and R2 of the Prophet model during pre-COVID-19 were 4.3-22.2% and 0.71-0.93, respectively. All three models showed satisfactory predictive results, especially during the pre-COVID-19 testing period. COVID-19 lockdowns and the associated regulatory measures appear to have affected MSW disposal behaviors, and all the univariate models failed to fully capture the abrupt changes in waste disposal behaviors. Modeling errors were largely attributed to data noise in seasonality and the unprecedented event of COVID-19 lockdowns. Overall, the modeling errors of the Prophet model were evenly distributed, with minimum modeling biases. The Prophet model also appeared to be versatile and successfully captured MSW disposal rates from 3000 to 39,000 tons/month. The study highlights the potential benefits of the use of univariate models in waste forecast.

Prediction model for spinal cord injury in spinal tuberculosis patients using multiple machine learning algorithms: a multicentric study

Spinal cord injury (SCI) is a prevalent and serious complication among patients with spinal tuberculosis (STB) that can lead to motor and sensory impairment and potentially paraplegia. This research aims to identify factors associated with SCI in STB patients and to develop a clinically significant predictive model. Clinical data from STB patients at a single hospital were collected and divided into training and validation sets. Univariate analysis was employed to screen clinical indicators in the training set. Multiple machine learning (ML) algorithms were utilized to establish predictive models. Model performance was evaluated and compared using receiver operating characteristic (ROC) curves, area under the curve (AUC), calibration curve analysis, decision curve analysis (DCA), and precision-recall (PR) curves. The optimal model was determined, and a prospective cohort from two other hospitals served as a testing set to assess its accuracy. Model interpretation and variable importance ranking were conducted using the DALEX R package. The model was deployed on the web by using the Shiny app. Ten clinical characteristics were utilized for the model. The random forest (RF) model emerged as the optimal choice based on the AUC, PRs, calibration curve analysis, and DCA, achieving a test set AUC of 0.816. Additionally, MONO was identified as the primary predictor of SCI in STB patients through variable importance ranking. The RF predictive model provides an efficient and swift approach for predicting SCI in STB patients.

Development and Validation of Contrast-Enhanced CT-Based Deep Transfer Learning and Combined Clinical-Radiomics Model to Discriminate Thymomas and Thymic Cysts: A Multicenter Study

RATIONALE AND OBJECTIVES

This study aims to evaluate the feasibility and effectiveness of deep transfer learning (DTL) and clinical-radiomics in differentiating thymoma from thymic cysts.

MATERIALS AND METHODS

Clinical and imaging data of 196 patients pathologically diagnosed with thymoma and thymic cysts were retrospectively collected from center 1. (training cohort: n = 137; internal validation cohort: n = 59). An independent external validation cohort comprised 68 thymoma and thymic cyst patients from center 2. Region of interest (ROI) delineation was performed on contrast-enhanced chest computed tomography (CT) images, and eight DTL models including Densenet 169, Mobilenet V2, Resnet 101, Resnet 18, Resnet 34, Resnet 50, Vgg 13, Vgg 16 were constructed. Radiomics features were extracted from the ROI on the CT images of thymoma and thymic cyst patients, and feature selection was performed using intra-observer correlation coefficient (ICC), Spearman correlation analysis, and least absolute shrinkage and selection operator (LASSO) algorithm. Univariate analysis and multivariable logistic regression (LR) were used to select clinical-radiological features. Six machine learning classifiers, including LR, support vector machine (SVM), k-nearest neighbors (KNN), Light Gradient Boosting Machine (LightGBM), Adaptive Boosting (AdaBoost), and Multilayer Perceptron (MLP), were used to construct Radiomics and Clinico-radiologic models. The selected features from the Radiomics and Clinico-radiologic models were fused to build a Combined model. Receiver operating characteristic curve (ROC), calibration curve, and decision curve analysis (DCA) were used to evaluate the discrimination, calibration, and clinical utility of the models, respectively. The Delong test was used to compare the AUC between different models. K-means clustering was used to subdivide the lesions of thymomas or thymic cysts into subregions, and traditional radiomics methods were used to extract features and compare the ability of Radiomics and DTL models to reflect intratumoral heterogeneity using correlation analysis.

RESULTS

The Densenet 169 based on DTL performed the best, with AUC of 0.933 (95% CI: 0.875-0.991) in the internal validation cohort and 0.962 (95% CI: 0.923-1.000) in the external validation cohort. The AdaBoost classifier achieved AUC of 0.965 (95% CI: 0.923-1.000) and 0.959 (95% CI: 0.919-1.000) in the internal and external validation cohorts, respectively, for the Radiomics model. The LightGBM classifier achieved AUC of 0.805 (95% CI: 0.690-0.920) and 0.839 (95% CI: 0.736-0.943) in the Clinico-radiologic model. The AUC of the Combined model in the internal and external validation cohorts was 0.933 (95% CI: 0.866-1.000) and 0.945 (95% CI: 0.897-0.994), respectively. The results of the Delong test showed that the Radiomics model, DTL model, and Combined model outperformed the Clinico-radiologic model in both internal and external validation cohorts (p-values were 0.002, 0.004, and 0.033 in the internal validation cohort, while in the external validation cohort, the p-values were 0.014, 0.006, and 0.015, respectively). But there was no statistical difference in performance among the three models (all p-values <0.05). Correlation analysis showed that radiomics performed better than DTL in quantifying intratumoral heterogeneity differences between thymoma and thymic cysts.

CONCLUSION

The developed DTL model and the Combined model based on radiomics and clinical-radiologic features achieved excellent diagnostic performance in differentiating thymic cysts from thymoma. They can serve as potential tools to assist clinical decision-making, particularly when endoscopic biopsy carries a high risk.

Predicting the risk of early intensive care unit admission for patients hospitalized with acute pancreatitis using supervised machine learning

Background

Acute pancreatitis (AP) is a complex and life-threatening disease. Early recognition of factors predicting morbidity and mortality is crucial. We aimed to develop and validate a pragmatic model to predict the individualized risk of early intensive care unit (ICU) admission for patients with AP.

Methods

The 2019 Nationwide Readmission Database was used to identify patients hospitalized with a primary diagnosis of AP without ICU admission. A matched comparison cohort of AP patients with ICU admission within 7 days of hospitalization was identified from the National Inpatient Sample after 1:N propensity score matching. The least absolute shrinkage and selection operator (LASSO) regression was used to select predictors and develop an ICU acute pancreatitis risk (IAPR) score validated by 10-fold cross-validation.

Results

A total of 1513 patients hospitalized for AP were included. The median age was 50.0 years (interquartile range: 39.0-63.0). The three predictors that were selected included hypoxia (area under the curve [AUC] 0.78), acute kidney injury (AUC 0.72), and cardiac arrhythmia (AUC 0.61). These variables were used to develop a nomogram that displayed excellent discrimination (AUC 0.874) (bootstrap bias-corrected 95% confidence interval 0.824-0.876). There was no evidence of miscalibration (test statistic = 2.88; P = 0.09). For high-risk patients (total score >6 points), the sensitivity was 68.94% and the specificity was 92.66%.

Conclusions

This supervised machine learning-based model can help recognize high-risk AP hospitalizations. Clinicians may use the IAPR score to identify patients with AP at high risk of ICU admission within the first week of hospitalization.

Network centrality and HIV prevention service use among people who inject drugs: Findings from a sociometric network cohort in New Delhi, India

BACKGROUND AND AIMS

Network centrality, an indicator of an individual's importance and potential to drive behavioral change, is rarely used to select peer educators. Individual-level predictors of network centrality might be useful to identify people who inject drugs (PWID) for potential roles as peer navigators or change agents in network-based interventions in settings where sociometric data are unavailable. We assessed the relationship between network centrality and HIV prevention service engagement to determine whether centrally-positioned PWID share measurable commonalities.

DESIGN

Observational study and survey using baseline data from a sociometric network cohort of PWID, enumerated using network software and biometric data (2017-2020). Network ties corresponded to direct injection partnerships in the prior month.

SETTING

New Delhi, India.

PARTICIPANTS

A total of 2512 PWID who were ≥18 years, provided written informed consent, and reported illicit injection drug use within the 24 months before study enrollment.

MEASUREMENTS

Interviewer-administered questionnaires measured demographics and substance use behaviors. Central versus peripheral network position was categorized using betweenness centrality 75th%ile . Logistic regression was used to estimate adjusted odds ratios (aOR) with 95% confidence intervals (95%CI) between network position and HIV testing, medication for opioid use disorder (MOUD), or syringe service use. Lasso models selected predictors of central network position among 20 covariates detailing demographic, biologic, and substance use information. Predictive accuracy was evaluated using model performance metrics.

FINDINGS

Overall, median age was 26 years (interquartile range 22-34); 99% were male; 628 were classified as central. Compared with PWID at the periphery, central PWID were more likely to use MOUD (aOR: 1.59, 95%CI: 1.30-1.94) and syringe services (aOR: 2.91, 95%CI: 2.25, 3.76) in the prior six months. Findings for HIV testing were inconclusive (aOR: 1.30, 95%CI: 1.00-1.69). The lasso variable selector identified several predictors of network centrality: HIV and hepatitis C infection, number of PWID seen in the prior month, injecting heroin and buprenorphine (vs. heroin only) six months prior, sharing injection equipment six months prior, experiencing drug overdose in the past year, and moderate/severe depression (vs. none/mild). Average agreement between model-predicted vs. observed values was 0.75; area under the receiver operator curve was 0.69.

CONCLUSIONS

In a socioeconomic network of people who inject drugs (PWID) in New Delhi, India, there are common characteristics among individuals based on their network position (central vs. peripheral) but individual-level predictors have only moderate predictive accuracy. Although central network members appear to be more likely to use HIV prevention services than peripheral network members, their potential as change agents may be limited by other factors that impede their ability to adopt or promote HIV prevention service use.

Construction and evaluation of neonatal respiratory failure risk prediction model for neonatal respiratory distress syndrome

BACKGROUND

Neonatal respiratory distress syndrome (NRDS) is a common respiratory disease in preterm infants, often accompanied by respiratory failure. The aim of this study was to establish and validate a nomogram model for predicting the probability of respiratory failure in NRDS patients.

METHODS

Patients diagnosed with NRDS were extracted from the MIMIC-iv database. The patients were randomly assigned to a training and a validation cohort. Univariate and stepwise Cox regression analyses were used to determine the prognostic factors of NRDS. A nomogram containing these factors was established to predict the incidence of respiratory failure in NRDS patients. The area under the receiver operating characteristic curve (AUC), receiver operating characteristic curve (ROC), calibration curves and decision curve analysis were used to determine the effectiveness of this model.

RESULTS

The study included 2,705 patients with NRDS. Univariate and multivariate stepwise Cox regression analysis showed that the independent risk factors for respiratory failure in NRDS patients were gestational age, pH, partial pressure of oxygen (PO2), partial pressure of carbon dioxide (PCO2), hemoglobin, blood culture, infection, neonatal intracranial hemorrhage, Pulmonary surfactant (PS), parenteral nutrition and respiratory support. Then, the nomogram was constructed and verified.

CONCLUSIONS

This study identified the independent risk factors of respiratory failure in NRDS patients and used them to construct and evaluate respiratory failure risk prediction model for NRDS. The present findings provide clinicians with the judgment of patients with respiratory failure in NRDS and help clinicians to identify and intervene in the early stage.

Development and validation of a noninvasive prediction model of autoimmune hepatitis in patients with liver diseases

BACKGROUND AND AIMS

There is no golden standard for the diagnosis of autoimmune hepatitis which still dependent on liver biopsy currently. So, we developed a noninvasive prediction model to help optimize the diagnosis of autoimmune hepatitis.

METHODS

From January 2017 to December 2019, 1739 patients who had undergone liver biopsy were seen in the second hospital of Nanjing, of which 128 were here for consultation. Clinical, laboratory, and histologic data were obtained retrospectively. Multivariable logistic regression analysis was employed to create a nomogram model that predicting the risk of autoimmune hepatitis. Internal and external validation was both performed to evaluate the model.

RESULTS

A total of 1288 patients with liver biopsy were enrolled (1184 from the second hospital of Nanjing, the remaining 104 from other centers). After the univariate and multivariate logistic regression analysis, nine variables including ALT, IgG, ALP/AST, ALB, ANA, AMA, HBsAg, age, and gender were selected to establish the noninvasive prediction model. The nomogram model exhibits good prediction in diagnosing autoimmune hepatitis with AUROC of 0.967 (95% CI: 0.776-0.891) in internal validation and 0.835 (95% CI: 0.752-0.919) in external validation.

CONCLUSIONS

ALT, IgG, ALP/AST, ALB, ANA, AMA, HBsAg, age, and gender are predictive factors for the diagnosis of autoimmune hepatitis in patients with unexplained liver diseases. The predictive nomogram model built by the nine predictors achieved good prediction for diagnosing autoimmune hepatitis.

The use of machine learning to predict somatic cell count status in dairy cows post-calving

Udder health remains a priority for the global dairy industry to reduce pain, economic losses, and antibiotic usage. The dry period is a critical time for the prevention of new intra-mammary infections and it provides a point for curing existing intra-mammary infections. Given the wealth of udder health data commonly generated through routine milk recording and the importance of udder health to the productivity and longevity of individual cows, an opportunity exists to extract greater value from cow-level data to undertake risk-based decision-making. The aim of this research was to construct a machine learning model, using routinely collected farm data, to make probabilistic predictions at drying off for an individual cow's risk of a raised somatic cell count (hence intra-mammary infection) post-calving. Anonymized data were obtained as a large convenience sample from 108 UK dairy herds that undertook regular milk recording. The outcome measure evaluated was the presence of a raised somatic cell count in the 30 days post-calving in this observational study. Using a 56-farm training dataset, machine learning analysis was performed using the extreme gradient boosting decision tree algorithm, XGBoost. External validation was undertaken on a separate 28-farm test dataset. Statistical assessment to evaluate model performance using the external dataset returned calibration plots, a Scaled Brier Score of 0.095, and a Mean Absolute Calibration Error of 0.009. Test dataset model calibration performance indicated that the probability of a raised somatic cell count post-calving was well differentiated across probabilities to allow an end user to apply group-level risk decisions. Herd-level new intra-mammary infection rate during the dry period was a key driver of the probability that a cow had a raised SCC post-calving, highlighting the importance of optimizing environmental hygiene conditions. In conclusion, this research has determined that probabilistic classification of the risk of a raised SCC in the 30 days post-calving is achievable with a high degree of certainty, using routinely collected data. These predicted probabilities provide the opportunity for farmers to undertake risk decision-making by grouping cows based on their probabilities and optimizing management strategies for individual cows immediately after calving, according to their likelihood of intra-mammary infection.

External validation of machine learning algorithm predicting prolonged opioid prescriptions in opioid-naïve lumbar spine surgery patients using a Taiwanese cohort

BACKGROUND/PURPOSE

Identifying patients at risk of prolonged opioid use after surgery prompts appropriate prescription and personalized treatment plans. The Skeletal Oncology Research Group machine learning algorithm (SORG-MLA) was developed to predict the risk of prolonged opioid use in opioid-naive patients after lumbar spine surgery. However, its utility in a distinct country remains unknown.

METHODS

A Taiwanese cohort containing 2795 patients who were 20 years or older undergoing primary surgery for lumbar decompression from 2010 to 2018 were used to validate the SORG-MLA. Discrimination (area under receiver operating characteristic curve [AUROC] and area under precision-recall curve [AUPRC]), calibration, overall performance (Brier score), and decision curve analysis were applied.

RESULTS

Among 2795 patients, the prolonged opioid prescription rate was 5.2%. The validation cohort were older, more inpatient disposition, and more common pharmaceutical history of NSAIDs. Despite the differences, the SORG-MLA provided a good discriminative ability (AUROC of 0.71 and AURPC of 0.36), a good overall performance (Brier score of 0.044 compared to that of 0.039 in the developmental cohort). However, the probability of prolonged opioid prescription tended to be overestimated (calibration intercept of -0.07 and calibration slope of 1.45). Decision curve analysis suggested greater clinical net benefit in a wide range of clinical scenarios.

CONCLUSION

The SORG-MLA retained good discriminative abilities and overall performances in a geologically and medicolegally different region. It was suitable for predicting patients in risk of prolonged postoperative opioid use in Taiwan.

Machine learning as a tool to identify inpatients who are not at risk of adverse drug events in a large dataset of a tertiary care hospital in the USA

AIMS

Adverse drug events (ADEs) are a major threat to inpatients in the United States of America (USA). It is unknown how well machine learning (ML) is able to predict whether or not a patient will suffer from an ADE during hospital stay based on data available at hospital admission for emergency department patients of all ages (binary classification task). It is further unknown whether ML is able to outperform logistic regression (LR) in doing so, and which variables are the most important predictors.

METHODS

In this study, 5 ML models- namely a random forest, gradient boosting machine (GBM), ridge regression, least absolute shrinkage and selection operator (LASSO) regression, and elastic net regression-as well as a LR were trained and tested for the prediction of inpatient ADEs identified using ICD-10-CM codes based on comprehensive previous work in a diverse population. In total, 210 181 observations from patients who were admitted to a large tertiary care hospital after emergency department stay between 2011 and 2019 were included. The area under the receiver operating characteristics curve (AUC) and AUC-precision-recall (AUC-PR) were used as primary performance indicators.

RESULTS

Tree-based models performed best with respect to AUC and AUC-PR. The gradient boosting machine (GBM) reached an AUC of 0.747 (95% confidence interval (CI): 0.735 to 0.759) and an AUC-PR of 0.134 (95% CI: 0.131 to 0.137) on unforeseen test data, while the random forest reached an AUC of 0.743 (95% CI: 0.731 to 0.755) and an AUC-PR of 0.139 (95% CI: 0.135 to 0.142), respectively. ML statistically significantly outperformed LR both on AUC and AUC-PR. Nonetheless, overall, models did not differ much with respect to their performance. Most important predictors were admission type, temperature and chief complaint for the best performing model (GBM).

CONCLUSIONS

The study demonstrated a first application of ML to predict inpatient ADEs based on ICD-10-CM codes, and a comparison with LR. Future research should address concerns arising from low precision and related problems.

Endoscopic sleeve gastroplasty reintervention score using supervised machine learning

BACKGROUND AND AIMS

Reintervention after endoscopic sleeve gastroplasty (ESG) can be indicated because of postprocedural adverse events from various preinterventional or postprocedural comorbidities. We developed and internally validated an ESG reintervention score (ESG-RS) that determines the individualized risk of reintervention within the first 30 days after ESG.

METHODS

We used data from a sample of 3583 patients who underwent ESG in the Metabolic and Bariatric Surgery Accreditation Quality Improvement Program database (2016-2021). The least absolute shrinkage and selection operator (LASSO)-penalized regression was used to select the most promising predictors of reintervention after ESG within 30 days. The predictive variables extracted by LASSO regression were entered into multivariate analysis to generate an ESG-RS by using the coefficients of the statistically significant variables. The model performance was assessed using receiver-operator curves by 10-fold cross-validation.

RESULTS

Eleven variables were selected by LASSO regression and used in the final multivariate analysis. The ESG-RS was inferred using 5 factors (history of previous foregut surgery, preoperative anticoagulation use, female gender, American Society of Anesthesiologists class ≥II, and hypertension) weighted by their regression coefficients in the multivariable logistic regression model. The area under the curve of the ESG-RS was .74 (95% confidence interval, .70-.78). For the ESG-RS, the optimal cutpoint was 67.9 (high risk vs low risk), with a sensitivity of .76 and specificity of .71.

CONCLUSIONS

The ESG-RS aids clinicians in preoperative risk stratification of patients undergoing ESG while clarifying factors contributing to a higher risk of reintervention.

Predicting post-stroke cognitive impairment using machine learning: A prospective cohort study

BACKGROUND

Post-stroke cognitive impairment (PSCI) is a serious complication of stroke that warrants prompt detection and management. Consequently, the development of a diagnostic prediction model holds clinical significance.

OBJECTIVE

Machine learning algorithms were employed to identify crucial variables and forecast PSCI occurrence within 3-6 months following acute ischemic stroke (AIS).

METHODS

A prospective study was conducted on a developed cohort (331 patients) utilizing data from the Affiliated Zhongda Hospital of Southeast University between January 2022 and August 2022, as well as an external validation cohort (66 patients) from December 2022 to January 2023. The optimal model was determined by integrating nine machine learning classification models, and personalized risk assessment was facilitated by a Shapley Additive exPlanations (SHAP) interpretation.

RESULTS

Age, education, baseline National Institutes of Health Scale (NIHSS), Cerebral white matter degeneration (CWMD), Homocysteine (Hcy), and C-reactive protein (CRP) were identified as predictors of PSCI occurrence. Gaussian Naïve Bayes (GNB) model was determined to be the optimal model, surpassing other classifier models in the validation set (area under the curve [AUC]: 0.925, 95 % confidence interval [CI]: 0.861 - 0.988) and achieving the lowest Brier score. The GNB model performed well in the test sets (AUC: 0.919, accuracy: 0.864, sensitivity: 0.818, and specificity: 0.932).

CONCLUSIONS

The present study involved the development of a GNB model and its elucidation through employment of the SHAP method. These findings provide compelling evidence for preventing PSCI, which could serve as a guide for high-risk patients to undertake appropriate preventive measures.

Development and External Validation of a Machine Learning Model for Prediction of Lymph Node Metastasis in Patients with Prostate Cancer

BACKGROUND

Pelvic lymph node dissection (PLND) is the gold standard for diagnosis of lymph node involvement (LNI) in patients with prostate cancer. The Roach formula, Memorial Sloan Kettering Cancer Center (MSKCC) calculator, and Briganti 2012 nomogram are elegant and simple traditional tools used to estimate the risk of LNI and select patients for PLND.

OBJECTIVE

To determine whether machine learning (ML) can improve patient selection and outperform currently available tools for predicting LNI using similar readily available clinicopathologic variables.

DESIGN, SETTING, AND PARTICIPANTS

Retrospective data for patients treated with surgery and PLND between 1990 and 2020 in two academic institutions were used.

OUTCOME MEASUREMENTS AND STATISTICAL ANALYSIS

We trained three models (two logistic regression models and one gradient-boosted trees-based model [XGBoost]) on data provided from one institution (n = 20267) with age, prostate-specific antigen (PSA) levels, clinical T stage, percentage positive cores, and Gleason scores as inputs. We externally validated these models using data from another institution (n = 1322) and compared their performance to that of the traditional models using the area under the receiver operating characteristic curve (AUC), calibration, and decision curve analysis (DCA).

RESULTS AND LIMITATIONS

LNI was present in 2563 patients (11.9%) overall, and in 119 patients (9%) in the validation data set. XGBoost had the best performance among all the models. On external validation, its AUC outperformed that of the Roach formula by 0.08 (95% confidence interval [CI] 0.042-0.12), the MSKCC nomogram by 0.05 (95% CI 0.016-0.070), and the Briganti nomogram by 0.03 (95% CI 0.0092-0.051; all p < 0.05). It also had better calibration and clinical utility in terms of net benefit on DCA across relevant clinical thresholds. The main limitation of the study is its retrospective design.

CONCLUSIONS

Taking all measures of performance together, ML using standard clinicopathologic variables outperforms traditional tools in predicting LNI.

PATIENT SUMMARY

Determining the risk of cancer spread to the lymph nodes in patients with prostate cancer allows surgeons to perform lymph node dissection only in patients who need it and avoid the side effects of the procedure in those who do not. In this study, we used machine learning to develop a new calculator to predict the risk of lymph node involvement that outperformed traditional tools currently used by oncologists.

Predicting whether patients will achieve minimal clinically important differences following hip or knee arthroplasty

Aims

A substantial fraction of patients undergoing knee arthroplasty (KA) or hip arthroplasty (HA) do not achieve an improvement as high as the minimal clinically important difference (MCID), i.e. do not achieve a meaningful improvement. Using three patient-reported outcome measures (PROMs), our aim was: 1) to assess machine learning (ML), the simple pre-surgery PROM score, and logistic-regression (LR)-derived performance in their prediction of whether patients undergoing HA or KA achieve an improvement as high or higher than a calculated MCID; and 2) to test whether ML is able to outperform LR or pre-surgery PROM scores in predictive performance.

Methods

MCIDs were derived using the change difference method in a sample of 1,843 HA and 1,546 KA patients. An artificial neural network, a gradient boosting machine, least absolute shrinkage and selection operator (LASSO) regression, ridge regression, elastic net, random forest, LR, and pre-surgery PROM scores were applied to predict MCID for the following PROMs: EuroQol five-dimension, five-level questionnaire (EQ-5D-5L), EQ visual analogue scale (EQ-VAS), Hip disability and Osteoarthritis Outcome Score-Physical Function Short-form (HOOS-PS), and Knee injury and Osteoarthritis Outcome Score-Physical Function Short-form (KOOS-PS).

Results

Predictive performance of the best models per outcome ranged from 0.71 for HOOS-PS to 0.84 for EQ-VAS (HA sample). ML statistically significantly outperformed LR and pre-surgery PROM scores in two out of six cases.

Conclusion

MCIDs can be predicted with reasonable performance. ML was able to outperform traditional methods, although only in a minority of cases.

Decreased Expression of GLYATL1 Predicts Poor Prognosis in Patients with Clear Cell Renal Cell Carcinoma

Background

GLYATL1 is a member of the glycine-N-acyltransferase family, which catalyses acyl group transfer. The role of GLYATL1 in cancer is largely unknown; therefore, the potential value of GLYATL1 in clear cell renal cell carcinoma (ccRCC) was explored.

Methods

The ccRCC gene expression profiles and clinical data were obtained from the University of California Santa Cruz Xena platform. Differential expression and survival analysis were performed using R software. Samples from the TIMER public database and real-world were used for validation. The potential molecular mechanism of GLYATL1 in ccRCC was explored using gene set enrichment analysis (GSEA).

Results

GLYATL1 was downregulated, indicating a poor prognosis in ccRCC. Low expression of GLYATL1 was significantly associated with advanced stage and higher histological grade ccRCC. The differential expression of GLYATL1 was validated at the protein level using clinical samples and tissue microarray. The results of GSEA showed that multiple crucial signalling pathways including fatty acid metabolism, adipogenesis, oxidative phosphorylation and epithelial-mesenchymal transition were enriched.

Conclusion

This study demonstrated that GLYATL1 downregulation has an unfavourable impact on the survival of patients with ccRCC. The resulting data indicated that GLYATL1 could be a potential new target for ccRCC therapy, which may be helpful for the personalized treatment of such individuals.

Can a Nomogram Predict Survival After Treatment for an Ankylosing Spondylitis Cervical Fracture in a Patient With Neurologic Impairment? A National, Multicenter Study

BACKGROUND

Ankylosing spondylitis-related cervical spine fracture with neurologic impairment (ASCF-NI) is a rare but often lethal injury. Factors independently associated with survival after treatment remain poorly defined, and identifying patients who are likely to survive the injury remains challenging.

QUESTIONS/PURPOSES

(1) What factors are independently associated with survival after treatment among patients with ASCF-NI? (2) Can a nomogram be developed that is sufficiently simple for clinicians to use that can identify patients who are the most likely to survive after injury?

METHODS

This retrospective study was conducted based on a multi-institutional group of patients admitted and treated at one of 29 tertiary hospitals in China between March 1, 2003, and July 31, 2019. A total of 363 patients with a mean age of 53 ± 12 years were eventually included, 343 of whom were male. According to the National Household Registration Management System, 17% (61 of 363) died within 5 years of injury. Patients were treated using nonsurgical treatment or surgery, including procedures using the anterior approach, posterior approach, or combined anterior and posterior approaches. Indications for surgery included three-column injury, unstable fracture displacement, neurologic impairment or continuous progress, and intervertebral disc incarceration. By contrast, patients generally received nonsurgical treatment when they had a relatively stable fracture or medical conditions that did not tolerate surgery. Demographic, clinical, and treatment data were collected. The primary study goal was to identify which factors are independently associated with death within 5 years of injury, and the secondary goal was the development of a clinically applicable nomogram. We developed a multivariable Cox hazards regression model, and independent risk factors were defined by backward stepwise selection with the Akaike information criterion. We used these factors to create a nomogram using a multivariate Cox proportional hazards regression analysis.

RESULTS

After controlling for potentially confounding variables, we found the following factors were independently associated with a lower likelihood of survival after injury: lower fracture site, more-severe peri-injury complications, poorer American Spinal Injury Association (ASIA) Impairment Scale, and treatment methods. We found that a C5 to C7 or T1 fracture (ref: C1 to C4 and 5; hazard ratio 1.7 [95% confidence interval 0.9 to 3.5]; p = 0.12), moderate peri-injury complications (ref: absence of or mild complications; HR 6.0 [95% CI 2.3 to 16.0]; p < 0.001), severe peri-injury complications (ref: absence of or mild complications; HR 30.0 [95% CI 11.5 to 78.3]; p < 0.001), ASIA Grade A (ref: ASIA Grade D; HR 2.8 [95% CI 1.1 to 7.0]; p = 0.03), anterior approach (ref: nonsurgical treatment; HR 0.5 [95% CI 0.2 to 1.0]; p = 0.04), posterior approach (ref: nonsurgical treatment; HR 0.4 [95% CI 0.2 to 0.8]; p = 0.006), and combined anterior and posterior approach (ref: nonsurgical treatment; HR 0.4 [95% CI 0.2 to 0.9]; p = 0.02) were associated with survival. Based on these factors, a nomogram was developed to predict the survival of patients with ASCF-NI after treatment. Tests revealed that the developed nomogram had good performance (C statistic of 0.91).

CONCLUSION

The nomogram developed in this study will allow us to classify patients with different mortality risk levels into groups. This, coupled with the factors we identified, was independently associated with survival, and can be used to guide more appropriate treatment and care strategies for patients with ASCF-NI.

LEVEL OF EVIDENCE

Level III, therapeutic study.

Classification of imbalanced data using machine learning algorithms to predict the risk of renal graft failures in Ethiopia

INTRODUCTION

The prevalence of end-stage renal disease has raised the need for renal replacement therapy over recent decades. Even though a kidney transplant offers an improved quality of life and lower cost of care than dialysis, graft failure is possible after transplantation. Hence, this study aimed to predict the risk of graft failure among post-transplant recipients in Ethiopia using the selected machine learning prediction models.

METHODOLOGY

The data was extracted from the retrospective cohort of kidney transplant recipients at the Ethiopian National Kidney Transplantation Center from September 2015 to February 2022. In response to the imbalanced nature of the data, we performed hyperparameter tuning, probability threshold moving, tree-based ensemble learning, stacking ensemble learning, and probability calibrations to improve the prediction results. Merit-based selected probabilistic (logistic regression, naive Bayes, and artificial neural network) and tree-based ensemble (random forest, bagged tree, and stochastic gradient boosting) models were applied. Model comparison was performed in terms of discrimination and calibration performance. The best-performing model was then used to predict the risk of graft failure.

RESULTS

A total of 278 completed cases were analyzed, with 21 graft failures and 3 events per predictor. Of these, 74.8% are male, and 25.2% are female, with a median age of 37. From the comparison of models at the individual level, the bagged tree and random forest have top and equal discrimination performance (AUC-ROC = 0.84). In contrast, the random forest has the best calibration performance (brier score = 0.045). Under testing the individual model as a meta-learner for stacking ensemble learning, the result of stochastic gradient boosting as a meta-learner has the top discrimination (AUC-ROC = 0.88) and calibration (brier score = 0.048) performance. Regarding feature importance, chronic rejection, blood urea nitrogen, number of post-transplant admissions, phosphorus level, acute rejection, and urological complications are the top predictors of graft failure.

CONCLUSIONS

Bagging, boosting, and stacking, with probability calibration, are good choices for clinical risk predictions working on imbalanced data. The data-driven probability threshold is more beneficial than the natural threshold of 0.5 to improve the prediction result from imbalanced data. Integrating various techniques in a systematic framework is a smart strategy to improve prediction results from imbalanced data. It is recommended for clinical experts in kidney transplantation to use the final calibrated model as a decision support system to predict the risk of graft failure for individual patients.

Development and internal validation of nomograms based on plasma metabolites to predict non-small cell lung cancer risk in smoking and nonsmoking populations

BACKGROUND

Lung cancer has significantly higher incidence and mortality rates worldwide. In this study, we analyzed the metabolic profiles of non-small cell lung cancer (NSCLC) patients and constructed prediction models for smokers and nonsmokers with internal validation.

METHODS

Plasma was collected from all patients enrolled for metabolic profiling by liquid chromatography-tandem mass spectrometry (LC-MS/MS). The total population was divided into two groups according to smoking or not. Statistical analysis of metabolites was performed separately for each group and prediction models were constructed.

RESULTS

A total of 1723 patients (1109 NSCLC patients and 614 healthy controls) were enrolled from the affiliated hospital during 2018 to 2021. After grouping by smoking history, each group was statistically analyzed and prediction models were constructed, which resulted in eight indicators (propionylcarnitine, arginine, citrulline, etc.) significantly associated with lung cancer risk for smokers and eight indicators (dodecanoylcarnitine, hydroxybutyrylcarnitine, asparagine, etc.) for nonsmokers (p < 0.05). The smoker model indicated an AUC of 0.860 in the training set and 0.850 in the validation set. The nonsmoker model showed an AUC of 0.783 in the training set and 0.762 in the validation set. Further calibration tests for both models indicated excellent goodness-of-fit results.

CONCLUSIONS

In this study, we found a series of metabolites significantly associated with lung cancer incidence and constructed respectively prediction models for NSCLC risk in smokers and nonsmokers, with internal validation to confirm the efficiency to discriminate lung cancer risk in both smoking and nonsmoking states.

Interpretation and Use of Applied/Operational Machine Learning and Artificial Intelligence in Surgery

Applications for artificial intelligence (AI) and machine learning in surgery include image interpretation, data summarization, automated narrative construction, trajectory and risk prediction, and operative navigation and robotics. The pace of development has been exponential, and some AI applications are working well. However, demonstrations of clinical utility, validity, and equity have lagged algorithm development and limited widespread adoption of AI into clinical practice. Outdated computing infrastructure and regulatory challenges which promote data silos are key barriers. Multidisciplinary teams will be needed to address these challenges and to build AI systems that are relevant, equitable, and dynamic.

Establishment and Validation of Predictive Model of Tophus in Gout Patients

(1) Background: A tophus is a clinical manifestation of advanced gout, and in some patients could lead to joint deformities, fractures, and even serious complications in unusual sites. Therefore, to explore the factors related to the occurrence of tophi and establish a prediction model is clinically significant. (2) Objective: to study the occurrence of tophi in patients with gout and to construct a predictive model to evaluate its predictive efficacy. (3) Methods: The clinical data of 702 gout patients were analyzed by using cross-sectional data of North Sichuan Medical College. The least absolute shrinkage and selection operator (LASSO) and multivariate logistic regression were used to analyze predictors. Multiple machine learning (ML) classification models are integrated to analyze and identify the optimal model, and Shapley Additive exPlanations (SHAP) interpretation was developed for personalized risk assessment. (4) Results: Compliance of urate-lowering therapy (ULT), Body Mass Index (BMI), course of disease, annual attack frequency, polyjoint involvement, history of drinking, family history of gout, estimated glomerular filtration rate (eGFR), and erythrocyte sedimentation rate (ESR) were the predictors of the occurrence of tophi. Logistic classification model was the optimal model, test set area under curve (AUC) (95% confidence interval, CI): 0.888 (0.839-0.937), accuracy: 0.763, sensitivity: 0.852, and specificity: 0.803. (5) Conclusions: We constructed a logistic regression model and explained it with the SHAP method, providing evidence for preventing tophus and guidance for individual treatment of different patients.

External validation of the SWEDEHEART score for predicting in-hospital major bleeding among East Asian patients with acute myocardial infarction

Background

Risk scores for predicting in-hospital major bleeding in patients with acute myocardial infarction (AMI) are rare. The Swedish web-system for the enhancement and development of evidence-based care in heart disease evaluated according to recommended therapies (SWEDEHEART) score (SS), consisting of five common clinical variables, is a novel model for predicting in-hospital major bleeding. External validation of SS has not yet been completed.

Methods and results

A retrospective study recruiting consecutive East Asian patients diagnosed with AMI was conducted in the Second Affiliated Hospital, Zhejiang University. The primary endpoint was the ability of SS to predict in-hospital major bleeding, which was defined as Bleeding Academic Research Consortium (BARC) type 3 or 5 bleeding. To validate SS, the discrimination and calibration were assessed in the overall population and several subgroups. The receiver operating characteristic (ROC) curves and the areas under ROC curves (AUCs) were calculated for discrimination. The calibration of SS was evaluated with the unreliability U test. A total of 2,841 patients diagnosed with AMI during hospitalization were included, and 1.94% (55) of them experienced in-hospital major bleeding events. The AUC of SS for the whole population was only 0.60 [95% confidence interval (CI), 0.52-0.67], without an acceptable calibration (p = 0.001). Meanwhile, the highest AUC (0.72; 95% CI, 0.61-0.82) of SS for the primary endpoint was found in the diabetes subgroup, with an acceptable calibration (p = 0.87).

Conclusion

This external validation study showed that SS failed to exhibit sufficient accuracy in predicting in-hospital major bleeding among East Asian patients with AMI despite demonstrating acceptable performance in the diabetic subgroup of patients. Studies to uncover optimal prediction tools for in-hospital major bleeding risk in AMI are urgently warranted.

Identifying adolescents at risk for depression: Assessment of a global prediction model in the Great Smoky Mountains Study

The Identifying Depression Early in Adolescence Risk Score (IDEA-RS) has been externally assessed in samples from four continents, but North America is lacking. Our aim here was to evaluate the performance of the IDEA-RS in predicting future onset of Major Depressive Disorder (MDD) in an adolescent population-based sample in the United States of America - the Great Smoky Mountains Study (GSMS). We applied the intercept and weights of the original IDEA-RS model developed in Brazil to generate individual probabilities for each participant of the GSMS at age 15 (N = 1029). We then evaluated the performance of such predictions against the diagnosis of MDD at age 19 using simple, case-mix corrected and refitted models. Furthermore, we compared how prioritizing the information provided by parents or by adolescents affected performance. The IDEA-RS exhibited a C-statistic of 0.63 (95% CI 0.53-0.74) to predict MDD in the GSMS when applying uncorrected weights. Case-mix corrected and refitted models enhanced performance to 0.69 and 0.67, respectively. No significant difference was found in performance by prioritizing the reports of adolescents or their parents. The IDEA-RS was able to parse out adolescents at risk for a later onset of depression in the GSMS cohort with above chance discrimination. The IDEA-RS has now showed above-chance performance in five continents.

Enhanced whale optimization algorithm for medical feature selection: A COVID-19 case study

The whale optimization algorithm (WOA) is a prominent problem solver which is broadly applied to solve NP-hard problems such as feature selection. However, it and most of its variants suffer from low population diversity and poor search strategy. Introducing efficient strategies is highly demanded to mitigate these core drawbacks of WOA particularly for dealing with the feature selection problem. Therefore, this paper is devoted to proposing an enhanced whale optimization algorithm named E-WOA using a pooling mechanism and three effective search strategies named migrating, preferential selecting, and enriched encircling prey. The performance of E-WOA is evaluated and compared with well-known WOA variants to solve global optimization problems. The obtained results proved that the E-WOA outperforms WOA's variants. After E-WOA showed a sufficient performance, then, it was used to propose a binary E-WOA named BE-WOA to select effective features, particularly from medical datasets. The BE-WOA is validated using medical diseases datasets and compared with the latest high-performing optimization algorithms in terms of fitness, accuracy, sensitivity, precision, and number of features. Moreover, the BE-WOA is applied to detect coronavirus disease 2019 (COVID-19) disease. The experimental and statistical results prove the efficiency of the BE-WOA in searching the problem space and selecting the most effective features compared to comparative optimization algorithms.

Development of a multivariable prediction model to identify dairy calves too young to be transported to auction markets in Canada using simple physical examination and body weight

Calves born on Eastern Canadian dairy farms that are not kept in the herds are traditionally sold through auction markets and are raised for meat purposes such as veal calves. Since February 2020, a new Canadian federal regulation has forbidden calves <9 d old to be sold through auction markets. However, in the absence of a real-time birth registry consultation system, it would be of interest to look for predictors that could be associated with age to allow identification of calves too young to be transported. In the current retrospective cross-sectional study, 1,178 calves with a declared birth date (411 calves aged <9 d old; 34.9%) were assessed in 2 large Québec auction sites. Easy-to-record covariates [body weight (BW), breed phenotype, and presence of an umbilical cord remnant] as well as other clinical signs (umbilical swelling, enlargement, umbilical pain, wet umbilicus, skin tent, sunken eyes, ocular and nasal secretion, and hide cleanliness) were assessed. Two logistic regression models using age as a dichotomous dependent variable (<9 d old vs ≥9 d old) were built. The first model (model 1) considered all covariates, which were selected after univariable analyses and a backward stepwise selection process, whereas a more pragmatic model (model 2) only included the 3 easy-to-record variables (i.e., BW, breed, umbilical cord). Both models had similar accuracy to detect calves <9 d old (sensitivity of 38.4 and 37.5%, and specificity of 85.7 and 84.6% for model 1 and 2, respectively). Model 2 was subsequently more specifically studied as it employs a faster and easier assessment. Decision thresholds were tested for their robustness based on misclassification cost term (MCT) analysis with various prevalence of calves <9 d old and various costs of false-negative:false-positive ratio. Despite statistical significance, model accuracy, even if refined with MCT analysis, was limited at the individual level, showing the limits of using physical signs and BW or their combination as a reliable proxy of age. The sensitivity of these models to find calves <9 d old was not to be used for monitoring compliance with the Canadian federal regulation. The relatively high model specificity may help to use this model as a rule-in test (i.e., targeting positive calves for further investigation) rather than a rule-out test (due to its low sensitivity).

Development and Validation of Nomograms to Predict Overall Survival and Cancer-Specific Survival in Patients With Pancreatic Adenosquamous Carcinoma

Background

Pancreatic adenosquamous carcinoma (PASC) is a heterogeneous group of primary pancreatic cancers characterized by the coexistence of both glandular and squamous differentiation. The aim of this study was to develop nomograms to predict survival outcomes in patients with PASC.

Methods

In this retrospective study, data on PASC, including clinicopathological characteristics, treatments, and survival outcomes, were collected from the SEER database between 2000 and 2018. The primary endpoints were overall survival (OS) and cancer-specific survival (CSS). The eligible patients were randomly divided into development cohort and validation cohort in a 7:3 ratio. The nomograms for prediction of OS and CSS were constructed by the development cohort using a LASSO-Cox regression model, respectively. Besides the model performance was internally and externally validated by examining the discrimination, calibration, and clinical utility.

Results

A total of 632 consecutive patients who had been diagnosed with PASC were identified and randomly divided into development (n = 444) and validation (n = 188) cohorts. In the development cohort, the estimated median OS was 7.0 months (95% CI: 6.19-7.82) and the median CSS was 7.0 months (95% CI: 6.15-7.85). In the validation cohort, the estimated median OS was 6.0 months (95% CI: 4.46-7.54) and the median CSS was 7.0 months (95% CI: 6.25-7.75). LASSO-penalized COX regression analysis identified 8 independent predictors in the OS prediction model and 9 independent risk factors in the CSS prediction model: age at diagnosis, gender, year of diagnosis, tumor location, grade, stage, size, lymph node metastasis, combined metastasis, surgery, radiation, and chemotherapy. The Harrell C index and time-dependent AUCs manifested satisfactory discriminative capabilities of the models. Calibration plots showed that both models were well calibrated. Furthermore, decision curves indicated good utility of the nomograms for decision-making.

Conclusion

Nomogram-based models to evaluate personalized OS and CSS in patients with PASC were developed and well validated. These easy-to-use tools will be useful methods to calculate individualized estimate of survival, assist in risk stratification, and aid clinical decision-making.

Downregulation of PPA2 expression correlates with poor prognosis of kidney renal clear cell carcinoma

Background

Kidney renal clear cell carcinoma (KIRC) is the most common subtype of kidney cancer. Inorganic pyrophosphatase (PPA2) is an enzyme that catalyzes the hydrolysis of pyrophosphate to inorganic phosphate; few studies have reported its significance in cancers. Therefore, we aimed to explore the prognostic value of PPA2 in KIRC.

Methods

PPA2 expression was detected via immunohistochemistry in a tissue chip containing specimens from 150 patients with KIRC. We evaluated the correlation between PPA2 expression, clinicopathological characteristics, and survival. Data from online databases and another cohort (paraffin-embedded specimens from 10 patients with KIRC) were used for external validation.

Results

PPA2 expression was significantly lower in KIRC tissues than in normal renal tissues (p < 0.0001). Low expression of PPA2 was significantly associated with a high histologic grade and poor prognosis. The differential expression of PPA2 was validated at the gene and protein levels. Multivariate Cox regression analysis showed that PPA2 expression was an independent prognostic factor in patients with KIRC. Gene set enrichment analysis suggested that decreased expression of PPA2 might be related to the epithelial-mesenchymal transition in KIRC.

Conclusions

Our study demonstrated that PPA2 is an important energy metabolism-associated biomarker correlated with a favorable prognosis in KIRC.

Development and external validation of a risk calculator to predict internalising symptoms among Ugandan youths affected by HIV

The aim of the study was to develop and externally validate a model to predict individualized risk of internalizing symptoms among AIDS-affected youths in low-resource settings in sub-Saharan Africa. Longitudinal data from 558 Ugandan adolescents orphaned by AIDS was used to develop our predictive model. Least Absolute Shrinkage and Selection Operator logistic regression was used to select the best subset of predictors using 10-fold cross-validation. External validation of the final model was conducted in a sample of 372 adolescents living with HIV in Uganda. Best predictors for internalizing symptoms were gender, family cohesion, social support, asset ownership, recent sexually transmitted infection (STI) diagnosis, physical health self-rating, and previous poor mental health; area under the curve (AUC)  = 72.2; 95% CI  = 67.9-76.5. For adolescents without history of internalizing symptoms, the AUC = 69.0, 95% CI = 63.4-74.6, and was best predicted by gender, drug use, social support, asset ownership, recent STI diagnosis, and physical health self-rating. Both models were well calibrated. External validation in adolescents living with HIV sample was similar, AUC = 69.7; 95% CI = 64.1-75.2. The model predicted internalizing symptoms among African AIDS-affected youth reasonably well and showed good generalizability. The model offers opportunities for the design of public health interventions addressing poor mental health among youth affected by HIV/AIDS.

Machine Learning Prediction of Death in Critically Ill Patients With Coronavirus Disease 2019

OBJECTIVES

Critically ill patients with coronavirus disease 2019 have variable mortality. Risk scores could improve care and be used for prognostic enrichment in trials. We aimed to compare machine learning algorithms and develop a simple tool for predicting 28-day mortality in ICU patients with coronavirus disease 2019.

DESIGN

This was an observational study of adult patients with coronavirus disease 2019. The primary outcome was 28-day inhospital mortality. Machine learning models and a simple tool were derived using variables from the first 48 hours of ICU admission and validated externally in independent sites and temporally with more recent admissions. Models were compared with a modified Sequential Organ Failure Assessment score, National Early Warning Score, and CURB-65 using the area under the receiver operating characteristic curve and calibration.

SETTING

Sixty-eight U.S. ICUs.

PATIENTS

Adults with coronavirus disease 2019 admitted to 68 ICUs in the United States between March 4, 2020, and June 29, 2020.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

The study included 5,075 patients, 1,846 (36.4%) of whom died by day 28. eXtreme Gradient Boosting had the highest area under the receiver operating characteristic curve in external validation (0.81) and was well-calibrated, while k-nearest neighbors were the lowest performing machine learning algorithm (area under the receiver operating characteristic curve 0.69). Findings were similar with temporal validation. The simple tool, which was created using the most important features from the eXtreme Gradient Boosting model, had a significantly higher area under the receiver operating characteristic curve in external validation (0.78) than the Sequential Organ Failure Assessment score (0.69), National Early Warning Score (0.60), and CURB-65 (0.65; p < 0.05 for all comparisons). Age, number of ICU beds, creatinine, lactate, arterial pH, and Pao2/Fio2 ratio were the most important predictors in the eXtreme Gradient Boosting model.

CONCLUSIONS

eXtreme Gradient Boosting had the highest discrimination overall, and our simple tool had higher discrimination than a modified Sequential Organ Failure Assessment score, National Early Warning Score, and CURB-65 on external validation. These models could be used to improve triage decisions and clinical trial enrichment.

Development of EndoScreen Chip, a Microfluidic Pre-Endoscopy Triage Test for Esophageal Adenocarcinoma

The current endoscopy and biopsy diagnosis of esophageal adenocarcinoma (EAC) and its premalignant condition Barrett's esophagus (BE) is not cost-effective. To enable EAC screening and patient triaging for endoscopy, we developed a microfluidic lectin immunoassay, the EndoScreen Chip, which allows sensitive multiplex serum biomarker measurements. Here, we report the proof-of-concept deployment for the EAC biomarker Jacalin lectin binding complement C9 (JAC-C9), which we previously discovered and validated by mass spectrometry. A monoclonal C9 antibody (m26 3C9) was generated and validated in microplate ELISA, and then deployed for JAC-C9 measurement on EndoScreen Chip. Cohort evaluation (n = 46) confirmed the expected elevation of serum JAC-C9 in EAC, along with elevated total serum C9 level. Next, we asked if the small panel of serum biomarkers improves detection of EAC in this cohort when used in conjunction with patient risk factors (age, body mass index and heartburn history). Using logistic regression modeling, we found that serum C9 and JAC-C9 significantly improved EAC prediction from AUROC of 0.838 to 0.931, with JAC-C9 strongly predictive of EAC (vs. BE OR = 4.6, 95% CI: 1.6-15.6, p = 0.014; vs. Healthy OR = 4.1, 95% CI: 1.2-13.7, p = 0.024). This proof-of-concept study confirms the microfluidic EndoScreen Chip technology and supports the potential utility of blood biomarkers in improving triaging for diagnostic endoscopy. Future work will expand the number of markers on EndoScreen Chip from our list of validated EAC biomarkers.

Predicting the individualized risk of poor adherence to ART medication among adolescents living with HIV in Uganda: the Suubi+Adherence study

INTRODUCTION

Achieving optimal adherence to antiretroviral therapy (ART) among adolescents living with HIV (ALWHIV) is challenging, especially in low-resource settings. To help accurately determine who is at risk of poor adherence, we developed and internally validated models comprising multi-level factors that can help to predict the individualized risk of poor adherence among ALWHIV in a resource-limited setting such as Uganda.

METHODS

We used data from a sample of 637 ALWHIV in Uganda who participated in a longitudinal study, "Suubi+Adherence" (2012 to 2018). The model was developed using the Least Absolute Shrinkage and Selection Operator (LASSO) penalized regression to select the best subset of multi-level predictors (individual, household, community or economic-related factors) of poor adherence in one year's time using 10-fold cross-validation. Seventeen potential predictors included in the model were assessed at 36 months of follow-up, whereas adherence was assessed at 48 months of follow-up. Model performance was evaluated using discrimination and calibration measures.

RESULTS

For the model predicting poor adherence, five of the 17 predictors (adherence history, adherence self-efficacy, family cohesion, child poverty and group assignment) were retained. Its ability to discriminate between individuals with and without poor adherence was acceptable; area under the curve (AUC) = 69.9; 95% CI: 62.7, 72.8. There was no evidence of possible areas of miscalibration (test statistic = 1.20; p = 0.273). The overall performance of the model was good.

CONCLUSIONS

Our findings support prediction modelling as a useful tool that can be leveraged to improve outcomes across the HIV care continuum. Utilizing information from multiple sources, the risk prediction score tool applied here can be refined further with the ultimate goal of being used in a screening tool by practitioners working with ALWHIV. Specifically, the tool could help identify and provide early interventions to adolescents at the highest risk of poor adherence and/or viral non-suppression. However, further fine-tuning and external validation may be required before wide-scale implementation.

A Validated Prognostic Model and Nomogram to Predict Early-Onset Complications Leading to Surgery in Patients With Crohn's Disease

BACKGROUND

Predicting aggressive Crohn's disease is crucial for determining therapeutic strategies.

OBJECTIVE

We aimed to develop a prognostic model to predict complications leading to surgery within 1 year after diagnosis of Crohn's disease and to create a nomogram to facilitate clinical decision making.

DESIGN

This is a retrospective study.

SETTING

This study was conducted from January 2012 to December 2016 in a single tertiary IBD center.

PATIENTS

Patients diagnosed with Crohn's disease showing B1 behavior according to the Montreal classification were included.

MAIN OUTCOME MEASURES

We measured the occurrence of complications that would ultimately lead to surgery, including severe GI bleeding (Glasgow-Blatchford score ≥6), stenosis, and perforations, confirmed by endoscopy, CT scan, and/or interventional radiology.

RESULTS

The mean follow-up period was 54 months (SD 13 months). Of the 614 eligible patients, 13.5% developed complications leading to surgery. Multivariable logistic regression revealed the independent predictors of early-onset complications to be age (adjusted odds ratio per 10-year increase in age = 0.4; 95% CI, 0.2-0.8; p = 0.004), disease duration (adjusted odds ratio = 2.7, 95% CI, 1.9-3.8; p < 0.001), perianal disease (adjusted odds ratio = 16.0; 95% CI, 4.3-59.9; p < 0.001), previous surgery (adjusted odds ratio = 3.7; 95% CI, 1.6-8.6; p = 0.003), and extraintestinal manifestations (adjusted odds ratio = 7.6; 95% CI, 2.3-24.9; p = 0.001). The specificity and sensitivity of the prognostic model were 88.3% (95% CI, 84.8%-91.2%) and 96.6% (95% CI, 88.1%-99.6%), and the area under the curve was 0.97 (95% CI, 0.95-0.98). This model was validated with good discrimination and excellent calibration using the Hosmer-Lemeshow goodness-of-fit test. A nomogram was created to facilitate clinical bedside practice.

LIMITATIONS

This was a retrospective design and included a small sample size from 1 center.

CONCLUSIONS

Our validated prognostic model effectively predicted early-onset complications leading to surgery and screened aggressive Crohn's disease, which will enable physicians to customize therapeutic strategies and monitor disease. See Video Abstract at http://links.lww.com/DCR/B442.Registered at Chinese Clinical Trial Registry (ChiCTR1900025751).

UN MODELO DE PRONSTICO VALIDADO Y UN NOMOGRAMA PARA PREDECIR COMPLICACIONES PRECOCES QUE REQUIRAN CIRUGA EN PACIENTES CON ENFERMEDAD DE CROHN

ANTECEDENTES:Predecir una enfermedad de Crohn muy agresiva es fundamental para determinar la estrategia terapéutica.OBJETIVO:Desarrollar un modelo de pronóstico para predecir las complicaciones que requieran cirugía dentro el primer año al diagnóstico de enfermedad de Crohn y crear un nomograma para facilitar la toma de decisiones clínicas.DISEÑO:El presente etudio es retrospectivo.AJUSTE:Estudio realizado entre Enero 2012 y Diciembre 2016, en un único centro terciario de tratamiento de enfermedad inflamatoria intestinal.PACIENTES:Se incluyeron todos aquellos pacientes diagnosticados de enfermedad de Crohn que mostraban manifestaciones tipo B1 según la clasificación de Montreal.PRINCIPALES MEDIDAS DE RESULTADO:Medimos la aparición de complicaciones que finalmente conducirían a una cirugía, incluida la hemorragia digestiva grave (puntuación de Glasgow-Blatchford ≥ 6), estenosis y perforaciones, confirmadas por endoscopía, tomografía computarizada y / o radiología intervencionista.RESULTADOS:El período medio de seguimiento fue de 54 meses (desviación estándar 13 meses). De los 614 pacientes elegibles, el 13,5% desarrolló complicaciones que llevaron a cirugía. La regresión logística multivariable reveló que los predictores independientes de complicaciones de inicio temprano eran la edad (razón de probabilidades ajustada [ORa] por aumento de 10 años en la edad = 0,4; intervalos de confianza del 95% [IC del 95%]: 0,2-0,8, p = 0,004), duración de la enfermedad (ORa = 2,7, IC del 95%: 1,9-3,8, p <0,001), enfermedad perianal (ORa = 16,0, IC del 95%: 4,3-59,9, p <0,001), cirugía previa (ORa = 3,7, 95% IC: 1,6-8,6, p = 0,003) y manifestaciones extraintestinales (ORa = 7,6, IC del 95%: 2,3-24,9, p = 0,001). La especificidad y sensibilidad del modelo pronóstico fueron 88,3% (IC 95%: 84,8% -91,2%) y 96,6% (IC 95%: 88,1% -99,6%), respectivamente, y el área bajo la curva fue 0,97 (95% % CI: 0,95-0,98). Este modelo fue validado con buena discriminación y excelente calibración utilizando la prueba de bondad de ajuste de Hosmer-Lemeshow. Se creó un nomograma para facilitar la práctica clínica al pié de la cama.LIMITACIONES:Diseño retrospectivo que incluyó un tamaño de muestra pequeña en un solo centro.CONCLUSIONES:Nuestro modelo de pronóstico validado predijo eficazmente las complicaciones precoces que conllevaron a cirugía y la detección de enfermedad de Crohn agresiva, lo que permitió a los médicos personalizar las estrategias terapéuticas y controlar la enfermedad. Consulte Video Resumen en http://links.lww.com/DCR/B442.Registrado en el Registro de Ensayos Clínicos de China (ChiCTR1900025751).

Derivation and validation of actionable quality indicators targeting reductions in complications for injury admissions

PURPOSE

Approximately, one out of five patients hospitalized following injury will develop at least one hospital complication, more than three times that observed for general admissions. We currently lack actionable Quality Indicators (QI) targeting specific complications in this population. We aimed to derive and validate QI targeting hospital complications for injury admissions and develop algorithms to identify patient charts to review.

METHODS

We conducted a retrospective cohort study including patients with major trauma admitted to any level I or II adult trauma center an integrated Canadian trauma system (2014-2019). We used the trauma registry to develop five QI targeting deep vein thrombosis/pulmonary embolism (DVT/PE), decubitus ulcers, delirium, pneumonia and urinary tract infection (UTI). We developed algorithms to identify patient charts to revise on consultation with a group of clinical experts.

RESULTS

The study population included 14,592 patients of whom 5.3% developed DVT or PE, 2.7% developed a decubitus ulcer, 8.6% developed delirium, 14.7% developed pneumonia and 7.3% developed UTI. The indicators demonstrated excellent predictive performance (Area Under the Curve 0.81-0.87). We identified 4 hospitals with a higher than average incidence of at least one of the targeted complications. The algorithms identified on average 50 and 20 charts to be reviewed per year for level I and II centers, respectively.

CONCLUSION

In line with initiatives to improve the quality of trauma care, we propose QI targeting reductions in hospital complications for injury admissions and algorithms to generate case lists to facilitate the review of patient charts.

Machine-learning based prediction of Cushing's syndrome in dogs attending UK primary-care veterinary practice

Cushing’s syndrome is an endocrine disease in dogs that negatively impacts upon the quality-of-life of affected animals. Cushing’s syndrome can be a challenging diagnosis to confirm, therefore new methods to aid diagnosis are warranted. Four machine-learning algorithms were applied to predict a future diagnosis of Cushing's syndrome, using structured clinical data from the VetCompass programme in the UK. Dogs suspected of having Cushing's syndrome were included in the analysis and classified based on their final reported diagnosis within their clinical records. Demographic and clinical features available at the point of first suspicion by the attending veterinarian were included within the models. The machine-learning methods were able to classify the recorded Cushing’s syndrome diagnoses, with good predictive performance. The LASSO penalised regression model indicated the best overall performance when applied to the test set with an AUROC = 0.85 (95% CI 0.80–0.89), sensitivity = 0.71, specificity = 0.82, PPV = 0.75 and NPV = 0.78. The findings of our study indicate that machine-learning methods could predict the future diagnosis of a practicing veterinarian. New approaches using these methods could support clinical decision-making and contribute to improved diagnosis of Cushing’s syndrome in dogs.

A tutorial on calibration measurements and calibration models for clinical prediction models

Our primary objective is to provide the clinical informatics community with an introductory tutorial on calibration measurements and calibration models for predictive models using existing R packages and custom implemented code in R on real and simulated data. Clinical predictive model performance is commonly published based on discrimination measures, but use of models for individualized predictions requires adequate model calibration. This tutorial is intended for clinical researchers who want to evaluate predictive models in terms of their applicability to a particular population. It is also for informaticians and for software engineers who want to understand the role that calibration plays in the evaluation of a clinical predictive model, and to provide them with a solid starting point to consider incorporating calibration evaluation and calibration models in their work. Covered topics include (1) an introduction to the importance of calibration in the clinical setting, (2) an illustration of the distinct roles that discrimination and calibration play in the assessment of clinical predictive models, (3) a tutorial and demonstration of selected calibration measurements, (4) a tutorial and demonstration of selected calibration models, and (5) a brief discussion of limitations of these methods and practical suggestions on how to use them in practice.

Prediction of lameness using automatically recorded activity, behavior and production data in post-parturient Irish dairy cows

BACKGROUND

Although visual locomotion scoring is inexpensive and simplistic, it is also time consuming and subjective. Automated lameness detection methods have been developed to replace the visual locomotion scoring and aid in early and accurate detection. Several types of sensors are measuring traits such as activity, lying behavior or temperature. Previous studies on automatic lameness detection have been unable to achieve high accuracy in combination with practical implementation in a on farm commercial setting. The objective of our research was to develop a prediction model for lameness in dairy cattle using a combination of remote sensor technology and other animal records that will translate sensor data into easy to interpret classified locomotion information for the farmer. During an 11-month period, data from 164 Holstein-Friesian dairy cows were gathered, housed at an Irish research farm. A neck-mounted accelerometer was used to gather behavioral metrics, additional automatically recorded data consisted of milk production and live weight. Locomotion scoring data were manually recorded, using a one-to-five scale (1 = non-lame, 5 = severely lame). Locomotion scores where then used to label the cows as sound (locomotion score 1) or unsound (locomotion score ≥ 2). Four supervised classification models, using a gradient boosted decision tree machine learning algorithm, were constructed to investigate whether cows could be classified as sound or unsound. Data available for model building included behavioral metrics, milk production and animal characteristics.

RESULTS

The resulting models were constructed using various combinations of the data sources. The accuracy of the models was then compared using confusion matrices, receiver-operator characteristic curves and calibration plots. The model which achieved the highest performance according to the accuracy measures, was the model combining all the available data, resulting in an area under the curve of 85% and a sensitivity and specificity of 78%.

CONCLUSION

These results show that 85% of this model's predictions were correct in identifying cows as sound or unsound, showing that the use of a neck-mounted accelerometer, in combination with production and other animal data, has potential to replace visual locomotion scoring as lameness detection method in dairy cows.

Assessing the clinical utility of genetic profiling in fracture risk prediction: a decision curve analysis

Using decision curve analysis on 2188 women and 1324 men, we found that an osteogenomic profile constructed from 62 genetic variants improved the clinical net benefit of fracture risk prediction over and above that of clinical risk factors and BMD.

INTRODUCTION

Genetic profiling is a promising tool for assessing fracture risk. This study sought to use the decision curve analysis (DCA), a novel approach to determine the impact of genetic profiling on fracture risk prediction.

METHODS

The study involved 2188 women and 1324 men, aged 60 years and above, who were followed for up to 23 years. Bone mineral density (BMD) and clinical risk factors were obtained at baseline. The incidence of fracture and mortality were recorded. A weighted individual genetic risk score (GRS) was constructed from 62 BMD-associated genetic variants. Four models were considered: CRF (clinical risk factors); CRF + GRS; Garvan model (GFRC) including CRF and femoral neck BMD; and GFRC + GRS. The DCA was used to evaluate the clinical net benefit of predictive models at a range of clinically reasonable risk thresholds.

RESULTS

In both women and men, the full model GFRC + GRS achieved the highest net benefits. For 10-year risk threshold > 18% for women and > 15% for men, the GRS provided net benefit above those of the CRF models. At 20% risk threshold, adding the GRS could help to avoid 1 additional treatment per 81 women or 1 per 24 men compared with the Garvan model. At lower risk thresholds, there was no significant difference between the four models.

CONCLUSIONS

The addition of genetic profiling into the clinical risk factors can improve the net clinical benefit at higher risk thresholds of fracture. Although the contribution of genetic profiling was modest in the presence of BMD + CRF, it appeared to be able to replace BMD for fracture prediction.

Predicting the risk of depression among adolescents in Nepal using a model developed in Brazil: the IDEA Project

The burden of adolescent depression is high in low- and middle-income countries (LMICs), yet research into prevention is lacking. Development and validation of models to predict individualized risk of depression among adolescents in LMICs is rare but crucial to ensure appropriate targeting of preventive interventions. We assessed the ability of a model developed in Brazil, a middle-income country, to predict depression in an existing culturally different adolescent cohort from Nepal, a low-income country with a large youth population with high rates of depression. Data were utilized from the longitudinal study of 258 former child soldiers matched with 258 war-affected civilian adolescents in Nepal. Prediction modelling techniques were employed to predict individualized risk of depression at age 18 or older in the Nepali cohort using a penalized logistic regression model. Following a priori exclusions for prior depression and age, 55 child soldiers and 71 war-affected civilians were included in the final analysis. The model was well calibrated, had good overall performance, and achieved good discrimination between depressed and non-depressed individuals with an area under the curve (AUC) of 0.73 (bootstrap-corrected 95% confidence interval 0.62-0.83). The Brazilian model comprising seven matching sociodemographic predictors, was able to stratify individualized risk of depression in a Nepali adolescent cohort. Further testing of the model's performance in larger socio-culturally diverse samples in other geographical regions should be attempted to test the model's wider generalizability.

Predicting the risk of future depression among school-attending adolescents in Nigeria using a model developed in Brazil

Depression commonly emerges in adolescence and is a major public health issue in low- and middle-income countries where 90% of the world's adolescents live. Thus efforts to prevent depression onset are crucial in countries like Nigeria, where two-thirds of the population are aged under 24. Therefore, we tested the ability of a prediction model developed in Brazil to predict future depression in a Nigerian adolescent sample. Data were obtained from school students aged 14-16 years in Lagos, who were assessed in 2016 and 2019 for depression using a self-completed version of the Mini International Neuropsychiatric Interview for Children and Adolescents. Only the 1,928 students free of depression at baseline were included. Penalized logistic regression was used to predict individualized risk of developing depression at follow-up for each adolescent based on the 7 matching baseline sociodemographic predictors from the Brazilian model. Discrimination between adolescents who did and did not develop depression was better than chance (area under the curve = 0.62 (bootstrap-corrected 95% CI: 0.58-0.66). However, the model was not well-calibrated even after adjustment of the intercept, indicating poorer overall performance compared to the original Brazilian cohort. Updating the model with context-specific factors may improve prediction of depression in this setting.

A model of twenty-three metabolic-related genes predicting overall survival for lung adenocarcinoma

Background

The highest rate of cancer-related deaths worldwide is from lung adenocarcinoma (LUAD) annually. Metabolism was associated with tumorigenesis and cancer development. Metabolic-related genes may be important biomarkers and metabolic therapeutic targets for LUAD.

Materials and Methods

In this study, the gleaned cohort included LUAD RNA-SEQ data from the Cancer Genome Atlas (TCGA) and corresponding clinical data (n = 445). The training cohort was utilized to model construction, and data from the Gene Expression Omnibus (GEO, GSE30219 cohort, n = 83; GEO, GSE72094, n = 393) were regarded as a testing cohort and utilized for validation. First, we used a lasso-penalized Cox regression analysis to build a new metabolic-related signature for predicting the prognosis of LUAD patients. Next, we verified the metabolic gene model by survival analysis, C-index, receiver operating characteristic (ROC) analysis. Univariate and multivariate Cox regression analyses were utilized to verify the gene signature as an independent prognostic factor. Finally, we constructed a nomogram and performed gene set enrichment analysis to facilitate subsequent clinical applications and molecular mechanism analysis.

Result

Patients with higher risk scores showed significantly associated with poorer survival. We also verified the signature can work as an independent prognostic factor for LUAD survival. The nomogram showed better clinical application performance for LUAD patient prognostic prediction. Finally, KEGG and GO pathways enrichment analyses suggested several especially enriched pathways, which may be helpful for us investigative the underlying mechanisms.

Development and internal validation of a prediction tool to aid the diagnosis of Cushing's syndrome in dogs attending primary-care practice

BACKGROUND

Novel methods to aid identification of dogs with spontaneous Cushing's syndrome are warranted to optimize case selection for diagnostics, avoid unnecessary testing, and ultimately aid decision-making for veterinarians.

HYPOTHESIS/OBJECTIVES

To develop and internally validate a prediction tool for dogs receiving a diagnosis of Cushing's syndrome using primary-care electronic health records.

ANIMALS

Three hundred and ninety-eight dogs diagnosed with Cushing's syndrome and 541 noncase dogs, tested for but not diagnosed with Cushing's syndrome, from a cohort of 905 544 dogs attending VetCompass participating practices.

METHODS

A cross-sectional study design was performed. A prediction model was developed using multivariable binary logistic regression taking the demography, presenting clinical signs and some routine laboratory results into consideration. Predictive performance of each model was assessed and internally validated through bootstrap resampling. A novel clinical prediction tool was developed from the final model.

RESULTS

The final model included predictor variables sex, age, breed, polydipsia, vomiting, potbelly/hepatomegaly, alopecia, pruritus, alkaline phosphatase, and urine specific gravity. The model demonstrated good discrimination (area under the receiver operating curve [AUROC] = 0.78 [95% CI = 0.75-0.81]; optimism-adjusted AUROC = 0.76) and calibration (C-slope = 0.86). A tool was developed from the model which calculates the predicted likelihood of a dog having Cushing's syndrome from 0% (score = -13) to 96% (score = 10).

CONCLUSIONS AND CLINICAL IMPORTANCE

A tool to predict a diagnosis of Cushing's syndrome at the point of first suspicion in dogs was developed, with good predictive performance. This tool can be used in practice to support decision-making and increase confidence in diagnosis.

Factors associated with 30-day and 1-year readmission among psychiatric inpatients in Beijing China: a retrospective, medical record-based analysis

BACKGROUND

Psychiatric readmissions negatively impact patients and their families while increasing healthcare costs. This study aimed at investigating factors associated with psychiatric readmissions within 30 days and 1 year of the index admissions and exploring the possibilities of monitoring and improving psychiatric care quality in China.

METHODS

Data on index admission, subsequent admission(s), clinical and hospital-related factors were extracted in the inpatient medical record database covering 10 secondary and tertiary psychiatric hospitals in Beijing, China. Logistic regressions were used to examine the associations between 30-day and 1-year readmissions plus frequent readmissions (≥3 times/year), and clinical variables as well as hospital characteristics.

RESULTS

The 30-day and 1-year psychiatric readmission rates were 16.69% (1289/7724) and 33.79% (2492/7374) respectively. 746/2492 patients (29.34%) were readmitted 3 times or more within a year (frequent readmissions). Factors significantly associated with the risk of both 30-day and 1-year readmission were residing in an urban area, having medical comorbidities, previous psychiatric admission(s), length of stay > 60 days in the index admission and being treated in tertiary hospitals (p < 0.001). Male patients were more likely to have frequent readmissions (OR 1.30, 95%CI 1.04-1.64). Receiving electroconvulsive therapy (ECT) was significantly associated with a lower risk of 30-day readmission (OR 0.72, 95%CI 0.56-0.91) and frequent readmissions (OR 0.60, 95%CI 0.40-0.91).

CONCLUSION

More than 30% of the psychiatric inpatients were readmitted within 1 year. Urban residents, those with medical comorbidities and previous psychiatric admission(s) or a longer length of stay were more likely to be readmitted, and men are more likely to be frequently readmitted. ECT treatment may reduce the likelihood of 30-day readmission and frequent admissions. Targeted interventions should be designed and piloted to effectively monitor and reduce psychiatric readmissions.

Developing an individualized risk calculator for psychopathology among young people victimized during childhood: A population-representative cohort study

BACKGROUND

Victimized children are at greater risk for psychopathology than non-victimized peers. However, not all victimized children develop psychiatric disorders, and accurately identifying which victimized children are at greatest risk for psychopathology is important to provide targeted interventions. This study sought to develop and internally validate individualized risk prediction models for psychopathology among victimized children.

METHODS

Participants were members of the Environmental Risk (E-Risk) Longitudinal Twin Study, a nationally-representative British birth cohort of 2,232 twins born in 1994-1995. Victimization exposure was measured prospectively between ages 5 and 12 years, alongside a comprehensive range of individual-, family-, and community-level predictors of psychopathology. Structured psychiatric interviews took place at age-18 assessment. Logistic regression models were estimated with Least Absolute Shrinkage and Selection Operator (LASSO) regularization to avoid over-fitting to the current sample, and internally validated using 10-fold nested cross-validation.

RESULTS

26.5% (n = 591) of E-Risk participants had been exposed to at least one form of severe childhood victimization, and 60.4% (n = 334) of victimized children met diagnostic criteria for any psychiatric disorder at age 18. Separate prediction models for any psychiatric disorder, internalizing disorders, and externalizing disorders selected parsimonious subsets of predictors. The three internally validated models showed adequate discrimination, based on area-under-the-curve estimates (range = =0.66-0.73), and good calibration.

LIMITATIONS

External validation in wholly-independent data is needed before clinical implementation.

CONCLUSIONS

Findings offer proof-of-principle evidence that prediction modeling can be useful in supporting identification of victimized children at greatest risk for psychopathology. This has the potential to inform targeted interventions and rational resource allocation.

Validation of serum gamma-glutamyl transferase activity and body weight information for identifying dairy calves that are too young to be transported to auction markets in Canada

Dairy calves are at risk of being stressed when transported during the first week of life. A new Canadian federal rule will forbid transportation of calves younger than 9 d old to auction market. However, in the absence of reliable information to determine birth date, other indirect methods would be of interest. This study aimed to determine the prediction accuracy of body weight, Brix refractometry, and serum gamma-glutamyl transferase (GGT) activity for determining if a calf was not fit to be transported (i.e., <9 d old). For this purpose, we used 284 calves with a known birth date from a cross-sectional and a prospective cohort study. A logistic regression model was built based on multivariable analysis as well as a misclassification cost term analysis. Because of the collinearity between GGT activity and Brix value and lower discrimination of Brix value, the GGT activity was retained for the main model. The final logistic regression model contained body weight and log-transformed GGT activity value. The misclassifications of the logistic model was minimized using a model probability threshold ≥0.55 with a sensitivity of 70.4% and a specificity of 77.3%. This probability threshold was relatively robust for various prevalence and false negative to false positive cost ratios. The prediction accuracy of this model was moderate at the individual level, but is helpful in calves with a reasonable suspicion of being less than 9 d old.