Net Reclassification Index and Integrated Discrimination Index Are Not Appropriate for Testing Whether a Biomarker Improves Predictive Performance

Prediction of acute kidney injury after cardiac surgery with fibrinogen-to-albumin ratio: a prospective observational study

Background

The occurrence of acute kidney injury (AKI) following cardiac surgery is common and linked to unfavorable consequences while identifying it in its early stages remains a challenge. The aim of this research was to examine whether the fibrinogen-to-albumin ratio (FAR), an innovative inflammation-related risk indicator, has the ability to predict the development of AKI in individuals after cardiac surgery.

Methods

Patients who underwent cardiac surgery from February 2023 to March 2023 and were admitted to the Cardiac Surgery Intensive Care Unit of a tertiary teaching hospital were included in this prospective observational study. AKI was defined according to the KDIGO criteria. To assess the diagnostic value of the FAR in predicting AKI, calculations were performed for the area under the receiver operating characteristic curve (AUC), continuous net reclassification improvement (NRI), and integrated discrimination improvement (IDI).

Results

Of the 260 enrolled patients, 85 developed AKI with an incidence of 32.7%. Based on the multivariate logistic analyses, FAR at admission [odds ratio (OR), 1.197; 95% confidence interval (CI), 1.064-1.347, p = 0.003] was an independent risk factor for AKI. The receiver operating characteristic (ROC) curve indicated that FAR on admission was a significant predictor of AKI [AUC, 0.685, 95% CI: 0.616-0.754]. Although the AUC-ROC of the prediction model was not substantially improved by adding FAR, continuous NRI and IDI were significantly improved.

Conclusions

FAR is independently associated with the occurrence of AKI after cardiac surgery and can significantly improve AKI prediction over the clinical prediction model.

Does the addition of serum antimüllerian hormone concentrations to the Framingham Risk Score and Pooled Cohort Equations improve the prediction of cardiovascular disease?

The present study revealed that the addition of serum antimüllerian hormone concentrations to Framingham Risk Score and Pooled Cohort Equations could potentially improve the risk prediction of cardiovascular disease.

Objective

The current study aimed to examine the added value of serum antimüllerian hormone (AMH) concentration to the Framingham Risk Score (FRS) and Pooled Cohort Equations (PCE) in predicting the risk of cardiovascular disease (CVD) in women of reproductive age.

Methods

Women 30 years and older were considered eligible for this population-based prospective study. The univariate and multivariate Cox proportional hazard models were used to evaluate the association between the serum concentrations of AMH and the risk of CVD.

Results

In the enhanced model, which integrated AMH into FRS and PCE and was adjusted for family history of premature CVD, AMH showed a significant association with the risk of CVD during a 19-year follow-up of 800 women (hazard ratio, 0.77 [95% CI, 0.60-0.99] and hazard ratio, 0.64 [95% CI, 0.48-0.84], respectively). According to the likelihood-ratio test, the addition of AMH measurements to FRS and PCE could significantly improve the risk prediction of CVD (P = 0.02 and P < 0.001, respectively); however, the integration of this biomarker did not improve the classification of risk categories.

Conclusions

The present findings revealed that the addition of serum AMH concentrations to FRS and PCE could potentially improve the risk prediction of CVD.

Cohen's Kappa Coefficient as a Measure to Assess Classification Improvement following the Addition of a New Marker to a Regression Model

The need to search for new measures describing the classification of a logistic regression model stems from the difficulty in searching for previously unknown factors that predict the occurrence of a disease. A classification quality assessment can be performed by testing the change in the area under the receiver operating characteristic curve (AUC). Another approach is to use the Net Reclassification Improvement (NRI), which is based on a comparison between the predicted risk, determined on the basis of the basic model, and the predicted risk that comes from the model enriched with an additional factor. In this paper, we draw attention to Cohen's Kappa coefficient, which examines the actual agreement in the correction of a random agreement. We proposed to extend this coefficient so that it may be used to detect the quality of a logistic regression model reclassification. The results provided by Kappa's reclassification were compared with the results obtained using NRI. The random variables' distribution attached to the model on the classification change, measured by NRI, Kappa, and AUC, was presented. A simulation study was conducted on the basis of a cohort containing 3971 Poles obtained during the implementation of a lower limb atherosclerosis prevention program.

Clinical implication of the advanced lung cancer inflammation index in patients with right-sided colon cancer after complete mesocolic excision: a propensity score-matched analysis

Background

This study aimed to assess the clinical implications of the advanced lung cancer inflammation index (ALI) in patients with right-sided colon cancer (RCC) after complete mesocolic excision (CME).

Methods

A total of 441 patients with RCC who underwent CME were included. The optimal cut-off value for the ALI was determined using the X-tile software. Logistic and Cox regression analyses were used to identify risk factors for postoperative complications and long-term outcomes. Predictive nomograms for overall survival (OS) and disease-free survival (DFS) were constructed after propensity score matching (PSM), and their performance was assessed using the net reclassification improvement index (NRI), integrated discrimination improvement index (IDI), and time-dependent receiver operating characteristic (time-ROC) curve analysis.

Results

The optimal preoperative ALI cut-off value was 36.3. After PSM, ASA classification 3/4, operative duration, and a low ALI were independently associated with postoperative complications in the multivariate analysis (all P<0.05). Cox regression analysis revealed that an age >60 years, a carbohydrate antigen 19-9 (CA19-9) level >37 U/mL, pathological N+ stage, and a low ALI were independently correlated with OS (all P<0.05). A CA19-9 level >37 U/mL, pathological N+ stage, lymphovascular invasion, and a low ALI were independent predictors of DFS (all P<0.05). Predictive nomograms for OS and DFS were constructed using PSM. Furthermore, a nomogram combined with the ALI was consistently superior to a non-ALI nomogram or the pathological tumor-node-metastasis classification based on the NRI, IDI, and time-ROC curve analysis after PSM (all P<0.05).

Conclusion

The ALI was an effective indicator for predicting short- and long-term outcomes in patients with RCC.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12957-022-02712-0.

FURIN Promoter Methylation Predicts the Risk of Incident Diabetes: A Prospective Analysis in the Gusu Cohort

Background

Furin has been associated with diabetes but the underlying mechanisms are unclear. As a mediator linking fixed genome and dynamic environment, DNA methylation of its coding gene FURIN may be involved. Here, we aimed to examine the prospective association between DNA methylation in FURIN promoter and incident diabetes during 4 years of follow-up in Chinese adults.

Methods

DNA methylation levels in FURIN promoter were quantified by target bisulfite sequencing using peripheral blood from 1836 participants in the Gusu cohort who were free of diabetes at baseline. To examine the association between DNA methylation levels in FURIN promoter and incident diabetes, we constructed a logistic regression model adjusting for the conventional factors. Multiple testing was controlled by adjusting for the total number of CpG sites assayed using the false-discovery rate approach.

Results

Among the 1836 participants free of diabetes at baseline, 109 (5.94%) participants developed diabetes during the average of 4 years of follow-up. Hypermethylation at two of the eight CpG sites assayed in the FURIN promoter was associated with an increased risk of diabetes, after multivariable adjustment and multiple testing correction. Every 5% increment in methylation levels at CpG1 and CpG2 were associated with a 22% (OR=1.22, 95%CI: 1.05-1.43, P=0.009, q=0.038) and 39% (OR=1.39, 95%CI: 1.08-1.77, P=0.009, q=0.038) higher risk of incident diabetes, respectively. The gene-based association analysis revealed that DNA methylation at multiple CpG loci was jointly associated with incident diabetes (P<0.001). Using the average methylation level of the 8 CpG loci in FURIN promoter revealed a similar association (OR=1.28, 95% CI: 1.02-1.62, P=0.037).

Conclusions

These results suggested that the hypermethylation levels in FURIN promoter were associated with an increased risk for incident diabetes in Chinese adults.

Biomarkers for Traumatic Brain Injury: Data Standards and Statistical Considerations

Recent biomarker innovations hold potential for transforming diagnosis, prognostic modeling, and precision therapeutic targeting of traumatic brain injury (TBI). However, many biomarkers, including brain imaging, genomics, and proteomics, involve vast quantities of high-throughput and high-content data. Management, curation, analysis, and evidence synthesis of these data are not trivial tasks. In this review, we discuss data management concepts and statistical and data sharing strategies when dealing with biomarker data in the context of TBI research. We propose that application of biomarkers involves three distinct steps-discovery, evaluation, and evidence synthesis. First, complex/big data has to be reduced to useful data elements at the stage of biomarker discovery. Second, inferential statistical approaches must be applied to these biomarker data elements for assessment of biomarker clinical utility and validity. Last, synthesis of relevant research is required to support practice guidelines and enable health decisions informed by the highest quality, up-to-date evidence available. We focus our discussion around recent experiences from the International Traumatic Brain Injury Research (InTBIR) initiative, with a specific focus on four major clinical projects (Transforming Research and Clinical Knowledge in TBI, Collaborative European NeuroTrauma Effectiveness Research in TBI, Collaborative Research on Acute Traumatic Brain Injury in Intensive Care Medicine in Europe, and Approaches and Decisions in Acute Pediatric TBI Trial), which are currently enrolling subjects in North America and Europe. We discuss common data elements, data collection efforts, data-sharing opportunities, and challenges, as well as examine the statistical techniques required to realize successful adoption and use of biomarkers in the clinic as a foundation for precision medicine in TBI.

Prognostic value of reverse remodelling criteria in heart failure with reduced or mid-range ejection fraction

Aims

Reverse remodelling (RR) is the recovery from left ventricular (LV) dilatation and dysfunction. Many arbitrary criteria for RR have been proposed. We searched the criteria with the strongest prognostic yield for the hard endpoint of cardiovascular death.

Methods and results

We performed a systematic literature search of diagnostic criteria for RR. We evaluated their prognostic significance in a cohort of 927 patients with LV ejection fraction (LVEF) < 50% undergoing two echocardiograms within 12 ± 2 months. These patients were followed for a median of 2.8 years (interquartile interval 1.3–4.9) after the second echocardiogram, recording 123 cardiovascular deaths. Two prognostic models were defined. Model 1 included age, LVEF, N‐terminal pro‐B‐type natriuretic peptide, ischaemic aetiology, cardiac resynchronization therapy, estimated glomerular filtration rate, New York Heart Association, and LV end‐systolic volume (LVESV) index, and Model 2 the validated Cardiac and Comorbid Conditions Heart Failure score. We identified 25 criteria for RR, the most used being LVESV reduction ≥15% (12 studies out of 42). In the whole cohort, two criteria proved particularly effective in risk reclassification over Model 1 and Model 2. These criteria were (i) LVEF increase >10 U and (ii) LVEF increase ≥1 category [severe (LVEF ≤ 30%), moderate (LVEF 31–40%), mild LV dysfunction (LVEF 41–55%), and normal LV function (LVEF ≥ 56%)]. The same two criteria yielded independent prognostic significance and improved risk reclassification even in patients with more severe systolic dysfunction, namely, those with LVEF < 40% or LVEF ≤ 35%. Furthermore, LVEF increase >10 U and LVEF increase ≥1 category displayed a greater prognostic value than LVESV reduction ≥15%, both in the whole cohort and in the subgroups with LVEF < 40% or LVEF ≤ 35%. For example, LVEF increase >10 U independently predicted cardiovascular death over Model 1 and LVESV reduction ≥15% (hazard ratio 0.40, 95% confidence interval 0.18–0.90, P = 0.026), while LVESV reduction ≥15% did not independently predict cardiovascular death (P = 0.112).

Conclusions

Left ventricular ejection fraction increase >10 U and LVEF increase ≥1 category are stronger predictors of cardiovascular death than the most commonly used criterion for RR, namely, LVESV reduction ≥15%.

Prognostic Value of Combined Detection of Preoperative Albumin-to-Fibrinogen Ratio and Neutrophil-to-Lymphocyte Ratio in Operable Esophageal Squamous Cell Carcinoma Patients without Neoadjuvant Therapy

Background

We retrospectively analyzed the prognostic value of the albumin-to-fibrinogen ratio (AFR)–neutrophil-to-lymphocyte ratio (NLR) score, comprising preoperative AFR and NLR, in esophageal squamous cell carcinoma (ESCC) patients after radical resection.

Patients and Methods

Overall, 215 patients were included. The optimal cutoff value was determined using the receiver operating characteristic (ROC) curve. Based on a low AFR (<12.06) and high NLR (≥1.78), the AFR–NLR score was classified as 2 (both hematological abnormalities present), 1 (one abnormality present), or 0 (both abnormalities absent). Kaplan–Meier curves, Cox regression, and predicted nomogram were used to evaluate the prognostic value of the score.

Results

The prognostic value of the AFR–NLR score was better than that of AFR or NLR alone (P <0.05). Multivariate analysis showed that a high AFR–NLR score was an independent predictor of poor prognosis for overall survival (P <0.001). Additionally, in the nomogram including the AFR–NLR score, the net reclassification improvement index increased by 35.5% (P <0.001), and the integrated discrimination improvement index increased by 9.0% (P <0.001). The predictive accuracy of the established nomogram model was proved using Harrell’s concordance index (0.811, 95% confidence interval: 0.765–0.856) and calibration curve. Notably, the decision analysis curve showed that the nomogram had a higher net benefit within most of the threshold probability range, indicating better clinical applicability.

Conclusion

The AFR–NLR score is a useful predictor of the prognosis of ESCC patients after radical resection, and the nomogram established on the basis of this score has a good prognostic value.

A novel clinical model for predicting malignancy of solitary pulmonary nodules: a multicenter study in chinese population

Background

This study aimed to establish and validate a novel clinical model to differentiate between benign and malignant solitary pulmonary nodules (SPNs).

Methods

Records from 295 patients with SPNs in Sun Yat-sen University Cancer Center were retrospectively reviewed. The novel prediction model was established using LASSO logistic regression analysis by integrating clinical features, radiologic characteristics and laboratory test data, the calibration of model was analyzed using the Hosmer-Lemeshow test (HL test). Subsequently, the model was compared with PKUPH, Shanghai and Mayo models using receiver-operating characteristics curve (ROC), decision curve analysis (DCA), net reclassification improvement index (NRI), and integrated discrimination improvement index (IDI) with the same data. Other 101 SPNs patients in Henan Tumor Hospital were used for external validation cohort.

Results

A total of 11 variables were screened out and then aggregated to generate new prediction model. The model showed good calibration with the HL test (P = 0.964). The AUC for our model was 0.768, which was higher than other three reported models. DCA also showed our model was superior to the other three reported models. In our model, sensitivity = 78.84%, specificity = 61.32%. Compared with the PKUPH, Shanghai and Mayo models, the NRI of our model increased by 0.177, 0.127, and 0.396 respectively, and the IDI changed − 0.019, -0.076, and 0.112, respectively. Furthermore, the model was significant positive correlation with PKUPH, Shanghai and Mayo models.

Conclusions

The novel model in our study had a high clinical value in diagnose of MSPNs.

Circulating tRNA Fragments as a Novel Biomarker Class to Distinguish Acute Stroke Subtypes

: Early blood biomarkers to diagnose acute stroke could drastically reduce treatment delays. We investigated whether circulating small non-coding RNAs can serve as biomarkers to distinguish between acute ischemic stroke (IS), intracerebral hemorrhage (ICH) and stroke mimics (SM). In an ongoing observational cohort study, we performed small RNA-sequencing in plasma obtained from a discovery cohort of 26 patients (9 IS, 8 ICH and 9 SM) presented to the emergency department within 6 h of symptom onset. We validated our results in an independent dataset of 20 IS patients and 20 healthy controls. ICH plasma had the highest abundance of ribosomal and tRNA-derived fragments, while microRNAs were most abundant in plasma of IS patients. Combinations of four to five tRNAs yielded diagnostic accuracies (areas under the receiver operating characteristics curve) up to 0.986 (ICH vs. IS and SM) in the discovery cohort. Validation of the IS and SM models in the independent dataset yielded diagnostic accuracies of 0.870 and 0.885 to distinguish IS from healthy controls. Thus, we identified tRNA-derived fragments as a promising novel class of biomarkers to distinguish between acute IS, ICH and SM, as well as healthy controls.

Establishment and validation of a mathematical diagnosis model to distinguish benign pulmonary nodules from early non-small cell lung cancer in Chinese people

Background

In this study, we aimed to establish and validate a mathematical diagnosis model to distinguish benign pulmonary nodules (BPNs) from early non-small cell lung cancer (eNSCLC) based on clinical characteristics, radiomics features, and hematological biomarkers.

Methods

Medical records from 81 patients (27 BPNs, 54 eNSCLC) were used to establish a novel mathematical diagnosis model and an additional 61 patients (21 BPNs, 40 eNSCLC) were used to validate this new model. To establish a clinical diagnosis model, a least absolute shrinkage and selection operator (LASSO) regression was applied to select predictors for eNSCLC, then multivariate logistic regression analysis was performed to determine independent predictors of the probability of eNSCLC, and to establish a clinical diagnosis model. The diagnostic accuracy and discriminative ability of our model were compared with the PKUPH and Mayo models using the following 4 indices: area under the receiver-operating characteristics curve (ROC), net reclassification improvement index (NRI), integrated discrimination improvement index (IDI), and decision curve analysis (DCA).

Results

Multivariate logistic regression analysis identified age, border, and albumin (ALB) as independent diagnostic markers of eNSCLC. In the training cohort, the AUC of our model was 0.740, which was larger than the AUCs for the PKUPH model (0.717, P=0.755) and the Mayo model (0.652, P=0.275). Compared with the PKUPH and Mayo models, the NRI of our model increased by 3.7% (P=0.731) and 27.78% (P=0.008), respectively, while the IDI changed −4.77% (P=0.437) and 11.67% (P=0.015), respectively. Moreover, the DCA demonstrated that our model had a higher overall net benefit compared to previously published models. Importantly, similar findings were confirmed in the validation cohort.

Conclusions

Age, border, and serum ALB levels were independent diagnostic markers of eNSCLC. Thus, our model could more accurately distinguish BPNs from eNSCLC and outperformed previously published models.

Impact of Annual Change in Geriatric Nutritional Risk Index on Mortality in Patients Undergoing Hemodialysis

Regular nutritional assessment may decrease the mortality rate in patients undergoing hemodialysis. This study aimed to evaluate whether annual change in geriatric nutritional risk index (ΔGNRI) can precisely predict mortality. We retrospectively examined 229 patients undergoing hemodialysis who measured geriatric nutritional risk index (GNRI). Patients were divided into four groups according to the baseline GNRI of 91.2, previously reported cutoff value, and declined or maintained GNRI during the first year (ΔGNRI < 0% vs. ΔGNRI ≥ 0%): Group 1 (G1), GNRI ≥ 91.2 and ΔGNRI ≥ 0%; G2, GNRI ≥ 91.2 and ΔGNRI < 0%; G3, GNRI < 91.2 and ΔGNRI ≥ 0%; and G4, GNRI < 91.2 and ΔGNRI < 0%. They were followed for mortality. During a median follow-up of 3.7 (1.9–6.9) years, 74 patients died, of which 35 had cardiovascular-specific causes. The GNRI significantly decreased from 94.8 ± 6.3 to 94.1 ± 6.7 in the first year (p = 0.035). ΔGNRI was negatively associated with baseline GNRI (ρ = −0.199, p = 0.0051). The baseline GNRI < 91.2 and ΔGNRI < 0% were independently associated with all-cause mortality (adjusted hazard ratio (aHR) 2.59, 95%, confidence interval (CI) 1.54–4.33, and aHR 2.33, 95% CI 1.32–4.32, respectively). The 10-year survival rates were 69.8%, 43.2%, 39.9%, and 19.2% in G1, G2, G3, and G4, respectively (p < 0.0001). The aHR value for G4 vs. G1 was 3.88 (95% CI 1.62–9.48). With regards to model discrimination, adding ΔGNRI to the baseline risk model including the baseline GNRI significantly improved the net reclassification improvement by 0.525 (p = 0.0005). With similar results obtained for cardiovascular mortality. We concluded that the ΔGNRI could not only predict all-cause and cardiovascular mortality but also improve predictability for mortality; therefore, GNRI might be proposed to be serially evaluated.

Methodological standards for the development and evaluation of clinical prediction rules: a review of the literature

Clinical prediction rules (CPRs) that predict the absolute risk of a clinical condition or future outcome for individual patients are abundant in the medical literature; however, systematic reviews have demonstrated shortcomings in the methodological quality and reporting of prediction studies. To maximise the potential and clinical usefulness of CPRs, they must be rigorously developed and validated, and their impact on clinical practice and patient outcomes must be evaluated. This review aims to present a comprehensive overview of the stages involved in the development, validation and evaluation of CPRs, and to describe in detail the methodological standards required at each stage, illustrated with examples where appropriate. Important features of the study design, statistical analysis, modelling strategy, data collection, performance assessment, CPR presentation and reporting are discussed, in addition to other, often overlooked aspects such as the acceptability, cost-effectiveness and longer-term implementation of CPRs, and their comparison with clinical judgement. Although the development and evaluation of a robust, clinically useful CPR is anything but straightforward, adherence to the plethora of methodological standards, recommendations and frameworks at each stage will assist in the development of a rigorous CPR that has the potential to contribute usefully to clinical practice and decision-making and have a positive impact on patient care.