The diagnostic accuracy of procalcitonin in infectious patients after cardiac surgery: a systematic review and meta-analysis

Development, validation, and clinical evaluation of a machine-learning based model for diagnosing early infection after cardiovascular surgery (DEICS): a multi-center cohort study

BACKGROUND

This study addresses the critical need for timely and accurate diagnosis of early postoperative infection (EPI) following cardiac surgery. EPI significantly impacts patient outcomes and healthcare costs, making its early detection vital.

OBJECTIVES

To develop, validate, and clinically implement a machine-learning-based model for diagnosing EPI post-cardiac surgery, enhancing postoperative care.

METHODS

In this multi-center cohort study spanning 2020 to 2022, data from four medical centers involved 2001 participants. Of these, 1400 were used for trainingand 601 for validation. Several machines-learning algorithms, including XGBoost, random forest, support vector machines, least absolute shrinkage and selection operator, and single-layer neural networks, were applied to develop predictive models. These were compared against a traditional logistic regression model. The model with the highest area under the receiver operating characteristic curve (AUROC) was deemed optimal. Implemented across four centers since 1 January 2023, a retrospective real-world study assessed its clinical applicability. Among 400 patients with an estimated EPI risk above 10%, identified by the optimal model, 55 followed its antibiotic upgrade recommendations (DEICS group). The remaining 345 patients upgraded antibiotics empirically, with 55 in the control group, matched 1:1 with the DEICS group. Clinical utility was evaluated through antibiotic use density (AUD), hospital costs, and ICU stay duration.

RESULTS

The XGBoost model achieved the highest performance with an AUROC of 0.96 (95% CI: 0.93-0.98). The calibration curve exhibited strong agreement with Brier scores of 0.02. According to the XGBoost model, the DEICS group significantly demonstrated reduced AUD ( P < 0.01) in the matched cohort, along with decreased ICU stay time (median: 5 vs. 6 days, P = 0.01) and hospital costs (median: ¥150 000 vs. median: ¥200 000, P = 0.01) in the EPI cohort.

CONCLUSION

The successful implementation of the XGBoost model facilitates accurate EPI diagnosis, improves postoperative recovery, and lowers hospital costs.

GRADE-based procalcitonin guideline for emergency departments

Procalcitonin is a useful biomarker for infection. Over the past two decades, there has been much research on the clinical applications of procalcitonin, yet the majority of these studies have been conducted in the intensive care setting. Despite the extensive use of procalcitonin in emergency departments, there have been no guidelines focusing specifically on these clinical settings. Additionally, previous guidelines were predominantly shaped by expert consensus and rarely incorporate evidence-based medicine concepts. To address these shortcomings, the current guideline adopts a novel approach. Initially, we identified the most critical questions regarding the use of procalcitonin in emergency settings through expert voting. This was followed by a systematic literature review and the evaluation of evidence levels using the GRADE (Grading of Recommendations, Assessment, Development, and Evaluation) methodology. Key characteristics of individual studies will be summarized and evaluated by the guideline development group to determine the overall quality of evidence. The GRADE working group's categorization system will be employed to rate evidence quality into four levels. Recommendations will be formulated based on explicit consideration of established criteria. This structured approach ensures that guideline recommendations are founded on robust evidence and transparently assessed for strength and potential caveats. This is the first guideline on the use of procalcitonin to be applied in emergency departments that adopts the principles of evidence-based medicine and encompasses the up-to-date literatures, and it marks an advancement in providing guidance on the utilization of procalcitonin in emergency departments.

Procalcitonin for the diagnosis of postoperative bacterial infection after adult cardiac surgery: a systematic review and meta-analysis

BACKGROUND AND AIMS

Patients undergoing cardiac surgery are subject to infectious complications that adversely affect outcomes. Rapid identification is essential for adequate treatment. Procalcitonin (PCT) is a noninvasive blood test that could serve this purpose, however its validity in the cardiac surgery population is still debated. We therefore performed a systematic review and meta-analysis to estimate the accuracy of PCT for the diagnosis of postoperative bacterial infection after cardiac surgery.

METHODS

We included studies on adult cardiac surgery patients, providing estimates of test accuracy. Search was performed on PubMed, EmBase and WebOfScience on April 12th, 2023 and rerun on September 15th, 2023, limited to the last 10 years. Study quality was assessed with the QUADAS-2 tool. The pooled measures of performance and diagnostic accuracy, and corresponding 95% Confidence Intervals (CI), were calculated using a bivariate regression model. Due to the variation in reported thresholds, we used a multiple-thresholds within a study random effects model for meta-analysis (diagmeta R-package).

RESULTS

Eleven studies were included in the systematic review, and 10 (2984 patients) in the meta-analysis. All studies were single-center with observational design, five of which with retrospective data collection. Quality assessment highlighted various issues, mainly concerning lack of prespecified thresholds for the index test in all studies. Results of bivariate model analysis using multiple thresholds within a study identified the optimal threshold at 3 ng/mL, with a mean sensitivity of 0.67 (0.47-0.82), mean specificity of 0.73 (95% CI 0.65-0.79), and AUC of 0.75 (IC95% 0.29-0.95). Given its importance for practice, we also evaluated PCT's predictive capability. We found that positive predictive value is at most close to 50%, also with a high prevalence (30%), and the negative predictive value was always > 90% when prevalence was < 20%.

CONCLUSIONS

These results suggest that PCT may be used to help rule out infection after cardiac surgery. The optimal threshold of 3 ng/mL identified in this work should be confirmed with large, well-designed randomized trials that evaluate the test's impact on health outcomes and on the use of antibiotic therapy. PROSPERO Registration number CRD42023415773. Registered 22 April 2023.

Diagnostic efficacy of serum presepsin for postoperative infectious complications: a meta-analysis

Background

Postoperative infectious complications (PICs) are major concerns. Early and accurate diagnosis is critical for timely treatment and improved outcomes. Presepsin is an emerging biomarker for bacterial infections. However, its diagnostic efficacy for PICs across surgical specialties remains unclear.

Methods

In this study, a systematic search on MEDLINE, Embase, Google Scholar, and Cochrane Library was performed on September 30, 2023, to identify studies that evaluated presepsin for diagnosing PICs. PIC is defined as the development of surgical site infection or remote infection. Pooled sensitivity, specificity, and hierarchical summary receiver operating characteristic (HSROC) curves were calculated. The primary outcome was the assessment of the efficacy of presepsin for PIC diagnosis, and the secondary outcome was the investigation of the reliability of procalcitonin or C-reactive protein (CRP) in the diagnosis of PICs.

Results

This meta-analysis included eight studies (n = 984) and revealed that the pooled sensitivity and specificity of presepsin for PIC diagnosis were 76% (95% confidence interval [CI] 68%-82%) and 83% (95% CI 75%-89%), respectively. The HSROC curve yielded an area under the curve (AUC) of 0.77 (95% CI 0.73-0.81). Analysis of six studies on procalcitonin showed a combined sensitivity of 78% and specificity of 77%, with an AUC of 0.83 derived from the HSROC. Meanwhile, data from five studies on CRP indicated pooled sensitivity of 84% and specificity of 79%, with the HSROC curve yielding an AUC of 0.89.

Conclusion

Presepsin exhibits moderate diagnostic accuracy for PIC across surgical disciplines. Based on the HSROC-derived AUC, CRP has the highest diagnostic efficacy for PICs, followed by procalcitonin and presepsin. Nonetheless, presepsin demonstrated greater specificity than the other biomarkers. Further study is warranted to validate the utility of and optimize the cutoff values for presepsin.

Systematic review registration

https://www.crd.york.ac.uk/prospero/, identifier CRD42023468358.

[Procalcitonin as sepsis predictor in cardiovascular surgery with cardiopulmonary bypass]

Background

Cardiopulmonary bypass generates an exacerbated response that may lead to sepsis.

Objective

To describe the association between procalcitonin levels and sepsis diagnosis in cardiovascular surgery subjects with cardiopulmonary bypass.

Methods

A case-series study was conducted in 142 patients. Serum procalcitonin levels were measured at 24 hours and at 72 hours after surgery using a point of care testing based on quantitative immunochromatographic method. To assess association between procalcitonin levels and sepsis status, we calculated area under the curve (AUC) and sensitivity, specificity, and predictive values for the best cut-off point.

Results

From 142 patients studied, 7 developed sepsis after surgery (4.9%). For 24-hours procalcitonin levels AUC was 0.921 and best cut-off point was 3.8 ng/mL (sensitivity 0.857 and specificity 0.904). In the case of 72-hours procalcitonin levels, we observed a value of 0.868 for AUC and best cut-off point was 8.4 ng/mL (sensitivity 0.86 and specificity 0.97).

Conclusions

Procalcitonin levels at 24 and 72 hours after cardiovascular surgery with cardiopulmonary bypass are associated with sepsis presence at cut-off points of 3.8 and 8.4 ng/mL respectively.

Diagnosis of Early Bacterial Pneumonia and Sepsis After Cardiovascular Surgery: A Diagnostic Prediction Model Based on LASSO Logistic Regression

Background

Early postoperative bacterial pneumonia and sepsis (ePOPS), which occurs within the first 48 hours after cardiovascular surgery, is a serious life-threatening complication. Diagnosis of ePOPS is extremely challenging, and the existing diagnostic tools are insufficient. The purpose of this study was to construct a novel diagnostic prediction model for ePOPS.

Methods

Least Absolute Shrinkage and Selection Operator (LASSO) with logistic regression was used to construct a model to diagnose ePOPS based on patients' comorbidities, medical history, and laboratory findings. The area under the receiver operating characteristic curve (AUC) was used to evaluate the model discrimination.

Results

A total of 1203 patients were recruited and randomly split into a training and validation set in a 7:3 ratio. By early morning on the 3rd postoperative day (POD3), 103 patients had experienced 133 episodes of bacterial pneumonia or sepsis (15 patients had both). LASSO logistic regression model showed that duration of mechanical ventilation (P=0.015), NYHA class ≥ III (P=0.001), diabetes (P<0.001), exudation on chest radiograph (P=0.011) and IL-6 on POD3 (P<0.001) were independent risk factors. Based on these factors, we created a nomogram named DICS-I with an AUC of 0.787 in the training set and 0.739 in the validation set.

Conclusion

The DICS-I model may be used to predict the risk of ePOPS after cardiovascular surgery, and is also especially suitable for predicting the risk of IRAO. The DICS-I model could help clinicians to adjust antibiotics on the POD3.

Calprotectin as a diagnostic marker for sepsis: A meta-analysis

Introduction

Sepsis is a life-threatening condition, and biomarkers are needed to diagnose sepsis fast and accurately. We aimed to perform this meta-analysis to investigate the diagnostic value of calprotectin on sepsis in critically ill patients.

Methods

The investigators searched MEDLINE, Embase, Web of Science and Cochrane Library. Studies were included if they assessed the diagnostic accuracy of serum calprotectin for sepsis in intensive care unit (ICU). We estimated its diagnostic value and explored the source of heterogeneity. The bivariate model and the hierarchical summary receiver operating characteristic (HSROC) curve were used in the meta-analysis.

Results

Six records assessing 821 patients were included in this meta-analysis. The pooled sensitivity, specificity, positive likelihood ratio (PLR), and diagnostic odds ratio (DOR) were separately as 0.77, 0.85, 5.20, 0.27, respectively. The Fagan's nomogram showed post-test probabilities of 91% and 35% for positive and negative outcomes, respectively. Subgroup analysis indicated that sepsis definition could be a possible source of heterogeneity, but there's no sufficient data to investigate sepsis-3 definition. Sensitivity analysis suggested that two studies could affect the stability of pooled results.

Conclusion

On the basis of our meta-analysis, calprotectin is a helpful marker for early diagnosis of sepsis on ICU admission.

Year in Review 2021: Noteworthy Literature in Cardiothoracic Critical Care

This year marked a number of milestones in critical care. As vaccines for the SARS-CoV-2 virus became widely available and were confirmed to be exceptionally effective against severe illness and hospitalization, we were then faced with new variants and the resource-intense responses necessary to combat them. Despite challenges new and old, we have persevered and continued to provide excellent care to our patients while pushing the boundaries of clinical research. This article is a collection of studies published in 2021 relevant to critical care, with a specific focus on cardiothoracic critical care. To ignore the impact of the COVID-19 pandemic would do a disservice to our colleagues, many of whom have made incredible breakthroughs in novel therapies to the coronavirus, and yet we present additional themes of delirium, acute kidney injury, lung transplant, advances in ECMO as well as biomarkers of sepsis.

Bacteremia detection from complete blood count and differential leukocyte count with machine learning: complementary and competitive with C-reactive protein and procalcitonin tests

Background

Biomarkers, such as leukocyte count, C-reactive protein (CRP), and procalcitonin (PCT), have been commonly used to predict the occurrence of life-threatening bacteremia and provide prognostic information, given the need for prompt intervention. However, such diagnosis methods require much time and money. Therefore, we propose a method with a high prediction capability using machine learning (ML) models based on complete blood count (CBC) and differential leukocyte count (DC) and compare its performance with traditional CRP or PCT biomarker methods and those of models incorporating CRP or PCT biomarkers.

Methods

We collected 366,586 daily blood culture (BC) results, of which 350,775 (93.2%), 308,803 (82.1%), and 23,912 (6.4%) cases were issued CBC/DC (CBC/DC group), CRP with CBC/DC (CRP&CBC/DC group), and PCT with CBC/DC (PCT&CBC/DC group), respectively. For the ML methods, conventional logistic regression and random forest models were selected, trained, applied, and validated for each group. Fivefold validation and prediction capability were also evaluated and reported.

Results

Overall, the ML methods, such as the random forest model, demonstrated promising performances. When trained with CBC/DC data, it achieved an area under the ROC curve (AUC) of 0.802, which is superior to the prediction conventionally made with CRP/PCT levels (0.699/0.731). Upon evaluating the performance enhanced by incorporating CRP or PCT biomarkers, it reported no substantial AUC increase with the addition of either CRP or PCT to CBC/DC data, which suggests the predicting power and applicability of using only CBC/DC data. Moreover, it showed competitive prognostic capability compared to the PCT test with similar all-cause in-hospital mortality (45.10% vs. 47.40%) and overall median survival time (27 vs. 25 days).

Conclusions

The ML models using only CBC/DC data yielded more accurate bacteremia predictions compared to those by methods using CRP and PCT data and reached similar prognostic performance as by PCT data. Thus, such models are potentially complementary and competitive with traditional CRP and PCT biomarkers for conducting and guiding antibiotic usage.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12879-022-07223-7.

Elevated Procalcitonin as a Risk Factor for Postoperative Delirium in the Elderly after Cardiac Surgery-A Prospective Observational Study

One of the most common complications after cardiac surgery with cardiopulmonary bypass (CBP) is delirium. The purpose of this study was to prospectively investigate the risk of developing postoperative delirium in a group of elderly patients using a multivariate assessment of preoperative and intraoperative risk factors. A total of 149 elderly patients were included. Thirty patients (20%) developed post-operative delirium. Preoperative procalcitonin (PCT) above the reference range (>0.05 ng/mL) was recorded more often in patients who postoperatively developed delirium than in the non-delirium group (50% vs. 27%, p = 0.019). After surgery, PCT was significantly higher in the delirium than the non-delirium group: ICU admission after surgery: 0.08 ng/mL vs. 0.05 ng/mL p = 0.011), and for consecutive days (day 1: 0.59 ng/mL vs. 0.25 ng/mL, p = 0.003; day 2: 1.21 ng/mL vs. 0.36 ng/mL, p = 0.006; day 3: 0.76 ng/mL vs. 0.34 ng/mL, p = 0.001). Patients with delirium were older (74 vs. 69 years, p = 0.038), more often had impaired daily functioning (47% vs. 28%, p = 0.041), depressive symptoms (40% vs. 17%, p = 0.005), and anemia (43% vs. 19%, p = 0.006). In a multivariable logistic regression model, preoperative procalcitonin (odds ratio (OR) = 3.05), depressive symptoms (OR = 5.02), age (OR = 1.14), impaired daily functioning (OR = 0.76) along with CPB time (OR = 1.04) were significant predictors of postoperative delirium.