Desirability of outcome ranking (DOOR) for comparing diagnostic tools and early therapeutic choices in patients with suspected candidemia

Early diagnosis of candidemia with explainable machine learning on automatically extracted laboratory and microbiological data: results of the AUTO-CAND project

BACKGROUND

Candidemia is associated with a heavy burden of morbidity and mortality in hospitalized patients. The availability of blood culture results could require up to 48-72 h after blood draw; thus, early treatment decisions are made in the absence of a definite diagnosis.

METHODS

In this retrospective study, we assessed the performance of different supervised machine learning algorithms for the early differential diagnosis of candidemia and bacteremia in adult patients on a large dataset automatically extracted within the AUTO-CAND project.

RESULTS

Overall, 12,483 episodes of candidemia (1275; 10%) or bacteremia (11,208; 90%) were included in the analysis. A random forest classifier achieved the best diagnostic performance for candidemia, with sensitivity 0.98 and specificity 0.65 on the training set (true skill statistic [TSS] = 0.63) and sensitivity 0.74 and specificity 0.57 on the test set (TSS = 0.31). Then, the random classifier was trained in the subgroup of patients with available serum β-D-glucan (BDG) and procalcitonin (PCT) values by exploiting the feature ranking learned in the entire dataset. Although no statistically significant differences were observed from the performance measures obtained by employing BDG and PCT alone, the performance measures of the classifier that included the features selected in the entire dataset, plus BDG and PCT, were the highest in most cases.

CONCLUSIONS

Random forest classifiers trained on large datasets of automatically extracted data have the potential to improve current diagnostic algorithms for candidemia. However, further development through implementation of automatically extracted clinical features may be necessary to achieve crucial improvements.

Validation of an Automated System for the Extraction of a Wide Dataset for Clinical Studies Aimed at Improving the Early Diagnosis of Candidemia

There is increasing interest in assessing whether machine learning (ML) techniques could further improve the early diagnosis of candidemia among patients with a consistent clinical picture. The objective of the present study is to validate the accuracy of a system for the automated extraction from a hospital laboratory software of a large number of features from candidemia and/or bacteremia episodes as the first phase of the AUTO-CAND project. The manual validation was performed on a representative and randomly extracted subset of episodes of candidemia and/or bacteremia. The manual validation of the random extraction of 381 episodes of candidemia and/or bacteremia, with automated organization in structured features of laboratory and microbiological data resulted in ≥99% correct extractions (with confidence interval < ±1%) for all variables. The final automatically extracted dataset consisted of 1338 episodes of candidemia (8%), 14,112 episodes of bacteremia (90%), and 302 episodes of mixed candidemia/bacteremia (2%). The final dataset will serve to assess the performance of different ML models for the early diagnosis of candidemia in the second phase of the AUTO-CAND project.

A Composite Endpoint for Treatment Benefit According to Patient Preference

Complex disorders usually affect multiple symptom domains measured by several outcomes. The importance of these outcomes is often different among patients. Current approaches integrate multiple outcomes without considering patient preferences at the individual level. In this paper, we propose a new composite Desirability of Outcome Ranking (DOOR) that integrates individual level ranking of outcome importance and define a winning probability measuring the overall treatment effect. Stratified randomization can be performed based on the participants' baseline outcome rankings. A Wilcoxon-Mann-Whitney U-statistic is used to average the pairwise DOOR between one treated and one control patient, considering the difference in these patients' ranking of outcome importance. We use both theoretical and empirical methods to examine the statistical properties of our method and to compare with conventional approaches. We conclude that the proposed composite DOOR properly reflects patient-level preferences and can be used in pivotal trials or comparative effectiveness trials for a patient-centered evaluation of overall treatment benefits.

The DOOR to Antibiotic Stewardship: Refining Assessments of Interventions With Desirability of Outcome Ranking

BACKGROUND

Limited sample size and disparate outcome measures can hinder the ability of antimicrobial stewardship programs to assess the utility of their quality improvement interventions. Desirability of outcome ranking (DOOR) is a novel methodology that incorporates multiple outcomes into a single value to more comprehensively compare therapeutic strategies. The objective of this study was to apply DOOR to a single center antibiotic stewardship intervention.

METHODS

A pre- and post-interventional study was conducted evaluating the impact of prospective pharmacist review of rapid molecular diagnostic testing (RDT) of blood cultures on antibiotic optimization. Outcomes included the percentage of patients who were switched to appropriate therapy, the time to appropriate therapy, and the percentage of patients who had missed de-escalation opportunities.

RESULTS

A total of 19 and 29 patients were included in the final analysis. The percentage of patients reaching appropriate therapy was 84% (16/19) and 97% ([28/29], p = 0.16) in the pre-intervention and post-intervention groups respectively. Median time to appropriate therapy was 26 hours and 36 minutes (IQR 13:05-50:45) and 22:40 (IQR 3:42-48:23, p = 0.32), respectively. One missed de-escalation opportunity was identified in the post-intervention group (0% vs 3%, p = 1.00). DOOR analysis indicated that the probability of a better outcome for the post-intervention group than the pre-intervention group was 58% (95% CI 54-62).

CONCLUSION

In this analysis, DOOR revealed a benefit that would not have been apparent with traditional outcomes assessments. Antimicrobial stewardship programs conducting quality improvement studies should consider incorporating DOOR into their methodology.

A look at clinical trial design for new antimicrobials for the adult population

Introduction: Antimicrobial resistance poses a substantial threat to global public health since it decreases the probability of effectively treating an infection and increases the risk of morbidity and mortality.Areas covered: In this review, the authors discuss the advantages and disadvantages of classical and novel trial designs for evaluating novel antibiotics for infections due to multidrug-resistant organisms (MDRO). An inductive literature search was performed using different keywords pertinent to the reviewed topics.Expert opinion: The need for active, effective compounds has strengthened regulatory, academic, and industry cooperation, leading to the recent approval of some novel anti-MDRO agents, with other promising compounds being also in the late phase of clinical development. Nonetheless, some important issues regarding the design of clinical trials have gained importance that are peculiar for novel anti-MDRO agents and should be addressed for continuing to guarantee the availability of effective treatments in the future. Very importantly, concerted cooperation with regulatory agencies will always be needed for continuously discussing and refining the acceptable level of evidence to be pursued through non-conventional and/or innovative trial designs or development strategies. Failure to do so would seriously pose the risk of perpetuating the unmet need for effective anti-MDRO agents.

Challenges and research priorities to progress the impact of antimicrobial stewardship

Antimicrobial stewardship programmes have been playing an important role in patient care and hospital policies. These programmes are now recognised as formal strategies for curbing the upward trend in antibiotic resistance and for improving the appropriate antimicrobial and antifungal use. The role of such programs in the era of antimicrobial resistance presents several unique challenges and opportunities, most notably in the diagnostic and therapeutic setting. Controversies remain regarding the most effective interventions and the appropriate design to evaluate their impact. In this review, based on rounds of discussion, we explain the most important challenges faced by antibiotic stewardship and antifungal stewardship programmes. We also try to suggest areas for further research.

Current and emerging pharmacotherapy for the treatment of infections following open-heart surgery

INTRODUCTION

Patients undergoing open-heart surgery may suffer from postoperative complications, including severe infections. Antimicrobials to treat infectious complications in this population should be selected thoughtfully, taking into account three different and fundamental issues: (i) the site of infection; (ii) the suspected or proven causative agent and its susceptibility pattern; and (iii) the risk of suboptimal pharmacokinetic characteristics and potential toxicity of the chosen drug/s.

AREAS COVERED

The present narrative review summarizes the current and future antimicrobial options for the treatment of infections developing after open-heart surgery.

EXPERT OPINION

The pharmacological treatment of infections developing in cardiac surgery patients poses peculiar challenges, including the need for an active empirical therapy for severe events such as bloodstream infections, deep sternal wound infections, or early-onset postoperative prosthetic endocarditis. In addition, the risk for multidrug-resistant pathogens should also be taken into account in endemic areas. A multidisciplinary evaluation on a patient-by-patient basis, deeply involving infectious diseases specialists and cardiothoracic surgeons, remains essential for appropriately balancing both short-term and long-term risks and benefits of any possible surgical reintervention in combination with adequate pharmacotherapy.