Quantifying uncertainty about future antimicrobial resistance: Comparing structured expert judgment and statistical forecasting methods

Long-term forecast for antibacterial drug consumption in Germany using ARIMA models

The increasing supply shortages of antibacterial drugs presents significant challenges to public health in Germany. This study aims to predict the future consumption of the ten most prescribed antibacterial drugs in Germany up to 2040 using ARIMA (Auto Regressive Integrated Moving Average) models, based on historical prescription data. This analysis also evaluates the plausibility of the forecasts. Our findings represent one of the first long-term national forecasts for antibacterial drug consumption. ARIMA(0,1,0), a random walk model with drift, is the best-fitting model to capture trends across all antibacterial drugs. While more complex models offer greater detail, they seem less suitable for long-term forecasting. In a short-term forecast of 5 and 10 years, predictions between significant models vary very little. Predictions indicate increasing DDD-prescriptions for amoxicillin, cefuroxime axetil, amoxicillin clavulanic acid, clindamycin, azithromycin, nitrofurantoin, and ciprofloxacin, while declines are forecasted for doxycycline, phenoxymethylpenicillin, and sulfamethoxazole-trimethoprim. The reliability of the predictions varies. Forecasts for azithromycin, phenoxymethylpenicillin, and sulfamethoxazole-trimethoprim are likely accurate, whereas uncertainties exist for doxycycline, amoxicillin clavulanic acid, nitrofurantoin, and ciprofloxacin, though general trends appear valid. Potential discrepancies may arise in the predictions for amoxicillin, cefuroxime axetil, and clindamycin. These forecasts highlight the urgent need for proactive healthcare planning to prevent future shortages, a problem underscored by recent supply disruptions in Germany. Future research should extend this analysis to the development of bacterial resistance and other frequently used drug classes.

Recommendations on the Use of Structured Expert Elicitation Protocols for Healthcare Decision Making: A Good Practices Report of an ISPOR Task Force

Healthcare decision making, including regulatory and reimbursement decisions, is based on uncertain assessments of clinical and economic value. This arises from the evidence supporting those assessments being uncertain, incomplete, or even absent. Qualitative, structured expert elicitation (SEE) is a valuable tool for extracting expert knowledge about an uncertain quantity and formulating that knowledge as a probability distribution. This creates a useful input to decision modeling and support, particularly in areas with limited evidence, such as advanced therapy products, precision medicine, rare diagnoses, and other areas with high uncertainty. Structured SEE protocols are used to improve the transparency, accuracy, and consistency of quantitative judgments from experts, limiting the effect of heuristics and biases. This task force report introduces 5 commonly used protocols for SEE (Sheffield elicitation framework; modified Delphi method; Cooke's classical method; investigate, discuss, estimate, aggregate protocol; and the Medical Research Council reference protocol). It describes the common elements of SEE, discusses how these protocols differ in their implementation of those elements and illustrates the use of the protocols. The report then reviews the relevant constraints on implementing SEE within the context of healthcare decision making and considers the strengths and weaknesses of these protocols in light of those considerations. Because this is an introductory report on an emerging topic, specific recommendations on practice are not made. However, there are broad recommendations based on the suitability of the different protocols in various decision contexts. The report concludes with recommendations for further research to better guide future practice.

Ceftazidime with avibactam for treating severe aerobic Gram-negative bacterial infections: technology evaluation to inform a novel subscription-style payment model

Background

To limit the use of antimicrobials without disincentivising the development of novel antimicrobials, there is interest in establishing innovative models that fund antimicrobials based on an evaluation of their value as opposed to the volumes used. The aim of this project was to evaluate the population-level health benefit of ceftazidime-avibactam in the NHS in England, for the treatment of severe aerobic Gram-negative bacterial infections when used within its licensed indications. The results were used to inform National Institute for Health and Care Excellence guidance in support of commercial discussions regarding contract value between the manufacturer and NHS England.

Methods

The health benefit of ceftazidime-avibactam was first derived for a series of high-value clinical scenarios. These represented uses that were expected to have a significant impact on patients' mortality risks and health-related quality of life. Patient-level costs and health-related quality of life of ceftazidime-avibactam under various usage scenarios compared with alternative management strategies in the high-value clinical scenarios were quantified using decision modelling. Results were reported as incremental net health effects expressed in quality-adjusted life-years, which were scaled to 20-year population in quality-adjusted life-years using infection number forecasts based on data from Public Health England. The outcomes estimated for the high-value clinical scenarios were extrapolated to other expected uses for ceftazidime-avibactam.

Results

The clinical effectiveness of ceftazidime-avibactam relative to its comparators was estimated by synthesising evidence on susceptibility of the pathogens of interest to the antimicrobials in a network meta-analysis. In the base case, ceftazidime-avibactam was associated with a statistically significantly higher susceptibility relative to colistin (odds ratio 7.24, 95% credible interval 2.58 to 20.94). The remainder of the treatments were associated with lower susceptibility than colistin (odds ratio < 1). The results were sensitive to the definition of resistance and the studies included in the analysis. In the base case, patient-level benefit of ceftazidime-avibactam was between 0.08 and 0.16 quality-adjusted life-years, depending on the site of infection and the usage scenario. There was a high degree of uncertainty surrounding the benefits of ceftazidime-avibactam across all subgroups, and the results were sensitive to assumptions in the meta-analysis used to estimate susceptibility. There was substantial uncertainty in the number of infections that are suitable for treatment with ceftazidime-avibactam, so population-level results are presented for a range of scenarios for the current infection numbers, the expected increases in infections over time, and rates of emergence of resistance. The population-level benefit varied substantially across the scenarios, from 531 to 2342 quality-adjusted life-years over 20 years.

Conclusion

This work has provided quantitative estimates of the value of ceftazidime-avibactam within its areas of expected usage within the NHS.

Limitations

Given existing evidence, the estimates of the value of ceftazidime-avibactam are highly uncertain.

Future work

Future evaluations of antimicrobials would benefit from improvements to NHS data linkages, research to support appropriate synthesis of susceptibility studies, and application of routine data and decision modelling to assess enablement value.

Study registration

No registration of this study was undertaken.

Funding

This award was funded by the National Institute for Health and Care Research (NIHR) Policy Research Programme (NIHR award ref: NIHR135592), conducted through the Policy Research Unit in Economic Methods of Evaluation in Health and Social Care Interventions, PR-PRU-1217-20401, and is published in full in Health Technology Assessment; Vol. 28, No. 73. See the NIHR Funding and Awards website for further award information.

Forecasting antimicrobial resistance evolution

Antimicrobial resistance (AMR) is a major global health issue. Current measures for tackling it comprise mainly the prudent use of drugs, the development of new drugs, and rapid diagnostics. Relatively little attention has been given to forecasting the evolution of resistance. Here, we argue that forecasting has the potential to be a great asset in our arsenal of measures to tackle AMR. We argue that, if successfully implemented, forecasting resistance will help to resolve the antibiotic crisis in three ways: it will (i) guide a more sustainable use (and therefore lifespan) of antibiotics and incentivize investment in drug development, (ii) reduce the spread of AMR genes and pathogenic microbes in the environment and between patients, and (iii) allow more efficient treatment of persistent infections, reducing the continued evolution of resistance. We identify two important challenges that need to be addressed for the successful establishment of forecasting: (i) the development of bespoke technology that allows stakeholders to empirically assess the risks of resistance evolving during the process of drug development and therapeutic/preventive use, and (ii) the transformative shift in mindset from the current praxis of mostly addressing the problem of antibiotic resistance a posteriori to a concept of a priori estimating, and acting on, the risks of resistance.

Cefiderocol for treating severe aerobic Gram-negative bacterial infections: technology evaluation to inform a novel subscription-style payment model

Background

To limit the use of antimicrobials without disincentivising the development of novel antimicrobials, there is interest in establishing innovative models that fund antimicrobials based on an evaluation of their value as opposed to the volumes used. The aim of this project was to evaluate the population-level health benefit of cefiderocol in the NHS in England, for the treatment of severe aerobic Gram-negative bacterial infections when used within its licensed indications. The results were used to inform the National Institute for Health and Care Excellence guidance in support of commercial discussions regarding contract value between the manufacturer and NHS England.

Methods

The health benefit of cefiderocol was first derived for a series of high-value clinical scenarios. These represented uses that were expected to have a significant impact on patients' mortality risks and health-related quality of life. The clinical effectiveness of cefiderocol relative to its comparators was estimated by synthesising evidence on susceptibility of the pathogens of interest to the antimicrobials in a network meta-analysis. Patient-level costs and health outcomes of cefiderocol under various usage scenarios compared with alternative management strategies were quantified using decision modelling. Results were reported as incremental net health effects expressed in quality-adjusted life-years, which were scaled to 20-year population values using infection number forecasts based on data from Public Health England. The outcomes estimated for the high-value clinical scenarios were extrapolated to other expected uses for cefiderocol.

Results

Among Enterobacterales isolates with the metallo-beta-lactamase resistance mechanism, the base-case network meta-analysis found that cefiderocol was associated with a lower susceptibility relative to colistin (odds ratio 0.32, 95% credible intervals 0.04 to 2.47), but the result was not statistically significant. The other treatments were also associated with lower susceptibility than colistin, but the results were not statistically significant. In the metallo-beta-lactamase Pseudomonas aeruginosa base-case network meta-analysis, cefiderocol was associated with a lower susceptibility relative to colistin (odds ratio 0.44, 95% credible intervals 0.03 to 3.94), but the result was not statistically significant. The other treatments were associated with no susceptibility. In the base case, patient-level benefit of cefiderocol was between 0.02 and 0.15 quality-adjusted life-years, depending on the site of infection, the pathogen and the usage scenario. There was a high degree of uncertainty surrounding the benefits of cefiderocol across all subgroups. There was substantial uncertainty in the number of infections that are suitable for treatment with cefiderocol, so population-level results are presented for a range of scenarios for the current infection numbers, the expected increases in infections over time and rates of emergence of resistance. The population-level benefits varied substantially across the base-case scenarios, from 896 to 3559 quality-adjusted life-years over 20 years.

Conclusion

This work has provided quantitative estimates of the value of cefiderocol within its areas of expected usage within the NHS.

Limitations

Given existing evidence, the estimates of the value of cefiderocol are highly uncertain.

Future work

Future evaluations of antimicrobials would benefit from improvements to NHS data linkages; research to support appropriate synthesis of susceptibility studies; and application of routine data and decision modelling to assess enablement value.

Study registration

No registration of this study was undertaken.

Funding

This award was funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment Policy Research Programme (NIHR award ref: NIHR135591), conducted through the Policy Research Unit in Economic Methods of Evaluation in Health and Social Care Interventions, PR-PRU-1217-20401, and is published in full in Health Technology Assessment; Vol. 28, No. 28. See the NIHR Funding and Awards website for further award information.

Innovations in genomic antimicrobial resistance surveillance

Whole-genome sequencing of antimicrobial-resistant pathogens is increasingly being used for antimicrobial resistance (AMR) surveillance, particularly in high-income countries. Innovations in genome sequencing and analysis technologies promise to revolutionise AMR surveillance and epidemiology; however, routine adoption of these technologies is challenging, particularly in low-income and middle-income countries. As part of a wider series of workshops and online consultations, a group of experts in AMR pathogen genomics and computational tool development conducted a situational analysis, identifying the following under-used innovations in genomic AMR surveillance: clinical metagenomics, environmental metagenomics, gene or plasmid tracking, and machine learning. The group recommended developing cost-effective use cases for each approach and mapping data outputs to clinical outcomes of interest to justify additional investment in capacity, training, and staff required to implement these technologies. Harmonisation and standardisation of methods, and the creation of equitable data sharing and governance frameworks, will facilitate successful implementation of these innovations.

Saliva culture as a predictive indicator for current blood infections and antimicrobial resistance in the ICU setting

Antimicrobial resistance is a worldwide health problem and patients in intensive care are more vulnerable, requiring strict control measures and early identification. Currently, clinical culture materials are used to identify the bacterial agent, but saliva culture is not validated, which has great clinical relevance because it participates in several pathophysiological processes. The aim of this study was to validate saliva culture in an intensive care unit environment, determining its diagnostic value for infection. For this purpose, the results of the 39-month surveillance cultures, from the database of a private hospital were evaluated. A total of 323 cultures were paired between saliva, tracheal secretions, blood and urine from patients who were hospitalized for more than 5 days. The search for correlations between the results was performed using the Spearman correlation test. Severity and evolution data were also correlated. It was possible to correlate the presence of Klebsiella spp. between blood culture and saliva culture in 25% of the results (r = 0.01) and the correlation between saliva and tracheal secretion was 33% (r = 0.33447) with p < 0.0001. In conclusion, saliva can be an excellent discriminator of systemic infections, and can be considered a useful culture in clinical practice.

The Resistance Patterns in E. coli Isolates among Apparently Healthy Adults and Local Drivers of Antimicrobial Resistance: A Mixed-Methods Study in a Suburban Area of Nepal

Evidence-based decision-making to combat antimicrobial resistance (AMR) mandates a well-built community-based surveillance system for assessing resistance patterns among commensals and pathogenic organisms. As there is no such surveillance system in Nepal, we attempted to describe the antimicrobial resistance pattern in E. coli isolated from the fecal samples of apparently healthy individuals in Dhulikhel municipality and also explored the local drivers of AMR. We used a mixed-method design with a cross-sectional quantitative component and a descriptive qualitative component, with focus group discussion and key informant interviews as the data collection method. Fecal samples were collected from 424 individuals randomly selected for the study. E. coli was isolated from 85.9% of human fecal samples, of which 14% were resistant to ≥3 class of antimicrobials (multidrug resistant). Of the 368 isolates, resistance to ampicillin (40.0%), tetracycline (20.7%) and cefotaxime (15.5%) were most prevalent. The major drivers of AMR were: lack of awareness of AMR, weak regulations on sales of antimicrobials, poor adherence to prescribed medications, and incomplete dosage due to financial constraints. These findings indicate the need for strict implementation of a national drug act to limit the over-the-counter sales of antimicrobials. Additionally, awareness campaigns with a multimedia mix are essential for educating people on AMR.

Resistance to extended-spectrum cephalosporins in Escherichia coli and Salmonella enterica isolated from food-producing animals: Ecological study from selected national surveillance programs

Extended-spectrum cephalosporins (ESC) are categorized by World Health Organization as critically important antimicrobials with limited therapeutic alternatives for the treatment of severe bacterial infections in humans. Preserving the effectiveness of ESC requires continuous monitoring of resistance and comparison of associated data across national surveillance programs in the face of globalization. In this ecological study, we compared ESC resistance in Escherichia coli and Salmonella enterica isolated from food-producing animals from 2003 to 2019 between nine countries (Canada, Denmark, Finland, Japan, Netherlands, Norway, Sweden, United Kingdom, and the United States). Using the beta-regression model, compared to Canada, non-selective ESC-R Salmonella enterica was less likely isolated from food producing animals in other eight countries (Odds ratio range: 0.07-0.76). We observed an interaction between the country and the year with a significantly decreased proportion (P < 0.05) of non-selective ESC-R Escherichia coli from the Netherlands, the United Kingdom, and the United States compared to Canada over the years. There was a linear correlation between non-selective ESC-R Escherichia coli and ESC use from Netherlands (Spearman's ρ = 0.91, P < 0.0001). For the six European countries, the interaction between the country and year showed a significant decrease in the proportion of selective ESC-R Escherichia coli over the years for the Netherlands compared to Denmark (P = 0.002). While there were variations in the proportion of beta-lactamase genes reported over the years, bla_CTX-M and bla_CMY-2 genes were commonly detected among the selective ESC-R Escherichia coli. This study reveals variability in the recovery of ESC-resistant bacteria among the countries that seems likely influenced by the individual country policy on the use of critically important antimicrobials and resistance surveillance programs. However, there is a need for harmonization and consistency in food animal sources of bacterial isolates used in surveillance programs within and between the countries for easy comparability.

Modelling the Future Clinical and Economic Burden of Antimicrobial Resistance: The Feasibility and Value of Models to Inform Policy

Due to the increasing threat to public health and the economy, governments internationally are interested in models to estimate the future clinical and economic burden of antimicrobial resistance (AMR) and to evaluate the cost-effectiveness of interventions to prevent or control resistance and to inform resource-allocation decision making. A widely cited UK report estimated that 10 million additional deaths will occur globally per annum due to AMR by 2050; however, the utility and accuracy of this prediction has been challenged. The precision of models predicting the future economic burden of AMR is dependent upon the accuracy of predicting future resistance rates. This paper reviews the feasibility and value of modelling to inform policy and resource allocation to manage and curb AMR. Here we describe methods used to estimate future resistance in published burden-of-disease models; the sources of uncertainty are highlighted, which could potentially mislead policy decision-making. While broad assumptions can be made regarding some predictable factors contributing to future resistance rates, the unexpected emergence, establishment and spread of new resistance genes introduces substantial uncertainty into estimates of future economic burden, and in models evaluating the effectiveness of interventions or policies to address AMR. Existing reporting standards for best practice in modelling should be adapted to guide the reporting of AMR economic models, to ensure model transparency and validation for interpretation by policymakers.

Antimicrobial Resistance: Is Health Technology Assessment Part of the Solution or Part of the Problem?

Antimicrobial resistance is a serious challenge to the success and sustainability of our healthcare systems. There has been increasing policy attention given to antimicrobial resistance in the last few years, and increased amounts of funding have been channeled into funding for research and development of antimicrobial agents. Nevertheless, manufacturers doubt whether there will be a market for new antimicrobial technologies sufficient to enable them to recoup their investment. Health technology assessment (HTA) has a critical role in creating confidence that if valuable technologies can be developed they will be reimbursed at a level that captures their true value. We identify 3 deficiencies of current HTA processes for appraising antimicrobial agents: a methods-centric approach rather than problem-centric approach for dealing with new challenges, a lack of tools for thinking about changing patterns of infection, and the absence of an approach to epidemiological risks. We argue that, to play their role more effectively, HTA agencies need to broaden their methodological tool kit, design and communicate their analysis to a wider set of users, and incorporate long-term policy goals, such as containing resistance, as part of their evaluation criteria alongside immediate health gains.

Developing a reference protocol for structured expert elicitation in health-care decision-making: a mixed-methods study

BACKGROUND

Many decisions in health care aim to maximise health, requiring judgements about interventions that may have higher health effects but potentially incur additional costs (cost-effectiveness framework). The evidence used to establish cost-effectiveness is typically uncertain and it is important that this uncertainty is characterised. In situations in which evidence is uncertain, the experience of experts is essential. The process by which the beliefs of experts can be formally collected in a quantitative manner is structured expert elicitation. There is heterogeneity in the existing methodology used in health-care decision-making. A number of guidelines are available for structured expert elicitation; however, it is not clear if any of these are appropriate for health-care decision-making.

OBJECTIVES

The overall aim was to establish a protocol for structured expert elicitation to inform health-care decision-making. The objectives are to (1) provide clarity on methods for collecting and using experts' judgements, (2) consider when alternative methodology may be required in particular contexts, (3) establish preferred approaches for elicitation on a range of parameters, (4) determine which elicitation methods allow experts to express uncertainty and (5) determine the usefulness of the reference protocol developed.

METHODS

A mixed-methods approach was used: systemic review, targeted searches, experimental work and narrative synthesis. A review of the existing guidelines for structured expert elicitation was conducted. This identified the approaches used in existing guidelines (the 'choices') and determined if dominant approaches exist. Targeted review searches were conducted for selection of experts, level of elicitation, fitting and aggregation, assessing accuracy of judgements and heuristics and biases. To sift through the available choices, a set of principles that underpin the use of structured expert elicitation in health-care decision-making was defined using evidence generated from the targeted searches, quantities to elicit experimental evidence and consideration of constraints in health-care decision-making. These principles, including fitness for purpose and reflecting individual expert uncertainty, were applied to the set of choices to establish a reference protocol. An applied evaluation of the developed reference protocol was also undertaken.

RESULTS

For many elements of structured expert elicitation, there was a lack of consistency across the existing guidelines. In almost all choices, there was a lack of empirical evidence supporting recommendations, and in some circumstances the principles are unable to provide sufficient justification for discounting particular choices. It is possible to define reference methods for health technology assessment. These include a focus on gathering experts with substantive skills, eliciting observable quantities and individual elicitation of beliefs. Additional considerations are required for decision-makers outside health technology assessment, for example at a local level, or for early technologies. Access to experts may be limited and in some circumstances group discussion may be needed to generate a distribution.

LIMITATIONS

The major limitation of the work conducted here lies not in the methods employed in the current work but in the evidence available from the wider literature relating to how appropriate particular methodological choices are.

CONCLUSIONS

The reference protocol is flexible in many choices. This may be a useful characteristic, as it is possible to apply this reference protocol across different settings. Further applied studies, which use the choices specified in this reference protocol, are required.

FUNDING

This project was funded by the NIHR Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 25, No. 37. See the NIHR Journals Library website for further project information. This work was also funded by the Medical Research Council (reference MR/N028511/1).

Antimicrobial Resistance in Escherichia coli Strains Isolated from Humans and Pet Animals

Throughout scientific literature, we can find evidence that antimicrobial resistance has become a big problem in the recent years on a global scale. Public healthcare systems all over the world are faced with a great challenge in this respect. Obviously, there are many bacteria that can cause infections in humans and animals alike, but somehow it seems that the greatest threat nowadays comes from the Enterobacteriaceae members, especially Escherichia coli. Namely, we are witnesses to the fact that the systems that these bacteria developed to fight off antibiotics are the strongest and most diverse in Enterobacteriaceae. Our great advantage is in understanding the systems that bacteria developed to fight off antibiotics, so these can help us understand the connection between these microorganisms and the occurrence of antibiotic-resistance both in humans and their pets. Furthermore, unfavorable conditions related to the ease of E. coli transmission via the fecal-oral route among humans, environmental sources, and animals only add to the problem. For all the above stated reasons, it is evident that the epidemiology of E. coli strains and resistance mechanisms they have developed over time are extremely significant topics and all scientific findings in this area will be of vital importance in the fight against infections caused by these bacteria.

Prevalence of Multi-Resistant Microorganisms and Antibiotic Stewardship among Hospitalized Patients Living in Residential Care Homes in Spain: A Cross-Sectional Study

Antimicrobial resistance is a growing global health problem. Patients living in care homes are a vulnerable high-risk population colonized by multidrug-resistant organisms (MDRO). We identified a case series of 116 residents of care homes from a cohort of 540 consecutive patients admitted to the internal medicine service of our hospital. We performed early diagnostic tests of MDRO through anal exudates in our sample. The prevalence of MDRO colonization was 34.5% of residents and 70% of them had not been previously identified in the clinical records. Previous hospitalizations and in-hospital antibiotic administration were significantly associated with the presence of MDRO. Our results emphasize the need to consider care homes in the planning of regional and national infection control measures and for implementing surveillance systems that monitor the spread of antimicrobial resistance in Spain. Systematic early testing upon admission to hospital services with a high prevalence of patients with MDRO colonization (e.g., internal medicine) could contribute to the adoption of adequate prevention measures. Specific educational programs for care home staff should also be implemented to address this increasing problem.