Sensitivity analysis and the expected value of perfect information

Scenario design for infectious disease projections: Integrating concepts from decision analysis and experimental design

Across many fields, scenario modeling has become an important tool for exploring long-term projections and how they might depend on potential interventions and critical uncertainties, with relevance to both decision makers and scientists. In the past decade, and especially during the COVID-19 pandemic, the field of epidemiology has seen substantial growth in the use of scenario projections. Multiple scenarios are often projected at the same time, allowing important comparisons that can guide the choice of intervention, the prioritization of research topics, or public communication. The design of the scenarios is central to their ability to inform important questions. In this paper, we draw on the fields of decision analysis and statistical design of experiments to propose a framework for scenario design in epidemiology, with relevance also to other fields. We identify six different fundamental purposes for scenario designs (decision making, sensitivity analysis, situational awareness, horizon scanning, forecasting, and value of information) and discuss how those purposes guide the structure of scenarios. We discuss other aspects of the content and process of scenario design, broadly for all settings and specifically for multi-model ensemble projections. As an illustrative case study, we examine the first 17 rounds of scenarios from the U.S. COVID-19 Scenario Modeling Hub, then reflect on future advancements that could improve the design of scenarios in epidemiological settings.

Value of Information for Clinical Trial Design: The Importance of Considering All Relevant Comparators

Value of Information (VOI) analyses calculate the economic value that could be generated by obtaining further information to reduce uncertainty in a health economic decision model. VOI has been suggested as a tool for research prioritisation and trial design as it can highlight economically valuable avenues for future research. Recent methodological advances have made it increasingly feasible to use VOI in practice for research; however, there are critical differences between the VOI approach and the standard methods used to design research studies such as clinical trials. We aimed to highlight key differences between the research design approach based on VOI and standard clinical trial design methods, in particular the importance of considering the full decision context. We present two hypothetical examples to demonstrate that VOI methods are only accurate when (1) all feasible comparators are included in the decision model when designing research, and (2) all comparators are retained in the decision model once the data have been collected and a final treatment recommendation is made. Omitting comparators from either the design or analysis phase of research when using VOI methods can lead to incorrect trial designs and/or treatment recommendations. Overall, we conclude that incorrectly specifying the health economic model by ignoring potential comparators can lead to misleading VOI results and potentially waste scarce research resources.

Scenario Design for Infectious Disease Projections: Integrating Concepts from Decision Analysis and Experimental Design

Across many fields, scenario modeling has become an important tool for exploring long-term projections and how they might depend on potential interventions and critical uncertainties, with relevance to both decision makers and scientists. In the past decade, and especially during the COVID-19 pandemic, the field of epidemiology has seen substantial growth in the use of scenario projections. Multiple scenarios are often projected at the same time, allowing important comparisons that can guide the choice of intervention, the prioritization of research topics, or public communication. The design of the scenarios is central to their ability to inform important questions. In this paper, we draw on the fields of decision analysis and statistical design of experiments to propose a framework for scenario design in epidemiology, with relevance also to other fields. We identify six different fundamental purposes for scenario designs (decision making, sensitivity analysis, value of information, situational awareness, horizon scanning, and forecasting) and discuss how those purposes guide the structure of scenarios. We discuss other aspects of the content and process of scenario design, broadly for all settings and specifically for multi-model ensemble projections. As an illustrative case study, we examine the first 17 rounds of scenarios from the U.S. COVID-19 Scenario Modeling Hub, then reflect on future advancements that could improve the design of scenarios in epidemiological settings.

Consolidated Health Economic Evaluation Reporting Standards - Value of Information (CHEERS-VOI): Explanation and Elaboration

OBJECTIVES

Although the ISPOR Value of Information (VOI) Task Force's reports outline VOI concepts and provide good-practice recommendations, there is no guidance for reporting VOI analyses. VOI analyses are usually performed alongside economic evaluations for which the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) 2022 Statement provides reporting guidelines. Thus, we developed the CHEERS-VOI checklist to provide reporting guidance and checklist to support the transparent, reproducible, and high-quality reporting of VOI analyses.

METHODS

A comprehensive literature review generated a list of 26 candidate reporting items. These candidate items underwent a Delphi procedure with Delphi participants through 3 survey rounds. Participants rated each item on a 9-point Likert scale to indicate its relevance when reporting the minimal, essential information about VOI methods and provided comments. The Delphi results were reviewed at 2-day consensus meetings and the checklist was finalized using anonymous voting.

RESULTS

We had 30, 25, and 24 Delphi respondents in rounds 1, 2, and 3, respectively. After incorporating revisions recommended by the Delphi participants, all 26 candidate items proceeded to the 2-day consensus meetings. The final CHEERS-VOI checklist includes all CHEERS items, but 7 items require elaboration when reporting VOI. Further, 6 new items were added to report information relevant only to VOI (eg, VOI methods applied).

CONCLUSIONS

The CHEERS-VOI checklist should be used when a VOI analysis is performed alongside economic evaluations. The CHEERS-VOI checklist will help decision makers, analysts and peer reviewers in the assessment and interpretation of VOI analyses and thereby increase transparency and rigor in decision making.

'First ensure no regret': a decision-theoretic approach to informed consent in clinical practice

Decision theorists recognise that information is valuable only insofar as it has the potential to change a decision. This means that since acquiring more information is time-consuming and sometimes expensive, judgements need to be made about what information is most valuable to acquire, and whether it is worth acquiring at all. In this article I apply this idea to informed consent and argue that the most valuable information relates not to what the best treatment option may be but to possible futures a patient may regret. I conclude by proposing a regret-minimisation framework for informed consent that I contend better captures the true nature of shared decision making than existing formulations.

Information Density in Decision Analysis

Information value has been proposed and used as a probabilistic sensitivity measure, the idea being that uncertain parameters having higher information value are precisely those to which an optimal decision is more sensitive. In this paper, we study the notion of information density as a graphical complement to information value analysis, one that augments an information value calculation with associated directions of information gain. We formally examine mathematical details absent from its earlier presentation that guarantee information density exists and is well posed and describe its relationship to alternate measures of information value. We present its application in the context of a realistic case study and discuss the associated insights.

Cost-effectiveness of adding oseltamivir to primary care for influenza-like-illness: economic evaluation alongside the randomised controlled ALIC4E trial in 15 European countries

BACKGROUND

Oseltamivir is usually not often prescribed (or reimbursed) for non-high-risk patients consulting for influenza-like-illness (ILI) in primary care in Europe. We aimed to evaluate the cost-effectiveness of adding oseltamivir to usual primary care in adults/adolescents (13 years +) and children with ILI during seasonal influenza epidemics, using data collected in an open-label, multi-season, randomised controlled trial of oseltamivir in 15 European countries.

METHODS

Direct and indirect cost estimates were based on patient reported resource use and official country-specific unit costs. Health-Related Quality of Life was assessed by EQ-5D questionnaires. Costs and quality adjusted life-years (QALY) were bootstrapped (N = 10,000) to estimate incremental cost-effectiveness ratios (ICER), from both the healthcare payers' and the societal perspectives, with uncertainty expressed through probabilistic sensitivity analysis and expected value for perfect information (EVPI) analysis. Additionally, scenario (self-reported spending), comorbidities subgroup and country-specific analyses were performed.

RESULTS

The healthcare payers' expected ICERs of oseltamivir were €22,459 per QALY gained in adults/adolescents and €13,001 in children. From the societal perspective, oseltamivir was cost-saving in adults/adolescents, but the ICER is €8,344 in children. Large uncertainties were observed in subgroups with comorbidities, especially for children. The expected ICERs and extent of decision uncertainty varied between countries (EVPI ranged €1-€35 per patient).

CONCLUSION

Adding oseltamivir to primary usual care in Europe is likely to be cost-effective for treating adults/adolescents and children with ILI from the healthcare payers' perspective (if willingness-to-pay per QALY gained > €22,459) and cost-saving in adults/adolescents from a societal perspective.

Value of Information Analysis in Models to Inform Health Policy

Value of information (VoI) is a decision-theoretic approach to estimating the expected benefits from collecting further information of different kinds, in scientific problems based on combining one or more sources of data. VoI methods can assess the sensitivity of models to different sources of uncertainty and help to set priorities for further data collection. They have been widely applied in healthcare policy making, but the ideas are general to a range of evidence synthesis and decision problems. This article gives a broad overview of VoI methods, explaining the principles behind them, the range of problems that can be tackled with them, and how they can be implemented, and discusses the ongoing challenges in the area.

Uncertainty and the Value of Information in Risk Prediction Modeling

Background

Because of the finite size of the development sample, predicted probabilities from a risk prediction model are inevitably uncertain. We apply value-of-information methodology to evaluate the decision-theoretic implications of prediction uncertainty.

Methods

Adopting a Bayesian perspective, we extend the definition of the expected value of perfect information (EVPI) from decision analysis to net benefit calculations in risk prediction. In the context of model development, EVPI is the expected gain in net benefit by using the correct predictions as opposed to predictions from a proposed model. We suggest bootstrap methods for sampling from the posterior distribution of predictions for EVPI calculation using Monte Carlo simulations. We used subsets of data of various sizes from a clinical trial for predicting mortality after myocardial infarction to show how EVPI changes with sample size.

Results

With a sample size of 1000 and at the prespecified threshold of 2% on predicted risks, the gains in net benefit using the proposed and the correct models were 0.0006 and 0.0011, respectively, resulting in an EVPI of 0.0005 and a relative EVPI of 87%. EVPI was zero only at unrealistically high thresholds (>85%). As expected, EVPI declined with larger samples. We summarize an algorithm for incorporating EVPI calculations into the commonly used bootstrap method for optimism correction.

Conclusion

The development EVPI can be used to decide whether a model can advance to validation, whether it should be abandoned, or whether a larger development sample is needed. Value-of-information methods can be applied to explore decision-theoretic consequences of uncertainty in risk prediction and can complement inferential methods in predictive analytics. R code for implementing this method is provided.

Prioritizing Research in an Era of Personalized Medicine: The Potential Value of Unexplained Heterogeneity

Background

Clinical care is moving from a “one size fits all” approach to a setting in which treatment decisions are based on individual treatment response, needs, preferences, and risk. Research into personalized treatment strategies aims to discover currently unknown markers that identify individuals who would benefit from treatments that are nonoptimal at the population level. Before investing in research to identify these markers, it is important to assess whether such research has the potential to generate value. Thus, this article aims to develop a framework to prioritize research into the development of new personalized treatment strategies by creating a set of measures that assess the value of personalizing care based on a set of unknown patient characteristics.

Methods

Generalizing ideas from the value of heterogeneity framework, we demonstrate 3 measures that assess the value of developing personalized treatment strategies. The first measure identifies the potential value of personalizing medicine within a given disease area. The next 2 measures highlight specific research priorities and subgroup structures that would lead to improved patient outcomes from the personalization of treatment decisions.

Results

We graphically present the 3 measures to perform sensitivity analyses around the key drivers of value, in particular, the correlation between the individual treatment benefits across the available treatment options. We illustrate these 3 measures using a previously published decision model and discuss how they can direct research in personalized medicine.

Conclusion

We discuss 3 measures that form the basis of a novel framework to prioritize research into novel personalized treatment strategies. Our novel framework ensures that research targets personalized treatment strategies that have high potential to improve patient outcomes and health system efficiency.

Highlights

The cost-utility of early use of high-flow nasal cannula in bronchiolitis

BACKGROUND

High-flow nasal cannula (HFNC) oxygen is a non-invasive ventilation system that was introduced as an alternative to CPAP (continuous positive airway pressure), with a marked increase in its use in pediatric care settings. This study aimed to evaluate the cost-effectiveness of early use of HFNC compared to oxygen by nasal cannula in an infant with bronchiolitis in the emergency setting.

METHODS

A decision tree model was used to estimate the cost-effectiveness of HFNC compared with oxygen by nasal cannula (control strategy) in an infant with bronchiolitis in the emergency setting. Cost data were obtained from a retrospective study on bronchiolitis from tertiary centers in Rionegro, Colombia, while utilities were collected from the literature.

RESULTS

The QALYs per patient calculated in the base-case model were 0.9141 (95% CI 0.913-0.915) in the HFNC and 0.9105 (95% CI 0.910-0.911) in control group. The cost per patient was US$368 (95% CI US$ 323-411) in HFNC and US$441 (95% CI US$ 384-498) per patient in the control group.

CONCLUSIONS

HFNC was cost-effective HFNC compared to oxygen by nasal cannula in an infant with bronchiolitis in the emergency setting. The use of this technology in emergency settings will allow a more efficient use of resources, especially in low-resource countries with high prevalence of bronchiolitis .

Generating, Presenting, and Interpreting Cost-Effectiveness Results in the Context of Uncertainty: A Tutorial for Deeper Knowledge and Better Practice

This tutorial aims to help make the best available methods for generating and presenting cost-effectiveness results with uncertainty common practice. We believe there is a need for such type of tutorial because some erroneous practices persist (e.g., identifying the cost-effective intervention as the one with the highest probability to be cost-effective), while some of the more advanced methods are hardly used (e.g., the net loss statistic ‘NL’, expected net loss curves and frontier). The tutorial explains with simple examples the pros and cons of using ICER, incremental net benefit and NL to identify the cost-effective intervention, both with and without uncertainty accounted for probabilistically. A flowchart provides practical guidance on when and how to use ICER, incremental net benefit or NL. Different ways to express and present uncertainty in the results are described, including confidence and credible intervals, the probability that a strategy is cost-effective (as usually shown with cost-effectiveness acceptability curves (CEACs)) and the expected value of perfect information (EVPI). The tutorial clarifies and illustrates why EVPI is the only measure accounting fully for decision uncertainty, and why NL curves and the NL frontier may be preferred over CEACs and other plots for presenting cost-effectiveness results in the context of uncertainty. The easy calculations and a worked-out real-life example will help users to thoroughly understand and correctly interpret key cost-effectiveness results. Examples with mathematical calculations, interpretation, plots and R code are provided.

Implementation Barriers to Value of Information Analysis in Health Technology Decision Making: Results From a Process Evaluation

OBJECTIVES

Value of information (VOI) analysis can support health technology assessment decision making, but it is a long way from being standard use. The objective of this study was to understand barriers to the implementation of VOI analysis and propose actions to overcome these.

METHODS

We performed a process evaluation of VOI analysis use within decision making on tomosynthesis versus digital mammography for use in the Dutch breast cancer population screening. Based on steering committee meeting attendance and regular meetings with analysts, we developed a list of barriers to VOI use, which were analyzed using an established diffusion model. We proposed actions to address these barriers. Barriers and actions were discussed and validated in a workshop with stakeholders representing patients, clinicians, regulators, policy advisors, researchers, and the industry.

RESULTS

Consensus was reached on groups of barriers, which included characteristics of VOI analysis itself, stakeholder's attitudes, analysts' and policy makers' skills and knowledge, system readiness, and implementation in the organization. Observed barriers did not only pertain to VOI analysis itself but also to formulating the objective of the assessment, economic modeling, and broader aspects of uncertainty assessment. Actions to overcome these barriers related to organizational changes, knowledge transfer, cultural change, and tools.

CONCLUSIONS

This in-depth analysis of barriers to implementation of VOI analysis and resulting actions and tools may be useful to health technology assessment organizations that wish to implement VOI analysis in technology assessment and research prioritization. Further research should focus on application and evaluation of the proposed actions in real-world assessment processes.

Weighing the unknowns: Value of Information for biological and operational uncertainty in invasion management

The management of biological invasions is a worldwide conservation priority. Unfortunately, decision-making on optimal invasion management can be impeded by lack of information about the biological processes that determine invader success (i.e. biological uncertainty) or by uncertainty about the effectiveness of candidate interventions (i.e. operational uncertainty). Concurrent assessment of both sources of uncertainty within the same framework can help to optimize control decisions.Here, we present a Value of Information (VoI) framework to simultaneously analyse the effects of biological and operational uncertainties on management outcomes. We demonstrate this approach with a case study: minimizing the long-term population growth of musk thistle Carduus nutans, a widespread invasive plant, using several insects as biological control agents, including Trichosirocalus horridus, Rhinocyllus conicus and Urophora solstitialis.The ranking of biocontrol agents was sensitive to differences in the target weed's demography and also to differences in the effectiveness of the different biocontrol agents. This finding suggests that accounting for both biological and operational uncertainties is valuable when making management recommendations for invasion control. Furthermore, our VoI analyses show that reduction of all uncertainties across all combinations of demographic model and biocontrol effectiveness explored in the current study would lead, on average, to a 15.6% reduction in musk thistle population growth rate. The specific growth reduction that would be observed in any instance would depend on how the uncertainties actually resolve. Resolving biological uncertainty (across demographic model combinations) or operational uncertainty (across biocontrol effectiveness combinations) alone would reduce expected population growth rate by 8.5% and 10.5% respectively.Synthesis and applications. Our study demonstrates that intervention rank is determined both by biological processes in the targeted invasive populations and by intervention effectiveness. Ignoring either biological uncertainty or operational uncertainty may result in a suboptimal recommendation. Therefore, it is important to simultaneously acknowledge both sources of uncertainty during the decision-making process in invasion management. The framework presented here can accommodate diverse data sources and modelling approaches, and has wide applicability to guide invasive species management and conservation efforts.

Cost-utility analysis and budget impact of benralizumab as add-on therapy to standard care for severe eosinophilic asthma in Colombia

Introduction:Benralizumab, amonoclonal antibody for human interleukin-5, has been associated with adecrease in asthma exacerbations. The introduction of this drug raises concerns about the economic impact in scenarios with constraints. This study aimed to estimate the cost-utility of benralizumab plus standard care (SoC) vs. SoC alone in adults with severe uncontrolled asthma with evidence of eosinophilic phenotype.Methods:We constructed aMarkov model with three health states (asthma on benralizumab and SOC, asthma on SOC alone, and death) from ahealthcare system perspective over alifetime horizon. The model was populated using local costs while utilities were derived from international literature. Cost and transition probabilities were obtained from amixture of Colombian-specific and internationally published data.Results:The incremental cost-effectiveness ratio (ICER) per patient peryear was $US 42,746per QALY gained. Benralizumab treatment would be cost-effective at the recommended societal US 18,000 WTP threshold if the cost of benralizumab is reduced by 41% more than the base case value.Conclusion:Benralizumab is not cost-effective using WTP of US$18,000per QALY threshold in Colombia. Our study provides evidence that should be used by decision-makers to improve clinical practice guidelines.

Early Cost-effectiveness Analysis of Risk-Based Selection Strategies for Adjuvant Treatment in Stage II Colon Cancer: The Potential Value of Prognostic Molecular Markers

BACKGROUND

To explore the potential value of consensus molecular subtypes (CMS) in stage II colon cancer treatment selection, we carried out an early cost-effectiveness assessment of a CMS-based strategy for adjuvant chemotherapy.

METHODS

We used a Markov cohort model to evaluate three selection strategies: (i) the Dutch guideline strategy (MSS+pT4), (ii) the mutation-based strategy (MSS plus a BRAF and/or KRAS mutation or MSS plus pT4), and (iii) the CMS-based strategy (CMS4 or pT4). Outcomes were number of colon cancer deaths per 1,000 patients, total discounted costs per patient (pp), and quality-adjusted life-years (QALY) pp. The analyses were conducted from a Dutch societal perspective. The robustness of model predictions was assessed in sensitivity analyses. To evaluate the value of future research, we performed a value of information (VOI) analysis.

RESULTS

The Dutch guideline strategy resulted in 8.10 QALYs pp and total costs of €23,660 pp. The CMS-based and mutation-based strategies were more effective and more costly, with 8.12 and 8.13 QALYs pp and €24,643 and €24,542 pp, respectively. Assuming a threshold of €50,000/QALY, the mutation-based strategy was considered as the optimal strategy in an incremental analysis. However, the VOI analysis showed substantial decision uncertainty driven by the molecular markers (expected value of partial perfect information: €18M).

CONCLUSIONS

On the basis of current evidence, our analyses suggest that the mutation-based selection strategy would be the best use of resources. However, the extensive decision uncertainty for the molecular markers does not allow selection of an optimal strategy at present.

IMPACT

Future research is needed to eliminate decision uncertainty driven by molecular markers.

Prioritisation and design of clinical trials

Clinical trials require participation of numerous patients, enormous research resources and substantial public funding. Time-consuming trials lead to delayed implementation of beneficial interventions and to reduced benefit to patients. This manuscript discusses two methods for the allocation of research resources and reviews a framework for prioritisation and design of clinical trials. The traditional error-driven approach of clinical trial design controls for type I and II errors. However, controlling for those statistical errors has limited relevance to policy makers. Therefore, this error-driven approach can be inefficient, waste research resources and lead to research with limited impact on daily practice. The novel value-driven approach assesses the currently available evidence and focuses on designing clinical trials that directly inform policy and treatment decisions. Estimating the net value of collecting further information, prior to undertaking a trial, informs a decision maker whether a clinical or health policy decision can be made with current information or if collection of extra evidence is justified. Additionally, estimating the net value of new information guides study design, data collection choices, and sample size estimation. The value-driven approach ensures the efficient use of research resources, reduces unnecessary burden to trial participants, and accelerates implementation of beneficial healthcare interventions.

Cost utility of fractional exhaled nitric oxide monitoring for the management of children asthma

Introduction

Fractional exhaled nitric oxide is a simple, non-invasive measurement of airway inflammation with minimal discomfort to the patient and with results available within a few minutes. This study aimed to evaluate the cost-effectiveness of asthma management using fractional exhaled nitric oxide monitoring in patients between 4 and 18 years of age.

Methods

A Markov model was used to estimate the cost-utility of asthma management using fractional exhaled nitric oxide monitoring versus asthma management without using fractional exhaled nitric oxide monitoring (standard therapy) in patients between 4 and 18 years of age. Cost data were obtained from a retrospective study on asthma from a tertiary center, in Medellin, Colombia, while probabilities of the Markov model and utilities were obtained from the systematic review of published randomized clinical trials. The analysis was carried out from a societal perspective.

Results

The model showed that fractional exhaled nitric oxide monitoring was associated with a lower total cost than standard therapy (US $1333 vs. US $1452 average cost per patient) and higher QALYs (0.93 vs. 0.92 average per patient). The probability that fractional exhaled nitric oxide monitoring provides a more cost-effective use of resources compared with standard therapy exceeds 99% for all willingness-to-pay thresholds.

Conclusion

Asthma management using fractional exhaled nitric oxide monitoring was cost-effective for treating patients between 4 and 18 years of age with mild to moderate allergic asthma. Our study suggests evidence that could be used by decision-makers to improve clinical practice guidelines, but this should be replicated in different clinical settings.

Optimal cholesterol treatment plans and genetic testing strategies for cardiovascular diseases

Atherosclerotic cardiovascular disease (ASCVD) is among the leading causes of death in the US. Although research has shown that ASCVD has genetic elements, the understanding of how genetic testing influences its prevention and treatment has been limited. To this end, we model the health trajectory of patients stochastically and determine treatment and testing decisions simultaneously. Since the cholesterol level of patients is one controllable risk factor for ASCVD events, we model cholesterol treatment plans as Markov decision processes. We determine whether and when patients should receive a genetic test using value of information analysis. By simulating the health trajectory of over 64 million adult patients, we find that 6.73 million patients undergo genetic testing. The optimal treatment plans informed with clinical and genetic information save 5,487 more quality-adjusted life-years while costing $1.18 billion less than the optimal treatment plans informed with clinical information only. As precision medicine becomes increasingly important, understanding the impact of genetic information becomes essential.

Fighting the Curse of Sparsity: Probabilistic Sensitivity Measures From Cumulative Distribution Functions

Quantitative models support investigators in several risk analysis applications. The calculation of sensitivity measures is an integral part of this analysis. However, it becomes a computationally challenging task, especially when the number of model inputs is large and the model output is spread over orders of magnitude. We introduce and test a new method for the estimation of global sensitivity measures. The new method relies on the intuition of exploiting the empirical cumulative distribution function of the simulator output. This choice allows the estimators of global sensitivity measures to be based on numbers between 0 and 1, thus fighting the curse of sparsity. For density-based sensitivity measures, we devise an approach based on moving averages that bypasses kernel-density estimation. We compare the new method to approaches for calculating popular risk analysis global sensitivity measures as well as to approaches for computing dependence measures gathering increasing interest in the machine learning and statistics literature (the Hilbert-Schmidt independence criterion and distance covariance). The comparison involves also the number of operations needed to obtain the estimates, an aspect often neglected in global sensitivity studies. We let the estimators undergo several tests, first with the wing-weight test case, then with a computationally challenging code with up to k = 30 , 000 inputs, and finally with the traditional Level E benchmark code.

Cost-utility of use of sputum eosinophil counts to guide management in children with asthma

BACKGROUND

Tailoring asthma interventions based on sputum eosinophils are beneficial in reducing the frequency of asthma exacerbations. The routine use of sputum eosinophils in asthma in children is not uniformly adopted. The main barriers to policymakers adopting new technologies are always doubts about their cost-utility in scenarios with scarce health resources. This study aimed to evaluate the cost-utility of sputum eosinophil counts to guide management in children with asthma, from a societal perspective.

METHODS

A Markov simulation with three mutually exclusive nonabsorbent states was used. The intervention evaluated was adjustment of asthma therapy based on sputum eosinophils to adjusting therapy based on clinical symptoms with or without spirometry/peak flow in children between 4 and 18 years of age (EO). The group comparison was adjusting therapy based on clinical symptoms with or without spirometry/peak flow (SC). The analysis was carried out from a societal perspective. The analytic horizon was 12 months.

RESULTS

The model showed that EO was associated with lower cost than SC (US $1375 vs US $1454 average annual cost per patient), and higher QALYs (0.95 vs 0.92 average per patient); showing dominance. The probability that EO provides a more cost-effective use of resources compared with standard therapy exceeds 99% for all willingness to pay thresholds.

CONCLUSION

EO was cost-effective for infants with asthma to guide asthma management in Children. Our study provides evidence that should be used by decision-makers to improve clinical practice guidelines and should be replicated to validate their results in other middle-income countries.

The cost-utility of intravenous magnesium sulfate for treating asthma exacerbations in children

INTRODUCTION

Although evidence supports the use of intravenous magnesium sulfate (MS) in asthma exacerbations, MS continues to be considered a second-line drug for managing pediatric asthma exacerbations. This study aimed to evaluate the cost-utility of MS in asthma exacerbations.

METHODS

We used a decision tree model to estimate the cost-utility of MS compared to treatment without MS (control group) in children with asthma exacerbations. Cost data were obtained from a retrospective study from tertiary centers in Rionegro, Colombia, while utilities were collected from the literature. Probabilistic sensitivity analysis was carried out using the Monte Carlo technique with a simulation of a hypothetical cohort of 10 000 patients to generate expected cost utilities with 95% confidence intervals. We used a cost-effectiveness acceptability curve to evaluate the uncertainty surrounding the cost-utility of MS.

RESULTS

The model showed that MS had a lower total cost than the control group (US $1149 vs US $1598 average cost per patient) and higher quality-adjusted life years (0.60 vs 0.52 average per patient), showing dominance. The probability that MS provides a more cost-effective use of resources compared with standard therapy exceeds 99% for all willingness-to-pay thresholds.

CONCLUSION

Intravenous MS was less expensive and more effective than treatment without intravenous MS in children with asthma exacerbations. Our study provides evidence that should be used by decision-makers to improve clinical practice guidelines and should be replicated to validate its results in other middle-income countries.

Informed and uninformed empirical therapy policies

We argue that a proper distinction must be made between informed and uninformed decision making when setting empirical therapy policies, as this allows one to estimate the value of gathering more information about the pathogens and their transmission and thus to set research priorities. We rely on the stochastic version of a compartmental model to describe the spread of an infecting organism in a health care facility and the emergence and spread of resistance to two drugs. We focus on information and uncertainty regarding the parameters of this model. We consider a family of adaptive empirical therapy policies. In the uninformed setting, the best adaptive policy allowsone to reduce the average cumulative infected patient days over 2 years by 39.3% (95% confidence interval (CI), 30.3-48.1%) compared to the combination therapy. Choosing empirical therapy policies while knowing the exact parameter values allows one to further decrease the cumulative infected patient days by 3.9% (95% CI, 2.1-5.8%) on average. In our setting, the benefit of perfect information might be offset by increased drug consumption.

Computing the Expected Value of Sample Information Efficiently: Practical Guidance and Recommendations for Four Model-Based Methods

Value of information (VOI) analyses can help policy makers make informed decisions about whether to conduct and how to design future studies. Historically a computationally expensive method to compute the expected value of sample information (EVSI) restricted the use of VOI to simple decision models and study designs. Recently, 4 EVSI approximation methods have made such analyses more feasible and accessible. Members of the Collaborative Network for Value of Information (ConVOI) compared the inputs, the analyst's expertise and skills, and the software required for the 4 recently developed EVSI approximation methods. Our report provides practical guidance and recommendations to help inform the choice between the 4 efficient EVSI estimation methods. More specifically, this report provides: (1) a step-by-step guide to the methods' use, (2) the expertise and skills required to implement the methods, and (3) method recommendations based on the features of decision-analytic problems.

Using value of information to prioritize research needs for migratory bird management under climate change: a case study using federal land acquisition in the United States

In response to global habitat loss, many governmental and non-governmental organizations have implemented land acquisition programs to protect critical habitats permanently for priority species. The ability of these protected areas to meet future management objectives may be compromised if the effects of climate change are not considered in acquisition decisions. Unfortunately, the effects of climate change on ecological systems are complex and plagued by uncertainty, making it difficult for organizations to prioritize research needs to improve decision-making. Herein, we demonstrate the use of qualitative value of information analysis to identify and prioritize which sources of uncertainty should be reduced to improve land acquisition decisions to protect migratory birds in the face of climate change. The qualitative value of information analysis process involves four steps: (i) articulating alternative hypotheses; (ii) determining the magnitude of uncertainty regarding each hypothesis; (iii) evaluating the relevance of each hypothesis to acquisition decision-making; and (iv) assessing the feasibility of reducing the uncertainty surrounding each hypothesis through research and monitoring. We demonstrate this approach using the objectives of 3 U.S. federal land acquisition programs that focus on migratory bird management. We used a comprehensive literature review, expert elicitation, and professional judgement to evaluate 11 hypotheses about the effect of climate change on migratory birds. Based on our results, we provide a list of priorities for future research and monitoring to reduce uncertainty and improve land acquisition decisions for the programs considered in our case study. Reducing uncertainty about how climate change will influence the spatial distribution of priority species and biotic homogenization were identified as the highest priorities for future research due to both the value of this information for improving land acquisition decisions and the feasibility of reducing uncertainty through research and monitoring. Research on how changes in precipitation patterns and winter severity will influence migratory bird abundance is also expected to benefit land acquisition decisions. By contrast, hypotheses about phenology and migration distance were identified as low priorities for research. By providing a rigorous and transparent approach to prioritizing research, we demonstrate that qualitative value of information is a valuable tool for prioritizing research and improving management decisions in other complex, high-uncertainty cases where traditional quantitative value of information analysis is not possible. Given the inherent complexity of ecological systems under climate change, and the difficulty of identifying management-relevant research priorities, we expect this approach to have wide applications within the field of natural resource management.

Calculating the Expected Value of Sample Information in Practice: Considerations from 3 Case Studies

Background. Investing efficiently in future research to improve policy decisions is an important goal. Expected value of sample information (EVSI) can be used to select the specific design and sample size of a proposed study by assessing the benefit of a range of different studies. Estimating EVSI with the standard nested Monte Carlo algorithm has a notoriously high computational burden, especially when using a complex decision model or when optimizing over study sample sizes and designs. Recently, several more efficient EVSI approximation methods have been developed. However, these approximation methods have not been compared, and therefore their comparative performance across different examples has not been explored. Methods. We compared 4 EVSI methods using 3 previously published health economic models. The examples were chosen to represent a range of real-world contexts, including situations with multiple study outcomes, missing data, and data from an observational rather than a randomized study. The computational speed and accuracy of each method were compared. Results. In each example, the approximation methods took minutes or hours to achieve reasonably accurate EVSI estimates, whereas the traditional Monte Carlo method took weeks. Specific methods are particularly suited to problems where we wish to compare multiple proposed sample sizes, when the proposed sample size is large, or when the health economic model is computationally expensive. Conclusions. As all the evaluated methods gave estimates similar to those given by traditional Monte Carlo, we suggest that EVSI can now be efficiently computed with confidence in realistic examples. No systematically superior EVSI computation method exists as the properties of the different methods depend on the underlying health economic model, data generation process, and user expertise.

Directly Acting Oral Anticoagulants for the Prevention of Stroke in Atrial Fibrillation in England and Wales: Cost-Effectiveness Model and Value of Information Analysis

Objectives. Determine the optimal, licensed, first-line anticoagulant for prevention of ischemic stroke in patients with non-valvular atrial fibrillation (AF) in England and Wales from the UK National Health Service (NHS) perspective and estimate value to decision making of further research. Methods. We developed a cost-effectiveness model to compare warfarin (international normalized ratio target range 2-3) with directly acting (or non-vitamin K antagonist) oral anticoagulants (DOACs) apixaban 5 mg, dabigatran 150 mg, edoxaban 60 mg, and rivaroxaban 20 mg, over 30 years post treatment initiation. In addition to death, the 17-state Markov model included the events stroke, bleed, myocardial infarction, and intracranial hemorrhage. Input parameters were informed by systematic literature reviews and network meta-analysis. Expected value of perfect information (EVPI) and expected value of partial perfect information (EVPPI) were estimated to provide an upper bound on value of further research. Results. At willingness-to-pay threshold £20,000, all DOACs have positive expected incremental net benefit compared to warfarin, suggesting they are likely cost-effective. Apixaban has highest expected incremental net benefit (£7533), followed by dabigatran (£6365), rivaroxaban (£5279), and edoxaban (£5212). There was considerable uncertainty as to the optimal DOAC, with the probability apixaban has highest net benefit only 60%. Total estimated population EVPI was £17.94 million (17.85 million, 18.03 million), with relative effect between apixaban versus dabigatran making the largest contribution with EVPPI of £7.95 million (7.66 million, 8.24 million). Conclusions. At willingness-to-pay threshold £20,000, all DOACs have higher expected net benefit than warfarin but there is considerable uncertainty between the DOACs. Apixaban had the highest expected net benefit and greatest probability of having highest net benefit, but there is considerable uncertainty between DOACs. A head-to-head apixaban versus dabigatran trial may be of value.

"Time Traveling Is Just Too Dangerous" but Some Methods Are Worth Revisiting: The Advantages of Expected Loss Curves Over Cost-Effectiveness Acceptability Curves and Frontier

BACKGROUND

Cost-effectiveness acceptability curves (CEACs) and the cost-effectiveness acceptability frontier (CEAF) are the recommended graphical representations of uncertainty in a cost-effectiveness analysis (CEA). Nevertheless, many limitations of CEACs and the CEAF have been recognized by others. Expected loss curves (ELCs) overcome these limitations by displaying the expected foregone benefits of choosing one strategy over others, the optimal strategy in expectation, and the value of potential future research all in a single figure.

OBJECTIVES

To revisit ELCs, illustrate their benefits using a case study, and promote their adoption by providing open-source code.

METHODS

We used a probabilistic sensitivity analysis of a CEA comparing 6 cerebrospinal fluid biomarker test-and-treat strategies in patients with mild cognitive impairment. We showed how to calculate ELCs for a set of decision alternatives. We used the probabilistic sensitivity analysis of the case study to illustrate the limitations of currently recommended methods for communicating uncertainty and then demonstrated how ELCs can address these issues.

RESULTS

ELCs combine the probability that each strategy is not cost-effective on the basis of current information and the expected foregone benefits resulting from choosing that strategy (ie, how much is lost if we recommended a strategy with a higher expected loss). ELCs display how the optimal strategy switches across willingness-to-pay thresholds and enables comparison between different strategies in terms of the expected loss.

CONCLUSIONS

ELCs provide a more comprehensive representation of uncertainty and overcome current limitations of CEACs and the CEAF. Communication of uncertainty in CEA would benefit from greater adoption of ELCs as a complementary method to CEACs, the CEAF, and the expected value of perfect information.

Value of Information: Sensitivity Analysis and Research Design in Bayesian Evidence Synthesis

Suppose we have a Bayesian model that combines evidence from several different sources. We want to know which model parameters most affect the estimate or decision from the model, or which of the parameter uncertainties drive the decision uncertainty. Furthermore, we want to prioritize what further data should be collected. These questions can be addressed by Value of Information (VoI) analysis, in which we estimate expected reductions in loss from learning specific parameters or collecting data of a given design. We describe the theory and practice of VoI for Bayesian evidence synthesis, using and extending ideas from health economics, computer modeling and Bayesian design. The methods are general to a range of decision problems including point estimation and choices between discrete actions. We apply them to a model for estimating prevalence of HIV infection, combining indirect information from surveys, registers, and expert beliefs. This analysis shows which parameters contribute most of the uncertainty about each prevalence estimate, and the expected improvements in precision from specific amounts of additional data. These benefits can be traded with the costs of sampling to determine an optimal sample size. Supplementary materials for this article, including a standardized description of the materials available for reproducing the work, are available as an online supplement.

The feasibility of early pulmonary rehabilitation and activity after COPD exacerbations: external pilot randomised controlled trial, qualitative case study and exploratory economic evaluation

BACKGROUND

Chronic obstructive pulmonary disease (COPD) affects > 3 million people in the UK. Acute exacerbations of COPD (AECOPD) are the second most common reason for emergency hospital admission in the UK. Pulmonary rehabilitation is usual care for stable COPD but there is little evidence for early pulmonary rehabilitation (EPR) following AECOPD, either in hospital or immediately post discharge.

OBJECTIVE

To assess the feasibility of recruiting patients, collecting data and delivering EPR to patients with AECOPD to evaluate EPR compared with usual care.

DESIGN

Parallel-group, pilot 2 × 2 factorial randomised trial with nested qualitative research and an economic analysis.

SETTING

Two acute hospital NHS trusts. Recruitment was carried out from September 2015 to April 2016 and follow-up was completed in July 2016.

PARTICIPANTS

Eligible patients were those aged ≥ 35 years who were admitted with AECOPD, who were non-acidotic and who maintained their blood oxygen saturation level (SpO₂) within a prescribed range. Exclusions included the presence of comorbidities that affected the ability to undertake the interventions.

INTERVENTIONS

(1) Hospital EPR: muscle training delivered at the patient's hospital bed using a cycle ergometer and (2) home EPR: a pulmonary rehabilitation programme delivered in the patient's home. Both interventions were delivered by trained physiotherapists. Participants were allocated on a 1 : 1 : 1 : 1 ratio to (1) hospital EPR (n = 14), (2) home EPR (n = 15), (3) hospital EPR and home EPR (n = 14) and (4) control (n = 15). Outcome assessors were blind to treatment allocation; it was not possible to blind patients.

MAIN OUTCOME MEASURES

Feasibility of recruiting 76 participants in 7 months at two centres; intervention delivery; views on intervention/research acceptability; clinical outcomes including the 6-minute walk distance (6WMD); and costs. Semistructured interviews with participants (n = 27) and research health professionals (n = 11), optimisation assessments and an economic analysis were also undertaken.

RESULTS

Over 7 months 449 patients were screened, of whom most were not eligible for the trial or felt too ill/declined entry. In total, 58 participants (76%) of the target 76 participants were recruited to the trial. The primary clinical outcome (6MWD) was difficult to collect (hospital EPR, n = 5; home EPR, n = 6; hospital EPR and home EPR, n = 5; control, n = 5). Hospital EPR was difficult to deliver over 5 days because of patient discharge/staff availability, with 34.1% of the scheduled sessions delivered compared with 78.3% of the home EPR sessions. Serious adverse events were experienced by 26 participants (45%), none of which was related to the interventions. Interviewed participants generally found both interventions to be acceptable. Home EPR had a higher rate of acceptability, mainly because patients felt too unwell when in hospital to undergo hospital EPR. Physiotherapists generally found the interventions to be acceptable and valued them but found delivery difficult because of staffing issues. The health economic analysis results suggest that there would be value in conducting a larger trial to assess the cost-effectiveness of the hospital EPR and hospital EPR plus home EPR trial arms and collect more information to inform the hospital cost and quality-adjusted life-year parameters, which were shown to be key drivers of the model.

CONCLUSIONS

A full-scale randomised controlled trial using this protocol would not be feasible. Recruitment and delivery of the hospital EPR intervention was difficult. The data obtained can be used to design a full-scale trial of home EPR. Because of the small sample and large confidence intervals, this study should not be used to inform clinical practice.

TRIAL REGISTRATION

Current Controlled Trials ISRCTN18634494.

FUNDING

This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 22, No. 11. See the NIHR Journals Library website for further project information.

Cost-Effectiveness of Alirocumab: A Just-in-Time Analysis Based on the ODYSSEY Outcomes Trial

BACKGROUND

The ODYSSEY Outcomes (Evaluation of Cardiovascular Outcomes After an Acute Coronary Syndrome During Treatment With Alirocumab) trial included participants with a recent acute coronary syndrome. Compared with participants receiving statins alone, those receiving a statin plus alirocumab had lower rates of a composite outcome including myocardial infarction (MI), stroke, and death.

OBJECTIVE

To determine the cost-effectiveness of alirocumab in these circumstances.

DESIGN

Decision analysis using the Cardiovascular Disease Policy Model.

DATA SOURCES

Data sources representative of the United States combined with data from the ODYSSEY Outcomes trial.

TARGET POPULATION

U.S. adults with a recent first MI and a baseline low-density lipoprotein cholesterol level of 1.81 mmol/L (70 mg/dL) or greater.

TIME HORIZON

Lifetime.

PERSPECTIVE

U.S. health system.

INTERVENTION

Alirocumab or ezetimibe added to statin therapy.

OUTCOME MEASURES

Incremental cost-effectiveness ratio in 2018 U.S. dollars per quality-adjusted life-year (QALY) gained.

RESULTS OF BASE-CASE ANALYSIS

Compared with a statin alone, the addition of ezetimibe cost $81 000 (95% uncertainty interval [UI], $51 000 to $215 000) per QALY. Compared with a statin alone, the addition of alirocumab cost $308 000 (UI, $197 000 to $678 000) per QALY. Compared with the combination of statin and ezetimibe, replacing ezetimibe with alirocumab cost $997 000 (UI, $254 000 to dominated) per QALY.

RESULTS OF SENSITIVITY ANALYSIS

The price of alirocumab would have to decrease from its original cost of $14 560 to $1974 annually to be cost-effective relative to ezetimibe.

LIMITATION

Effectiveness estimates were based on a single randomized trial with a median follow-up of 2.8 years and should not be extrapolated to patients with stable coronary heart disease.

CONCLUSION

The price of alirocumab would have to be reduced considerably to be cost-effective. Because substantial reductions already have occurred, we believe that timely, independent cost-effectiveness analyses can inform clinical and policy discussions of new drugs as they enter the market.

PRIMARY FUNDING SOURCE

University of California, San Francisco, and Institute for Clinical and Economic Review.

Paramedic Acute Stroke Treatment Assessment (PASTA): study protocol for a randomised controlled trial

BACKGROUND

Despite evidence from clinical trials that intravenous (IV) thrombolysis is a cost-effective treatment for selected acute ischaemic stroke patients, there remain large variations in the rate of IV thrombolysis delivery between stroke services. This study is evaluating whether an enhanced care pathway delivered by paramedics (the Paramedic Acute Stroke Treatment Assessment (PASTA)) could increase the number of patients who receive IV thrombolysis treatment.

METHODS

Study design: Cluster randomised trial with economic analysis and parallel process evaluation.

SETTING

National Health Service ambulance services, emergency departments and hyper-acute stroke units within three geographical regions of England and Wales. Randomisation: Ambulance stations within each region are the units of randomisation. According to station allocation, paramedics based at a station deliver the PASTA pathway (intervention) or continue with standard stroke care (control). Study intervention: The PASTA pathway includes structured pre-hospital information collection, prompted pre-notification, structured handover of information in hospital and assistance with simple tasks during the initial hospital assessment. Study-trained intervention group paramedics deliver this pathway to adults within 4 h of suspected stroke onset. Study control: Standard stroke care according to national and local guidelines for the pre-hospital and hospital assessment of suspected stroke.

PARTICIPANTS

Participants enrolled in the study are adults with confirmed stroke who were assessed by a study paramedic within 4 h of symptom onset.

PRIMARY OUTCOME

Proportion of participants receiving IV thrombolysis.

SAMPLE SIZE

1297 participants provide 90% power to detect a 10% difference in the proportion of patients receiving IV thrombolysis.

DISCUSSION

The results from this trial will determine whether an enhanced care pathway delivered by paramedics can increase thrombolysis delivery rates.

TRIAL REGISTRATION

ISRCTN registry, ISRCTN12418919 . Registered on 5 November 2015.

On the Relationship between Safety and Decision Significance

Risk analysts are often concerned with identifying key safety drivers, that is, the systems, structures, and components (SSCs) that matter the most to safety. SSCs importance is assessed both in the design phase (i.e., before a system is built) and in the implementation phase (i.e., when the system has been built) using the same importance measures. However, in a design phase, it would be necessary to appreciate whether the failure/success of a given SSC can cause the overall decision to change from accept to reject (decision significance). This work addresses the search for the conditions under which SSCs that are safety significant are also decision significant. To address this issue, the work proposes the notion of a θ-importance measure. We study in detail the relationships among risk importance measures to determine which properties guarantee that the ranking of SSCs does not change before and after the decision is made. An application to a probabilistic safety assessment model developed at NASA illustrates the risk management implications of our work.

Estimating the Economic Value of Information for Screening in Disseminating and Targeting Effective School-based Preventive Interventions: An Illustrative Example

When candidates for school-based preventive interventions are heterogeneous in their risk of poor outcomes, an intervention's expected economic net benefits may be maximized by targeting candidates for whom the intervention is most likely to yield benefits, such as those at high risk of poor outcomes. Although increasing amounts of information about candidates may facilitate more accurate targeting, collecting information can be costly. We present an illustrative example to show how cost-benefit analysis results from effective intervention demonstrations can help us to assess whether improved targeting accuracy justifies the cost of collecting additional information needed to make this improvement.

A systematic review, evidence synthesis and meta-analysis of quantitative and qualitative studies evaluating the clinical effectiveness, the cost-effectiveness, safety and acceptability of interventions to prevent postnatal depression

BACKGROUND

Postnatal depression (PND) is a major depressive disorder in the year following childbirth, which impacts on women, their infants and their families. A range of interventions has been developed to prevent PND.

OBJECTIVES

To (1) evaluate the clinical effectiveness, cost-effectiveness, acceptability and safety of antenatal and postnatal interventions for pregnant and postnatal women to prevent PND; (2) apply rigorous methods of systematic reviewing of quantitative and qualitative studies, evidence synthesis and decision-analytic modelling to evaluate the preventive impact on women, their infants and their families; and (3) estimate cost-effectiveness.

DATA SOURCES

We searched MEDLINE, EMBASE, Science Citation Index and other databases (from inception to July 2013) in December 2012, and we were updated by electronic alerts until July 2013.

REVIEW METHODS

Two reviewers independently screened titles and abstracts with consensus agreement. We undertook quality assessment. All universal, selective and indicated preventive interventions for pregnant women and women in the first 6 postnatal weeks were included. All outcomes were included, focusing on the Edinburgh Postnatal Depression Scale (EPDS), diagnostic instruments and infant outcomes. The quantitative evidence was synthesised using network meta-analyses (NMAs). A mathematical model was constructed to explore the cost-effectiveness of interventions contained within the NMA for EPDS values.

RESULTS

From 3072 records identified, 122 papers (86 trials) were included in the quantitative review. From 2152 records, 56 papers (44 studies) were included in the qualitative review. The results were inconclusive. The most beneficial interventions appeared to be midwifery redesigned postnatal care [as shown by the mean 12-month EPDS score difference of -1.43 (95% credible interval -4.00 to 1.36)], person-centred approach (PCA)-based and cognitive-behavioural therapy (CBT)-based intervention (universal), interpersonal psychotherapy (IPT)-based intervention and education on preparing for parenting (selective), promoting parent-infant interaction, peer support, IPT-based intervention and PCA-based and CBT-based intervention (indicated). Women valued seeing the same health worker, the involvement of partners and access to several visits from a midwife or health visitor trained in person-centred or cognitive-behavioural approaches. The most cost-effective interventions were estimated to be midwifery redesigned postnatal care (universal), PCA-based intervention (indicated) and IPT-based intervention in the sensitivity analysis (indicated), although there was considerable uncertainty. Expected value of partial perfect information (EVPPI) for efficacy data was in excess of £150M for each population. Given the EVPPI values, future trials assessing the relative efficacies of promising interventions appears to represent value for money.

LIMITATIONS

In the NMAs, some trials were omitted because they could not be connected to the main network of evidence or did not provide EPDS scores. This may have introduced reporting or selection bias. No adjustment was made for the lack of quality of some trials. Although we appraised a very large number of studies, much of the evidence was inconclusive.

CONCLUSIONS

Interventions warrant replication within randomised controlled trials (RCTs). Several interventions appear to be cost-effective relative to usual care, but this is subject to considerable uncertainty.

FUTURE WORK RECOMMENDATIONS

Several interventions appear to be cost-effective relative to usual care, but this is subject to considerable uncertainty. Future research conducting RCTs to establish which interventions are most clinically effective and cost-effective should be considered.

STUDY REGISTRATION

This study is registered as PROSPERO CRD42012003273.

FUNDING

The National Institute for Health Research Health Technology Assessment programme.

The Impact of Decision Makers' Constraints on the Outcome of Value of Information Analysis

BACKGROUND

When proven effective, decision making regarding reimbursement of new health technology typically involves ethical, social, legal, and health economic aspects and constraints. Nevertheless, when applying standard value of information (VOI) analysis, the value of collecting additional evidence is typically estimated assuming that only cost-effectiveness outcomes guide such decisions.

OBJECTIVES

To illustrate how decision makers' constraints can be incorporated into VOI analyses and how these may influence VOI outcomes.

METHODS

A simulation study was performed to estimate the cost-effectiveness of a new hypothetical technology compared with usual care. Constraints were defined for the new technology on 1) the maximum acceptable rate of complications and 2) the maximum acceptable additional budget. The expected value of perfect information (EVPI) for the new technology was estimated in various scenarios, both with and without incorporating these constraints.

RESULTS

For a willingness-to-pay threshold of €20,000 per quality-adjusted life-year, the probability that the new technology was cost-effective equaled 57%, with an EVPI of €1868 per patient. Applying the complication rate constraint reduced the EVPI to €1137. Similarly, the EVPI reduced to €770 when applying the budget constraint. Applying both constraints simultaneously further reduced the EVPI to €318.

CONCLUSIONS

When decision makers explicitly apply additional constraints, beyond a willingness-to-pay threshold, to reimbursement decisions, these constraints can and should be incorporated into VOI analysis as well, because they may influence VOI outcomes. This requires continuous interaction between VOI analysts and decision makers and is expected to improve both the relevance and the acceptance of VOI outcomes.

Efficient Monte Carlo Estimation of the Expected Value of Sample Information Using Moment Matching

BACKGROUND

The Expected Value of Sample Information (EVSI) is used to calculate the economic value of a new research strategy. Although this value would be important to both researchers and funders, there are very few practical applications of the EVSI. This is due to computational difficulties associated with calculating the EVSI in practical health economic models using nested simulations.

METHODS

We present an approximation method for the EVSI that is framed in a Bayesian setting and is based on estimating the distribution of the posterior mean of the incremental net benefit across all possible future samples, known as the distribution of the preposterior mean. Specifically, this distribution is estimated using moment matching coupled with simulations that are available for probabilistic sensitivity analysis, which is typically mandatory in health economic evaluations.

RESULTS

This novel approximation method is applied to a health economic model that has previously been used to assess the performance of other EVSI estimators and accurately estimates the EVSI. The computational time for this method is competitive with other methods.

CONCLUSION

We have developed a new calculation method for the EVSI which is computationally efficient and accurate.

LIMITATIONS

This novel method relies on some additional simulation so can be expensive in models with a large computational cost.

Deciding with Thresholds: Importance Measures and Value of Information

Risk-informed decision making is often accompanied by the specification of an acceptable level of risk. Such target level is compared against the value of a risk metric, usually computed through a probabilistic safety assessment model, to decide about the acceptability of a given design, the launch of a space mission, etc. Importance measures complement the decision process with information about the risk/safety significance of events. However, importance measures do not tell us whether the occurrence of an event can change the overarching decision. By linking value of information and importance measures for probabilistic risk assessment models, this work obtains a value-of-information-based importance measure that brings together the risk metric, risk importance measures, and the risk threshold in one expression. The new importance measure does not impose additional computational burden because it can be calculated from our knowledge of the risk achievement and risk reduction worth, and complements the insights delivered by these importance measures. Several properties are discussed, including the joint decision worth of basic event groups. The application to the large loss of coolant accident sequence of the Advanced Test Reactor helps us in illustrating the risk analysis insights.

Economic Evaluations Alongside Efficient Study Designs Using Large Observational Datasets: the PLEASANT Trial Case Study

BACKGROUND

Large observational datasets such as Clinical Practice Research Datalink (CPRD) provide opportunities to conduct clinical studies and economic evaluations with efficient designs.

OBJECTIVES

Our objectives were to report the economic evaluation methodology for a cluster randomised controlled trial (RCT) of a UK NHS-delivered public health intervention for children with asthma that was evaluated using CPRD and describe the impact of this methodology on results.

METHODS

CPRD identified eligible patients using predefined asthma diagnostic codes and captured 1-year pre- and post-intervention healthcare contacts (August 2012 to July 2014). Quality-adjusted life-years (QALYs) 4 months post-intervention were estimated by assigning utility values to exacerbation-related contacts; a systematic review identified these utility values because preference-based outcome measures were not collected. Bootstrapped costs were evaluated 12 months post-intervention, both with 1-year regression-based baseline adjustment (BA) and without BA (observed).

RESULTS

Of 12,179 patients recruited, 8190 (intervention 3641; control 4549) were evaluated in the primary analysis, which included patients who received the protocol-defined intervention and for whom CPRD data were available. The intervention's per-patient incremental QALY loss was 0.00017 (bias-corrected and accelerated 95% confidence intervals [BCa 95% CI] -0.00051 to 0.00018) and cost savings were £14.74 (observed; BCa 95% CI -75.86 to 45.19) or £36.07 (BA; BCa 95% CI -77.11 to 9.67), respectively. The probability of cost savings was much higher when accounting for BA versus observed costs due to baseline cost differences between trial arms (96.3 vs. 67.3%, respectively).

CONCLUSION

Economic evaluations using data from a large observational database without any primary data collection is feasible, informative and potentially efficient. Clinical Trials Registration Number: ISRCTN03000938.

The Cost-Effectiveness of Antibiotic Prophylaxis for Patients at Risk of Infective Endocarditis

BACKGROUND

In March 2008, the National Institute for Health and Care Excellence recommended stopping antibiotic prophylaxis (AP) for those at risk of infective endocarditis (IE) undergoing dental procedures in the United Kingdom, citing a lack of evidence of efficacy and cost-effectiveness. We have performed a new economic evaluation of AP on the basis of contemporary estimates of efficacy, adverse events, and resource implications.

METHODS

A decision analytic cost-effectiveness model was used. Health service costs and benefits (measured as quality-adjusted life-years) were estimated. Rates of IE before and after the National Institute for Health and Care Excellence guidance were available to estimate prophylactic efficacy. AP adverse event rates were derived from recent UK data, and resource implications were based on English Hospital Episode Statistics.

RESULTS

AP was less costly and more effective than no AP for all patients at risk of IE. The results are sensitive to AP efficacy, but efficacy would have to be substantially lower for AP not to be cost-effective. AP was even more cost-effective in patients at high risk of IE. Only a marginal reduction in annual IE rates (1.44 cases in high-risk and 33 cases in all at-risk patients) would be required for AP to be considered cost-effective at £20 000 ($26 600) per quality-adjusted life-year. Annual cost savings of £5.5 to £8.2 million ($7.3-$10.9 million) and health gains >2600 quality-adjusted life-years could be achieved from reinstating AP in England.

CONCLUSIONS

AP is cost-effective for preventing IE, particularly in those at high risk. These findings support the cost-effectiveness of guidelines recommending AP use in high-risk individuals.

A Review of Methods for Analysis of the Expected Value of Information

In recent years, value-of-information analysis has become more widespread in health economic evaluations, specifically as a tool to guide further research and perform probabilistic sensitivity analysis. This is partly due to methodological advancements allowing for the fast computation of a typical summary known as the expected value of partial perfect information (EVPPI). A recent review discussed some approximation methods for calculating the EVPPI, but as the research has been active over the intervening years, that review does not discuss some key estimation methods. Therefore, this paper presents a comprehensive review of these new methods. We begin by providing the technical details of these computation methods. We then present two case studies in order to compare the estimation performance of these new methods. We conclude that a method based on nonparametric regression offers the best method for calculating the EVPPI in terms of accuracy, computational time, and ease of implementation. This means that the EVPPI can now be used practically in health economic evaluations, especially as all the methods are developed in parallel with R functions and a web app to aid practitioners.

A Common Rationale for Global Sensitivity Measures and Their Estimation

Measures of sensitivity and uncertainty have become an integral part of risk analysis. Many such measures have a conditional probabilistic structure, for which a straightforward Monte Carlo estimation procedure has a double-loop form. Recently, a more efficient single-loop procedure has been introduced, and consistency of this procedure has been demonstrated separately for particular measures, such as those based on variance, density, and information value. In this work, we give a unified proof of single-loop consistency that applies to any measure satisfying a common rationale. This proof is not only more general but invokes less restrictive assumptions than heretofore in the literature, allowing for the presence of correlations among model inputs and of categorical variables. We examine numerical convergence of such an estimator under a variety of sensitivity measures. We also examine its application to a published medical case study.

An Iterative Bayesian Approach to Health Technology Assessment: Application to a Policy of Preoperative Optimization for Patients Undergoing Major Elective Surgery

Purpose. This article presents an iterative framework for managing the dynamic process of health technology assessment. The framework uses Bayesian statistical decision theory and value of information (VOI) analysis to inform decision making regarding appropriate patient management and to direct future research effort over the lifetime of a technology. Within the article, the framework is applied to a policy decision regarding preoperative patient management before major elective surgery, for which trial data are available.

Method. The evidence available prior to the trial is used to determine the appropriate method of patient management and to ascertain whether, at the time of commissioning, the trial was potentially worthwhile. The prior information is then updated with the trial data via a Bayesian analysis using informative priors. This post trial information set is then used to reassess the appropriate method for patient management and to determine whether there is a requirement for any further research.

Results. Prior to the trial, preoperative optimization with dopexamine is identified as the appropriate method of patient management. The results of the VOI analysis suggest that a short-term trial was potentially worthwhile (population expected value of perfect information [EVPI] = £48 million). Following the trial, the uncertainty surrounding the choice of appropriate patient management and the potential worth of further research had increased (population EVPI = £67 million).

Conclusions. The article demonstrates the value and practicality of applying the iterative framework to the dynamic process of health technology assessment. It is only by formally incorporating all of the information available to decision makers, through informed priors, that the appropriate decisions can be made.

Calculating Partial Expected Value of Perfect Information via Monte Carlo Sampling Algorithms

Partial expected value of perfect information (EVPI) calculations can quantify the value of learning about particular subsets of uncertain parameters in decision models. Published case studies have used different computational approaches. This article examines the computation of partial EVPI estimates via Monte Carlo sampling algorithms. The mathematical definition shows 2 nested expectations, which must be evaluated separately because of the need to compute a maximum between them. A generalized Monte Carlo sampling algorithm uses nested simulation with an outer loop to sample parameters of interest and, conditional upon these, an inner loop to sample remaining uncertain parameters. Alternative computation methods and shortcut algorithms are discussed and mathematical conditions for their use considered. Maxima of Monte Carlo estimates of expectations are biased upward, and the authors show that the use of small samples results in biased EVPI estimates. Three case studies illustrate 1) the bias due to maximization and also the inaccuracy of shortcut algorithms 2) when correlated variables are present and 3) when there is nonlinearity in net benefit functions. If relatively small correlation or nonlinearity is present, then the shortcut algorithm can be substantially inaccurate. Empirical investigation of the numbers of Monte Carlo samples suggests that fewer samples on the outer level and more on the inner level could be efficient and that relatively small numbers of samples can sometimes be used. Several remaining areas for methodological development are set out. A wider application of partial EVPI is recommended both for greater understanding of decision uncertainty and for analyzing research priorities.

A Graphical Decision-Theoretic Model for Neonatal Jaundice

Background. Neonatal jaundice is treated daily at all hospitals. However, the routine, urgency, and case load of most doctors stop them from carefully analyzing all the factors that they would like to (and should) take into account. This article develops a complex decision support system for neonatal jaundice management. Methods. The problem is represented by means of an influence diagram, including admission and treatment decisions. The corresponding uncertainty model is built with the aid of both historical data and subjective judgments. Parents and doctors were interviewed to elicit a multiattribute utility function. The decision analysis cycle is completed with sensitivity analyses and explanations of the results. Results. The construction and use of this decision support system for jaundice management have induced a profound change in daily medical practice, avoiding aggressive treatments—there have been no exchange transfusions in the past 3 years—and reducing the lengths of stay at the hospital. More information is now taken into account to decide on treatments. Interestingly, after embarking on this modeling effort, physicians came to view jaundice as a much more difficult problem than they had initially thought. Comparisons between real cases and system proposals revealed that treatments by nonexpert doctors tend to be longer than what expert doctors would administer. Conclusion. The system is especially designed to help neonatologists in situations in which their lack of experience may lead to unnecessary treatments. Different points of view from several expert doctors and, more interestingly, from parents are taken into account. This knowledge gives a broader picture of the medical problem— incorporating new action criteria, new agents to intervene, more uncertainty variables—to get an insight into the suitability of each therapeutic decision for each patient situation. The benefits gained and the usefulness perceived by neonatologists are worth the increased and time-consuming effort of developing this complex system. Although specially designed for a specific hospital and for neonatal jaundice management, it can be easily adapted to other hospitals and problems.

Large-Sample Bayesian Posterior Distributions for Probabilistic Sensitivity Analysis

In probabilistic sensitivity analyses, analysts assign probability distributions to uncertain model parameters and use Monte Carlo simulation to estimate the sensitivity of model results to parameter uncertainty. The authors present Bayesian methods for constructing large-sample approximate posterior distributions for probabilities, rates, and relative effect parameters, for both controlled and uncontrolled studies, and discuss how to use these posterior distributions in a probabilistic sensitivity analysis. These results draw on and extend procedures from the literature on large-sample Bayesian posterior distributions and Bayesian random effects meta-analysis. They improve on standard approaches to probabilistic sensitivity analysis by allowing a proper accounting for heterogeneity across studies as well as dependence between control and treatment parameters, while still being simple enough to be carried out on a spreadsheet. The authors apply these methods to conduct a probabilistic sensitivity analysis for a recently published analysis of zidovudine prophylaxis following rapid HIV testing in labor to prevent vertical HIV transmission in pregnant women.

Expected Value of Sample Information Calculations in Medical Decision Modeling

There has been an increasing interest in using expected value of information (EVI) theory in medical decision making, to identify the need for further research to reduce uncertainty in decision and as a tool for sensitivity analysis. Expected value of sample information (EVSI) has been proposed for determination of optimum sample size and allocation rates in randomized clinical trials. This article derives simple Monte Carlo, or nested Monte Carlo, methods that extend the use of EVSI calculations to medical decision applications with multiple sources of uncertainty, with particular attention to the form in which epidemiological data and research findings are structured. In particular, information on key decision parameters such as treatment efficacy are invariably available on measures of relative efficacy such as risk differences or odds ratios, but not on model parameters themselves. In addition, estimates of model parameters and of relative effect measures in the literature may be heterogeneous, reflecting additional sources of variation besides statistical sampling error. The authors describe Monte Carlo procedures for calculating EVSI for probability, rate, or continuous variable parameters in multi parameter decision models and approximate methods for relative measures such as risk differences, odds ratios, risk ratios, and hazard ratios. Where prior evidence is based on a random effects meta-analysis, the authors describe different ESVI calculations, one relevant for decisions concerning a specific patient group and the other for decisions concerning the entire population of patient groups. They also consider EVSI methods for new studies intended to update information on both baseline treatment efficacy and the relative efficacy of 2 treatments. Although there are restrictions regarding models with prior correlation between parameters, these methods can be applied to the majority of probabilistic decision models. Illustrative worked examples of EVSI calculations are given in an appendix.