Equitable access to quality injury care; Equi-Injury project protocol for prioritizing interventions in four low- or middle-income countries: a mixed method study

BACKGROUND

Equitable access to quality care after injury is an essential step for improved health outcomes in low- and middle-income countries (LMICs). We introduce the Equi-Injury project, in which we will use integrated frameworks to understand how to improve equitable access to quality care after injury in four LMICs: Ghana, Pakistan, Rwanda and South Africa.

METHODS

This project has 5 work packages (WPs) as well as essential cross-cutting pillars of community engagement, capacity building and cross-country learning. In WP1, we will identify needs, barriers, and facilitators to impactful stakeholder engagement in developing and prioritising policy solutions. In WP2, we will collect data on patient care and outcomes after injuries. In WP3, we will develop an injury pathway model to understand which elements in the pathway of injury response, care and treatment have the biggest impact on health and economic outcomes. In WP4, we will work with stakeholders to gain consensus on solutions to address identified issues; these solutions will be implemented and tested in future research. In WP5, in order to ascertain where learning is transferable across contexts, we will identify which outcomes are shared across countries. The study has received approval from ethical review boards (ERBs) of all partner countries in South Africa, Rwanda, Ghana, Pakistan and the University of Birmingham.

DISCUSSION

This health system evaluation project aims to provide a deeper understanding of injury care and develop evidence-based interventions within and across partner countries in four diverse LMICs. Strong partnership with multiple stakeholders will facilitate utilisation of the results for the co-development of sustainable interventions.

Anti-Vascular Endothelial Growth Factor Drugs Compared With Panretinal Photocoagulation for the Treatment of Proliferative Diabetic Retinopathy: A Cost-Effectiveness Analysis

OBJECTIVES

This study aimed to evaluate the cost-effectiveness of anti-vascular endothelial growth factor drugs (anti-VEGFs) compared with panretinal photocoagulation (PRP) for treating proliferative diabetic retinopathy (PDR) in the United Kingdom.

METHODS

A discrete event simulation model was developed, informed by individual participant data meta-analysis. The model captures treatment effects on best corrected visual acuity in both eyes, and the occurrence of diabetic macular edema and vitreous hemorrhage. The model also estimates the value of undertaking further research to resolve decision uncertainty.

RESULTS

Anti-VEGFs are unlikely to generate clinically meaningful benefits over PRP. The model predicted anti-VEGFs be more costly and similarly effective as PRP, generating 0.029 fewer quality-adjusted life-years at an additional cost of £3688, with a net health benefit of -0.214 at a £20 000 willingness-to-pay threshold. Scenario analysis results suggest that only under very select conditions may anti-VEGFs offer potential for cost-effective treatment of PDR. The consequences of loss to follow-up were an important driver of model outcomes.

CONCLUSIONS

Anti-VEGFs are unlikely to be a cost-effective treatment for early PDR compared with PRP. Anti-VEGFs are generally associated with higher costs and similar health outcomes across various scenarios. Although anti-VEGFs were associated with lower diabetic macular edema rates, the number of cases avoided is insufficient to offset the additional treatment costs. Key uncertainties relate to the long-term comparative effectiveness of anti-VEGFs, particularly considering the real-world rates and consequences of treatment nonadherence. Further research on long-term visual acuity and rates of vision-threatening complications may be beneficial in resolving uncertainties.

Reflecting Parameter Uncertainty in Addition to Variability in Constrained Healthcare Resource Discrete Event Simulations: Worth Going the Extra Mile or a Road to Nowhere?

OBJECTIVES

Probabilistic sensitivity analysis (PSA) has been shown to reduce bias in outcomes of health economic models. However, only 1 existing study has been identified that incorporates PSA within a resource-constrained discrete event simulation (DES) model. This article aims to assess whether it is feasible and appropriate to use PSA to characterize parameter uncertainty in DES models that are primarily constructed to explore the impact of constrained resources.

METHODS

PSA is incorporated into a new case study of an Emergency Department DES. Structured expert elicitation is used to derive the variability and uncertainty input distributions associated with length of time taken to complete key activities within the Emergency Department. Potential challenges of implementation and analysis are explored.

RESULTS

The results of a trial of the model, which used the best estimates of the elicited means and variability around the time taken to complete activities, provided a reasonable fit to the data for length of time within the Emergency Department. However, there was substantial and skewed uncertainty around the activity times estimated from the elicitation exercise. This led to patients taking almost 3 weeks to leave the Emergency Department in some PSA runs, which would not occur in practice.

CONCLUSIONS

Structured expert elicitation can be used to derive plausible estimates of activity times and their variability, but experts' uncertainty can be substantial. For parameters that have an impact on interactions within a resource-constrained simulation model, PSA can lead to implausible model outputs; hence, other methods may be needed.

Priorities for research to support local authority action on health and climate change: a study in England

BACKGROUND

Evidence is needed to support local action to reduce the adverse health impacts of climate change and maximise the health co-benefits of climate action. Focused on England, the study identifies priority areas for research to inform local decision making.

METHODS

Firstly, potential priority areas for research were identified from a brief review of UK policy documents, and feedback invited from public and policy stakeholders. This included a survey of Directors of Public Health (DsPH) in England, the local government officers responsible for public health. Secondly, rapid reviews of research evidence examined whether there was UK evidence relating to the priorities identified in the survey.

RESULTS

The brief policy review pointed to the importance of evidence in two broad areas: (i) community engagement in local level action on the health impacts of climate change and (ii) the economic (cost) implications of such action. The DsPH survey (n = 57) confirmed these priorities. With respect to community engagement, public understanding of climate change's health impacts and the public acceptability of local climate actions were identified as key evidence gaps. With respect to economic implications, the gaps related to evidence on the health and non-health-related costs and benefits of climate action and the short, medium and longer-term budgetary implications of such action, particularly with respect to investments in the built environment. Across both areas, the need for evidence relating to impacts across income groups was highlighted, a point also emphasised by the public involvement panel. The rapid reviews confirmed these evidence gaps (relating to public understanding, public acceptability, economic evaluation and social inequalities). In addition, public and policy stakeholders pointed to other barriers to action, including financial pressures, noting that better evidence is insufficient to enable effective local action.

CONCLUSIONS

There is limited evidence to inform health-centred local action on climate change. More evidence is required on public perspectives on, and the economic dimensions of, local climate action. Investment in locally focused research is urgently needed if local governments are to develop and implement evidence-based policies to protect public health from climate change and maximise the health co-benefits of local action.

Achieving cardiac rehabilitation uptake targets: What is the value case for commissioners? A UK case-study

Cardiac Rehabilitation (CR) has become an established intervention to support patient recovery after a cardiac event, with evidence supporting its effectiveness and cost-effectiveness in improving patient health and reducing future burden on healthcare systems. However, this evidence has focussed on the national value case for CR rather than at the point at which it is commissioned. This analysis uses the UK as a case-study to explore variation in current CR engagement and disassemble the value case from a commissioner perspective. Using data collected by the National Audit of CR (NACR), and an existing model of cost-effectiveness, we present details on the current level of CR uptake by commissioning region (Specialist Clinical Networks) in light of the current UK target of achieving 85% uptake. We then interrogate the value case for achieving the target at a commissioner level, highlighting the expected profile of health benefits and healthcare system costs over the long-term. Importantly we consider where this may differ from the national value case. Each commissioning region has a unique level of CR uptake and sociodemographic profile. Concurrently, the value case for commissioning CR relies on the upfront cost of the service being offset by long-term healthcare savings, and health improvements. The shift in the UK and internationally to more localised commissioning necessitates evidence of cost-effectiveness that better reflects the realities of those decision makers. This paper provides vital additional data to facilitate such commissioners to understand the value case in increasing CR uptake in line with national policy.

Study Protocol. Evaluating the life-course health impact of a city-wide system approach to improve air quality in Bradford, UK: A quasi-experimental study with implementation and process evaluation

BACKGROUND

Air quality is a major public health threat linked to poor birth outcomes, respiratory and cardiovascular disease, and premature mortality. Deprived groups and children are disproportionately affected. Bradford will implement a Clean Air Zone (CAZ) as part of the Bradford Clean Air Plan (B-CAP) in 2022 to reduce pollution, providing a natural experiment. The aim of the current study is to evaluate the impact of the B-CAP on health outcomes and air quality, inequalities and explore value for money. An embedded process and implementation evaluation will also explore barriers and facilitators to implementation, impact on attitudes and behaviours, and any adverse consequences.

METHODS

The study is split into 4 work packages (WP). WP1A: 20 interviews with decision makers, 20 interviews with key stakeholders; 10 public focus groups and documentary analysis of key reports will assess implementation barriers, acceptability and adverse or unanticipated consequences at 1 year post-implementation (defined as point at which charging CAZ goes 'live'). WP1B: A population survey (n = 2000) will assess travel behaviour and attitudes at baseline and change at 1 year post-implementation). WP2: Routine air quality measurements will be supplemented with data from mobile pollution sensors in 12 schools collected by N = 240 pupil citizen scientists (4 within, 4 bordering and 4 distal to CAZ boundary). Pupils will carry sensors over four monitoring periods over a 12 month period (two pre, and two post-implementation). We will explore whether reductions in pollution vary by CAZ proximity. WP3A: We will conduct a quasi-experimental interrupted time series analysis using a longitudinal routine health dataset of > 530,000 Bradford residents comparing trends (3 years prior vs 3 years post) in respiratory health (assessed via emergency/GP attendances. WP3B: We will use the richly-characterised Born in Bradford cohort (13,500 children) to explore health inequalities in respiratory health using detailed socio-economic data. WP4: will entail a multi-sectoral health economic evaluation to determine value for money of the B-CAP.

DISCUSSION

This will be first comprehensive quasi-experimental evaluation of a city-wide policy intervention to improve air quality. The findings will be of value for other areas implementing this type of approach.

TRIAL REGISTRATION

ISRCTN67530835 https://doi.org/10.1186/ISRCTN67530835.

Modelling decay in effectiveness for evaluation of behaviour change interventions: a tutorial for public health economists

BACKGROUND AND PURPOSE

Recent methodological reviews of evaluations of behaviour change interventions in public health have highlighted that the decay in effectiveness over time has been mostly overlooked, potentially leading to suboptimal decision-making. While, in principle, discrete-time Markov chains-the most commonly used modelling approach-can be adapted to account for decay in effectiveness, this framework inherently lends itself to strong model simplifications. The application of formal and more appropriate modelling approaches has been supported, but limited progress has been made to date. The purpose of this paper is to encourage this shift by offering a practical guide on how to model decay in effectiveness using a continuous-time Markov chain (CTMC)-based approach.

METHODS

A CTMC approach is demonstrated, with a contextualized tutorial being presented to facilitate learning and uptake. A worked example based on the stylized case study in physical activity promotion is illustrated with accompanying R code.

DISCUSSION

The proposed framework presents a relatively small incremental change from the current modelling practice. CTMC represents a technical solution which, in absence of relevant data, allows for formally testing the sensitivity of results to assumptions regarding the long-term sustainability of intervention effects and improving model transparency.

CONCLUSIONS

The use of CTMC should be considered in evaluations where decay in effectiveness is likely to be a key factor to consider. This would enable more robust model-based evaluations of population-level programmes to promote behaviour change and reduce the uncertainty surrounding the decision to invest in these public health interventions.

Cost Effectiveness of Digital Interventions for Generalised Anxiety Disorder: A Model-Based Analysis

BACKGROUND

Digital interventions (DIs) are increasingly being used in mental health care, despite limited evidence regarding their value for money. This study aimed to evaluate the cost effectiveness of DIs for generalised anxiety disorder (GAD), in comparison with alternative care options, from the perspective of the UK health care system.

METHODS

An open-source decision analytic cohort model was used to extrapolate the results of a network meta-analysis over a patient's lifetime and estimate the costs and outcomes (quality-adjusted life-years) of DIs and their comparators. The net monetary benefit (NMB) and probability of cost effectiveness was estimated for each comparator, and we conducted a Value of Information analysis to evaluate the scale and drivers of uncertainty.

RESULTS

DIs were associated with lower NMB compared with medication and with group therapy, but greater NMB compared with non-therapeutic controls and with usual care. DIs that were supported by a clinician, an assistant or a lay person had higher delivery costs than purely patient-self-directed DIs, yielding a greater NMB when opportunity cost was above £3000/QALY. There was considerable uncertainty in the findings driven largely by uncertainty in the estimated treatment effects. The value of further research to establish the effectiveness of DIs for GAD was substantial, at least £12.9 billion.

CONCLUSIONS

The high uncertainty about these results does not allow for recommendations based on the cost effectiveness of DIs. However, the analysis highlights areas for future research, and demonstrates that apparent cost savings associated with DIs can be offset by reduced effectiveness.

Reference Case Methods for Expert Elicitation in Health Care Decision Making

BACKGROUND

The evidence used to inform health care decision making (HCDM) is typically uncertain. In these situations, the experience of experts is essential to help decision makers reach a decision. Structured expert elicitation (referred to as elicitation) is a quantitative process to capture experts' beliefs. There is heterogeneity in the existing elicitation methodology used in HCDM, and it is not clear if existing guidelines are appropriate for use in this context. In this article, we seek to establish reference case methods for elicitation to inform HCDM.

METHODS

We collated the methods available for elicitation using reviews and critique. In addition, we conducted controlled experiments to test the accuracy of alternative methods. We determined the suitability of the methods choices for use in HCDM according to a predefined set of principles for elicitation in HCDM, which we have also generated. We determined reference case methods for elicitation in HCDM for health technology assessment (HTA).

RESULTS

In almost all methods choices available for elicitation, we found a lack of empirical evidence supporting recommendations. Despite this, it is possible to define reference case methods for HTA. The reference methods include a focus on gathering experts with substantive knowledge of the quantities being elicited as opposed to those trained in probability and statistics, eliciting quantities that the expert might observe directly, and individual elicitation of beliefs, rather than solely consensus methods. It is likely that there are additional considerations for decision makers in health care outside of HTA.

CONCLUSIONS

The reference case developed here allows the use of different methods, depending on the decision-making setting. Further applied examples of elicitation methods would be useful. Experimental evidence comparing methods should be generated.

Digital interventions in mental health: evidence syntheses and economic modelling

BACKGROUND

Economic evaluations provide evidence on whether or not digital interventions offer value for money, based on their costs and outcomes relative to the costs and outcomes of alternatives.

OBJECTIVES

(1) Evaluate and summarise published economic studies about digital interventions across different technologies, therapies, comparators and mental health conditions; (2) synthesise clinical evidence about digital interventions for an exemplar mental health condition; (3) construct an economic model for the same exemplar mental health condition using the previously synthesised clinical evidence; and (4) consult with stakeholders about how they understand and assess the value of digital interventions.

METHODS

We completed four work packages: (1) a systematic review and quality assessment of economic studies about digital interventions; (2) a systematic review and network meta-analysis of randomised controlled trials on digital interventions for generalised anxiety disorder; (3) an economic model and value-of-information analysis on digital interventions for generalised anxiety disorder; and (4) a series of knowledge exchange face-to-face and digital seminars with stakeholders.

RESULTS

In work package 1, we reviewed 76 economic evaluations: 11 economic models and 65 within-trial analyses. Although the results of the studies are not directly comparable because they used different methods, the overall picture suggests that digital interventions are likely to be cost-effective, compared with no intervention and non-therapeutic controls, whereas the value of digital interventions compared with face-to-face therapy or printed manuals is unclear. In work package 2, we carried out two network meta-analyses of 20 randomised controlled trials of digital interventions for generalised anxiety disorder with a total of 2350 participants. The results were used to inform our economic model, but when considered on their own they were inconclusive because of the very wide confidence intervals. In work package 3, our decision-analytic model found that digital interventions for generalised anxiety disorder were associated with lower net monetary benefit than medication and face-to-face therapy, but greater net monetary benefit than non-therapeutic controls and no intervention. Value for money was driven by clinical outcomes rather than by intervention costs, and a value-of-information analysis suggested that uncertainty in the treatment effect had the greatest value (£12.9B). In work package 4, stakeholders identified several areas of benefits and costs of digital interventions that are important to them, including safety, sustainability and reducing waiting times. Four factors may influence their decisions to use digital interventions, other than costs and outcomes: increasing patient choice, reaching underserved populations, enabling continuous care and accepting the 'inevitability of going digital'.

LIMITATIONS

There was substantial uncertainty around effect estimates of digital interventions compared with alternatives. This uncertainty was driven by the small number of studies informing most comparisons, the small samples in some of these studies and the studies' high risk of bias.

CONCLUSIONS

Digital interventions may offer good value for money as an alternative to 'doing nothing' or 'doing something non-therapeutic' (e.g. monitoring or having a general discussion), but their added value compared with medication, face-to-face therapy and printed manuals is uncertain. Clinical outcomes rather than intervention costs drive 'value for money'.

FUTURE WORK

There is a need to develop digital interventions that are more effective, rather than just cheaper, than their alternatives.

STUDY REGISTRATION

This study is registered as PROSPERO CRD42018105837.

FUNDING

This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 26, No. 1. See the NIHR Journals Library website for further project information.

Does providing everyone with free-of-charge organised exercise opportunities work in public health?

BACKGROUND AND PURPOSE

Population-level initiatives of free-of-charge organised exercise have been implemented to encourage residents to take up regular physical activity. However, there exists a paucity of evidence on the ability of these interventions to attract and engage residents, especially targeted subgroups. Seeking to contribute to this evidence base, we evaluated a proportionate universal programme providing free exercise sessions, Leeds Let's Get Active.

METHODS

Descriptive statistics were used to summarise the programme data and participants. Time to event, count and logistic regression models examined how different population subgroups engaged with the programme in terms of number of entries, weekly participation rates and drop-off patterns.

RESULTS

51,874 adult residents registered to the programme and provided baseline data (2013-2016). A small proportion (1.6%) attended the free sessions on a weekly basis. Higher participation rates were estimated for the groups of males, retired and non-inactive participants. A neighbourhood-level deprivation status was found to have no marginal effect on the level and frequency of participation, but to be negatively associated with participation drop-off (HR 0.93, 95% CI 0.89-0.97, p = 0.001).

CONCLUSIONS

Providing everyone with free-of-charge organised exercise opportunities in public leisure centres located in deprived areas can attract large volumes of residents, but may not sufficiently encourage adults, especially inactive residents and those living in disadvantaged neighbourhoods, to take up regular exercise.

Digital Interventions for Generalized Anxiety Disorder (GAD): Systematic Review and Network Meta-Analysis

Background: Generalized anxiety disorder is the most common mental health condition based on weekly prevalence. Digital interventions have been used as alternatives or as supplements to conventional therapies to improve access, patient choice, and clinical outcomes. Little is known about their comparative effectiveness for generalized anxiety disorder. Methods: We conducted a systematic review and network meta-analysis of randomized controlled trials comparing digital interventions with medication, non-digital interventions, non-therapeutic controls, and no intervention. Results: We included 21 randomized controlled trials with a total of 2,350 participants from generalized anxiety disorder populations. Pooled outcomes using analysis of Covariance and rankograms based on the surface under the cumulative ranking curves indicated that antidepressant medication and group therapy had a higher probability than digital interventions of being the "best" intervention. Supported digital interventions were not necessarily "better" than unsupported (pure self-help) ones. Conclusions: Due to very wide confidence intervals, network meta-analysis results were inconclusive as to whether digital interventions are better than no intervention and non-therapeutic active controls, or whether they confer an additional benefit to standard therapy. Future research needs to compare digital interventions with one-to-one therapy and with manualized non-digital self-help and to include antidepressant medication as a treatment comparator and effect modifier.

The Cost Effectiveness of Ecotherapy as a Healthcare Intervention, Separating the Wood from the Trees

Internationally, shifts to more urbanised populations, and resultant reductions in engagements with nature, have been a contributing factor to the mental health crisis facing many developed and developing countries. While the COVID-19 pandemic reinforced recent trends in many countries to give access to green spaces more weight in political decision making, nature-based activities as a form of intervention for those with mental health problems constitute a very small part of patient pathways of care. Nature-based interventions, such as ecotherapy, are increasingly used as therapeutic solutions for people with common mental health problems. However, there is little data about the potential costs and benefits of ecotherapy, making it difficult to offer robust assessments of its cost-effectiveness. This paper explores the capacity for ecotherapy to be cost-effective as a healthcare intervention. Using a pragmatic scoping review of the literature to understand where the potential costs and health benefit lie, we applied value of information methodology to identify what research is needed to inform future cost-effectiveness assessments. We show that there is the potential for ecotherapy for people with mild to moderate common mental health problems to be cost-effective but significant further research is required. Furthermore, nature-based interventions such as ecotherapy also confer potential social and wider returns on investment, strengthening the case for further research to better inform robust commissioning.

Delayed transfers of care for older people: a wider perspective

Delayed transfers of care (DTOC), often unhelpfully referred to as 'bed blocking', has become a byword for waste and inefficiency in healthcare systems throughout the world. An estimated 2.7 million bed days are occupied each year in England by older people no longer in need of acute treatment, estimated to cost £820 million (2014/15) in inpatient care. Policy and media attention have often been drawn to this narrative of financial waste, resulting in policy setting that directly targets the level of DTOC, but has done little to put patient health first. These figures and policies portray a misleading image of the delays as primarily of concern in terms of their financial burden on acute hospital care, with little consideration given to the quantification on patient health or wider societal impacts. In spite of the multi-factorial decision-making process that occurs for each patient discharge, current evaluation frameworks and national policy setting fail to reflect the complexity of the process. In this commentary, we interrogate the current approach to the quantification of the DTOC impact and explore how policies and evaluation methods can do more to reflect the true impact of the delays.

Developing a reference protocol for structured expert elicitation in health-care decision-making: a mixed-methods study

BACKGROUND

Many decisions in health care aim to maximise health, requiring judgements about interventions that may have higher health effects but potentially incur additional costs (cost-effectiveness framework). The evidence used to establish cost-effectiveness is typically uncertain and it is important that this uncertainty is characterised. In situations in which evidence is uncertain, the experience of experts is essential. The process by which the beliefs of experts can be formally collected in a quantitative manner is structured expert elicitation. There is heterogeneity in the existing methodology used in health-care decision-making. A number of guidelines are available for structured expert elicitation; however, it is not clear if any of these are appropriate for health-care decision-making.

OBJECTIVES

The overall aim was to establish a protocol for structured expert elicitation to inform health-care decision-making. The objectives are to (1) provide clarity on methods for collecting and using experts' judgements, (2) consider when alternative methodology may be required in particular contexts, (3) establish preferred approaches for elicitation on a range of parameters, (4) determine which elicitation methods allow experts to express uncertainty and (5) determine the usefulness of the reference protocol developed.

METHODS

A mixed-methods approach was used: systemic review, targeted searches, experimental work and narrative synthesis. A review of the existing guidelines for structured expert elicitation was conducted. This identified the approaches used in existing guidelines (the 'choices') and determined if dominant approaches exist. Targeted review searches were conducted for selection of experts, level of elicitation, fitting and aggregation, assessing accuracy of judgements and heuristics and biases. To sift through the available choices, a set of principles that underpin the use of structured expert elicitation in health-care decision-making was defined using evidence generated from the targeted searches, quantities to elicit experimental evidence and consideration of constraints in health-care decision-making. These principles, including fitness for purpose and reflecting individual expert uncertainty, were applied to the set of choices to establish a reference protocol. An applied evaluation of the developed reference protocol was also undertaken.

RESULTS

For many elements of structured expert elicitation, there was a lack of consistency across the existing guidelines. In almost all choices, there was a lack of empirical evidence supporting recommendations, and in some circumstances the principles are unable to provide sufficient justification for discounting particular choices. It is possible to define reference methods for health technology assessment. These include a focus on gathering experts with substantive skills, eliciting observable quantities and individual elicitation of beliefs. Additional considerations are required for decision-makers outside health technology assessment, for example at a local level, or for early technologies. Access to experts may be limited and in some circumstances group discussion may be needed to generate a distribution.

LIMITATIONS

The major limitation of the work conducted here lies not in the methods employed in the current work but in the evidence available from the wider literature relating to how appropriate particular methodological choices are.

CONCLUSIONS

The reference protocol is flexible in many choices. This may be a useful characteristic, as it is possible to apply this reference protocol across different settings. Further applied studies, which use the choices specified in this reference protocol, are required.

FUNDING

This project was funded by the NIHR Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 25, No. 37. See the NIHR Journals Library website for further project information. This work was also funded by the Medical Research Council (reference MR/N028511/1).

Taking a local government perspective for economic evaluation of a population-level programme to promote exercise

BACKGROUND

In order to tackle the issue of physical inactivity, local governments have implemented population-level programmes to promote exercise. While evidence is accumulating on the cost-effectiveness of these interventions, studies have typically adopted a health sector perspective for economic evaluation. This approach has been challenged as it does not allow for key concerns by local governments, which are primary stakeholders, to be addressed.

OBJECTIVES

To show how taking a local government perspective for economic evaluation can be implemented in practice and this may affect the economic conclusions.

METHODS

Based on data from a case study, the health equity impact of the intervention and its opportunity cost from a service provider viewpoint were assessed. The cost-effectiveness implications of a change in perspective were subsequently estimated by means of scenario analysis.

FINDINGS

The intervention was found to provide adult residents living in the most deprived city areas with greater health benefits compared with the rest of the population. However, a negative net equity impact was found in the short-term. The opportunity cost of the intervention was estimated to be substantially lower than its financial cost (£2.77 per person/year), with significant implications for decision-making.

CONCLUSIONS

Taking a local government perspective can affect the conclusions drawn from the economic evaluation of population-level programmes to promote exercise, and therefore influence decision making.

Model to Determine the Cost-Effectiveness of Screening Psoriasis Patients for Psoriatic Arthritis

OBJECTIVE

Screening psoriasis patients for psoriatic arthritis (PsA) is intended to identify patients at earlier stages of the disease. Early treatment is expected to slow disease progression and delay the need for biologic therapy. Our objective was to determine the cost-effectiveness of screening for PsA in patients with psoriasis in Canada.

METHODS

A Markov model was built to estimate the costs and quality-adjusted life years (QALYs) of screening tools for PsA in psoriasis patients. The screening tools included the Toronto Psoriatic Arthritis Screen, Psoriasis Epidemiology Screening Tool, Psoriatic Arthritis Screening and Evaluation, and Early Psoriatic Arthritis Screening Questionnaire (EARP) questionnaires. States of health were defined by disability levels as measured by the Health Assessment Questionnaire. State transitions were modeled based on annual disease progression. Incremental cost-effectiveness ratios and incremental net monetary benefits were estimated. Sensitivity analyses were undertaken to account for parameter uncertainty and to test model assumptions.

RESULTS

Screening was cost-effective compared to no screening. The EARP tool had the lowest total cost ($2,000 per patient per year saved compared to no screening) and the highest total QALYs (additional 0.18 per patient compared to no screening). The results were most sensitive to test accuracy and the efficacy of disease-modifying antirheumatic drugs (DMARDs). No screening was cost-effective (at $50,000 per QALY) relative to screening when DMARDs failed to slow disease progression.

CONCLUSION

If early therapy with DMARDs delays biologic treatment, implementing screening in patients with psoriasis in Canada is expected to represent a cost savings of $220 million per year and improve the quality of life.

Systematic Review and Critique of Methods for Economic Evaluation of Digital Mental Health Interventions

OBJECTIVES

Investment in digital interventions for mental health conditions is growing rapidly, offering the potential to elevate systems that are currently overstretched. Despite a growing literature on economic evaluation of digital mental health interventions (DMHIs), including several systematic reviews, there is no conclusive evidence regarding their cost-effectiveness. This paper reviews the methodology used to determine their cost-effectiveness and assesses whether this meets the requirements for decision-making. In doing so we consider the challenges specific to the economic evaluation of DMHIs, and identify where consensus and possible further research is warranted.

METHODS

A systematic review was conducted to identify all economic evaluations of DMHIs published between 1997 and December 2018. The searches included databases of published and unpublished research, reference lists and citations of all included studies, forward citations on all identified protocols and conference abstracts, and contacting authors researchers in the field. The identified studies were critiqued against a published set of requirements for decision-making in healthcare, identifying methodological challenges and areas where consensus is required.

RESULTS

The review identified 67 papers evaluating DMHIs. The majority of the evaluations were conducted alongside trials, failing to capture all relevant available evidence and comparators, and long-term impact of mental health disorders. The identified interventions are complex and heterogeneous. As a result, there are a number of challenges specific to their evaluation, including estimation of all costs and outcomes, conditional on analysis viewpoint, and identification of relevant comparators. A taxonomy for DMHIs may be required to inform what interventions can reasonably be pooled and compared.

CONCLUSIONS

This study represents the first attempt to understand the appropriateness of the methodologies used to evaluate the value for money of DMHIs, helping work towards consensus and methods' harmonisation on these complex interventions.

Antimicrobial-impregnated central venous catheters for preventing neonatal bloodstream infection: the PREVAIL RCT

BACKGROUND

Clinical trials show that antimicrobial-impregnated central venous catheters reduce catheter-related bloodstream infection in adults and children receiving intensive care, but there is insufficient evidence for use in newborn babies.

OBJECTIVES

The objectives were (1) to determine clinical effectiveness by conducting a randomised controlled trial comparing antimicrobial-impregnated peripherally inserted central venous catheters with standard peripherally inserted central venous catheters for reducing bloodstream or cerebrospinal fluid infections (referred to as bloodstream infections); (2) to conduct an economic evaluation of the costs, cost-effectiveness and value of conducting additional research; and (3) to conduct a generalisability analysis of trial findings to neonatal care in the NHS.

DESIGN

Three separate studies were undertaken, each addressing one of the three objectives. (1) This was a multicentre, open-label, pragmatic randomised controlled trial; (2) an analysis was undertaken of hospital care costs, lifetime cost-effectiveness and value of information from an NHS perspective; and (3) this was a retrospective cohort study of bloodstream infection rates in neonatal units in England.

SETTING

The randomised controlled trial was conducted in 18 neonatal intensive care units in England.

PARTICIPANTS

Participants were babies who required a peripherally inserted central venous catheter (of 1 French gauge in size).

INTERVENTIONS

The interventions were an antimicrobial-impregnated peripherally inserted central venous catheter (coated with rifampicin-miconazole) or a standard peripherally inserted central venous catheter, allocated randomly (1 : 1) using web randomisation.

MAIN OUTCOME MEASURE

Study 1 - time to first bloodstream infection, sampled between 24 hours after randomisation and 48 hours after peripherally inserted central venous catheter removal. Study 2 - cost-effectiveness of the antimicrobial-impregnated peripherally inserted central venous catheter compared with the standard peripherally inserted central venous catheters. Study 3 - risk-adjusted bloodstream rates in the trial compared with those in neonatal units in England. For study 3, the data used were as follows: (1) case report forms and linked death registrations; (2) case report forms and linked death registrations linked to administrative health records with 6-month follow-up; and (3) neonatal health records linked to infection surveillance data.

RESULTS

Study 1, clinical effectiveness - 861 babies were randomised (antimicrobial-impregnated peripherally inserted central venous catheter, n = 430; standard peripherally inserted central venous catheter, n = 431). Bloodstream infections occurred in 46 babies (10.7%) randomised to antimicrobial-impregnated peripherally inserted central venous catheters and in 44 (10.2%) babies randomised to standard peripherally inserted central venous catheters. No difference in time to bloodstream infection was detected (hazard ratio 1.11, 95% confidence interval 0.73 to 1.67; p = 0.63). Secondary outcomes of rifampicin resistance in positive blood/cerebrospinal fluid cultures, mortality, clinical outcomes at neonatal unit discharge and time to peripherally inserted central venous catheter removal were similar in both groups. Rifampicin resistance in positive peripherally inserted central venous catheter tip cultures was higher in the antimicrobial-impregnated peripherally inserted central venous catheter group (relative risk 3.51, 95% confidence interval 1.16 to 10.57; p = 0.02) than in the standard peripherally inserted central venous catheter group. Adverse events were similar in both groups. Study 2, economic evaluation - the mean cost of babies' hospital care was £83,473. Antimicrobial-impregnated peripherally inserted central venous catheters were not cost-effective. Given the increased price, compared with standard peripherally inserted central venous catheters, the minimum reduction in risk of bloodstream infection for antimicrobial-impregnated peripherally inserted central venous catheters to be cost-effective was 3% and 15% for babies born at 23-27 and 28-32 weeks' gestation, respectively. Study 3, generalisability analysis - risk-adjusted bloodstream infection rates per 1000 peripherally inserted central venous catheter days were similar among babies in the trial and in all neonatal units. Of all bloodstream infections in babies receiving intensive or high-dependency care in neonatal units, 46% occurred during peripherally inserted central venous catheter days.

LIMITATIONS

The trial was open label as antimicrobial-impregnated and standard peripherally inserted central venous catheters are different colours. There was insufficient power to determine differences in rifampicin resistance.

CONCLUSIONS

No evidence of benefit or harm was found of peripherally inserted central venous catheters impregnated with rifampicin-miconazole during neonatal care. Interventions with small effects on bloodstream infections could be cost-effective over a child's life course. Findings were generalisable to neonatal units in England. Future research should focus on other types of antimicrobial impregnation of peripherally inserted central venous catheters and alternative approaches for preventing bloodstream infections in neonatal care.

TRIAL REGISTRATION

Current Controlled Trials ISRCTN81931394.

FUNDING

This project was funded by the National Institute for Health Research Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 24, No. 57. See the NIHR Journals Library website for further project information.

Estimating the health loss due to poor engagement with cardiac rehabilitation in Australia

BACKGROUND

Cardiac rehabilitation (CR) programs are effective in reducing cardiovascular mortality and readmissions. However, most patients are denied the benefits of CR due to low referral rates. Of those patients referred, commencement rates vary from 28.4% to 60%. This paper quantifies the scale of health loss in Australia due to poor engagement with the program, and estimates how much public funding can be justifiably reallocated to address the problem.

METHODS

Economic decision modelling was undertaken to estimate the expected lifetime health loss and costs to Medicare. Key parameters were derived from Australian databases, CR registries and meta-analyses. Population health gains associated with uptake rates of 60%, and 85% were calculated.

RESULTS

CR was associated with a 99.9% probability of being cost-effective, even at a cost-effectiveness threshold lower than conventionally applied. Importantly, an average of 0.52 years of life expectancy are lost due to national uptake being below 60% achieved in some best performing programs in Australia, equivalent to 0.28 quality adjusted life years. The analysis indicates that $12.9 million/year could be justifiably reallocated from public funds to achieve a national uptake rate of 60%, while maintaining cost-effectiveness of CR due to the large health gains that would be expected.

CONCLUSION

CR is a cost-effective service for patients with coronary heart disease. In Australia, less than a third of patients commence CR, potentially resulting in avoidable patient harm. Additional investment in CR is vital and should be a national priority as the health gains for patients far outweigh the costs.

Cost-effectiveness of a proportionate universal offer of free exercise: Leeds Let’s Get Active

Background

The purpose of this paper is to assess the cost-effectiveness of a proportionate universal programme to reduce physical inactivity (Leeds Let us Get Active (LLGA)) in adults.

Methods

A continuous-time Markov chain model was developed to assess the cost implications and QALY gains associated with increases in physical activity levels across the adult population. A parametric survival analysis approach was applied to estimate the decay of intervention effect over time. Baseline model data were obtained from previous economic models, population-based surveys and other published literature. A cost-utility analysis was conducted from a health care sector perspective over the programme duration (39 months). Scenario and probabilistic sensitivity analyses were performed to test the robustness of cost-effectiveness results.

Results

In total, 51 874 adult residents registered to the programme and provided baseline data,19.5% of which were living in deprived areas. Under base case assumptions, LLGA was found to be likely to be cost-effective. However, variations in key structural assumptions showed sensitivity of the results.

Conclusions

Results from this study suggest a non-negligible level of uncertainty regarding the effectiveness, and therefore, cost-effectiveness of a universal offer of free leisure centre-based exercise that targets hard to reach groups. Further data collection and a shift towards prospective evaluations are needed.

The Relevant Perspective of Economic Evaluations Informing Local Decision Makers: An Exploration in Weight Loss Services

Since 2013, obesity services in the UK National Health Service (NHS) have focused on a tiered structure, with tiers 3 (specialist weight management services) and 4 (primarily bariatric surgery) commissioned by Clinical Commissioning Groups (CCGs) and widely reported as cost effective and recommended by national guidelines. However, CCGs have been reluctant to fully conform to the guidance. We explore how the different evaluative perspective of those generating evidence from local decision makers has contributed to this failure of the CCGs to provide services considered cost effective. We explore four elements where the conventional economic evaluation framework, as applied by the National Institute for Health and Care Excellence (NICE), differ from the reality faced by local decision makers: the cost-effectiveness threshold, the implications of decision uncertainty and budgetary excess, the valuation of future costs and outcomes, and the scope of included costs. We argue that the failure of the conventional framework to reflect the reality faced by local decision makers is rendering much of the existing literature and guidance inappropriate to the key commissioners. Our analysis demonstrates that it is not reasonable to assume that the framework of economic evaluation used to inform national guidance applies to local decision makers, such as in the commissioning of weight loss services. This failure is likely to apply to the majority of cases where evidence is generated to inform national decision makers but commissioning is at a local level.

Cost-effectiveness of strategies preventing late-onset infection in preterm infants

OBJECTIVE

Developing a model to analyse the cost-effectiveness of interventions preventing late-onset infection (LOI) in preterm infants and applying it to the evaluation of anti-microbial impregnated peripherally inserted central catheters (AM-PICCs) compared with standard PICCs (S-PICCs).

DESIGN

Model-based cost-effectiveness analysis, using data from the Preventing infection using Antimicrobial Impregnated Long Lines (PREVAIL) randomised controlled trial linked to routine healthcare data, supplemented with published literature. The model assumes that LOI increases the risk of neurodevelopmental impairment (NDI).

SETTING

Neonatal intensive care units in the UK National Health Service (NHS).

PATIENTS

Infants born ≤32 weeks gestational age, requiring a 1 French gauge PICC.

INTERVENTIONS

AM-PICC and S-PICC.

MAIN OUTCOME MEASURES

Life expectancy, quality-adjusted life years (QALYs) and healthcare costs over the infants' expected lifetime.

RESULTS

Severe NDI reduces life expectancy by 14.79 (95% CI 4.43 to 26.68; undiscounted) years, 10.63 (95% CI 7.74 to 14.02; discounted) QALYs and costs £19 057 (95% CI £14 197; £24697; discounted) to the NHS. If LOI causes NDI, the maximum acquisition price of an intervention reducing LOI risk by 5% is £120. AM-PICCs increase costs (£54.85 (95% CI £25.95 to £89.12)) but have negligible impact on health outcomes (-0.01 (95% CI -0.09 to 0.04) QALYs), compared with S-PICCs. The NHS can invest up to £2.4 million in research to confirm that AM-PICCs are not cost-effective.

CONCLUSIONS

The model quantifies health losses and additional healthcare costs caused by NDI and LOI during neonatal care. Given these consequences, interventions preventing LOI, even by a small extent, can be cost-effective. AM-PICCs, being less effective and more costly than S-PICC, are not likely to be cost-effective.

TRIAL REGISTRATION NUMBER

NCT03260517.

Quantifying the impact of delayed delivery of cardiac rehabilitation on patients' health

Background

Despite its role as an effective intervention to improve the long-term health of patients with cardiovascular disease and existence of national guidelines on timeliness, many health services still fail to offer cardiac rehabilitation in a timely manner after referral. The impact of this failure on patient health and the additional burden on healthcare providers in an English setting is quantified in this article.

Methods

Two logistic regressions are conducted, using the British Heart Foundation National Audit of Cardiac Rehabilitation dataset, to estimate the impact of delayed cardiac rehabilitation initiation on the level of uptake and completion. The results of these regressions are applied to a decision model to estimate the long-term implications of these factors on patient health and National Health Service expenditure.

Results

We demonstrate that the failure of 43.6% of patients in England to start cardiac rehabilitation within the recommended timeframe results in a 15.3% reduction in uptake, and 7.4% in completion. These combine to cause an average lifetime loss of 0.08 years of life expectancy per person. Scaled up to an annual cohort this implies 10,753 patients not taking up cardiac rehabilitation due to the delay, equating to a loss of 3936 years of life expectancy. We estimate that an additional £12.3 million of National Health Service funding could be invested to alleviate the current delay.

Conclusions

The current delay in many patients starting cardiac rehabilitation is causing quantifiable and avoidable harm to their long-term health; policy and research must now look at both supply and demand solutions in tackling this issue.

Understanding and addressing the challenges of conducting quantitative evaluation at a local level: a worked example of the available approaches

OBJECTIVES

In the context of tightening fiscal budgets and increased commissioning responsibility, local decision-makers across the UK healthcare sector have found themselves in charge of the implementation and evaluation of a greater range of healthcare interventions and services. However, there is often little experience, guidance or funding available at a local level to ensure robust evaluations are conducted. In this paper, we evaluate the possible scenarios that could occur when seeking to conduct a quantitative evaluation of a new intervention, specifically with regards to the availability of evidence.

DESIGN

We outline the full set of possible data scenarios that could occur if the decision-maker seeks to explore the impact of the launch of a new intervention on some relevant quantifiable outcomes. In each case we consider the implicit assumptions associated with conducting an evaluation, exploring possible situations where such scenarios may occur. We go on to apply the scenarios to a simulated dataset to explore how each scenario can result in different conclusions as to the effectiveness of the new intervention.

RESULTS

We demonstrate that, across the full set of scenarios, differences in the scale of the estimated effectiveness of a new intervention and even the direction of effect are possible given different data availability and analytical approaches.

CONCLUSIONS

When conducting quantitative evaluations of new interventions, the availability of data on the outcome of interest and the analytical approach can have profound effects on the conclusions of the evaluation. Although it will not always be possible to obtain a complete set of data and conduct extensive analysis, it is vital to understand the implications of the data used and consider the implicit assumptions made through its use.

Does the integration of response services lead to meaningful change in healthcare activity? A case study evaluation

Purpose

The aim of the NHS England Vanguards of new care models was to improve healthcare provision and integration through the coordination of services, seeking to deliver the Five Year Forward View. The purpose of this paper is to report on an extensive analysis of one of the Vanguard programmes, exploring whether the implemented integrated response service (IRS) based in Harrogate, England, resulted in any meaningful change in secondary healthcare activity.

Design/methodology/approach

The authors used an interrupted time series framework applied to aggregate secondary care data, specifically emergency attendances for patients 65+, emergency bed days for all adults and non-elective admissions for 65+. Synthetic and geographic comparator data were employed to inform additional scenario analyses.

Findings

The majority of the analyses conducted found no statistically significant effect of the IRS team in either direction, suggesting that there was no change in the metrics that could be separated from natural variation. The data correlated with the findings of a qualitative analysis and challenges faced in staffing the team towards the end of the analysis period and the eventual disbanding of the IRS.

Research limitations/implications

The analysis was partially hampered by data access challenges, limited to poorly specified aggregate secondary care data, and a poorly specified intervention. Furthermore, the follow-up period was limited by the disbanding of the service.

Originality/value

This analysis indicates that the Harrogate-based IRS team is unlikely to have delivered any sustained quantifiable impact on the intended secondary care outcomes. While this does not necessarily demonstrate a failure of the core principle behind the drive for integrated care, it is an important exploration of the challenges of evaluating such a service.

Correction to: Which Costs Matter? Costs Included in Economic Evaluation and their Impact on Decision Uncertainty for Stable Coronary Artery Disease

The original article can be found online.

Improving cardiac rehabilitation uptake: Potential health gains by socioeconomic status

BACKGROUND

Globally, cardiac rehabilitation (CR) is recommended as soon as possible after admission from an acute myocardial infarction (MI) or revascularisation. However, uptake is consistently poor internationally, ranging from 10% to 60%. The low level of uptake is compounded by variation across different socioeconomic groups. Policy recommendations continue to focus on increasing uptake and addressing inequalities in participation; however, to date, there is a paucity of economic evidence evaluating higher CR participation rates and their relevance to socioeconomic inequality.

METHODS

This study constructed a de-novo cost-effectiveness model of CR, utilising the results from the latest Cochrane review and national CR audit data. We explore the role of socioeconomic status by incorporating key deprivation parameters and determine the population health gains associated with achieving an uptake target of 65%.

RESULTS

We find that the low cost of CR and the potential for reductions in subsequent MI and revascularisation rates combine to make it a highly cost-effective intervention. While CR is less cost-effective for more deprived groups, the lower level of uptake in these groups makes the potential health gains, from achieving the target, greater. Using England as a model, we estimate the expenditure that could be justified while maintaining the cost-effectiveness of CR at £68.4 m per year.

CONCLUSIONS

Increasing CR uptake is cost-effective and can also be implemented to reduce known socioeconomic inequalities. Using an estimation of potential population health gains and justifiable expenditure, we have produced tools with which policymakers and commissioners can encourage greater utilisation of CR services.

Which Costs Matter? Costs Included in Economic Evaluation and their Impact on Decision Uncertainty for Stable Coronary Artery Disease

BACKGROUND

Variation exists in the resource categories included in economic evaluations, and National Institute for Health and Care Excellence (NICE) guidance suggests the inclusion only of costs related to the index condition or intervention. However, there is a growing consensus that all healthcare costs should be included in economic evaluations for Health Technology Assessments (HTAs), particularly those related to extended years of life.

OBJECTIVE AND METHODS

We aimed to quantify the impact of a range of cost categories on the adoption decision about a hypothetical intervention, and uncertainty around that decision, for stable coronary artery disease (SCAD) based on a dataset comprising 94,966 patients. Three costing scenarios were considered: coronary heart disease (CHD) costs only, cardiovascular disease (CVD) costs and all costs. The first two illustrate different interpretations of what might be regarded as related costs.

RESULTS

Employing a 20-year time horizon, the highest mean expected incremental cost was when all costs were included (£2468) and the lowest when CVD costs only were included (£2377). The probability of the treatment being cost effective, estimating health opportunity costs using a ratio of £30,000 per quality-adjusted life-year (QALY), was different for each of the CHD (70%) costs, CVD costs (73%) and all costs (56%) scenarios. The results concern a hypothetical intervention and are illustrative only, as such they cannot necessarily be generalised to all interventions and diseases.

CONCLUSIONS

Cost categories included in an economic evaluation of SCAD impact on estimates of both cost effectiveness and decision uncertainty. With an aging and co-morbid population, the inclusion of all healthcare costs may have important ramifications for the selection of healthcare provision on economic grounds.

Model Structuring for Economic Evaluations of New Health Technologies

In countries such as Australia, the UK and Canada, decisions on whether to fund new health technologies are commonly informed by decision analytic models. While the impact of making inappropriate structural choices/assumptions on model predictions is well noted, there is a lack of clarity about the definition of key structural aspects, the process of developing model structure (including the development of conceptual models) and uncertainty associated with the structuring process (structural uncertainty) in guidelines developed by national funding bodies. This forms the focus of this article. Building on the reports of good modelling practice, and recognising the fundamental role of model structuring within the model development process, we specified key structural choices and provided ideas about model structuring for the future direction. This will help to further standardise guidelines developed by national funding bodies, with potential impact on transparency, comprehensiveness and consistency of model structuring. We argue that the process of model structuring and structural sensitivity analysis should be documented in a more systematic and transparent way in submissions to national funding bodies. Within the decision-making process, the development of conceptual models and presentation of all key structural choices would mean that national funding bodies could be more confident of maximising value for money when making public funding decisions.

Impact of specialist rehabilitation services on hospital length of stay and associated costs

BACKGROUND

Provision of specialist rehabilitation services in North Yorkshire and Humberside may be suboptimal. Local commissioning bodies need to prioritise investments in health care, but previous studies provide limited evidence to inform the decision to expand existing services on the basis of cost-effectiveness. We examine the impact of specialist rehabilitation services in the subregion on hospital length of stay (LoS) and associated costs compared to routine care.

METHODS

Comparison of hospital LoS and associated costs in centres with greater access (Hull) and limited access (i.e. routine care, York and Northern Lincolnshire), to specialist rehabilitation services for patients with complex disabilities following illness or injury, using Hospital Episodes Statistics data.

RESULTS

Average LoS and duration costs by Healthcare Resource Group (HRG) were lower for the majority of patients with greater access to specialist rehabilitation compared to routine care. Difference in LoS between groups widened with level of complexity within each HRG. For the more frequent HRG codes, the LoS difference was as high as 34 days longer for York compared to Hull and £7900 more costly.

CONCLUSION

Rehabilitation patients within York and Northern Lincolnshire areas appear to have longer LoS and higher associated costs compared to those admitted to the Hull Trust. This analysis suggests that specialist rehabilitation may be cost saving compared to routine care and supports the case for expansion of the existing services to improve coverage in the area.

Certolizumab pegol and secukinumab for treating active psoriatic arthritis following inadequate response to disease-modifying antirheumatic drugs: a systematic review and economic evaluation

BACKGROUND

Several biologic therapies are approved by the National Institute for Health and Care Excellence (NICE) for psoriatic arthritis (PsA) patients who have had an inadequate response to two or more synthetic disease-modifying antirheumatic drugs (DMARDs). NICE does not specifically recommend switching from one biologic to another, and only ustekinumab (UST; STELARA^®, Janssen Pharmaceuticals, Inc., Horsham, PA, USA) is recommended after anti-tumour necrosis factor failure. Secukinumab (SEC; COSENTYX^®, Novartis International AG, Basel, Switzerland) and certolizumab pegol (CZP; CIMZIA^®, UCB Pharma, Brussels, Belgium) have not previously been appraised by NICE.

OBJECTIVE

To determine the clinical effectiveness and cost-effectiveness of CZP and SEC for treating active PsA in adults in whom DMARDs have been inadequately effective.

DESIGN

Systematic review and economic model.

DATA SOURCES

Fourteen databases (including MEDLINE and EMBASE) were searched for relevant studies from inception to April 2016 for CZP and SEC studies; update searches were run to identify new comparator studies.

REVIEW METHODS

Clinical effectiveness data from randomised controlled trials (RCTs) were synthesised using Bayesian network meta-analysis (NMA) methods to investigate the relative efficacy of SEC and CZP compared with comparator therapies. A de novo model was developed to assess the cost-effectiveness of SEC and CZP compared with the other relevant comparators. The model was specified for three subpopulations, in accordance with the NICE scope (patients who have taken one prior DMARD, patients who have taken two or more prior DMARDs and biologic-experienced patients). The models were further classified according to the level of concomitant psoriasis.

RESULTS

Nineteen eligible RCTs were included in the systematic review of short-term efficacy. Most studies were well conducted and were rated as being at low risk of bias. Trials of SEC and CZP demonstrated clinically important efficacy in all key clinical outcomes. At 3 months, patients taking 150 mg of SEC [relative risk (RR) 6.27, 95% confidence interval (CI) 2.55 to 15.43] or CZP (RR 3.29, 95% CI 1.94 to 5.56) were more likely to be responders than patients taking placebo. The NMA results for the biologic-naive subpopulations indicated that the effectiveness of SEC and CZP relative to other biologics and each other was uncertain. Limited data were available for the biologic-experienced subpopulation. Longer-term evidence suggested that these newer biologics reduced disease progression, with the benefits being similar to those seen for older biologics. The de novo model generated incremental cost-effectiveness ratios (ICERs) for three subpopulations and three psoriasis subgroups. In subpopulation 1 (biologic-naive patients who had taken one prior DMARD), CZP was the optimal treatment in the moderate-severe psoriasis subgroup and 150 mg of SEC was optimal in the subgroups of patients with mild-moderate psoriasis or no concomitant psoriasis. In subpopulation 2 (biologic-naive patients who had taken two or more prior DMARDs), etanercept (ETN; ENBREL^®, Pfizer Inc., New York City, NY, USA) is likely to be the optimal treatment in all subgroups. The ICERs for SEC and CZP versus best supportive care are in the region of £20,000-30,000 per quality-adjusted life-year (QALY). In subpopulation 3 (biologic-experienced patients or patients in whom biologics are contraindicated), UST is likely to be the optimal treatment (ICERs are in the region of £21,000-27,000 per QALY). The optimal treatment in subpopulation 2 was sensitive to the choice of evidence synthesis model. In subpopulations 2 and 3, results were sensitive to the algorithm for Health Assessment Questionnaire-Disability Index costs. The optimal treatment is not sensitive to the use of biosimilar prices for ETN and infliximab (REMICADE^®, Merck Sharp & Dohme, Kenilworth, NJ, USA).

CONCLUSIONS

SEC and CZP may be an effective use of NHS resources, depending on the subpopulation and subgroup of psoriasis severity. There are a number of limitations to this assessment, driven mainly by data availability.

FUTURE WORK

Trials are needed to inform effectiveness of biologics in biologic-experienced populations.

STUDY REGISTRATION

This study is registered as PROSPERO CRD42016033357.

FUNDING

The National Institute for Health Research Health Technology Assessment programme.

Experiences of Structured Elicitation for Model-Based Cost-Effectiveness Analyses

BACKGROUND

Empirical evidence supporting the cost-effectiveness estimates of particular health care technologies may be limited, or it may even be missing entirely. In these situations, additional information, often in the form of expert judgments, is needed to reach a decision. There are formal methods to quantify experts' beliefs, termed as structured expert elicitation (SEE), but only limited research is available in support of methodological choices. Perhaps as a consequence, the use of SEE in the context of cost-effectiveness modelling is limited.

OBJECTIVES

This article reviews applications of SEE in cost-effectiveness modelling with the aim of summarizing the basis for methodological choices made in each application and recording the difficulties and challenges reported by the authors in the design, conduct, and analyses.

METHODS

The methods used in each application were extracted along with the criteria used to support methodological and practical choices and any issues or challenges discussed in the text. Issues and challenges were extracted using an open field, and then categorised and grouped for reporting.

RESULTS

The review demonstrates considerable heterogeneity in methods used, and authors acknowledge great methodological uncertainty in justifying their choices. Specificities of the context area emerging as potentially important in determining further methodological research in elicitation are between- expert variation and its interpretation, the fact that substantive experts in the area may not be trained in quantitative subjects, that judgments are often needed on various parameter types, the need for some form of assessment of validity, and the need for more integration with behavioural research to devise relevant debiasing strategies.

CONCLUSIONS

This review of experiences of SEE highlights a number of specificities/constraints that can shape the development of guidance and target future research efforts in this area.

Using Cost-Effectiveness Analysis to Quantify the Value of Genomic-Based Diagnostic Tests: Recommendations for Practice and Research

AIMS

New sequencing technologies allow increased opportunities to use genomic-based diagnostic tests (genomic tests) in routine clinical practice, which will impact healthcare budgets and patients' outcomes. This article aims to generate a list of recommendations on how the principles and methods of cost-effectiveness analysis (CEA) can be used to quantify the costs and benefits of genomic tests.

METHODS

A systematic literature search identified publications describing the use of CEA to evaluate genomic tests. Data were extracted as key concepts to produce a thematic list of previously described challenges and solutions to using CEA to evaluate genomic tests. Defining features of evaluating genomic tests were categorized into a list of key recommendations for applying methods in practice and for research needs.

RESULTS

Features producing challenges in the implementation of CEA to evaluate genomic tests were as follows: the ability of the tests to diagnose multiple disorders; potential consequences for future generations suggesting an infinite time horizon; and the potential need to consider nonhealth benefits.

CONCLUSIONS

CEA was identified as an appropriate evaluative framework for genomic tests, although standard methods may need modification and important method research questions remain. Key recommendations suggest a need for research to reflect: sharing genomic information across generations; genomic tests for multiple disorders; and health and nonhealth benefits.

Informing Reimbursement Decisions Using Cost-Effectiveness Modelling: A Guide to the Process of Generating Elicited Priors to Capture Model Uncertainties

In informing decisions, utilising health technology assessment (HTA), expert elicitation can provide valuable information, particularly where there is a less-developed evidence-base at the point of market access. In these circumstances, formal methods to elicit expert judgements are preferred to improve the accountability and transparency of the decision-making process, help reduce bias and the use of heuristics, and also provide a structure that allows uncertainty to be expressed. Expert elicitation is the process of transforming the subjective and implicit knowledge of experts into their quantifiable expressions. The use of expert elicitation in HTA is gaining momentum, and there is particular interest in its application to diagnostics, medical devices and complex interventions such as in public health or social care. Compared with the gathering of experimental evidence, elicitation constitutes a reasonably low-cost source of evidence. Given its inherent subject nature, the potential biases in elicited evidence cannot be ignored and, due to its infancy in HTA, there is little guidance to the analyst wishing to conduct a formal elicitation exercise. This article attempts to summarise the stages of designing and conducting an expert elicitation, drawing on key literature and examples, most of which are not in HTA. In addition, we critique their applicability to HTA, given its distinguishing features. There are a number of issues that the analyst should be mindful of, in particular the need to appropriately characterise the uncertainty associated with model inputs and the fact that there are often numerous parameters required, not all of which can be defined using the same quantities. This increases the need for the elicitation task to be as straightforward as possible for the expert to complete.

How to Appropriately Extrapolate Costs and Utilities in Cost-Effectiveness Analysis

Costs and utilities are key inputs into any cost-effectiveness analysis. Their estimates are typically derived from individual patient-level data collected as part of clinical studies the follow-up duration of which is often too short to allow a robust quantification of the likely costs and benefits a technology will yield over the patient's entire lifetime. In the absence of long-term data, some form of temporal extrapolation-to project short-term evidence over a longer time horizon-is required. Temporal extrapolation inevitably involves assumptions regarding the behaviour of the quantities of interest beyond the time horizon supported by the clinical evidence. Unfortunately, the implications for decisions made on the basis of evidence derived following this practice and the degree of uncertainty surrounding the validity of any assumptions made are often not fully appreciated. The issue is compounded by the absence of methodological guidance concerning the extrapolation of non-time-to-event outcomes such as costs and utilities. This paper considers current approaches to predict long-term costs and utilities, highlights some of the challenges with the existing methods, and provides recommendations for future applications. It finds that, typically, economic evaluation models employ a simplistic approach to temporal extrapolation of costs and utilities. For instance, their parameters (e.g. mean) are typically assumed to be homogeneous with respect to both time and patients' characteristics. Furthermore, costs and utilities have often been modelled to follow the dynamics of the associated time-to-event outcomes. However, cost and utility estimates may be more nuanced, and it is important to ensure extrapolation is carried out appropriately for these parameters.

Evaluating the cost-effectiveness of biologic treatments for psoriatic arthritis: can we make better use of patient data registries?

The primary aim of this study is to explore the extent to which registry data may fulfill the evidence requirements of cost-effectiveness analysis (CEA) studies evaluating biologic therapies for the treatment of psoriatic arthritis (PsA), where trial data are lacking or insufficient. In addition, the paper aims to identify how future data collection in PsA registries might be better tailored to inform CEA research. A review of the literature was performed to identify existing registries containing PsA patients. Where possible, information was extracted on the design and characteristics of the registries. The registries were then appraised according to a set of criteria that was formulated based on the methods currently used to model PsA in the CEA literature. A review of the literature identified 21 potentially relevant registries from around the world containing patients with PsA. There was substantial variation regarding the extent to which the registries, as a whole, were useful for the purposes of CEA studies. There were also notable disparities found in terms of the accessibility of the registries to researchers. The critical review conducted in this study showed that all of the registries identified are potentially useful, at least in some degree, for the purposes of informing CEA studies in PsA. However, no individual registry on its own was found to meet all of the evidence requirements when considering how the disease has been modeled previously.

The Clinical and Cost Effectiveness of Apremilast for Treating Active Psoriatic Arthritis: A Critique of the Evidence

As part of the National Institute for Health and Clinical Excellence (NICE) single technology appraisal (STA) process, the manufacturer of apremilast was invited to submit evidence for its clinical and cost effectiveness for the treatment of active psoriatic arthritis (PsA) for whom disease-modifying anti-rheumatic drugs (DMARDs) have been inadequately effective, not tolerated or contraindicated. The Centre for Reviews and Dissemination and Centre for Health Economics at the University of York were commissioned to act as the independent Evidence Review Group (ERG). This paper provides a description of the ERG review of the company's submission, the ERG report and submission and summarises the NICE Appraisal Committee's subsequent guidance (December 2015). In the company's initial submission, the base-case analysis resulted in an incremental cost-effectiveness ratio (ICER) of £14,683 per quality-adjusted life-year (QALY) gained for the sequence including apremilast (positioned before tumour necrosis factor [TNF]-α inhibitors) versus a comparator sequence without apremilast. However, the ERG considered that the base-case sequence proposed by the company represented a limited set of potentially relevant treatment sequences and positions for apremilast. The company's base-case results were therefore not a sufficient basis to inform the most efficient use and position of apremilast. The exploratory ERG analyses indicated that apremilast is more effective (i.e. produces higher health gains) when positioned after TNF-α inhibitor therapies. Furthermore, assumptions made regarding a potential beneficial effect of apremilast on long-term Health Assessment Questionnaire (HAQ) progression, which cannot be substantiated, have a very significant impact on results. The NICE Appraisal Committee (AC), when taking into account their preferred assumptions for HAQ progression for patients on treatment with apremilast, placebo response and monitoring costs for apremilast, concluded that the addition of apremilast resulted in cost savings but also a QALY loss. These cost savings were not high enough to compensate for the clinical effectiveness that would be lost. The AC thus decided that apremilast alone or in combination with DMARD therapy is not recommended for treating adults with active PsA that has not responded to prior DMARD therapy, or where such therapy is not tolerated.

Designing and Undertaking a Health Economics Study of Digital Health Interventions

This paper introduces and discusses key issues in the economic evaluation of digital health interventions. The purpose is to stimulate debate so that existing economic techniques may be refined or new methods developed. The paper does not seek to provide definitive guidance on appropriate methods of economic analysis for digital health interventions. This paper describes existing guides and analytic frameworks that have been suggested for the economic evaluation of healthcare interventions. Using selected examples of digital health interventions, it assesses how well existing guides and frameworks align to digital health interventions. It shows that digital health interventions may be best characterized as complex interventions in complex systems. Key features of complexity relate to intervention complexity, outcome complexity, and causal pathway complexity, with much of this driven by iterative intervention development over time and uncertainty regarding likely reach of the interventions among the relevant population. These characteristics imply that more-complex methods of economic evaluation are likely to be better able to capture fully the impact of the intervention on costs and benefits over the appropriate time horizon. This complexity includes wider measurement of costs and benefits, and a modeling framework that is able to capture dynamic interactions among the intervention, the population of interest, and the environment. The authors recommend that future research should develop and apply more-flexible modeling techniques to allow better prediction of the interdependency between interventions and important environmental influences.

A discrete choice experiment to explore patients' willingness to risk disease relapse from treatment withdrawal in psoriatic arthritis

The objective of this study is to assess patient preferences for treatment-related benefits and risk of disease relapse in the management of low disease states of psoriatic arthritis (PsA). Focus groups with patients and a literature review were undertaken to determine the characteristics of treatment and symptoms of PsA important to patients. Patient preferences were assessed using a discrete choice experiment which compared hypothetical treatment profiles of the risk and benefits of treatment withdrawal. The risk outcome included increased risk of disease relapse, while benefit outcomes included reduced sickness/nausea from medication and changes in health-related quality of life. Each patient completed 12 choice sets comparing treatment profiles. Preference weights were estimated using a logic regression model, and the maximum acceptable risk in disease relapse for a given improvement in benefit outcomes was elicited. Final sample included 136 patients. Respondents attached the greatest importance to eliminating severe side effects of sickness/nausea and the least importance to a change in risk of relapse. Respondents were willing to accept an increase in the risk of relapse of 32.6 % in order to eliminate the side effects of sickness/nausea. For improvements in health status, the maximum acceptable risk in relapse was comparable to a movement from some to no sickness/nausea. The study suggests that patients in low disease states of PsA are willing to accept greater risks of relapse for improvements in side effects of sickness/nausea and overall health status, with the most important benefit attribute being the elimination of severe sickness or nausea.

A Comprehensive Algorithm for Approval of Health Technologies With, Without, or Only in Research: The Key Principles for Informing Coverage Decisions

BACKGROUND

The value of evidence about the performance of a technology and the value of access to a technology are central to policy decisions regarding coverage with, without, or only in research and managed entry (or risk-sharing) agreements.

OBJECTIVES

We aim to outline the key principles of what assessments are needed to inform "only in research" (OIR) or "approval with research" (AWR) recommendations, in addition to approval or rejection.

METHODS

We developed a comprehensive algorithm to inform the sequence of assessments and judgments that lead to different types of guidance: OIR, AWR, Approve, or Reject. This algorithm identifies the order in which assessments might be made, how similar guidance might be arrived at through different combinations of considerations, and when guidance might change.

RESULTS

The key principles are whether the technology is expected to be cost-effective; whether the technology has significant irrecoverable costs; whether additional research is needed; whether research is possible with approval and whether there are opportunity costs that once committed by approval cannot be recovered; and whether there are effective price reductions. Determining expected cost-effectiveness is only a first step. In addition to AWR for technologies expected to be cost-effective and OIR for those not expected to be cost-effective, there are other important circumstances when OIR should be considered.

CONCLUSIONS

These principles demonstrate that cost-effectiveness is a necessary but not sufficient condition for approval. Even when research is possible with approval, OIR may be appropriate when a technology is expected to be cost-effective due to significant irrecoverable costs.

Extrapolating Survival from Randomized Trials Using External Data: A Review of Methods

This article describes methods used to estimate parameters governing long-term survival, or times to other events, for health economic models. Specifically, the focus is on methods that combine shorter-term individual-level survival data from randomized trials with longer-term external data, thus using the longer-term data to aid extrapolation of the short-term data. This requires assumptions about how trends in survival for each treatment arm will continue after the follow-up period of the trial. Furthermore, using external data requires assumptions about how survival differs between the populations represented by the trial and external data. Study reports from a national health technology assessment program in the United Kingdom were searched, and the findings were combined with “pearl-growing” searches of the academic literature. We categorized the methods that have been used according to the assumptions they made about how the hazards of death vary between the external and internal data and through time, and we discuss the appropriateness of the assumptions in different circumstances. Modeling choices, parameter estimation, and characterization of uncertainty are discussed, and some suggestions for future research priorities in this area are given.

A pharmacoeconomic approach to assessing the costs and benefits of air quality interventions that improve health: a case study

OBJECTIVE

This paper explores the use of pharmacoeconomic methods of valuation to health impacts resulting from exposure to poor air quality. In using such methods, interventions that reduce exposure to poor air quality can be directly compared, in terms of value for money (or cost-effectiveness), with competing demands for finite resources, including other public health interventions.

DESIGN

Using results estimated as part of a health impact assessment regarding a West Yorkshire Low Emission Zone strategy, this paper quantifies cost-saving and health-improving implications of transport policy through its impact on air quality.

DATA SOURCE

Estimates of health-related quality of life and the National Health Service (NHS)/Personal Social Services (PSS) costs for identified health events were based on data from Leeds and Bradford using peer-reviewed publications or Office for National Statistics releases.

POPULATION

Inhabitants of the area within the outer ring roads of Leeds and Bradford.

MAIN OUTCOMES MEASURES

NHS and PSS costs and quality-adjusted life years (QALYs).

RESULTS

Averting an all-cause mortality death generates 8.4 QALYs. Each coronary event avoided saves £28 000 in NHS/PSS costs and generates 1.1 QALYs. For every fewer case of childhood asthma, there will be NHS/PSS cost saving of £3000 and a health benefit of 0.9 QALYs. A single term, low birthweight birth avoided saves £2000 in NHS/PSS costs. Preventing a preterm birth saves £24 000 in NHS/PSS costs and generates 1.3 QALYs. A scenario modelled in the West Yorkshire Low Emission Zone Feasibility Study, where pre-EURO 4 buses and HGVs are upgraded to EURO 6 by 2016 generates an annual benefit of £2.08 million and a one-off benefit of £3.3 million compared with a net present value cost of implementation of £6.3 million.

CONCLUSIONS

Interventions to improve air quality and health should be evaluated and where improvement of population health is the primary objective, cost-effectiveness analysis using a NHS/PSS costs and QALYs framework is an appropriate methodology.

Unifying Research and Reimbursement Decisions: Case Studies Demonstrating the Sequence of Assessment and Judgments Required

BACKGROUND

The key principles regarding what assessments lead to different types of guidance about the use of health technologies (Only in Research, Approval with Research, Approve, or Reject) provide an explicit and transparent framework for technology appraisal.

OBJECTIVE

We aim to demonstrate how these principles and assessments can be applied in practice through the use of a seven-point checklist of assessment.

METHODS

The value of access to a technology and the value of additional evidence are explored through the application of the checklist to the case studies of enhanced external counterpulsation for chronic stable angina and clopidogrel for the management of patients with non-ST-segment elevation acute coronary syndromes.

RESULTS

The case studies demonstrate the importance of considering 1) the expected cost-effectiveness and population net health effects; 2) the need for evidence and whether the type of research required can be conducted once a technology is approved for widespread use; 3) whether there are sources of uncertainty that cannot be resolved by research but only over time; and 4) whether there are significant (opportunity) costs that once committed by approval cannot be recovered.

CONCLUSIONS

The checklist demonstrates that cost-effectiveness is a necessary but not sufficient condition for approval. Only in Research may be appropriate when a technology is expected to be cost-effective due to significant irrecoverable costs. It is only approval that can be ruled out if a technology is not expected to be cost-effective. Lack of cost-effectiveness is not a necessary or sufficient condition for rejection.

Belief elicitation to populate health economic models of medical diagnostic devices in development

BACKGROUND AND OBJECTIVE

Bayesian methods can be used to elicit experts' beliefs about the clinical value of healthcare technologies. This study investigates a belief-elicitation method for estimating diagnostic performance in an early stage of development of photoacoustic mammography (PAM) imaging versus magnetic resonance imaging (MRI) for detecting breast cancer.

RESEARCH DESIGN

Eighteen experienced radiologists ranked tumor characteristics regarding their importance to detect malignancies. With reference to MRI, radiologists estimated the true positives and negatives of PAM using the variable interval method. An overall probability density function was determined using linear opinion pooling, weighted for individual experts' experience.

RESULT

The most important tumor characteristics are mass margins and mass shape. Respondents considered MRI the better technology to visualize these characteristics. Belief elicitation confirmed this by providing an overall sensitivity of PAM ranging from 58.9 to 85.1% (mode 75.6%) and specificity ranging from 52.2 to 77.6% (mode 66.5%).

CONCLUSION

Belief elicitation allowed estimates to be obtained for the expected diagnostic performance of PAM, although radiologists expressed difficulties in doing so. Heterogeneity within and between experts reflects this uncertainty and the infancy of PAM. Further clinical trials are required to validate the extent to which this belief-elicitation method is predictive for observed test performance.

Clinical and economic evaluation of laparoscopic surgery compared with medical management for gastro-oesophageal reflux disease: 5-year follow-up of multicentre randomised trial (the REFLUX trial)

BACKGROUND

Despite promising evidence that laparoscopic fundoplication provides better short-term relief of gastro-oesophageal reflux disease (GORD) than continued medical management, uncertainty remains about whether benefits are sustained and outweigh risks.

OBJECTIVE

To evaluate the long-term clinical effectiveness, cost-effectiveness and safety of laparoscopic surgery among people with GORD requiring long-term medication and suitable for both surgical and medical management.

DESIGN

Five-year follow-up of a randomised trial (with parallel non-randomised preference groups) comparing a laparoscopic surgery-based policy with a continued medical management policy. Cost-effectiveness was assessed alongside the trial using a NHS perspective for costs and expressing health outcomes in terms of quality-adjusted life-years (QALYs).

SETTING

Follow-up was by annual postal questionnaire and selective hospital case notes review; initial recruitment in 21 UK hospitals.

PARTICIPANTS

Questionnaire responders among the 810 original participants. At entry, all had documented evidence of GORD and symptoms for > 12 months. Questionnaire response rates (years 1-5) were from 89.5% to 68.9%.

INTERVENTIONS

Three hundred and fifty-seven participants were recruited to the randomised comparison (178 randomised to surgical management and 179 randomised to continued medical management) and 453 to the preference groups (261 surgical management and 192 medical management). The surgeon chose the type of fundoplication.

MAIN OUTCOME MEASURES

Primary: disease-specific outcome measure (the REFLUX questionnaire); secondary: Short Form questionnaire-36 items (SF-36), European Quality of Life-5 Dimensions (EQ-5D), NHS resource use, reflux medication, complications.

RESULTS

The randomised groups were well balanced. By 5 years, 63% in the randomised surgical group and 13% in the randomised medical management group had received a total or partial wrap fundoplication (85% and 3% in the preference groups), with few perioperative complications and no associated deaths. At 1 year (and 5 years) after surgery, 36% (41%) in the randomised surgical group - 15% (26%) of those who had surgery - were taking proton pump inhibitor medication compared with 87% (82%) in the randomised medical group. At each year, differences in the REFLUX score significantly favoured the randomised surgical group (a third of a SD; p< 0.01 at 5 years). SF-36 and EQ-5D scores also favoured surgery, but differences attenuated over time and were generally not statistically significant at 5 years. The worse the symptoms at trial entry, the larger the benefit observed after surgery. Those randomised to medical management who subsequently had surgery had low baseline scores that markedly improved after surgery. Following fundoplication, 3% had surgical treatment for a complication and 4% had subsequent reflux-related operations - most often revision of the wrap. Dysphagia, flatulence and inability to vomit were similar in the two randomised groups. The economic analysis indicated that surgery was the more cost-effective option for this patient group. The incremental cost-effectiveness ratio for surgery in the base case was £7028 per additional QALY; these findings were robust to changes in approaches and assumptions. The probability of surgery being cost-effective at a threshold of £20,000 per additional QALY was > 0.80 for all analyses.

CONCLUSIONS

After 5 years, laparoscopic fundoplication continues to provide better relief of GORD symptoms with associated improved health-related quality of life. Complications of surgery were uncommon. Despite being initially more costly, a surgical policy is highly likely to be cost-effective.

TRIAL REGISTRATION

Current Controlled Trials ISRCTN15517081.

FUNDING

This project was funded by the NIHR Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 17, No. 22. See the HTA programme website for further project information.

Informing a decision framework for when NICE should recommend the use of health technologies only in the context of an appropriately designed programme of evidence development

BACKGROUND

The general issue of balancing the value of evidence about the performance of a technology and the value of access to a technology can be seen as central to a number of policy questions. Establishing the key principles of what assessments are needed, as well as how they should be made, will enable them to be addressed in an explicit and transparent manner.

OBJECTIVES

The aims of this research are to (1) establish the key principles of what assessments are needed to inform an 'only in research' (OIR) or 'approval with research' (AWR) recommendation, (2) evaluate previous National Institute for Health and Clinical Evidence (NICE) guidance in which OIR or AWR recommendations were made or considered and (3) evaluate a range of alternative options to establish criteria, additional information and/or analysis that could be made available to inform the assessments needed.

DATA SOURCES

All NICE draft and final guidance up to January 2010 was considered in the review of NICE technology appraisal guidance. Four case studies were used to evaluate the range of options of what information and analysis could be made available to inform the assessment required. These were based on a reanalysis of existing health technology appraisals for NICE or the Health Technology Assessment programme.

REVIEW METHODS

A critical review of policies, practice and literature was undertaken using traditional systematic searching based on initial search terms informed by key publications. An iterative approach was adopted using 'pearl growing' evaluated through capture-recapture methods. In addition, grey literature, policy documents and other sources, such as special interest groups and the expertise of the Advisory Group for the project, were used to contribute to this process.

RESULTS

A series of recommendations, or options, for NICE to consider were developed with the involvement of key stakeholders. These establish the key principles and associated criteria that might guide OIR and AWR recommendations and identify what, if any, additional information or analysis might be included in the technology appraisal process, including how such recommendations might be more likely to be implemented through publically funded and sponsored research. To meet these aims the research is broadly structured as follows. A critical review of policy, practice and literature in this area informs the development of a coherent conceptual framework to establish the key principles and the sequence of assessment and judgements required. This sequence of assessment and judgement is represented as an algorithm, which can also be summarised as a simple set of explicit criteria or a 7-point checklist of assessments. A review of previous NICE guidance in which OIR or AWR recommendations were either made or considered was undertaken to examine the extent to which the key principles are evident. The application of the checklist of assessment to a series of four case studies informs considerations of whether or not such assessments can be made based on existing information and analysis in current NICE appraisal and in what circumstances could additional information and/or analysis be useful. Finally, some of the implications that this more explicit assessment of OIR and AWR might have for policy (e.g. NICE guidance and drug pricing), the process of appraisal (e.g. greater involvement of research commissioners) and methods of appraisal (e.g. should additional information, evidence and analysis be required) are drawn together. At each stage this research has been informed by a diverse and international Advisory Group and the feedback from participants at two workshops involving a wide range of key stakeholders, which included members of NICE and its Advisory Committees (including lay members and other NICE programmes), patient advocates, manufacturers, and research and NHS commissioners, as well as relevant academics.

LIMITATIONS

Further research is required to establish how these considerations could be integrated within a practical value-based pricing scheme. In addition, irrecoverable opportunity costs are commonly associated with many health technologies that offer future benefits following treatment. The significance of these types of irrecoverable costs is not widely recognised and further research to demonstrate their potential impact more generally is needed.

CONCLUSIONS

The categories of guidance available to NICE have a wider application than is reflected in the review of previous guidance. Importantly, determining which category of guidance will be appropriate depends only partly on an assessment of expected cost-effectiveness. As well as AWR for technologies expected to be cost-effective and OIR for those not expected to be cost-effective, there are other important circumstances when OIR should be considered. In particular, for technologies expected to be cost-effective, OIR rather than approve may be appropriate when research is not possible with approval and OIR or even reject, rather than AWR or approve, may be appropriate even if research is possible with approval when there are significant irrecoverable costs.

FUNDING

The National Institute for Health Research Health Technology Assessment programme.