Economic evaluation of the NET intervention versus guideline dissemination for management of mild head injury in hospital emergency departments

BACKGROUND

Evidence-based guidelines for the management of mild traumatic brain injury (mTBI) in the emergency department (ED) are now widely available, and yet, clinical practice remains inconsistent with the guidelines. The Neurotrauma Evidence Translation (NET) intervention was developed to increase the uptake of guideline recommendations and improve the management of minor head injury in Australian emergency departments (EDs). However, the adoption of this type of intervention typically entails an upfront investment that may or may not be fully offset by improvements in clinical practice, health outcomes and/or reductions in health service utilisation. The present study estimates the cost and cost-effectiveness of the NET intervention, as compared to the passive dissemination of the guideline, to evaluate whether any improvements in clinical practice or health outcomes due to the NET intervention can be obtained at an acceptable cost.

METHODS AND FINDINGS

Study setting: The NET cluster randomised controlled trial [ACTRN12612001286831].

STUDY SAMPLE

Seventeen EDs were randomised to the control condition and 14 to the intervention. One thousand nine hundred forty-three patients were included in the analysis of clinical practice outcomes (NET sample). A total of 343 patients from 14 control and 10 intervention EDs participated in follow-up interviews and were included in the analysis of patient-reported health outcomes (NET-Plus sample).

OUTCOME MEASURES

Appropriate post-traumatic amnesia (PTA) screening in the ED (primary outcome). Secondary clinical practice outcomes: provision of written information on discharge (INFO) and safe discharge (defined as CT scan appropriately provided plus PTA plus INFO). Secondary patient-reported, post-discharge health outcomes: anxiety (Hospital Anxiety and Depression Scale), post-concussive symptoms (Rivermead), and preference-based health-related quality of life (SF6D).

METHODS

Trial-based economic evaluations from a health sector perspective, with time horizons set to coincide with the final follow-up for the NET sample (2 months post-intervention) and to 1-month post-discharge for the NET-Plus sample.

RESULTS

Intervention and control groups were not significantly different in health service utilisation received in the ED/inpatient ward following the initial mTBI presentation (adjusted mean difference $23.86 per patient; 95%CI - $106, $153; p = 0.719) or over the longer follow-up in the NET-plus sample (adjusted mean difference $341.78 per patient; 95%CI - $58, $742; p = 0.094). Savings from lower health service utilisation are therefore unlikely to offset the significantly higher upfront cost of the intervention (mean difference $138.20 per patient; 95%CI $135, $141; p < 0.000). Estimates of the net effect of the intervention on total cost (intervention cost net of health service utilisation) suggest that the intervention entails significantly higher costs than the control condition (adjusted mean difference $169.89 per patient; 95%CI $43, $297, p = 0.009). This effect is larger in absolute magnitude over the longer follow-up in the NET-plus sample (adjusted mean difference $505.06; 95%CI $96, $915; p = 0.016), mostly due to additional health service utilisation. For the primary outcome, the NET intervention is more costly and more effective than passive dissemination; entailing an additional cost of $1246 per additional patient appropriately screened for PTA ($169.89/0.1363; Fieller's 95%CI $525, $2055). For NET to be considered cost-effective with 95% confidence, decision-makers would need to be willing to trade one quality-adjusted life year (QALY) for 25 additional patients appropriately screened for PTA. While these results reflect our best estimate of cost-effectiveness given the data, it is possible that a NET intervention that has been scaled and streamlined ready for wider roll-out may be more or less cost-effective than the NET intervention as delivered in the trial.

CONCLUSIONS

While the NET intervention does improve the management of mTBI in the ED, it also entails a significant increase in cost and-as delivered in the trial-is unlikely to be cost-effective at currently accepted funding thresholds. There may be a scope for a scaled-up and streamlined NET intervention to achieve a better balance between costs and outcomes.

TRIAL REGISTRATION

Australian New Zealand Clinical Trials Registry ACTRN12612001286831 , date registered 12 December 2012.

An investigation of the impact of using different methods for network meta-analysis: a protocol for an empirical evaluation

BACKGROUND

Network meta-analysis, a method to synthesise evidence from multiple treatments, has increased in popularity in the past decade. Two broad approaches are available to synthesise data across networks, namely, arm- and contrast-synthesis models, with a range of models that can be fitted within each. There has been recent debate about the validity of the arm-synthesis models, but to date, there has been limited empirical evaluation comparing results using the methods applied to a large number of networks. We aim to address this gap through the re-analysis of a large cohort of published networks of interventions using a range of network meta-analysis methods.

METHODS

We will include a subset of networks from a database of network meta-analyses of randomised trials that have been identified and curated from the published literature. The subset of networks will include those where the primary outcome is binary, the number of events and participants are reported for each direct comparison, and there is no evidence of inconsistency in the network. We will re-analyse the networks using three contrast-synthesis methods and two arm-synthesis methods. We will compare the estimated treatment effects, their standard errors, treatment hierarchy based on the surface under the cumulative ranking (SUCRA) curve, the SUCRA value, and the between-trial heterogeneity variance across the network meta-analysis methods. We will investigate whether differences in the results are affected by network characteristics and baseline risk.

DISCUSSION

The results of this study will inform whether, in practice, the choice of network meta-analysis method matters, and if it does, in what situations differences in the results between methods might arise. The results from this research might also inform future simulation studies.

Assessing the complexity of interventions within systematic reviews: development, content and use of a new tool (iCAT_SR)

BACKGROUND

Health interventions fall along a spectrum from simple to more complex. There is wide interest in methods for reviewing 'complex interventions', but few transparent approaches for assessing intervention complexity in systematic reviews. Such assessments may assist review authors in, for example, systematically describing interventions and developing logic models. This paper describes the development and application of the intervention Complexity Assessment Tool for Systematic Reviews (iCAT_SR), a new tool to assess and categorise levels of intervention complexity in systematic reviews.

METHODS

We developed the iCAT_SR by adapting and extending an existing complexity assessment tool for randomized trials. We undertook this adaptation using a consensus approach in which possible complexity dimensions were circulated for feedback to a panel of methodologists with expertise in complex interventions and systematic reviews. Based on these inputs, we developed a draft version of the tool. We then invited a second round of feedback from the panel and a wider group of systematic reviewers. This informed further refinement of the tool.

RESULTS

The tool comprises ten dimensions: (1) the number of active components in the intervention; (2) the number of behaviours of recipients to which the intervention is directed; (3) the range and number of organizational levels targeted by the intervention; (4) the degree of tailoring intended or flexibility permitted across sites or individuals in applying or implementing the intervention; (5) the level of skill required by those delivering the intervention; (6) the level of skill required by those receiving the intervention; (7) the degree of interaction between intervention components; (8) the degree to which the effects of the intervention are context dependent; (9) the degree to which the effects of the interventions are changed by recipient or provider factors; (10) and the nature of the causal pathway between intervention and outcome. Dimensions 1-6 are considered 'core' dimensions. Dimensions 7-10 are optional and may not be useful for all interventions.

CONCLUSIONS

The iCAT_SR tool facilitates more in-depth, systematic assessment of the complexity of interventions in systematic reviews and can assist in undertaking reviews and interpreting review findings. Further testing of the tool is now needed.