Trying to do better than average: a commentary on 'statistical inference for cost-effectiveness ratios'

Cost-Effectiveness Analyses of Lung Cancer Screening Using Low-Dose Computed Tomography: A Systematic Review Assessing Strategy Comparison and Risk Stratification

OBJECTIVES

Our first study objective was to assess the range of lung cancer screening intervals compared within cost-effectiveness analyses (CEAs) of low-dose computed tomography (LDCT) and to examine the implications for the strategies identified as optimally cost effective; the second objective was to examine if and how risk subgroup-specific policies were considered.

METHODS

PubMed, Embase and Web of Science were searched for model-based CEAs of LDCT lung screening. The retrieved studies were assessed to examine if the analyses considered sufficient strategy variation to permit incremental estimation of cost effectiveness. Regarding risk selection, we examined if analyses considered alternative risk strata in separate analyses or as alternative risk-based eligibility criteria for screening.

RESULTS

The search identified 33 eligible CEAs, 23 of which only considered one screening frequency. Of the 10 analyses considering multiple screening intervals, only 4 included intervals longer than 2 years. Within the 10 studies considering multiple intervals, the optimal policy choice would differ in 5 if biennial intervals or longer had not been considered. Nineteen studies conducted risk subgroup analyses, 12 of which assumed that subgroup-specific policies were possible and 7 of which assumed that a common screening policy applies to all those screened.

CONCLUSIONS

The comparison of multiple strategies is recognised as good practice in CEA when seeking optimal policies. Studies that do include multiple intervals indicate that screening intervals longer than 1 year can be relevant. The omission of intervals of 2 years or longer from CEAs of LDCT screening could lead to the adoption of sub-optimal policies. There also is scope for greater consideration of risk-stratified policies which tailor screening intensity to estimated disease risk. Policy makers should take care when interpreting current evidence before implementing lung screening.

Understanding implementation costs of a pediatric weight management intervention: an economic evaluation protocol

Background

Understanding the cost and/or cost-effectiveness of implementation strategies is crucial for organizations to make informed decisions about the resources needed to implement and sustain evidence-based interventions (EBIs). This economic evaluation protocol describes the methods and processes that will be used to assess costs and cost-effectiveness across implementation strategies used to improve the reach, adoption, implementation, and organizational maintenance of an evidence-based pediatric weight management intervention- Building Health Families (BHF).

Methods

A within-trial cost and cost-effectiveness analysis (CEA) will be completed as part of a hybrid type III effectiveness-implementation trial (HEI) designed to examine the impact of an action Learning Collaborative (LC) strategy consisting of network weaving, consultee-centered training, goal-setting and feedback, and sustainability action planning to improve the adoption, implementation, organizational maintenance, and program reach of BHF in micropolitan and surrounding rural communities in the USA, over a 12-month period. We discuss key features of implementation strategy components and the associated cost collection and outcome measures and present brief examples on what will be included in the CEA for each discrete implementation strategy and how the results will be interpreted. The cost data will be collected by identifying implementation activities associated with each strategy and using a digital-based time tracking tool to capture the time associated with each activity. Costs will be assessed relative to the BHF program implementation and the multicomponent implementation strategy, included within and external to a LC designed to improve reach, effectiveness, adoption, implementation, and maintenance (RE-AIM) of BHF. The CEA results will be reported by RE-AIM outcomes, using the average cost-effectiveness ratio or incremental cost-effectiveness ratio. All the CEAs will be performed from the community perspective.

Discussion

The proposed costing approach and economic evaluation framework for dissemination and implementation strategies and EBI implementation will contribute to the evolving but still scant literature on economic evaluation of implementation and strategies used and facilitate the comparative economic analysis.

Trial registration

ClinicalTrials.gov NCT04719442. Registered on January 22, 2021.

Supplementary Information

The online version contains supplementary material available at 10.1186/s43058-022-00287-1.

Operational safety economics: Foundations, current approaches and paths for future research

Due to the COVID-19 pandemic in 2020, the trade-off between economics and epidemic prevention (safety) has become painfully clear worldwide. This situation thus highlights the significance of balancing the economy with safety and health. Safety economics, considering the interdependencies between safety and micro-economics, is ideal for supporting this kind of decision-making. Although economic approaches such as cost-benefit analysis and cost-effectiveness analysis have been used in safety management, little attention has been paid to the fundamental issues and the primary methodologies in safety economics. Therefore, this paper presents a systematic study on safety economics to analyze the foundational issues and explore the possible approaches. Firstly, safety economics is defined as a transdisciplinary and interdisciplinary field of academic research focusing on the interdependencies and coevolution of micro-economies and safety. Then we explore the role of safety economics in safety management and production investment. Furthermore, to make decisions more profitable, economic approaches are summarized and analyzed for decision-making about prevention investments and/or safety strategies. Finally, we discuss some open issues in safety economics and possible pathways to improve this research field, such as security economics, risk perception, and multi-criteria analysis.

Calculating and Interpreting ICERs and Net Benefit

For several decades, the incremental cost-effectiveness ratio has been routinely used by health technology assessment agencies around the world to summarise the results of economic evaluations of health interventions. Yet reporting and considering incremental cost-effectiveness ratios is unnecessary. Alternative summary measures exist, based on the concept of 'net benefit'. The incremental cost-effectiveness ratio and measures of net benefit share several commonalities but some important distinctions. As a result, different methods are required to calculate and interpret incremental cost-effectiveness ratios compared to measures of net benefit. The aim of this practical application is to introduce readers to these methods, using a hypothetical example to illustrate key issues. First, the methods used to calculate each measure are described. Next, for each measure, consideration is made of whether and how each measure may be interpreted to perform the following tasks, each of which may be of interest to health technology assessment agencies: (1) identifying the single most cost-effective strategy; (2) ranking strategies from 'most' to 'least' cost-effective (on an ordinal scale); (3) determining the magnitude to which a strategy is more or less cost-effective than another strategy (on a cardinal scale); and (4) determining whether a strategy is more or less cost-effective following a sensitivity or scenario analysis. This practical application also introduces a novel approach for visually interpreting measures of net benefit using the cost-effectiveness plane, which addresses a number of limitations of the conventional cost-effectiveness 'efficiency frontier'. By the end of this practical application, readers should have an understanding of how to calculate and interpret each measure, as well as the relative strengths and limitations of each.

Advantages of the net benefit regression framework for economic evaluations of interventions in the workplace: a case study of the cost-effectiveness of a collaborative mental health care program for people receiving short-term disability benefits for psychiatric disorders

OBJECTIVE

Economic evaluations commonly accompany trials of new treatments or interventions; however, regression methods and their corresponding advantages for the analysis of cost-effectiveness data are not well known.

METHODS

To illustrate regression-based economic evaluation, we present a case study investigating the cost-effectiveness of a collaborative mental health care program for people receiving short-term disability benefits for psychiatric disorders. We implement net benefit regression to illustrate its strengths and limitations.

RESULTS

Net benefit regression offers a simple option for cost-effectiveness analyses of person-level data. By placing economic evaluation in a regression framework, regression-based techniques can facilitate the analysis and provide simple solutions to commonly encountered challenges.

CONCLUSIONS

Economic evaluations of person-level data (eg, from a clinical trial) should use net benefit regression to facilitate analysis and enhance results.

Cost-effectiveness analysis: a proposal of new reporting standards in statistical analysis

Cost-effectiveness analysis (CEA) is a method for evaluating the outcomes and costs of competing strategies designed to improve health, and has been applied to a variety of different scientific fields. Yet there are inherent complexities in cost estimation and CEA from statistical perspectives (e.g., skewness, bidimensionality, and censoring). The incremental cost-effectiveness ratio that represents the additional cost per unit of outcome gained by a new strategy has served as the most widely accepted methodology in the CEA. In this article, we call for expanded perspectives and reporting standards reflecting a more comprehensive analysis that can elucidate different aspects of available data. Specifically, we propose that mean- and median-based incremental cost-effectiveness ratios and average cost-effectiveness ratios be reported together, along with relevant summary and inferential statistics, as complementary measures for informed decision making.

Average cost-effectiveness ratio with censored data

In cost-effectiveness analysis, interest could lie foremost in the incremental cost-effectiveness ratio (ICER), which is the ratio of the incremental cost to the incremental benefit of two competing interventions. The average cost-effectiveness ratio (ACER) is the ratio of the cost to benefit of an intervention without reference to a comparator. A vast literature is available for statistical inference of the ICERs, but limited methods have been developed for the ACERs, particularly in the presence of censoring. Censoring is a common feature in prospective studies, and valid analyses should properly adjust for censoring in cost as well as in effectiveness. In this article, we propose statistical methods for constructing a confidence interval for the ACER from censored data. Different methods-Fieller, Taylor, bootstrap-are proposed, and through simulation studies and data analysis, we address the performance characteristics of these methods.

Individualized cost-effectiveness analysis

John Ioannidis and Alan Garber discuss how to use incremental cost-effectiveness ratios (ICER) and related metrics so they can be useful for decision-making at the individual level, whether used by clinicians or individual patients.

Regression models for analyzing costs and their determinants in health care: an introductory review

OBJECTIVE

This article aims to describe the various approaches in multivariable modelling of healthcare costs data and to synthesize the respective criticisms as proposed in the literature.

METHODS

We present regression methods suitable for the analysis of healthcare costs and then apply them to an experimental setting in cardiovascular treatment (COSTAMI study) and an observational setting in diabetes hospital care.

RESULTS

We show how methods can produce different results depending on the degree of matching between the underlying assumptions of each method and the specific characteristics of the healthcare problem.

CONCLUSIONS

The matching of healthcare cost models to the analytic objectives and characteristics of the data available to a study requires caution. The study results and interpretation can be heavily dependent on the choice of model with a real risk of spurious results and conclusions.

A clinician's guide to correct cost-effectiveness analysis: think incremental not average

OBJECTIVE

To explain how to correctly report the results from a cost-effectiveness analysis (CEA).

METHODS

Results were used from a hypothetical clinical trial to illustrate how different ways of reporting economic results affect both presentation of findings and formulation of conclusions. To provide context, we reviewed some high-profile exchanges in the scientific literature.

RESULTS

The critical issue with which decision makers must grapple involves the trade-offs introduced by a new treatment or intervention. Specifically, are decision makers willing to pay the additional cost for the additional outcomes? This question cannot be considered without estimates of the additional cost and additional outcomes. Correct cost-effectiveness measures, such as the incremental cost-effectiveness ratio or the incremental net benefit, address this issue.

CONCLUSIONS

As decision makers face the challenge of balancing increasing health care demand with cost containment, it will be crucial to identify cost-effective ways of providing care. Health care providers and other decision makers should not be misled by the results of improperly reported CEAs. Decisions around adoption of pharmaceuticals or implementation of new programs or interventions may be affected by which cost-effectiveness summary measure is reported. Thus consumers of CEA must have a basic understanding of why different methods give different results, and how the results should be interpreted.

The Application of Economic Evaluation Methods in the Chronic Pain Medicine Literature

BACKGROUND

The relative efficiency of a health care intervention or health status improvement realized for a given amount of resources expended can be determined using cost-effectiveness analysis or cost-utility analysis.

METHODS

An extensive chronic pain-focused search was undertaken of the MEDLINE, EMBASE, and SCI-EXPANDED databases. A total of 1822 unique citations were generated, with 142 studies subsequently categorized as incorporating one of seven recognized types of health care economic evaluation.

RESULTS

Of the 142 identified chronic pain-related economic evaluations published between 1988 and 2006, 30 incorporated a cost-effectiveness analysis and 29 incorporated a cost-utility analysis. The data are consistent with the previously reported chronological pattern of an increased overall diffusion of cost-utility analysis studies from the general medical and health services research literature into the medical subspecialty journals. However, only a few studies combined the economic analysis alongside a randomized controlled trial, the economic end-points in the trials had limited time horizons, and there was failure to address the protracted costs versus benefits of treating long-term and often recurrent chronic pain conditions.

CONCLUSIONS

Although it would appear worthwhile for researchers and clinicians to consider cost-effectiveness analysis and cost-utility analysis in their trial designs and treatment algorithms for chronic pain conditions, methodological improvements can be made in trial designs.

Modeling the 5-year cost effectiveness of treatment strategies including tumor necrosis factor-blocking agents and leflunomide for treating rheumatoid arthritis in the Netherlands

OBJECTIVE

To determine the cost effectiveness of treatment strategies for rheumatoid arthritis patients satisfying the indication for tumor necrosis factor (TNF)-blocking treatment.

METHODS

A Markov model study was performed. The following treatment strategies were considered: 1) usual treatment; 2) treatment with leflunomide, in the case of nonresponse after 3 months, switch to usual treatment; 3) TNF-blocking treatment, in the case of nonresponse after 3 months, switch to usual treatment; 4) treatment with leflunomide, in the case of nonresponse, switch to TNF-blocking treatment, in the case of nonresponse to TNF-blocking treatment, switch to usual treatment; 5) TNF-blocking treatment, in the case of nonresponse, switch to leflunomide treatment, in the case of nonresponse to leflunomide, switch to usual treatment. Expected patient-years in the different Markov states, costs, and quality-adjusted life years (QALYs) were compared between the treatment strategies; incremental cost-effectiveness ratios (ICERs) were calculated.

RESULTS

Over the 5-year period, the expected effect on disease activity and QALYs was better for treatment strategies that included TNF-blocking treatment than for the other treatment strategies. The greater effectiveness of these treatment strategies reduced medical and nonmedical costs compared with usual treatment by about 16% and 33%, respectively, omitting the costs of medication. When the costs of medication were included, the costs of strategies that started with TNF-blocking treatment were higher than those of the other treatment strategies. Treatment strategy 4 had the most favorable ICER of the treatment strategies that included TNF-blocking treatment: 163,556/QALY compared with usual treatment.

CONCLUSION

Among strategies that include TNF-blocking agents, one starting with leflunomide and, in the case of nonresponse, switching to TNF-blocking treatment probably results in the most favorable ratio between incremental costs and effects.

Optimisation versus certainty: understanding the issue of heterogeneity in economic evaluation

This paper considers how the information provided by economic evaluation for decision-makers can fail to optimise use of health resources and how assessment of the relative cost-effectiveness of health care interventions can be misleading unless heterogeneity within populations is taken into account. The cost-effectiveness of an intervention is not a point estimate but an average chosen from within a distribution of different results. The normal interpretation of the distribution around that point is often mistakenly assumed to be the 'white noise' of measurement error. In reality this variance is a combination of measurement error and true heterogeneity of results. There remains an overemphasis on pursuing certainty which stems from the fact that the methods involved were originally devised to measure dichotomous outcomes not continuous ones such as cost-effectiveness ratios. It is argued in this paper that more consideration be given to the heterogeneous nature of costs and effects across populations in analysis and policy making.

Cost effectiveness of initial endoscopy for dyspepsia in patients over age 50 years: a randomised controlled trial in primary care

BACKGROUND

Dyspepsia can be managed by initial endoscopy and treatment based on endoscopic findings, or by empirical prescribing. We aimed to determine the cost effectiveness of initial endoscopy compared with usual management in patients with dyspepsia over age 50 years presenting to their primary care physician.

METHODS

422 patients were recruited and randomly assigned to initial endoscopy or usual management. Primary outcomes were effect of treatment on dyspepsia symptoms and cost effectiveness. Secondary outcomes were quality of life and patient satisfaction. Total costs were calculated from individual patient's use of resources with unit costs applied from national data. Statistical analysis of uncertainty on incremental cost-effectiveness ratio (ICER) was done along with a sensitivity analysis on unit costs with cost-effectiveness acceptability curves.

FINDINGS

In the 12 months following recruitment, 213 (84%) patients had an endoscopy compared with 75 (41%) controls. Initial endoscopy resulted in a significant improvement in symptom score (p=0.03), and quality of life pain dimension (p=0.03), and a 48% reduction in the use of proton pump inhibitors (p=0.005). The ICER was Pound Sterling1728 (UK Pound Sterling) per patient symptom-free at 12 months. The ICER was very sensitive to the cost of endoscopy, and could be reduced to Pound Sterling165 if the unit cost of this procedure fell from Pound Sterling246 to Pound Sterling100.

INTERPRETATION

Initial endoscopy in dyspeptic patients over age 50 might be a cost-effective intervention.

Cost-effectiveness analysis of oral N-acetylcysteine as a preventive treatment in chronic bronchitis

Chronic bronchitis has a prevalence of approximately 11% in the population aged over 35 years and its frequent acute exacerbations (AECBs) are an important cause of morbidity and costs in health-care resources. Oral N -acetylcysteine (NAC) is administered during the winter months as a way of reducing AECBs. This cost-effectiveness analysis was done from the payers' point of view in the Swiss health-care system, based on a retrospective analysis of published placebo-controlled studies. The pooled data show that continuous administration of 400 mg day(-1)per os of NAC leads to a significant reduction in the number of AECBs (NAC: 16.2 vs 25.2% AECBs per month); a significantly smaller percentage of days of sick leave (NAC: 3.6 vs 5.3%) and a lower rate of hospitalizations (NAC: 1.5 vs 3.5% over a period of 6 months). Taking into account the poor compliance of these patients, calculations assumed a compliance of 80%. Direct costs were those of an NAC treatment, the management of an AECB (biological tests in 59%, X-rays in 65% and pulmonary function tests in 45%; antibiotics 70%, bronchodilators in 89%, corticosteroids in 24% and 'others' in 25% of the patients), and of hospitalizations (estimated at 10 days per case). Based on these figures, the mean direct costs of an untreated patient were CHF 869 vs CHF 700 in the NAC-treated patient. Univariate sensitivity analysis indicated that cost neutrality is reached with 0.6 (<0.25-1. 94, 95% CI) AECBs per 6 months. Indirect costs (based on sick leave) were also significantly different; the mean in untreated patients was CHF 1324 vs CHF 779 in the NAC-treated patients.

CONCLUSION

Treating chronic bronchitis patients with NAC during the winter months is cost-effective both from the payer's and a social point of view.

Stochastic cost-effectiveness analysis: a simultaneous marginal-effect approach

OBJECTIVE

The purpose of this study is to develop a cost-effectiveness methodology in the context of a simultaneous modeling framework that provides consistent point and interval estimates.

METHODS

A simultaneous model of cost and effectiveness functions was developed to measure the incremental cost-effectiveness ratio for competing medical interventions. A feasible nonlinear least-squares method was suggested to estimate the simultaneous model. Using a series of hypothetical data, a simulation analysis was performed to show the superior performance of the proposed model, relative to the average-effect model, a widely used approach to cost-effectiveness estimation.

RESULTS

The traditional average-effect approach has two shortcomings. First, it assumes two strong conditions: truly random distributions of all the significant nontreatment variables (both observed and unobserved) across study groups, and the independence of cost and effectiveness variables. Second, it does not give the confidence interval, an important measure to assess the stochastic nature and robustness of point estimates. In contrast, the simultaneous modeling approach provides marginal-effect estimates, imposing no restrictions on the random distributions of the individual characteristics across study groups. Furthermore, it takes into account the simultaneity of cost and effectiveness functions being estimated. The simulation analysis showed that the simultaneous modeling approach is significantly more unbiased and efficient in predicting the true cost-effectiveness ratio.

CONCLUSION

The simultaneous modeling approach is superior to the average-effect approach in the estimation of incremental cost-effectiveness ratios using data with significant nontreatment confounding factors. The advantages of the simultaneous modeling approach are particularly appealing for evaluative studies dealing with large-scale retrospective data at the patient level.