Power and sample assessments for tests of hypotheses on cost-effectiveness ratios

Mobile contingency management for smoking cessation among socioeconomically disadvantaged adults: Protocol for a randomized trial

BACKGROUND

Smoking rates remain high among socioeconomically disadvantaged adults. Offering small escalating financial incentives for abstinence (i.e., contingency management [CM]), alongside clinic-based treatment dramatically increases cessation rates in this vulnerable population. However, innovative approaches are needed for those who are less able to attend office visits. The current study will evaluate an automated mobile phone-based CM approach that will allow socioeconomically disadvantaged individuals to remotely earn financial incentives for smoking cessation.

METHODS

The investigators have previously combined technologies, including 1) carbon monoxide monitors that connect with mobile phones to remotely verify abstinence, 2) facial recognition software to confirm identity during breath sample submissions, and 3) automated delivery of incentives triggered by biochemical abstinence confirmation. This automated CM approach will be evaluated in a randomized controlled trial of 532 low-income adults seeking cessation treatment. Participants will be randomly assigned to telephone counseling and nicotine replacement therapy (standard care [SC]) or SC plus mobile financial incentives (CM) for abstinence.

RESULTS

Biochemically-verified 7-day point prevalence abstinence at 26 weeks post-quit is the primary outcome. The cost-effectiveness of the interventions will be evaluated. Potential treatment mechanisms, including self-efficacy, motivation, and treatment engagement, will be explored to optimize future interventions.

DISCUSSION

Automated mobile CM may offer a low-cost approach to smoking cessation that can be combined with telephone counseling and pharmacological interventions. This approach represents a critical step toward the widespread dissemination of CM treatment to real-world settings, to reduce tobacco-related disease and disparities.

Tolerance limits and tolerance intervals for ratios of normal random variables using a bootstrap calibration

This paper addresses the problem of deriving one-sided tolerance limits and two-sided tolerance intervals for a ratio of two random variables that follow a bivariate normal distribution, or a lognormal/normal distribution. The methodology that is developed uses nonparametric tolerance limits based on a parametric bootstrap sample, coupled with a bootstrap calibration in order to improve accuracy. The methodology is also adopted for computing confidence limits for the median of the ratio random variable. Numerical results are reported to demonstrate the accuracy of the proposed approach. The methodology is illustrated using examples where ratio random variables are of interest: an example on the radioactivity count in reverse transcriptase assays and an example from the area of cost-effectiveness analysis in health economics.

No economic benefit of early knee reconstruction over optional delayed reconstruction for ACL tears: registry enriched randomised controlled trial data

BACKGROUND

To analyse 5-year cost-effectiveness of early versus optional delayed acute anterior cruciate ligament (ACL) reconstruction.

METHODS

121 young, active adults with acute ACL injury to a previously uninjured knee were randomised to early ACL reconstruction (n=62, within 10 weeks of injury) or optional delayed ACL reconstruction (n=59; 30 with ACL reconstruction within 6-55 months); all patients received similar structured rehabilitation. Real life data on health care utilisation and sick leave were obtained from regional and national registers. Costs and quality-adjusted life years (QALYs) were discounted at 3%. Full-analysis set (based on study randomisation) and as-treated analysis (according to actual treatment over 5 years) principles were applied.

RESULTS

Mean cost of early ACL reconstruction was €4695 higher than optional delayed ACL reconstruction (p=0.19) and provided an additional 0.13 QALYs (p=0.11). Full-analysis set showed incremental net benefit of early versus optional delayed ACL reconstruction was not statistically significantly different from zero at any level. As-treated analysis showed that costs for rehabilitation alone were €13 650 less than early ACL reconstruction (p<0.001). Results were robust to sensitivity analyses.

CONCLUSIONS

In young active adults with acute ACL injury, a strategy of early ACL reconstruction did not provide extra economic value over a strategy of optional delayed ACL reconstruction over a 5-year period. Results from this and previous reports of the KANON-trial imply that early identification of individuals who would benefit from either early ACL reconstruction or rehabilitation alone might reduce resource consumption and decrease risk of unnecessary overtreatment.

TRIAL REGISTRATION

ISRCTN84752559.

Neuro-Bio-Behavioral Mechanisms of Placebo and Nocebo Responses: Implications for Clinical Trials and Clinical Practice

The placebo effect has often been considered a nuisance in basic and particularly clinical research. This view has gradually changed in recent years due to deeper insight into the neuro-bio-behavioral mechanisms steering both the placebo and nocebo responses, the evil twin of placebo. For the neuroscientist, placebo and nocebo responses have evolved as indispensable tools to understand brain mechanisms that link cognitive and emotional factors with symptom perception as well as peripheral physiologic systems and end organ functioning. For the clinical investigator, better understanding of the mechanisms driving placebo and nocebo responses allow the control of these responses and thereby help to more precisely define the efficacy of a specific pharmacological intervention. Finally, in the clinical context, the systematic exploitation of these mechanisms will help to maximize placebo responses and minimize nocebo responses for the patient's benefit. In this review, we summarize and critically examine the neuro-bio-behavioral mechanisms underlying placebo and nocebo responses that are currently known in terms of different diseases and physiologic systems. We subsequently elaborate on the consequences of this knowledge for pharmacological treatments of patients and the implications for pharmacological research, the training of healthcare professionals, and for the health care system and future research strategies on placebo and nocebo responses.

Economic evaluation of pediatric cancer treatment: a systematic literature review

OBJECTIVE

Although there is a growing national focus on health care cost containment and accountability in resource utilization, childhood cancer therapy costs continue to increase without proportionate survival improvements. Economic evaluations (EEs) such as cost and/or cost effectiveness analysis may identify areas to improve resource efficiency. This review aims to identify and characterize the EE studies performed in this field.

METHODS

We performed a structured literature search of the Medline, PubMed, and the National Health Service EE databases from 2000 to 2011. Concepts for the search included "cost analyses," "child," and "cancer." Studies were limited to original research, comparison of 2 or more treatments using monetary units, English language, and originating from economically developed countries. Identified studies were assessed by the Drummond checklist and characterized by the therapy studied, data sources, and research perspectives.

RESULTS

Forty studies met inclusion criteria. Eleven studied chemotherapy, surgery, or radiation. Twenty-nine studied supportive measures such as growth factor support or treatment of infection. The median Drummond score was 6 of 10 (range, 2-9). Only 15 (36%) included treatment outcomes when comparing costs. Methodological limitations were common.

CONCLUSIONS

A wide variety of topics and methodological limitations made comparisons between studies difficult. Strategies for increasing the generalizability of future EE studies are presented. Substantial opportunity exists for EE research in childhood cancer.

Cost–utility analysis conducted alongside randomized controlled trials: Are economic end points considered in sample size calculations and does it matter?

Background

Many randomized controlled trials (RCTs) collect cost-effectiveness data. Without appropriate sample size calculations, patient recruitment may cease before the cost-effectiveness of the intervention can be established or continue after the cost-effectiveness of the intervention is established beyond doubt.

Purpose

We determined the frequency with which cost-effectiveness is considered in sample size calculations and whether RCT-based economic evaluations are likely to come to inconclusive results at odds with the clinical findings.

Methods

We searched the National Health Service Economic Evaluation Database (NHS EED) to identify RCT-based cost-utility analyses. RCTs that collected individual patient data on costs and quality-adjusted life years (QALYs) were eligible. Studies using models to extrapolate the results of RCTs or with insufficient information on incremental costs and QALYs were excluded.

Results

In total, 38 trials met eligibility criteria. Only one considered cost-effectiveness in sample size calculations. RCTs were less likely to reach definitive conclusions based on the cost-effectiveness results than the primary clinical outcome (15.8% vs. 42.1%; McNemar; p = 0.01). In trials that provided sufficient data, exploratory analysis indicated that the median power to detect important differences was 29.5% for QALYs, 94.1% for costs, and 78.7% for the primary clinical outcome. In three trials (7.9%), a definitely more effective intervention was found to be expensive and probably not cost-effective.

Limitations

Our results reflect trials where authors considered within-trial estimates of cost-effectiveness to be meaningful. In focusing on one primary clinical outcome from each RCT, we have simplified the clinical effectiveness results, although the primary outcome will usually be one that policy makers use in judging the ‘success’ of the intervention.

Conclusions

Economic evaluations conducted alongside RCTs are valuable, but often present inconclusive evidence. Trial results may lead to discordant messages when the most effective intervention is probably not the most cost-effective. Despite methodological advances, trialists rarely assessed the extent to which their trial might resolve the key uncertainties about the cost-effectiveness of interventions. We recommend that grant funders should do more to encourage trialists to include economic end points in sample size calculations, particularly when the majority of costs and benefits of the intervention occur within the time frame of the trial.

Sample size determination for cost-effectiveness trials

Methods for determining sample size requirements for cost-effectiveness studies are reviewed and illustrated. Traditional methods based on tests of hypothesis and power arguments are given for the incremental cost-effectiveness ratio and incremental net benefit (INB). In addition, a full Bayesian approach using decision theory to determine optimal sample size is given for INB. The full Bayesian approach, based on the value of information, is proposed in reaction to concerns that traditional methods rely on arbitrarily chosen error probabilities and an ill-defined notion of the smallest clinically important difference. Furthermore, the results of cost-effectiveness studies are used for decision making (e.g. should a new intervention be adopted or the old one retained), and employing decision theory, which permits optimal use of current information and the optimal design of new studies, provides a more consistent approach.

Sample size and power for cost-effectiveness analysis (part 1)

Basic sample size and power formulae for cost-effectiveness analysis have been established in the literature. These formulae are reviewed and the similarities and differences between sample size and power for cost-effectiveness analysis and for the analysis of other continuous variables such as changes in blood pressure or weight are described. The types of sample size and power tables that are commonly calculated for cost-effectiveness analysis are also described and the impact of varying the assumed parameter values on the resulting sample size and power estimates is discussed. Finally, the way in which the data for these calculations may be derived are discussed.

Manipulative therapy in addition to usual medical care accelerates recovery of shoulder complaints at higher costs: economic outcomes of a randomized trial

Background

Shoulder complaints are common in primary care and have unfavourable long term prognosis. Our objective was to evaluate the clinical effectiveness of manipulative therapy of the cervicothoracic spine and the adjacent ribs in addition to usual medical care (UMC) by the general practitioner in the treatment of shoulder complaints.

Methods

This economic evaluation was conducted alongside a randomized trial in primary care. Included were 150 patients with shoulder complaints and a dysfunction of the cervicothoracic spine and adjacent ribs. Patients were treated with UMC (NSAID's, corticosteroid injection or referral to physical therapy) and were allocated at random (yes/no) to manipulative therapy (manipulation and mobilization). Patient perceived recovery, severity of main complaint, shoulder pain, disability and general health were outcome measures. Data about direct and indirect costs were collected by means of a cost diary.

Results

Manipulative therapy as add-on to UMC accelerated recovery on all outcome measures included. At 26 weeks after randomization, both groups reported similar recovery rates (41% vs. 38%), but the difference between groups in improvement of severity of the main complaint, shoulder pain and disability sustained. Compared to the UMC group the total costs were higher in the manipulative group (€1167 vs. €555). This is explained mainly by the costs of the manipulative therapy itself and the higher costs due sick leave from work. The cost effectiveness ratio showed that additional manipulative treatment is more costly but also more effective than UMC alone. The cost-effectiveness acceptability curve shows that a 50%-probability of recovery with AMT within 6 months after initiation of treatment is achieved at €2876.

Conclusion

Manipulative therapy in addition to UMC accelerates recovery and is more effective than UMC alone on the long term, but is associated with higher costs.

International Standard Randomized Controlled Trial Number Register

ISRCTN11216

A behavioral Bayes method to determine the sample size of a clinical trial considering efficacy and safety

It is necessary for the calculation of sample size to achieve the best balance between the cost of a clinical trial and the possible benefits from a new treatment. Gittins and Pezeshk developed an innovative (behavioral Bayes) approach, which assumes that the number of users is an increasing function of the difference in performance between the new treatment and the standard treatment. The better a new treatment, the more the number of patients who want to switch to it. The optimal sample size is calculated in this framework. This BeBay approach takes account of three decision-makers, a pharmaceutical company, the health authority and medical advisers. Kikuchi, Pezeshk and Gittins generalized this approach by introducing a logistic benefit function, and by extending to the more usual unpaired case, and with unknown variance. The expected net benefit in this model is based on the efficacy of the new drug but does not take account of the incidence of adverse reactions. The present paper extends the model to include the costs of treating adverse reactions and focuses on societal cost-effectiveness as the criterion for determining sample size. The main application is likely to be to phase III clinical trials, for which the primary outcome is to compare the costs and benefits of a new drug with a standard drug in relation to national health-care.

Costs of an intervention for primary care patients with medically unexplained symptoms: a randomized controlled trial

OBJECTIVE

This study sought to determine whether an intervention for patients with medically unexplained symptoms in primary care reduced total costs, components of cost, and longer-term costs and whether it led to decreased service use outside the health maintenance organization (HMO).

METHODS

A randomized controlled trial involving 206 patients with medically unexplained symptoms was conducted in a staff-model HMO. The protocol emphasized the provider-patient relationship and included cognitive-behavioral therapy and pharmacological management. Cost data for medical treatments were derived from the HMO's electronic database. Patients were interviewed about work days lost and out-of-pocket expenses for medical care outside the HMO.

RESULTS

The difference in total costs ($1,071) for the 12-month intervention was not significant. The treatment group had significantly higher costs for antidepressants than the usual-care group ($192 higher) during the intervention, and a larger proportion received antidepressants. The intervention group used less medical care outside the HMO and missed one less work day per month on average (1.23 days), indicating a slight improvement in productivity, but the difference was not significant. The between-group difference in estimated total cost was smaller in the year after the intervention (difference of $341) but were not significant.

CONCLUSIONS

The total costs for the intervention group were not significantly different, but the group had greater use of antidepressants. Coupled with findings of improved mental health outcomes for this group in a previous study, the results indicate that the intervention may be cost-effective. The longer-term impact needs to be further studied.

Estimation, power and sample size calculations for stochastic cost and effectiveness analysis

Various methods have been proposed to address uncertainty in economic evaluations of healthcare programmes. One approach suggested in the literature is to estimate separate confidence intervals for the incremental costs and effects of a new health programme in comparison with an existing programme. These intervals are then combined to generate a rectangular confidence region in the cost-effectiveness plane that implicitly defines a corresponding confidence interval for the incremental cost-effectiveness ratio (ICER). The same approach has been used to calculate sample sizes and study power. This application of the rectangle method is consistent with the adoption of ICERs and a threshold as a decision rule, this being the most commonly used approach in empirical applications of cost-effectiveness analysis, as well as the one recommended by agencies that assess medical technology around the world. In this paper, we first outline the rectangle method, and then propose a modification that recognises that separate inferences are being drawn on the cost and effectiveness domains, and that corrects for multiple statistical comparisons. The confidence rectangle is otherwise too small, the corresponding confidence interval for the ICER is too narrow and sample sizes are under-estimated. Our modification corrects these problems. A further difficulty is that the placement of the confidence rectangle around the null value is somewhat arbitrary, and does not correspond to a unique value of ICERs. As a result, different values of sample size and power for the estimation of ICERs can be obtained, depending on the null values of the cost and effectiveness. We conclude that it is important to clearly identify the analytic goal in terms of estimating differential costs, differential effects or a combination of the two using the ICER index. These ideas are illustrated using numerical examples.

Incremental net benefit in randomized clinical trials with quality-adjusted survival

Owing to induced dependent censoring, estimating mean costs and quality-adjusted survival in a cost-effectiveness comparison of two groups using standard life-table methods leads to biased results. In this paper we propose methods for estimating the difference in mean costs and the difference in mean effectiveness, together with their respective variances and covariance in the presence of dependent censoring. We consider the situation in which the measure of effectiveness is either the probability of surviving a duration of interest or mean quality-adjusted survival time over a duration of interest. The methods are illustrated in an example using an incremental net benefit analysis.

Bayesian methods for design and analysis of cost-effectiveness trials in the evaluation of health care technologies

We review the development of Bayesian statistical methods for the design and analysis of randomized controlled trials in the assessment of the cost-effectiveness of health care technologies. We place particular emphasis on the benefits of the Bayesian approach; the implications of skew cost data; the need to model the data appropriately to generate efficient and robust inferences instead of relying on distribution-free methods; the importance of making full use of quantitative and structural prior information to produce realistic inferences; and issues in the determination of sample size. Several new examples are presented to illustrate the methods. We conclude with a discussion of the key areas for future research.

Using inverse-weighting in cost-effectiveness analysis with censored data

Due to induced dependent censoring, estimating mean costs and quality-adjusted survival in a cost-effectiveness analysis using standard life-table methods leads to biased results. In this paper we propose methods for estimating the difference in mean costs and the difference in effectiveness, together with their respective variances and covariance in the presence of dependent censoring. We consider the situation in which the measure of effectiveness is either the probability of patients surviving a duration of interest, mean survival time over a duration of interest or mean quality-adjusted survival time over a duration of interest. The method of inverse-weighting is used for censored cost and quality of life data. The methods are illustrated in an example using an incremental net benefit analysis.

The practicality and validity of directly elicited and SF-36 derived health state preferences in patients with low back pain

Recent research has derived preference scores from the SF-36. We compare the practicality and construct validity of SF-36 derived preference scores with directly elicited time trade off (TTO) and visual analogue scale (VAS) scores. In this observational study, low back pain (LBP), patients were asked to complete disease specific, generic (SF-36), and health state preference (VAS and TTO) instruments. Baseline SF-36 responses were converted to preference scores using six published algorithms. Response rates for the SF-36 derived and TTO preference values were 354 of 379 (93%) and 303 of 379 (80%), respectively. Thirty patients were excluded from the TTO exercise because of difficulties comprehending the scaling task. Choice based methods (standard gamble, TTO) yielded higher and more uniform estimates of preference (0.77-0.79) than non-choice based methods (VAS) (0.42-0.70). Directly elicited TTO values were variable and had less power to distinguish among patients with differing severity of LBP. All SF-36 derived preferences exhibited a minimum threshold implying a potential floor effect for severely ill patients. SF-36 derived preferences demonstrated good practicality and construct validity in this setting, however different methods will yield disparate estimates of marginal benefit. This emphasises the need for a standardised algorithm for deriving SF-36 preference scores.

Use of randomised controlled trials for producing cost-effectiveness evidence: potential impact of design choices on sample size and study duration

BACKGROUND

A number of approaches to conducting economic evaluations could be adopted. However, some decision makers have a preference for wholly stochastic cost-effectiveness analyses, particularly if the sampled data are derived from randomised controlled trials (RCTs). Formal requirements for cost-effectiveness evidence have heightened concerns in the pharmaceutical industry that development costs and times might be increased if formal requirements increase the number, duration or costs of RCTs. Whether this proves to be the case or not will depend upon the timing, nature and extent of the cost-effectiveness evidence required.

OBJECTIVE

To illustrate how different requirements for wholly stochastic cost-effectiveness evidence could have a significant impact on two of the major determinants of new drug development costs and times, namely RCT sample size and study duration.

DESIGN

Using data collected prospectively in a clinical evaluation, sample sizes were calculated for a number of hypothetical cost-effectiveness study design scenarios. The results were compared with a baseline clinical trial design.

RESULTS

The sample sizes required for the cost-effectiveness study scenarios were mostly larger than those for the baseline clinical trial design. Circumstances can be such that a wholly stochastic cost-effectiveness analysis might not be a practical proposition even though its clinical counterpart is. In such situations, alternative research methodologies would be required. For wholly stochastic cost-effectiveness analyses, the importance of prior specification of the different components of study design is emphasised. However, it is doubtful whether all the information necessary for doing this will typically be available when product registration trials are being designed.

CONCLUSIONS

Formal requirements for wholly stochastic cost-effectiveness evidence based on the standard frequentist paradigm have the potential to increase the size, duration and number of RCTs significantly and hence the costs and timelines associated with new product development. Moreover, it is possible to envisage situations where such an approach would be impossible to adopt. Clearly, further research is required into the issue of how to appraise the economic consequences of alternative economic evaluation research strategies.

The effect of outliers on confidence interval procedures for cost-effectiveness ratios

Cost-effectiveness ratio (CER) is defined as the ratio of the difference in cost between a test and standard health care programme to the difference in benefit, respectively. Methods to obtain confidence intervals for CERs are either variants of Fieller's method (1954), or bootstrap methods. We study the effect of outliers in cost measures on the precision of confidence interval procedures for CERs. In particular the performance of the procedures under single and multiple case influential deletion diagnostics, respectively, are evaluated. Simulation studies suggest that the bias-corrected percentile bootstrap procedure gives better precision and coverage under either diagnostic.