A probabilistic bias analysis on the magnitude of unmeasured confounding: The impact of driving mileage on road traffic crashes

INTRODUCTION

Unmeasured confounding can lead to biased interpretations of empirical findings. This paper aimed to assess the magnitude of suspected unmeasured confounding due to driving mileage and simulate the statistical power required to detect a discrepancy in the effect of polypharmacy on road traffic crashes (RTCs) among older adults.

METHODS

Based on Monte Carlo Simulation (MCS) approach, we estimated 1) the magnitude of confounding of driving mileage on the association of polypharmacy and RTCs and 2) the statistical power of to detect a discrepancy from no adjusted effect. A total of 1000 studies, each of 500000 observations, were simulated.

RESULTS

Under the assumption of a modest adjusted exposure-outcome odds ratio of 1.35, the magnitude of confounding bias by driving mileage was estimated to be 16% higher with a statistical power of 50%. Only an adjusted odds ratio of at least 1.60 would be associated with a statistical power of about 80% CONCLUSION: This applied probabilistic bias analysis showed that not adjusting for driving mileage as a confounder can lead to an overestimation of the effect of polypharmacy on RTCs in older adults. Even considering a large sample, small to moderate adjusted exposure effects were difficult to be detected.

Cost-effectiveness of adding Sativex® spray to spasticity care in Belgium: using bootstrapping instead of Monte Carlo simulation for probabilistic sensitivity analyses

BACKGROUND

Uncertainty in model-based cost-utility analyses is commonly assessed in a probabilistic sensitivity analysis. Model parameters are implemented as distributions and values are sampled from these distributions in a Monte Carlo simulation. Bootstrapping is an alternative method that requires fewer assumptions and incorporates correlations between model parameters.

METHODS

A Markov model-based cost-utility analysis comparing oromucosal spray containing delta-9-tetrahidrocannabinol + cannabidiol (Sativex®, nabiximols) plus standard care versus standard spasticity care alone in the management of multiple sclerosis spasticity was performed over a 5-year time horizon from the Belgian healthcare payer perspective. The probabilistic sensitivity analysis was implemented using a bootstrap approach to ensure that the correlations present in the source clinical trial data were incorporated in the uncertainty estimates.

RESULTS

Adding Sativex® spray to standard care was found to dominate standard spasticity care alone, with cost savings of €6,068 and a quality-adjusted life year gain of 0.145 per patient over the 5-year analysis. The probability of dominance increased from 29% in the first year to 94% in the fifth year, with the probability of QALY gains in excess of 99% for all years considered.

CONCLUSIONS

Adding Sativex® spray to spasticity care was found to dominate standard spasticity care alone in the Belgian healthcare setting. This study showed the use of bootstrapping techniques in a Markov model probabilistic sensitivity analysis instead of Monte Carlo simulations. Bootstrapping avoided the need to make distributional assumptions and allowed the incorporation of correlating structures present in the original clinical trial data in the uncertainty assessment.

An algorithm to generate correlated input-parameters to be used in probabilistic sensitivity analyses

Background: Assessment of uncertainty in cost-effectiveness analyses (CEAs) is paramount for decision-making. Probabilistic sensitivity analysis (PSA) estimates uncertainty by varying all input parameters simultaneously within predefined ranges; however, PSA often ignores correlations between parameters. Objective: To implement an efficient algorithm that integrates parameter correlation in PSA. Study design: An algorithm based on Cholesky decomposition was developed to generate multivariate non-normal parameter distributions for the age-dependent incidence of herpes zoster (HZ). The algorithm was implemented in an HZ CEA model and evaluated for gamma and beta distributions. The incremental cost-effectiveness ratio (ICER) and the probability of being cost-effective at a given ICER threshold were calculated for different levels of correlation. Five thousand Monte Carlo simulations were carried out. Results: Correlation coefficients between parameters sampled from the distribution generated by the algorithm matched the desired correlations for both distribution functions. With correlations set to 0.0, 0.5, and 0.9, 90% of the simulations showed ICERs below $25,000, $33,000, and $38,000 per quality-adjusted life-year (QALY), respectively, varying incidence only; and below $38,000, $48,000, and $58,000 per QALY, respectively, varying most parameters. Conclusion: Parameter correlation may impact the uncertainty of CEA results. We implemented an efficient method for generating correlated non-normal distributions for use in PSA.

Increasing the information provided by probabilistic sensitivity analysis: The relative density plot

BACKGROUND

Results of probabilistic sensitivity analyses (PSA) are frequently visualized as a scatterplot, which is limited through overdrawing and a lack of insight in relative density. To overcome these limitations, we have developed the Relative Density plot (PSA-ReD).

METHODS

The PSA-ReD combines a density plot and a contour plot to visualize and quantify PSA results. Relative density, depicted using a color gradient, is transformed to a cumulative probability. Contours are then plotted over regions with a specific cumulative probability. We use two real-world case studies to demonstrate the value of the PSA-ReD plot.

RESULTS

The PSA-ReD method demonstrates proof-of-concept and feasibility. In the real-world case-studies, PSA-ReD provided additional visual information that could not be understood from the traditional scatterplot. High density areas were identified by color-coding and the contour plot allowed for quantification of PSA iterations within areas of the cost-effectiveness plane, diminishing overdrawing and putting infrequent iterations in perspective. Critically, the PSA-ReD plot informs modellers about non-linearities within their model.

CONCLUSIONS

The PSA-ReD plot is easy to implement, presents more of the information enclosed in PSA data, and prevents inappropriate interpretation of PSA results. It gives modelers additional insight in model functioning and the distribution of uncertainty around the cost-effectiveness estimate.

Misleading meta-analyses of observational studies may generate unjustified alarms: The case of medications for nausea and vomiting in pregnancy

OBJECTIVES

Because observational studies often use imperfect measurements, results are prone to misclassification errors. We used as a motivating example the possible teratogenic risks of antiemetic agents in pregnancy since a large observational study recently showed that first-trimester exposure to doxylamine-pyridoxine was associated with significantly increased risk of congenital malformations as a whole, as well as central nervous system defects, and previous observational studies did not show such associations. A meta-analysis on this issue was carried out with the aim to illustrate how differential exposure and outcome misclassifications may lead to uncertain conclusions.

METHODS

Medline, searched to October 2019 for full text papers in English. Summary Odds Ratios (ORs) with confidence intervals (CIs) were calculated using random-effect models. Probabilistic sensitivity analyses were performed for evaluating the extension of differential misclassification required to account for the exposure-outcome association.

RESULTS

Summary ORs were 1.02 (95 % CI, 0.92-1.15), 0.99 (0.82-1.19) and 1.25 (1.08-1.44) for overall congenital, cardiocirculatory, and central nervous system malformations respectively. By assuming exposure and outcome bias factor respectively of 0.95 (i.e., newborns with congenital defects had exposure specificity 5% lower than healthy newborns) and 1.12 (i.e., exposed newborns had outcome sensitivity 12 % higher than unexposed newborns), summary OR of central nervous system defects became 1.13 (95 % CI, 0.99-1.29) and 1.17 (95 % CI, 0.99-1.38).

CONCLUSION

Observational investigations and meta-analyses of observational studies need cautious interpretations. Their susceptibility to several, often sneaky, sources of bias should be carefully evaluated.

Health and economic burden of respiratory syncytial virus (RSV) disease and the cost-effectiveness of potential interventions against RSV among children under 5 years in 72 Gavi-eligible countries

BACKGROUND

Respiratory syncytial virus (RSV) frequently causes acute lower respiratory infection in children under 5, representing a high burden in Gavi-eligible countries (mostly low-income and lower-middle-income). Since multiple RSV interventions, including vaccines and monoclonal antibody (mAb) candidates, are under development, we aim to evaluate the key drivers of the cost-effectiveness of maternal vaccination and infant mAb for 72 Gavi countries.

METHODS

A static Multi-Country Model Application for RSV Cost-Effectiveness poLicy (MCMARCEL) was developed to follow RSV-related events monthly from birth until 5 years of age. MCMARCEL was parameterised using country- and age-specific demographic, epidemiological, and cost data. The interventions' level and duration of effectiveness were guided by the World Health Organization's preferred product characteristics and other literature. Maternal vaccination and mAb were assumed to require single-dose administration at prices assumed to align with other Gavi-subsidised technologies. The effectiveness and the prices of the interventions were simultaneously varied in extensive scenario analyses. Disability-adjusted life years (DALYs) were the primary health outcomes for cost-effectiveness, integrated with probabilistic sensitivity analyses and Expected Value of Partially Perfect Information analysis.

RESULTS

The RSV-associated disease burden among children in these 72 countries is estimated at an average of 20.8 million cases, 1.8 million hospital admissions, 40 thousand deaths, 1.2 million discounted DALYs, and US$611 million discounted direct costs. Strategy 'mAb' is more effective due to its assumed longer duration of protection versus maternal vaccination, but it was also assumed to be more expensive. Given all parameterised uncertainty, the optimal strategy of choice tends to change for increasing willingness to pay (WTP) values per DALY averted from the current situation to maternal vaccination (at WTP > US$1000) to mAB (at WTP > US$3500). The age-specific proportions of cases that are hospitalised and/or die cause most of the uncertainty in the choice of optimal strategy. Results are broadly similar across countries.

CONCLUSIONS

Both the maternal and mAb strategies need to be competitively priced to be judged as relatively cost-effective. Information on the level and duration of protection is crucial, but also more and better disease burden evidence-especially on RSV-attributable hospitalisation and death rates-is needed to support policy choices when novel RSV products become available.

A Non-inferiority Framework for Cost-Effectiveness Analysis

BACKGROUND

Traditional decision rules have limitations when a new technology is less effective and less costly than a comparator. We propose a new probabilistic decision framework to examine non-inferiority in effectiveness and net monetary benefit (NMB) simultaneously. We illustrate this framework using the example of repetitive transcranial magnetic stimulation (rTMS) and electroconvulsive therapy (ECT) for treatment-resistant depression.

METHODS

We modeled the quality-adjusted life-years (QALYs) associated with the new intervention (rTMS), an active control (ECT), and a placebo control, and we estimated the fraction of effectiveness preserved by the new intervention through probabilistic sensitivity analysis (PSA). We then assessed the probability of cost-effectiveness using a traditional cost-effectiveness acceptability curve (CEAC) and our new decision-making framework. In our new framework, we considered the new intervention cost-effective in each simulation of the PSA if it preserved at least 75 percent of the effectiveness of the active control (thus demonstrating non-inferiority) and had a positive NMB at a given willingness-to-pay threshold (WTP).

RESULTS

rTMS was less effective (i.e., associated with fewer QALYs) and less costly than ECT. The traditional CEAC approach showed that the probabilities of rTMS being cost-effective were 100 percent, 39 percent, and 14 percent at WTPs of $0, $50,000, and $100,000 per QALY gained, respectively. In the new decision framework, the probabilities of rTMS being cost-effective were reduced to 23 percent, 21 percent, and 13 percent at WTPs of $0, $50,000, and $100,000 per QALY, respectively.

CONCLUSIONS

This new framework provides a different perspective for decision making with considerations of both non-inferiority and WTP thresholds.

Cost-effectiveness of different treat-to-target strategies in rheumatoid arthritis: results from the DREAM registry

Background

Adjusting medication of patients with rheumatoid arthritis (RA) until predefined disease activity targets are met, i.e. Treat-to-Target (T2T), is the currently recommended treatment approach. However, not much is known about long-term cost-effectiveness of different T2T strategies.

We model the 5-year costs and effects of a step-up approach (MTX mono - > MTX + csDMARD combination - > Adalimumab - > second anti-TNF) and an initial combination therapy approach (MTX + csDMARD - > MTX + csDMARD higher dose - > anti-TNFs) from the healthcare and societal perspectives, by adapting a previously validated Markov model.

Methods

We constructed a Markov model in which 3-monthly transitions between DAS28-defined health states of remission (≤2.6), low (2.6 < DAS28 ≤ 3.2), moderate (3.2 < DAS28 ≤ 5.1), and high disease activity (DAS28 > 5.1) were simulated. Modelled patients proceeded to subsequent treatments in case of non-remission at each (3-month) cycle start. In case of remission for two consecutive cycles medication was tapered, until medication-free remission was achieved. Transition probabilities for individual treatment steps were estimated using data of Dutch Rheumatology Monitoring registry Remission Induction Cohort I (step-up) and II (initial combination). Expected costs, utility, and ICER after 5 years were compared between the two strategies. To account for parameter uncertainty, probabilistic sensitivity analysis was employed through Gamma, Normal, and Dirichlet distributions. All utilities, costs, and transition probabilities were replaced by fitted distributions.

Results

Over a 5-year timespan, initial combination therapy was less costly and more effective than step-up therapy. Initial combination therapy accrued €16,226.3 and 3.552 QALY vs €20,183.3 and 3.517 QALYs for step-up therapy. This resulted in a negative ICER, indicating that initial combination therapy was both less costly and more effective in terms of utility gained. This can be explained by higher (±5%) remission percentages in initial combination strategy at all time points. More patients in remission generates less healthcare and productivity loss costs and higher utility. Additionally, higher remission percentages caused less bDMARD use in the initial combination strategy, lowering overall costs.

Conclusion

Initial combination therapy was found favourable over step-up therapy in the treatment of Rheumatoid Arthritis, when considering cost-effectiveness. Initial combination therapy resulted in more utility at a lower cost over 5 years.

Electronic supplementary material

The online version of this article (10.1186/s41927-019-0064-9) contains supplementary material, which is available to authorized users.

A dynamic Bayesian Markov model for health economic evaluations of interventions in infectious disease

BACKGROUND

Health economic evaluations of interventions in infectious disease are commonly based on the predictions of ordinary differential equation (ODE) systems or Markov models (MMs). Standard MMs are static, whereas ODE systems are usually dynamic and account for herd immunity which is crucial to prevent overestimation of infection prevalence. Complex ODE systems including distributions on model parameters are computationally intensive. Thus, mainly ODE-based models including fixed parameter values are presented in the literature. These do not account for parameter uncertainty. As a consequence, probabilistic sensitivity analysis (PSA), a crucial component of health economic evaluations, cannot be conducted straightforwardly.

METHODS

We present a dynamic MM under a Bayesian framework. We extend a static MM by incorporating the force of infection into the state allocation algorithm. The corresponding output is based on dynamic changes in prevalence and thus accounts for herd immunity. In contrast to deterministic ODE-based models, PSA can be conducted straightforwardly. We introduce a case study of a fictional sexually transmitted infection and compare our dynamic Bayesian MM to a deterministic and a Bayesian ODE system. The models are calibrated to simulated time series data.

RESULTS

By means of the case study, we show that our methodology produces outcome which is comparable to the "gold standard" of the Bayesian ODE system.

CONCLUSIONS

In contrast to ODE systems in the literature, the dynamic MM includes distributions on all model parameters at manageable computational effort (including calibration). The run time of the Bayesian ODE system is 15 times longer.

Factors affecting the cost-effectiveness of on-farm culture prior to the treatment of clinical mastitis in dairy cows

The objective of this study was to use probabilistic sensitivity analysis to evaluate the cost-effectiveness of using an on-farm culture (OFC) approach to the treatment of clinical mastitis in dairy cows and compare this to a ‘standard’ treatment approach. A specific aim was to identify the herd circumstances under which an OFC approach would be most likely to be cost-effective. A stochastic Monte Carlo model was developed to simulate 5000 cases of clinical mastitis at the cow level and to calculate the associated costs simultaneously when treated according to 2 different treatment protocols; i) a 'conventional' approach (3 tubes of intramammary antibiotic) and ii) an OFC programme, whereby cows are treated according to the results of OFC. Model parameters were taken from recent peer reviewed literature on the use of OFC prior to treatment of clinical mastitis. Spearman rank correlation coefficients were used to evaluate the relationships between model input values and the estimated difference in cost between the standard and OFC treatment protocols. The simulation analyses revealed that both the difference in the bacteriological cure rate due to a delay in treatment when using OFC and the proportion of Gram-positive cases that occur on a dairy unit would have a fundamental impact on whether OFC would be cost-effective. The results of this study illustrated that an OFC approach for the treatment of clinical mastitis would probably not be cost-effective in many circumstances, in particular, not those in which Gram-positive pathogens were responsible for more than 20% of all clinical cases. The results highlight an ethical dilemma surrounding reduced use of antimicrobials for clinical mastitis since it may be associated with financial losses and poorer cow welfare in many instances.

Dairy herd mastitis and reproduction: using simulation to aid interpretation of results from discrete time survival analysis

Probabilistic sensitivity analysis (PSA) is a simulation-based technique for evaluating the relative importance of different inputs to a complex process model. It is commonly employed in decision analysis and for evaluation of the potential impact of uncertainty in research findings on clinical practice, but has a wide variety of other possible applications. In this example, it was used to evaluate the association between herd-level udder health and reproductive performance in dairy herds. Although several recent studies have found relatively large associations between mastitis and fertility at the level of individual inseminations or lactations, the current study demonstrated that herd-level intramammary infection status is highly unlikely to have a clinically significant impact on the overall reproductive performance of a dairy herd under typical conditions. For example, a large increase in incidence rate of clinical mastitis (from 92 to 131 cases per 100 cows per year) would be expected to increase a herd's modified FERTEX score (a cost-based measure of overall reproductive performance) by just £4.50(1) per cow per year. The herd's background level of submission rate (proportion of eligible cows served every 21 days) and pregnancy risk (proportion of inseminations leading to a pregnancy) correlated strongly with overall reproductive performance and explained a large proportion of the between-herd variation in performance. PSA proved to be a highly useful technique to aid understanding of results from a complex statistical model, and has great potential for a wide variety of applications within the field of veterinary science.

Denosumab: a cost-effective alternative for older men with osteoporosis from a Swedish payer perspective

OBJECTIVE

To assess the cost-effectiveness of denosumab versus other treatments in men with osteoporosis who are ≥75years old from a payer perspective in Sweden.

METHODS

A lifetime cohort Markov model was developed with seven health states: well, hip fracture, vertebral fracture, other osteoporotic fracture, post-hip fracture, post-vertebral fracture, and dead. During each cycle, patients could have a fracture, remain healthy, remain in a post-fracture state or die. Background fracture risks, mortality rates, persistence rates, utilities, medical and drug costs were derived using published sources. Estimates of fracture efficacy were drawn from available studies in post-menopausal osteoporotic (PMO) women as BMD improvements have been shown to be similar across male osteoporosis (MOP) and PMO populations, and a recent clinical trial suggested that the fracture risk reduction from bisphosphonate therapy in men is similar to that seen in women in comparable studies. Lifetime expected costs and quality-adjusted life-years (QALYs) were estimated for denosumab, generic alendronate, generic risedronate, ibandronate, zoledronate, strontium ranelate and teriparatide. On average, patients in the model were 78years old, with bone mineral density T-score at the femoral neck of -2.12. Prevalent vertebral fractures were present in 23% of patients. In the base-case, the model assumed that patients would experience treatment-related effects up to 2years after discontinuation. Costs and QALYs were discounted at 3% annually. Extensive sensitivity analyses were conducted.

RESULTS

Total lifetime costs for denosumab, alendronate, strontium ranelate, zoledronate, risedronate, ibandronate and teriparatide were €31,004, €33,731, €34,788, €34,796, €34,826, €35,983 and €37,461, respectively. Total QALYs were 5.23, 5.15, 5.15, 5.17, 5.13, 5.12 and 5.22, respectively. Compared to other treatments, denosumab had the lowest costs and highest QALYs. In the one-way sensitivity analyses, when compared to alendronate (next least expensive strategy), the ICER for denosumab was most sensitive to the relative risk of hip fracture on denosumab. The probability of denosumab being cost-effective compared to the other treatments at a threshold of €66,000/QALY was 96.1%.

CONCLUSION

Denosumab dominated all comparators, including generic bisphosphonates, in the treatment of osteoporosis in men who were ≥75years old in Sweden.

Optimal timing of influenza vaccination in patients with human immunodeficiency virus: a Markov cohort model based on serial study participant hemoagglutination inhibition titers

BACKGROUND

Seasonal influenza vaccination offers one of the best population-level protections against influenza-like illness (ILI). For most people, a single dose prior to the flu season offers adequate immunogenicity. HIV+ patients, however, tend to exhibit a shorter period of clinical protection, and therefore may not retain immunogenicity for the entire season. Building on the work of Nosyk et al. (2011) that determined a single dose is the optimal dosing strategy for HIV+ patients, we investigate the optimal time to administer this vaccination.

METHODS

Using data from the "single dose" treatment arm of an RCT conducted at 12 CIHR Canadian HIV Trials Network sites we estimated semimonthly clinical seroprotection levels for a cohort (N=93) based on HAI titer levels. These estimates were combined with CDC attack rate data for the three main strains of seasonal influenza to estimate instances of ILI over different vaccination timing strategies. Using bootstrap resampling of the cohort, nine years of CDC data, and parameter distributions, we developed a Markov cohort model that included probabilistic sensitivity analysis. Cost, quality adjusted life-years (QALYs), and net monetary benefits are presented for each timing strategy.

RESULTS

The beginning of December is the optimal time for HIV+ patients to receive the seasonal influenza vaccine. Assuming a willingness-to-pay threshold of $50,000, the net monetary benefit associated with a Dec 1 vaccination date is $19,501.49 and the annual QALY was 0.833744.

INTERPRETATION

Our results support a policy of administering the seasonal influenza vaccination for this population in the middle of November or beginning of December, assuming nothing is know about the upcoming flu season. But because the difference in between this strategy and the CDC guideline is small-12 deaths averted per year and a savings of $60 million across the HIV+ population in the US-more research is needed concerning strategies for subpopulations.