Computing Normalizing Constants for Finite Mixture Models via Incremental Mixture Importance Sampling (IMIS)

Cost-effectiveness of flexible take-home buprenorphine-naloxone versus methadone for treatment of prescription-type opioid use disorder

BACKGROUND

Our objective was to examine the cost-effectiveness of flexible take-home buprenorphine-naloxone (BNX) versus methadone alongside the OPTIMA trial in Canada.

METHODS

The OPTIMA study was a pragmatic, open-label, noninferiority, two-arm randomized controlled trial, to assess the comparative effectiveness of flexible take-home BNX vs. methadone in routine clinical care for individuals with prescription-type opioid use disorder. We evaluated cost-effectiveness using a semi-Markov cohort model. Probabilities of overdose were calibrated, accounting for fentanyl prevalence and other overdose risk factors such as naloxone availability. We considered health sector and societal cost perspectives, including costs (2020 CAD) for treatment, health resource use, criminal activity, and health state-specific preference weights as outcomes to calculate incremental cost-effectiveness ratios. Six-month and lifetime (3% annual discount rate) time-horizons were explored.

RESULTS

Over a lifetime time horizon, individuals accumulated -0.144 [CI: -0.302, -0.025] incremental quality-adjusted life years (QALYs) in BNX compared with methadone. Incremental costs were -$2047 [CI: -$39,197, $24,250] from a societal perspective, and -$4549 [CI: -$6332, -$3001] from a health sector perspective. Over a six-month time-horizon, individuals accumulated 0.002 [credible interval (CI): -0.011, 0.016] incremental QALYs in BNX compared with methadone. Incremental costs were -$307 [CI: -$10,385, $8466] from a societal perspective and -$1111 [CI: -$1517, -$631] from a health sector perspective. BNX was dominated (costlier, less effective) in 49.7% of simulations when adopting a societal perspective over a lifetime time horizon.

CONCLUSIONS

Flexible take-home BNX was not cost-effective versus methadone over a lifetime time horizon, resulting from better treatment retention in methadone compared to BNX.

Characterization and Valuation of the Uncertainty of Calibrated Parameters in Microsimulation Decision Models

Background: We evaluated the implications of different approaches to characterize the uncertainty of calibrated parameters of microsimulation decision models (DMs) and quantified the value of such uncertainty in decision making.

Methods: We calibrated the natural history model of CRC to simulated epidemiological data with different degrees of uncertainty and obtained the joint posterior distribution of the parameters using a Bayesian approach. We conducted a probabilistic sensitivity analysis (PSA) on all the model parameters with different characterizations of the uncertainty of the calibrated parameters. We estimated the value of uncertainty of the various characterizations with a value of information analysis. We conducted all analyses using high-performance computing resources running the Extreme-scale Model Exploration with Swift (EMEWS) framework.

Results: The posterior distribution had a high correlation among some parameters. The parameters of the Weibull hazard function for the age of onset of adenomas had the highest posterior correlation of −0.958. When comparing full posterior distributions and the maximum-a-posteriori estimate of the calibrated parameters, there is little difference in the spread of the distribution of the CEA outcomes with a similar expected value of perfect information (EVPI) of $653 and $685, respectively, at a willingness-to-pay (WTP) threshold of $66,000 per quality-adjusted life year (QALY). Ignoring correlation on the calibrated parameters’ posterior distribution produced the broadest distribution of CEA outcomes and the highest EVPI of $809 at the same WTP threshold.

Conclusion: Different characterizations of the uncertainty of calibrated parameters affect the expected value of eliminating parametric uncertainty on the CEA. Ignoring inherent correlation among calibrated parameters on a PSA overestimates the value of uncertainty.

CDX2 Biomarker Testing and Adjuvant Therapy for Stage II Colon Cancer: An Exploratory Cost-Effectiveness Analysis

OBJECTIVES

Adjuvant chemotherapy is not recommended for patients with average-risk stage II (T3N0) colon cancer. Nevertheless, a subgroup of these patients who are CDX2-negative might benefit from adjuvant chemotherapy. We evaluated the cost-effectiveness of testing for the absence of CDX2 expression followed by adjuvant chemotherapy (fluorouracil combined with oxaliplatin [FOLFOX]) for patients with stage II colon cancer.

METHODS

We developed a decision model to simulate a hypothetical cohort of 65-year-old patients with average-risk stage II colon cancer with 7.2% of these patients being CDX2-negative under 2 different interventions: (1) test for the absence of CDX2 expression followed by adjuvant chemotherapy for CDX2-negative patients and (2) no CDX2 testing and no adjuvant chemotherapy for any patient. We derived disease progression parameters, adjuvant chemotherapy effectiveness and utilities from published analyses, and cancer care costs from the Surveillance, Epidemiology, and End Results (SEER)-Medicare data. Sensitivity analyses were conducted.

RESULTS

Testing for CDX2 followed by FOLFOX for CDX2-negative patients had an incremental cost-effectiveness ratio of $5500/quality-adjusted life-years (QALYs) compared with no CDX2 testing and no FOLFOX (6.874 vs 6.838 discounted QALYs and $89 991 vs $89 797 discounted US dollar lifetime costs). In sensitivity analyses, considering a cost-effectiveness threshold of $100 000/QALY, testing for CDX2 followed by FOLFOX on CDX2-negative patients remains cost-effective for hazard ratios of <0.975 of the effectiveness of FOLFOX in CDX2-negative patients in reducing the rate of developing a metastatic recurrence.

CONCLUSIONS

Testing tumors of patients with stage II colon cancer for CDX2 and administration of adjuvant treatment to the subgroup found CDX2-negative is a cost-effective and high-value management strategy across a broad range of plausible assumptions.

Sequentially calibrating a Bayesian microsimulation model to incorporate new information and assumptions

BACKGROUND

Microsimulation models are mathematical models that simulate event histories for individual members of a population. They are useful for policy decisions because they simulate a large number of individuals from an idealized population, with features that change over time, and the resulting event histories can be summarized to describe key population-level outcomes. Model calibration is the process of incorporating evidence into the model. Calibrated models can be used to make predictions about population trends in disease outcomes and effectiveness of interventions, but calibration can be challenging and computationally expensive.

METHODS

This paper develops a technique for sequentially updating models to take full advantage of earlier calibration results, to ultimately speed up the calibration process. A Bayesian approach to calibration is used because it combines different sources of evidence and enables uncertainty quantification which is appealing for decision-making. We develop this method in order to re-calibrate a microsimulation model for the natural history of colorectal cancer to include new targets that better inform the time from initiation of preclinical cancer to presentation with clinical cancer (sojourn time), because model exploration and validation revealed that more information was needed on sojourn time, and that the predicted percentage of patients with cancers detected via colonoscopy screening was too low.

RESULTS

The sequential approach to calibration was more efficient than recalibrating the model from scratch. Incorporating new information on the percentage of patients with cancers detected upon screening changed the estimated sojourn time parameters significantly, increasing the estimated mean sojourn time for cancers in the colon and rectum, providing results with more validity.

CONCLUSIONS

A sequential approach to recalibration can be used to efficiently recalibrate a microsimulation model when new information becomes available that requires the original targets to be supplemented with additional targets.

Application of Distributed Agent-based Modeling to Investigate Opioid Use Outcomes in Justice Involved Populations

Criminal justice involved (CJI) individuals with a history of opioid use disorder (OUD) are at high risk of overdose and death in the weeks following release from jail. We developed the Justice-Community Circulation Model (JCCM) to investigate OUD/CJI dynamics post-release and the effects of interventions on overdose deaths. The JCCM uses a synthetic agent-based model population of approximately 150,000 unique individuals that is generated using demographic information collected from multiple Chicago-area studies and data sets. We use a high-performance computing (HPC) workflow to implement a sequential approximate Bayesian computation algorithm for calibrating the JCCM. The calibration results in the simulated joint posterior distribution of the JCCM input parameters. The calibrated model is used to investigate the effects of a naloxone intervention for a mass jail release. The simulation results show the degree to which a targeted intervention focusing on recently released jail inmates can help reduce the risk of death from opioid overdose.

CaliPro: A Calibration Protocol That Utilizes Parameter Density Estimation to Explore Parameter Space and Calibrate Complex Biological Models

Introduction

Mathematical and computational modeling have a long history of uncovering mechanisms and making predictions for biological systems. However, to create a model that can provide relevant quantitative predictions, models must first be calibrated by recapitulating existing biological datasets from that system. Current calibration approaches may not be appropriate for complex biological models because: 1) many attempt to recapitulate only a single aspect of the experimental data (such as a median trend) or 2) Bayesian techniques require specification of parameter priors and likelihoods to experimental data that cannot always be confidently assigned. A new calibration protocol is needed to calibrate complex models when current approaches fall short.

Methods

Herein, we develop CaliPro, an iterative, model-agnostic calibration protocol that utilizes parameter density estimation to refine parameter space and calibrate to temporal biological datasets. An important aspect of CaliPro is the user-defined pass set definition, which specifies how the model might successfully recapitulate experimental data. We define the appropriate settings to use CaliPro.

Results

We illustrate the usefulness of CaliPro through four examples including predator-prey, infectious disease transmission, and immune response models. We show that CaliPro works well for both deterministic, continuous model structures as well as stochastic, discrete models and illustrate that CaliPro can work across diverse calibration goals.

Conclusions

We present CaliPro, a new method for calibrating complex biological models to a range of experimental outcomes. In addition to expediting calibration, CaliPro may be useful in already calibrated parameter spaces to target and isolate specific model behavior for further analysis.

Methods for Model Calibration under High Uncertainty: Modeling Cholera in Bangladesh

Background. Published data on a disease do not always correspond directly to the parameters needed to simulate natural history. Several calibration methods have been applied to computer-based disease models to extract needed parameters that make a model's output consistent with available data. Objective. To assess 3 calibration methods and evaluate their performance in a real-world application. Methods. We calibrated a model of cholera natural history in Bangladesh, where a lack of active surveillance biases available data. We built a cohort state-transition cholera natural history model that includes case hospitalization to reflect the passive surveillance data-generating process. We applied 3 calibration techniques: incremental mixture importance sampling, sampling importance resampling, and random search with rejection sampling. We adapted these techniques to the context of wide prior uncertainty and many degrees of freedom. We evaluated the resulting posterior parameter distributions using a range of metrics and compared predicted cholera burden estimates. Results. All 3 calibration techniques produced posterior distributions with a higher likelihood and better fit to calibration targets as compared with prior distributions. Incremental mixture importance sampling resulted in the highest likelihood and largest number of unique parameter sets to better inform joint parameter uncertainty. Compared with naïve uncalibrated parameter sets, calibrated models of cholera in Bangladesh project substantially more cases, many of which are not detected by passive surveillance, and fewer deaths. Limitations. Calibration cannot completely overcome poor data quality, which can leave some parameters less well informed than others. Calibration techniques may perform differently under different circumstances. Conclusions. Incremental mixture importance sampling, when adapted to the context of high uncertainty, performs well. By accounting for biases in data, calibration can improve model projections of disease burden.

MICROSIMULATION MODEL CALIBRATION USING INCREMENTAL MIXTURE APPROXIMATE BAYESIAN COMPUTATION

Microsimulation models (MSMs) are used to inform policy by predicting population-level outcomes under different scenarios. MSMs simulate individual-level event histories that mark the disease process (such as the development of cancer) and the effect of policy actions (such as screening) on these events. MSMs often have many unknown parameters; calibration is the process of searching the parameter space to select parameters that result in accurate MSM prediction of a wide range of targets. We develop Incremental Mixture Approximate Bayesian Computation (IMABC) for MSM calibration, which results in a simulated sample from the posterior distribution of model parameters given calibration targets. IMABC begins with a rejection-based ABC step, drawing a sample of points from the prior distribution of model parameters and accepting points that result in simulated targets that are near observed targets. Next, the sample is iteratively updated by drawing additional points from a mixture of multivariate normal distributions and accepting points that result in accurate predictions. Posterior estimates are obtained by weighting the final set of accepted points to account for the adaptive sampling scheme. We demonstrate IMABC by calibrating CRC-SPIN 2.0, an updated version of a MSM for colorectal cancer (CRC) that has been used to inform national CRC screening guidelines.

Modeling malaria genomics reveals transmission decline and rebound in Senegal

To study the effects of malaria-control interventions on parasite population genomics, we examined a set of 1,007 samples of the malaria parasite Plasmodium falciparum collected in Thiès, Senegal between 2006 and 2013. The parasite samples were genotyped using a molecular barcode of 24 SNPs. About 35% of the samples grouped into subsets with identical barcodes, varying in size by year and sometimes persisting across years. The barcodes also formed networks of related groups. Analysis of 164 completely sequenced parasites revealed extensive sharing of genomic regions. In at least two cases we found first-generation recombinant offspring of parents whose genomes are similar or identical to genomes also present in the sample. An epidemiological model that tracks parasite genotypes can reproduce the observed pattern of barcode subsets. Quantification of likelihoods in the model strongly suggests a reduction of transmission from 2006-2010 with a significant rebound in 2012-2013. The reduced transmission and rebound were confirmed directly by incidence data from Thiès. These findings imply that intensive intervention to control malaria results in rapid and dramatic changes in parasite population genomics. The results also suggest that genomics combined with epidemiological modeling may afford prompt, continuous, and cost-effective tracking of progress toward malaria elimination.

Tuberculosis control strategies to reach the 2035 global targets in China: the role of changing demographics and reactivation disease

BACKGROUND

In the last 20 years, China ramped up a DOTS (directly observed treatment, short-course)-based tuberculosis (TB) control program with 80% population coverage, achieving the 2015 Millennium Development Goal of a 50% reduction in TB prevalence and mortality. Recently, the World Health Organization developed the End TB Strategy, with an overall goal of a 90% reduction in TB incidence and a 95% reduction in TB deaths from 2015-2035. As the TB burden shifts to older individuals and China's overall population ages, it is unclear if maintaining the current DOTS strategy will be sufficient for China to reach the global targets.

METHODS

We developed an individual-based computational model of TB transmission, implementing realistic age demographics and fitting to country-level data of age-dependent prevalence over time. We explored the trajectory of TB burden if the DOTS strategy is maintained or if new interventions are introduced using currently available and soon-to-be-available tools. These interventions include increasing population coverage of DOTS, reducing time to treatment, increasing treatment success, and active case finding among elders > 65 years old. We also considered preventative therapy in latently infected elders, a strategy limited by resource constraints and the risk of adverse events.

RESULTS

Maintenance of the DOTS strategy reduces TB incidence and mortality by 42% (95% credible interval, 27-59%) and 41% (5-64%), respectively, between 2015 and 2035. A combination of all feasible interventions nears the 2035 mortality target, reducing TB incidence and mortality by 59% (50-76%) and 83% (73-94%). Addition of preventative therapy for elders would enable China to nearly reach both the incidence and mortality targets, reducing incidence and mortality by 84% (78-93%) and 92% (86-98%).

CONCLUSIONS

The current decline in incidence is driven by two factors: maintaining a low level of new infections in young individuals and the aging out of older latently infected individuals who contribute incidence due to reactivation disease. While further reducing the level of new infections has a modest effect on burden, interventions that limit reactivation have a greater impact on TB burden. Tools that make preventative therapy more feasible on a large scale and in elders will help China achieve the global targets.

Calibration of an intrahost malaria model and parameter ensemble evaluation of a pre-erythrocytic vaccine

Background

A pre-erythrocytic vaccine could provide a useful tool for burden reduction and eventual eradication of malaria. Mathematical malaria models provide a mechanism for evaluating the effective burden reduction across a range of transmission conditions where such a vaccine might be deployed.

Methods

The EMOD model is an individual-based model of malaria transmission dynamics, including vector lifecycles and species-specific behaviour, coupled to a mechanistic intrahost model of malaria parasite and host immune system dynamics. The present work describes the extension of the EMOD model to include diagnoses of severe malaria and iterative calibration of the immune system parameters and parasite antigenic variation to age-stratified prevalence, incidence and severe disease incidence data obtained from multiple regions with broadly varying transmission conditions in Africa. An ensemble of calibrated model parameter sets is then employed to evaluate the potential impact of routine immunization with a pre-erythrocytic vaccine.

Results

The reduction in severe malaria burden exhibits a broad peak at moderate transmission conditions. Under sufficiently intense transmission, a vaccine that reduces but does not eliminate the probability of acquisition from a single challenge bite may delay infections but produces minimal or no net reduction. Conversely, under sufficiently weak transmission conditions, a vaccine can provide a high fractional reduction but avert a relatively low absolute number of cases due to low baseline burden.

Conclusions

Roll-out of routine immunization with pre-erythrocytic malaria vaccines can provide substantial burden reduction across a range of transmission conditions typical to many regions in Africa.

Electronic supplementary material

The online version of this article (doi:10.1186/1475-2875-14-6) contains supplementary material, which is available to authorized users.

Estimates of age-specific reductions in HIV prevalence in Uganda: Bayesian melding estimation and probabilistic population forecast with an HIV-enabled cohort component projection model

BACKGROUND

Much of our knowledge of the epidemiology and demography of HIV epidemics in Africa is derived from models fit to sparse, non-representative data. These often average over age and other important dimensions, rarely quantify uncertainty, and typically do not impose consistency on the epidemiology and the demography of the population.

OBJECTIVE

This work conducts an empirical investigation of the history of the HIV epidemic in Uganda and Tanzania through the late 1990s, focusing on sex-age-specific incidence, uses those results to produce probabilistic forecasts of HIV prevalence ten years later, and compares those to measures of HIV prevalence at the later time to describe the sex-age pattern of changes in prevalence over the intervening period.

METHODS

We adapt an epidemographic model of a population affected by HIV so that its parameters can be estimated using both the Bayesian melding with IMIS estimation method and maximum likelihood methods. Using the Bayesian version of the model we produce probabilistic forecasts of the population with HIV.

RESULTS

We produce estimates of sex-age-specific HIV incidence in Uganda and Tanzania in the late 1990s, produce probabilistic forecasts of the HIV epidemics in Uganda and Tanzania during the early 2000s, describe the sex-age pattern of changes in HIV prevalence in Uganda during the early 2000s, and compare the performance and results of the Bayesian and maximum likelihood estimation procedures.

CONCLUSIONS

We demonstrate that: (1) it is possible to model HIV epidemics in Africa taking account of sex and age, (2) there are important advantages to the Bayesian estimation method, including rigorous quantification of uncertainty and the ability to make probabilistic forecasts, and (3) that there were important age-specific changes in HIV incidence in Uganda during the early 2000s.

Estimating and Projecting Trends in HIV/AIDS Generalized Epidemics Using Incremental Mixture Importance Sampling

The Joint United Nations Programme on HIV/AIDS (UNAIDS) has decided to use Bayesian melding as the basis for its probabilistic projections of HIV prevalence in countries with generalized epidemics. This combines a mechanistic epidemiological model, prevalence data, and expert opinion. Initially, the posterior distribution was approximated by sampling-importance-resampling, which is simple to implement, easy to interpret, transparent to users, and gave acceptable results for most countries. For some countries, however, this is not computationally efficient because the posterior distribution tends to be concentrated around nonlinear ridges and can also be multimodal. We propose instead incremental mixture importance sampling (IMIS), which iteratively builds up a better importance sampling function. This retains the simplicity and transparency of sampling importance resampling, but is much more efficient computationally. It also leads to a simple estimator of the integrated likelihood that is the basis for Bayesian model comparison and model averaging. In simulation experiments and on real data, it outperformed both sampling importance resampling and three publicly available generic Markov chain Monte Carlo algorithms for this kind of problem.