Calibrating models in economic evaluation: a seven-step approach

Development and calibration of a mathematical model of HIV outcomes among Rwandan adults: informing equitable achievement of targets in Rwanda

Background

We developed and calibrated the Central Africa-International epidemiology Databases to Evaluate AIDS (CA-IeDEA) HIV policy model to inform equitable achievement of global goals, overall and across sub-populations, in Rwanda.

Methods

We created a deterministic dynamic model to project adult HIV epidemic and care continuum outcomes, overall and for 25 subpopulations (age group, sex, HIV acquisition risk, urbanicity). Data came from the Rwanda cohort of CA-IeDEA, 2004-2020; Rwanda Demographic and Health Surveys, 2005, 2010, 2015; Rwanda Population-based HIV Impact Assessment, 2019; and the literature and reports. We calibrated the model to 47 targets by selecting the 50 best-fitting parameter sets among 20,000 simulations. Calibration targets reflected epidemic (HIV prevalence, incidence), global goals (percentage on antiretroviral therapy (ART) among diagnosed, percentage virally suppressed among on ART) and other (number on ART, percentage virally suppressed) indicators, overall and by sex. Best-fitting sets minimized the summed absolute value of the percentage deviation (AVPD) between model projections and calibration targets. Good model performance was mean AVPD < 5% across the 50 best-fitting sets and/or projections within the target confidence intervals; acceptable was mean AVPD >5% and < 15%.

Results

Across indicators, 1,841 of 2,350 (78.3%) model projections were a good or acceptable fit to calibration targets. For HIV epidemic indicators, 256 of 300 (85.3%) projections were a good fit to targets, with the model performing better for women (83.3% a good fit) than for men (71.7% a good fit). For global goals indicators, 96 of 100 (96.0%) projections were a good fit; model performance was similar for women and men. For other indicators, 653 of 950 (68.7%) projections were a good or acceptable fit. Fit was better for women than for men (percentage virally suppressed only) and when restricting targets for number on ART to 2013 and beyond.

Conclusions

The CA-IeDEA HIV policy model fits historical data and can inform policy solutions for equitably achieving global goals to end the HIV epidemic in Rwanda. High-quality, unbiased population-based data, as well as novel approaches that account for calibration target quality, are critical to ongoing use of mathematical models for programmatic planning.

Cost-Utility Analysis of Maintenance Pemetrexed Plus Best Supportive Care Compared With Best Supportive Care Alone in Treating Patients With Non-Small Cell Lung Cancer in Jordan

OBJECTIVES

To assess the cost-effectiveness of maintenance pemetrexed plus best supportive care (BSC) in non-small cell lung cancer patients from a Jordanian healthcare system perspective.

METHODS

A Markov model with 4 health states was developed to estimate life years, quality-adjusted life-years (QALY), costs, and the incremental cost-utility ratio of pemetrexed plus BSC versus BSC. A lifelong time horizon was used in the base-case analysis. The transition probabilities were estimated from the PARAMOUNT trial, the utility weights were taken from published literature, and costs were based on data and unit costs at King Hussein Cancer Center and the Jordan Food and Drug Administration. Both costs and outcomes were discounted using a 3%. The parameter uncertainty was tested using deterministic and probabilistic sensitivity analyses.

RESULTS

The base-case analysis showed that pemetrexed plus BSC increased QALYs and cost compared with BSC. Pemetrexed plus BSC leads to incremental 0.255 QALYs and incremental costs of US $30 826, resulting in an incremental cost-utility ratio of US $120 886/QALY. The results were sensitive to changes in the utility estimates during the progression-free health state, the progression health state, and the cost of postprogression medications The probabilistic sensitivity analysis showed that the probability of pemetrexed plus BSC being a cost-effective option compared with BSC is 0 at a threshold of $56 000.

CONCLUSIONS

Maintenance pemetrexed for non-small cell lung cancer is not a cost-effective option compared with BSC from a healthcare system perspective based on the listed price at a threshold of $56 000/QALY.

Challenges and Opportunities in Interdisciplinary Research and Real-World Data for Treatment Sequences in Health Technology Assessments

With an ever-increasing number of treatment options, the assessment of treatment sequences has become crucial in health technology assessment (HTA). This review systematically explores the multifaceted challenges inherent in evaluating sequences, delving into their interplay and nuances that go beyond economic model structures. We synthesised a 'roadmap' of literature from key methodological studies, highlighting the evolution of recent advances and emerging research themes. These insights were compared against HTA guidelines to identify potential avenues for future research. Our findings reveal a spectrum of challenges in sequence evaluation, encompassing selecting appropriate decision-analytic modelling approaches and comparators, deriving appropriate clinical effectiveness evidence in the face of data scarcity, scrutinising effectiveness assumptions and statistical adjustments, considering treatment displacement, and optimising model computations. Integrating methodologies from diverse disciplines-statistics, epidemiology, causal inference, operational research and computer science-has demonstrated promise in addressing these challenges. An updated review of application studies is warranted to provide detailed insights into the extent and manner in which these methodologies have been implemented. Data scarcity on the effectiveness of treatment sequences emerged as a dominant concern, especially because treatment sequences are rarely compared in clinical trials. Real-world data (RWD) provide an alternative means for capturing evidence on effectiveness and future research should prioritise harnessing causal inference methods, particularly Target Trial Emulation, to evaluate treatment sequence effectiveness using RWD. This approach is also adaptable for analysing trials harbouring sequencing information and adjusting indirect comparisons when collating evidence from heterogeneous sources. Such investigative efforts could lend support to reviews of HTA recommendations and contribute to synthesising external control arms involving treatment sequences.

Informing Public Health Policies with Models for Disease Burden, Impact Evaluation, and Economic Evaluation

Conducting real-world public health experiments is often costly, time-consuming, and ethically challenging, so mathematical models have a long-standing history of being used to inform policy. Applications include estimating disease burden, performing economic evaluation of interventions, and responding to health emergencies such as pandemics. Models played a pivotal role during the COVID-19 pandemic, providing early detection of SARS-CoV-2's pandemic potential and informing subsequent public health measures. While models offer valuable policy insights, they often carry limitations, especially when they depend on assumptions and incomplete data. Striking a balance between accuracy and timely decision-making in rapidly evolving situations such as disease outbreaks is challenging. Modelers need to explore the extent to which their models deviate from representing the real world. The uncertainties inherent in models must be effectively communicated to policy makers and the public. As the field becomes increasingly influential, it needs to develop reporting standards that enable rigorous external scrutiny.

A Framework for Assessing the Impact of Outbreak Response Immunization Programs

The impact of outbreak response immunization (ORI) can be estimated by comparing observed outcomes to modelled counterfactual scenarios without ORI, but the most appropriate metrics depend on stakeholder needs and data availability. This study developed a framework for using mathematical models to assess the impact of ORI for vaccine-preventable diseases. Framework development involved (1) the assessment of impact metrics based on stakeholder interviews and literature reviews determining data availability and capacity to capture as model outcomes; (2) mapping investment in ORI elements to model parameters to define scenarios; (3) developing a system for engaging stakeholders and formulating model questions, performing analyses, and interpreting results; and (4) example applications for different settings and pathogens. The metrics identified as most useful were health impacts, economic impacts, and the risk of severe outbreaks. Scenario categories included investment in the response scale, response speed, and vaccine targeting. The framework defines four phases: (1) problem framing and data sourcing (identification of stakeholder needs, metrics, and scenarios); (2) model choice; (3) model implementation; and (4) interpretation and communication. The use of the framework is demonstrated by application to two outbreaks, measles in Papua New Guinea and Ebola in the Democratic Republic of the Congo. The framework is a systematic way to engage with stakeholders and ensure that an analysis is fit for purpose, makes the best use of available data, and uses suitable modelling methodology.

A multimorbidity model for estimating health outcomes from the syndemic of injection drug use and associated infections in the United States

BACKGROUND

Fatal drug overdoses and serious injection-related infections are rising in the US. Multiple concurrent infections in people who inject drugs (PWID) exacerbate poor health outcomes, but little is known about how the synergy among infections compounds clinical outcomes and costs. Injection drug use (IDU) converges multiple epidemics into a syndemic in the US, including opioid use and HIV. Estimated rates of new injection-related infections in the US are limited due to widely varying estimates of the number of PWID in the US, and in the absence of clinical trials and nationally representative longitudinal observational studies of PWID, simulation models provide important insights to policymakers for informed decisions.

METHODS

We developed and validated a MultimorbiditY model to Reduce Infections Associated with Drug use (MYRIAD). This microsimulation model of drug use and associated infections (HIV, hepatitis C virus [HCV], and severe bacterial infections) uses inputs derived from published data to estimate national level trends in the US. We used Latin hypercube sampling to calibrate model output against published data from 2015 to 2019 for fatal opioid overdose rates. We internally validated the model for HIV and HCV incidence and bacterial infection hospitalization rates among PWID. We identified best fitting parameter sets that met pre-established goodness-of-fit targets using the Pearson's chi-square test. We externally validated the model by comparing model output to published fatal opioid overdose rates from 2020.

RESULTS

Out of 100 sample parameter sets for opioid use, the model produced 3 sets with well-fitting results to key calibration targets for fatal opioid overdose rates with Pearson's chi-square test ranging from 1.56E-5 to 2.65E-5, and 2 sets that met validation targets. The model produced well-fitting results within validation targets for HIV and HCV incidence and serious bacterial infection hospitalization rates. From 2015 to 2019, the model estimated 120,000 injection-related overdose deaths, 17,000 new HIV infections, and 144,000 new HCV infections among PWID.

CONCLUSIONS

This multimorbidity microsimulation model, populated with data from national surveillance data and published literature, accurately replicated fatal opioid overdose, incidence of HIV and HCV, and serious bacterial infections hospitalization rates. The MYRIAD model of IDU could be an important tool to assess clinical and economic outcomes related to IDU behavior and infections with serious morbidity and mortality for PWID.

Modeling the Natural History and Screening Effects of Colorectal Cancer Using Both Adenoma and Serrated Neoplasia Pathways: The Development, Calibration, and Validation of a Discrete Event Simulation Model

Background. Existing colorectal cancer (CRC) screening models mostly focus on the adenoma pathway of CRC development, overlooking the serrated neoplasia pathway, which might result in overly optimistic screening predictions. In addition, Bayesian inference methods have not been widely used for model calibration. We aimed to develop a CRC screening model accounting for both pathways, calibrate it with approximate Bayesian computation (ABC) methods, and validate it with large CRC screening trials. Methods. A discrete event simulation (DES) of the CRC natural history (DECAS) was constructed using the adenoma and serrated pathways in R software. The model simulates CRC-related events in a specific birth cohort through various natural history states. Calibration took advantage of 74 prevalence data points from the German screening colonoscopy program of 5.2 million average-risk participants using an ABC method. CRC incidence outputs from DECAS were validated with the German national cancer registry data; screening effects were validated using 17-y data from the UK Flexible Sigmoidoscopy Screening sigmoidoscopy trial and a German screening colonoscopy cohort study. Results. The Bayesian calibration rendered 1,000 sets of posterior parameter samples. With the calibrated parameters, the observed age- and sex-specific CRC prevalences from the German registries were within the 95% DECAS-predicted intervals. Regarding screening effects, DECAS predicted a 41% (95% intervals 30%-51%) and 62% (95% intervals 55%-68%) reduction in 17-y cumulative CRC mortality for a single screening sigmoidoscopy and colonoscopy, respectively, falling within 95% confidence intervals reported in the 2 clinical studies used for validation. Conclusions. We presented DECAS, the first Bayesian-calibrated DES model for CRC natural history and screening, accounting for 2 CRC tumorigenesis pathways. The validated model can serve as a valid tool to evaluate the (cost-)effectiveness of CRC screening strategies.

Highlights

This article presents a new discrete event simulation model, DECAS, which models both adenoma-carcinoma and serrated neoplasia pathways for colorectal cancer (CRC) development and CRC screening effects.DECAS is calibrated based on a Bayesian inference method using the data from German screening colonoscopy program, which consists of more than 5 million first-time average-risk participants aged 55 years and older in 2003 to 2014.DECAS is flexible for evaluating various CRC screening strategies and can differentiate screening effects in different parts of the colon.DECAS is validated with large screening sigmoidoscopy and colonoscopy clinical study data and can be further used to evaluate the (cost-)effectiveness of German colorectal cancer screening strategies.

Understanding the risk of developing weight-related complications associated with different body mass index categories: a systematic review

BACKGROUND

Obesity and overweight are major risk factors for several chronic diseases. There is limited systematic evaluation of risk equations that predict the likelihood of developing an obesity or overweight associated complication. Predicting future risk is essential for health economic modelling. Availability of future treatments rests upon a model's ability to inform clinical and decision-making bodies.  This systematic literature review aimed to identify studies reporting (1) equations that calculate the risk for individuals with obesity, or overweight with a weight-related complication (OWRC), of developing additional complications, namely T2D, cardiovascular (CV) disease (CVD), acute coronary syndrome, stroke, musculoskeletal disorders, knee replacement/arthroplasty, or obstructive sleep apnea; (2) absolute or proportional risk for individuals with severe obesity, obesity or OWRC developing T2D, a CV event or mortality from knee surgery, stroke, or an acute CV event.

METHODS

Databases (MEDLINE and Embase) were searched for English language reports of population-based cohort analyses or large-scale studies in Australia, Canada, Europe, the UK, and the USA between January 1, 2011, and March 29, 2021. Included reports were quality assessed using an adapted version of the Newcastle Ottawa Scale.

RESULTS

Of the 60 included studies, the majority used European cohorts. Twenty-nine reported a risk prediction equation for developing an additional complication. The most common risk prediction equations were logistic regression models that did not differentiate between body mass index (BMI) groups (particularly above 40 kg/m²) and lacked external validation. The remaining included studies (31 studies) reported the absolute or proportional risk of mortality (29 studies), or the risk of developing T2D in a population with obesity and with prediabetes or normal glucose tolerance (NGT) (three studies), or a CV event in populations with severe obesity with NGT or T2D (three studies). Most reported proportional risk, predominantly a hazard ratio.

CONCLUSION

More work is needed to develop and validate these risk equations, specifically in non-European cohorts and that distinguish between BMI class II and III obesity. New data or adjustment of the current risk equations by calibration would allow for more accurate decision making at an individual and population level.

EarlyCDT Lung blood test for risk classification of solid pulmonary nodules: systematic review and economic evaluation

BACKGROUND

EarlyCDT Lung (Oncimmune Holdings plc, Nottingham, UK) is a blood test to assess malignancy risk in people with solid pulmonary nodules. It measures the presence of seven lung cancer-associated autoantibodies. Elevated levels of these autoantibodies may indicate malignant disease. The results of the test might be used to modify the risk of malignancy estimated by existing risk calculators, including the Brock and Herder models.

OBJECTIVES

The objectives were to determine the diagnostic accuracy, clinical effectiveness and cost-effectiveness of EarlyCDT Lung; and to develop a conceptual model and identify evidence requirements for a robust cost-effectiveness analysis.

DATA SOURCES

MEDLINE (including Epub Ahead of Print, In-Process & Other Non-Indexed Citations, Ovid MEDLINE Daily and Ovid MEDLINE), EMBASE, Cochrane Central Register of Controlled Trials, Science Citation Index, EconLit, Cochrane Database of Systematic Reviews, Database of Abstracts of Reviews of Effects, Health Technology Assessment database, NHS Economic Evaluation Database ( NHS EED ) and the international Health Technology Assessment database were searched on 8 March 2021.

REVIEW METHODS

A systematic review was performed of evidence on EarlyCDT Lung, including diagnostic accuracy, clinical effectiveness and cost-effectiveness. Study quality was assessed with the quality assessment of diagnostic accuracy studies-2 tool. Evidence on other components of the pulmonary nodule diagnostic pathway (computerised tomography surveillance, Brock risk, Herder risk, positron emission tomography-computerised tomography and biopsy) was also reviewed. When feasible, bivariate meta-analyses of diagnostic accuracy were performed. Clinical outcomes were synthesised narratively. A simulation study investigated the clinical impact of using EarlyCDT Lung. Additional reviews of cost-effectiveness studies evaluated (1) other diagnostic strategies for lung cancer and (2) screening approaches for lung cancer. A conceptual model was developed.

RESULTS

A total of 47 clinical publications on EarlyCDT Lung were identified, but only five cohorts (695 patients) reported diagnostic accuracy data on patients with pulmonary nodules. All cohorts were small or at high risk of bias. EarlyCDT Lung on its own was found to have poor diagnostic accuracy, with a summary sensitivity of 20.2% (95% confidence interval 10.5% to 35.5%) and specificity of 92.2% (95% confidence interval 86.2% to 95.8%). This sensitivity was substantially lower than that estimated by the manufacturer (41.3%). No evidence on the clinical impact of EarlyCDT Lung was identified. The simulation study suggested that EarlyCDT Lung might potentially have some benefit when considering intermediate risk nodules (10-70% risk) after Herder risk analysis. Two cost-effectiveness studies on EarlyCDT Lung for pulmonary nodules were identified; none was considered suitable to inform the current decision problem. The conceptualisation process identified three core components for a future cost-effectiveness assessment of EarlyCDT Lung: (1) the features of the subpopulations and relevant heterogeneity, (2) the way EarlyCDT Lung test results affect subsequent clinical management decisions and (3) how changes in these decisions can affect outcomes. All reviewed studies linked earlier diagnosis to stage progression and stage shift to final outcomes, but evidence on these components was sparse.

LIMITATIONS

The evidence on EarlyCDT Lung among patients with pulmonary nodules was very limited, preventing meta-analyses and economic analyses.

CONCLUSIONS

The evidence on EarlyCDT Lung among patients with pulmonary nodules is insufficient to draw any firm conclusions as to its diagnostic accuracy or clinical or economic value.

FUTURE WORK

Prospective cohort studies, in which EarlyCDT Lung is used among patients with identified pulmonary nodules, are required to support a future assessment of the clinical and economic value of this test. Studies should investigate the diagnostic accuracy and clinical impact of EarlyCDT Lung in combination with Brock and Herder risk assessments. A well-designed cost-effectiveness study is also required, integrating emerging relevant evidence with the recommendations in this report.

STUDY REGISTRATION

This study is registered as PROSPERO CRD42021242248.

FUNDING

This project was funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 26, No. 49. See the NIHR Journals Library website for further project information.

The Cost-Effectiveness of Vaccination of Older Adults with an MF59-Adjuvanted Quadrivalent Influenza Vaccine Compared to Other Available Quadrivalent Vaccines in Germany

Enhanced quadrivalent influenza vaccines that include an adjuvant (aQIV) or a high dose of antigen (QIV-HD), which stimulate a stronger immune response in older adults than the standard vaccine (QIVe), are now approved. The objective of this research is to compare available vaccines and determine the cost-effectiveness of immunizing persons aged 65 years and above with aQIV compared to QIVe and QIV-HD in Germany. A compartmental transmission model calibrated to outpatient visits for influenza in Germany was used to predict the number of medically attended infections using the three vaccines. The rates of hospitalizations, deaths, and other economic consequences were estimated with a decision tree using German data where available. Based on meta-analysis, the rVE of −2.5% to 8.9% for aQIV versus QIV-HD, the vaccines are similar clinically, but aQIV is cost saving compared to QIV-HD (unit cost of EUR 40.55). All results were most sensitive to changes in vaccine effectiveness. aQIV may be cost-effective compared to QIVe depending on the willingness to pay for additional benefits in Germany. As aQIV and QIV-HD are similar in terms of effectiveness, aQIV is cost saving compared to QIV-HD at current unit prices.

Validation of the Decision model of the Burden of Hearing loss Across the Lifespan (DeciBHAL) in Chile, India, and Nigeria

Background

There is no published decision model for informing hearing health care resource allocation across the lifespan in low- and middle-income countries. We sought to validate the Decision model of the Burden of Hearing loss Across the Lifespan International (DeciBHAL-I) in Chile, India, and Nigeria.

Methods

DeciBHAL-I simulates bilateral sensorineural hearing loss (SNHL) and conductive hearing loss (CHL) acquisition, SNHL progression, and hearing loss treatment. To inform model inputs, we identified setting-specific estimates including SNHL prevalence from the Global Burden of Disease (GBD) studies, acute otitis media (AOM) incidence and prevalence of otitis-media related CHL from a systematic review, and setting-specific pediatric and adult hearing aid use prevalence. We considered a coefficient of variance root mean square error (CV-RMSE) of ≤15% to indicate good model fit.

Findings

The model-estimated prevalence of bilateral SNHL closely matched GBD estimates, (CV-RMSEs: 3.2-7.4%). Age-specific AOM incidences from DeciBHAL-I also achieved good fit (CV-RMSEs=5.0-7.5%). Model-projected chronic suppurative otitis media prevalence (1.5% in Chile, 4.9% in India, and 3.4% in Nigeria) was consistent with setting-specific estimates, and the incidence of otitis media-related CHL was calibrated to attain adequate model fit. DeciBHAL-projected adult hearing aid use in Chile (3.2-19.7% ages 65-85 years) was within the 95% confidence intervals of published estimates. Adult hearing aid prevalence from the model in India was 1.4-2.3%, and 1.1-1.3% in Nigeria, consistent with literature-based and expert estimates.

Interpretation

DeciBHAL-I reasonably simulates hearing loss natural history, detection, and treatment in Chile, India, and Nigeria. Future cost-effectiveness analyses might use DeciBHAL-I to inform global hearing health policy.

Funding

National Institutes of Health (3UL1-TR002553-03S3 and F30 DC019846).

Cervical cancer screening in low- and middle-income countries: A systematic review of economic evaluation studies

Economic assessments are relevant to support the decision to incorporate more cost-effective strategies to reduce Cervical Cancer (CC) mortality. This systematic review analyzes the economic evaluation studies of CC prevention strategies (HPV DNA-based tests and conventional cytology) in low- and middle-income countries. Medline, EMBASE, CRD, and LILACS were searched for economic evaluation studies that reported cost and effectiveness measures of HPV DNA-based tests for CC screening and conventional cytology in women, without age, language, or publication date restrictions. Selection and data extraction were carried out independently. For comparability of results, cost-effectiveness measures were converted to international dollars (2019). Report quality was assessed using the CHEERS checklist. The Dominance Matrix Ranking (DRM) was used to analyze and interpret the results. The review included 15 studies from 12 countries, with cost-effectiveness analyzes from the health system's perspective and a 3% discount rate. The strategies varied in age and frequency of screening. Most studies used the Markov analytical model, and the cost-benefit threshold was based on the per capita GDP of each country. The sensitivity analysis performed in most studies was deterministic. The completeness of the report was considered sufficient in most of the items evaluated by CHEERS. The Dominance Interpretation (DRM) varied; in 6 studies, the HPV test was dominant, 5 studies showed a weak dominance evaluating greater effectiveness of the HPV test at a higher cost, yet in 2 studies conventional cytology was dominant. Although the context-dependent nature of economic evaluations, this review points out the challenge of methodological standardization in the analytical models.

Clinical Benefit and Cost Effectiveness of Risk-Stratified Gastric Cancer Screening Strategies in China: A Modeling Study

BACKGROUND AND OBJECTIVE

A new gastric cancer screening scoring system (NGCS) strategy was recommended for the early gastric cancer (GC) screening process in China. The current study aimed to assess the clinical benefits and the cost effectiveness of the NGCS strategy in GC high-risk areas of China from a societal perspective.

METHODS

A Markov microsimulation model was developed to evaluate 30 alternative screening strategies with varying initiation age, including the NGCS strategy, the modified NGCS strategy, and the endoscopic screening strategy with various screening intervals. The primary outcomes included GC mortality, number of endoscopies, quality-adjusted life-years (QALYs), costs, and incremental cost-effectiveness ratios (ICERs). Cost estimates were reported in 2021 USD (US$) and both costs and benefits were discounted at 5% annually. Deterministic and probabilistic sensitivity analyses were performed to evaluate model uncertainty.

RESULTS

Screening with the NGCS strategy from age 40 years (40-NGCS) reduced the GC incidence by 86.4%, which provided the greatest benefit across strategies. Compared with all strategies, at a willingness-to pay threshold of US$17,922 per QALY, the 40-NGCS strategy was a leading cost-effective strategy, with an ICER of US$15,668 per QALY. Results were robust in univariate and probabilistic sensitivity analyses. The probability of the 40-NGCS strategy being cost effective was 0.863.

CONCLUSIONS

The 40-NGCS strategy was an effective and cost-effective strategy to reduce GC incidence and mortality in China. The findings provide important evidence for decision makers to formulate and optimize targeted approaches for GC prevention and control policies in China.

Cost-Effectiveness Analysis of Negative Pressure Wound Therapy to Prevent Surgical Site Infection After Elective Colorectal Surgery

BACKGROUND

Surgical site infection is common after colorectal surgery and is associated with increased costs. Prophylactic negative pressure wound therapy has previously been shown to reduce surgical site infection compared with conventional dressings. However, negative pressure wound therapy application is met with hesitancy because of its additional cost.

OBJECTIVE

This study aims to determine whether the application of prophylactic negative pressure wound therapy after elective colorectal surgery is cost-effective.

DESIGN

A cost-effectiveness analysis comparing prophylactic negative pressure wound therapy versus conventional dressing was completed using a Markov microsimulation model. A publicly funded single health care payer perspective was adopted across a lifetime horizon.

SETTING

This study was conducted using in-hospital elective colorectal surgery.

PATIENTS

The base case was an age-, sex-, and comorbidity-standardized patient undergoing open elective colorectal surgery.

INTERVENTION

Negative pressure wound therapy was applied postoperatively over closed incisions.

MAIN OUTCOMES

The primary outcomes of interest were the number of surgical site infections, total costs, and quality-adjusted life-years gained. Secondary outcomes included emergency department presentation, hospital readmission, nursing wound care utilization, fascial dehiscence, incisional hernia, and non-surgical site infection-related complications.

RESULTS

We found that prophylactic negative pressure wound therapy, standardized to 1000 patients, prevented 51 surgical site infections, 3 fascial dehiscences, 10 incisional hernias, 22 emergency department presentations, and 6 hospital readmissions. This resulted in a total cost saving of $17,066 and 92.2 quality-adjusted life-years gained ($17.07 and 0.09 quality-adjusted life-years gained on average per patient). When the patients' risk of surgical site infections was greater than 3.2%, negative pressure wound therapy was a cost-effective strategy at a willingness to pay of $50,000/quality-adjusted life-years.

LIMITATIONS

We did not model for societal perspective, emergent presentations of incarcerated hernias, or complications with hernia repair. The results of this model are reliant on the published negative pressure wound therapy efficacy and may change when additional data arise.

CONCLUSION

The use of negative pressure wound therapy is the dominant strategy with improved outcomes and reduced costs compared with conventional dressing in patients undergoing colorectal surgery, particularly in at-risk patients. See Video Abstract at http://links.lww.com/DCR/B782.

ANLISIS DE RENTABILIDAD DE LA TERAPIA DE PRESIN NEGATIVA PARA PREVENIR INFECCIN DEL SITIO QUIRRGICO DESPUS DE CIRUGA COLORRECTAL ELECTIVA

ANTECEDENTES:La infección del sitio quirúrgico es común después de la cirugía colorrectal y se asocia con un aumento de los costos. Anteriormente se demostró que la terapia profiláctica con presión negativa reduce la infección del sitio quirúrgico en comparación con los apósitos convencionales. Sin embargo, el uso de la terapia de presión negativa se encuentra en dudas debido a su costo adicional.OBJETIVO:Determinar si la aplicación de la terapia profiláctic con presión negativa después de la cirugía colorrectal electiva es rentable.DISEÑO:Se completó un análisis de costo-efectividad comparando la terapia profiláctica con presión negativa versus apósito convencional utilizando un modelo de microsimulación de Markov. Se adoptó una perspectiva de pagador único de asistencia sanitaria financiada con fondos públicos a lo largo de toda la vida.AJUSTE:Cirugía colorrectal electiva intrahospitalaria.PACIENTES:El caso base fue un paciente estandarizado por edad, sexo y comorbilidad sometido a cirugía colorrectal abierta electiva.INTERVENCIÓN:Aplicación postoperatoria de terapia de presión negativa sobre incisiones cerradas.RESULTADOS PRINCIPALES:Los resultados primarios de interés fueron el número de infecciones del sitio quirúrgico, los costos totales y los años de vida ganados ajustados por calidad. Los resultados secundarios incluyeron presentación en la sala de emergencias, reingreso al hospital, la utilización del cuidado de heridas por enfermería, dehiscencia fascial, hernia incisional y complicaciones relacionadas con infecciones del sitio no quirúrgico.RESULTADOS:Estandarizado para 1,000 pacientes, encontramos que la terapia profiláctica con presión negativa previno 51 infecciones del sitio quirúrgico, 3 dehiscencias fasciales, 10 hernias incisionales, 22 presentaciones en la sala de emergencias y 6 reingresos al hospital. Esto resultó en un ahorro total de costos de $ 17.066 y 92.2 años de vida ganados ajustados por calidad ($ 17.07 y 0.09 años de vida ganados ajustados por calidad en promedio por paciente). Cuando el riesgo de infección del sitio quirúrgico de los pacientes era superior al 3,2%, la terapia de presión negativa era una estrategia rentable con una disposición a pagar de 50.000 dólares por años de vida ajustados por calidad.LIMITACIONES:No modelamos para la perspectiva social, presentaciones emergentes de hernias encarceladas o complicaciones con la reparación de hernias. Los resultados de este modelo dependen de la eficacia publicada de la terapia de presión negativa y pueden cambiar cuando surjan más datos.CONCLUSIONES:El uso de la terapia de presión negativa es la estrategia dominante con mejores resultados y costos reducidos en comparación con el apósito convencional en pacientes sometidos a cirugía colorrectal, particularmente en pacientes de riesgo. Consulte Video Resumen en http://links.lww.com/DCR/B782. (Traducción- Dr. Francisco M. Abarca-Rendon).

Calibrating Natural History of Cancer Models in the Presence of Data Incompatibility: Problems and Solutions

The calibration of cancer natural history models is often challenged by a lack of representative calibration targets, forcing modellers to rely on potentially incompatible datasets. Using a microsimulation colorectal cancer model as an example, the purposes of this paper are to (1) highlight the reasons for uncertainty in calibration targets, (2) illustrate practical and generalisable approaches for dealing with incompatibility in calibration targets, and (3) discuss the importance of future research in the area of incorporating uncertainty in calibration. The low quality of data and differences in populations, outcome definitions, and healthcare systems may result in incompatibility between the model and the data. Acknowledging reasons for data incompatibility allows assessment of the risk of incompatibility before calibrating the model. Only a few approaches are available to address data incompatibility, for instance addressing biases in calibration targets and their adjustment, relaxing the goodness-of-fit metric, and validation of the calibration targets to the data not used in the calibration. However, these approaches lack explicit comparison and validation, and so more research is needed to describe the nature and causes of indirect uncertainty (i.e. uncertainty that cannot be expressed in absolute quantitative forms) and identify methods for managing this uncertainty in healthcare modelling.

Modeling Chronic Kidney Disease in Type 2 Diabetes Mellitus: A Systematic Literature Review of Models, Data Sources, and Derivation Cohorts

INTRODUCTION

As novel therapies for chronic kidney disease (CKD) in type 2 diabetes mellitus (T2DM) become available, their long-term benefits should be evaluated using CKD progression models. Existing models offer different modeling approaches that could be reused, but it may be challenging for modelers to assess commonalities and differences between the many available models. Additionally, the data and underlying population characteristics informing model parameters may not always be evident. Therefore, this study reviewed and summarized existing modeling approaches and data sources for CKD in T2DM, as a reference for future model development.

METHODS

This systematic literature review included computer simulation models of CKD in T2DM populations. Searches were implemented in PubMed (including MEDLINE), Embase, and the Cochrane Library, up to October 2021. Models were classified as cohort state-transition models (cSTM) or individual patient simulation (IPS) models. Information was extracted on modeled kidney disease states, risk equations for CKD, data sources, and baseline characteristics of derivation cohorts in primary data sources.

RESULTS

The review identified 49 models (21 IPS, 28 cSTM). A five-state structure was standard among state-transition models, comprising one kidney disease-free state, three kidney disease states [frequently including albuminuria and end-stage kidney disease (ESKD)], and one death state. Five models captured CKD regression and three included cardiovascular disease (CVD). Risk equations most commonly predicted albuminuria and ESKD incidence, while the most predicted CKD sequelae were mortality and CVD. Most data sources were well-established registries, cohort studies, and clinical trials often initiated decades ago in predominantly White populations in high-income countries. Some recent models were developed from country-specific data, particularly for Asian countries, or from clinical outcomes trials.

CONCLUSION

Modeling CKD in T2DM is an active research area, with a trend towards IPS models developed from non-Western data and single data sources, primarily recent outcomes trials of novel renoprotective treatments.

Microsimulation Model Calibration with Approximate Bayesian Computation in R: A Tutorial

Mathematical health policy models, including microsimulation models (MSMs), are widely used to simulate complex processes and predict outcomes consistent with available data. Calibration is a method to estimate parameter values such that model predictions are similar to observed outcomes of interest. Bayesian calibration methods are popular among the available calibration techniques, given their strong theoretical basis and flexibility to incorporate prior beliefs and draw values from the posterior distribution of model parameters and hence the ability to characterize and evaluate parameter uncertainty in the model outcomes. Approximate Bayesian computation (ABC) is an approach to calibrate complex models in which the likelihood is intractable, focusing on measuring the difference between the simulated model predictions and outcomes of interest in observed data. Although ABC methods are increasingly being used, there is limited practical guidance in the medical decision-making literature on approaches to implement ABC to calibrate MSMs. In this tutorial, we describe the Bayesian calibration framework, introduce the ABC approach, and provide step-by-step guidance for implementing an ABC algorithm to calibrate MSMs, using 2 case examples based on a microsimulation model for dementia. We also provide the R code for applying these methods.

Development and validation of DeciBHAL-US: A novel microsimulation model of hearing loss across the lifespan in the United States

BACKGROUND

Hearing loss affects over 50% of people in the US across their lifespan and there is a lack of decision modeling frameworks to inform optimal hearing healthcare delivery. Our objective was to develop and validate a microsimulation model of hearing loss across the lifespan in the US.

METHODS

We collaborated with the Lancet Commission on Hearing Loss to outline model structure, identify input data sources, and calibrate/validate DeciBHAL-US (Decision model of the Burden of Hearing loss Across the Lifespan). We populated the model with literature-based estimates and validated the conceptual model with key informants. We validated key model endpoints to the published literature, including: 1) natural history of sensorineural hearing loss (SNHL), 2) natural history of conductive hearing loss (CHL), and 3) the hearing loss cascade of care. We reported the coefficient of variance root mean square error (CV-RMSE), considering values ≤15% to indicate adequate fit.

FINDINGS

For SNHL prevalence, the CV-RMSE for model projected male and female age-specific prevalence compared to sex-adjusted National Health and Nutrition Examination Survey (NHANES) data was 4.9 and 5.7%, respectively. Incorporating literature-based age-related decline in SNHL, we validated mean four-frequency average hearing loss in the better ear (dB) among all persons to longitudinal data (CV-RMSE=11.3%). We validated the age-stratified prevalence of CHL to adjusted NHANES data (CV-RMSE=10.9%). We incorporated age- and severity-stratified time to first hearing aid (HA) use data and HA discontinuation data (adjusted for time-period of use) and validated to NHANES estimates on the prevalence of adult HA use (CV-RMSE=10.3%).

INTERPRETATION

Our results indicate adequate model fit to internal and external validation data. Future incorporation of cost and severity-stratified utility data will allow for cost-effectiveness analysis of US hearing healthcare interventions across the lifespan. Further research might expand the modeling framework to international settings.

FUNDING

This study was funded by the National Institute on Deafness and Other Communication Disorders and the National Institute on Aging (3UL1-TR002553-03S3 and F30 DC019846).

Simulating Study Data to Support Expected Value of Sample Information Calculations: A Tutorial

The expected value of sample information (EVSI) can be used to prioritize avenues for future research and design studies that support medical decision making and offer value for money spent. EVSI is calculated based on 3 key elements. Two of these, a probabilistic model-based economic evaluation and updating model uncertainty based on simulated data, have been frequently discussed in the literature. By contrast, the third element, simulating data from the proposed studies, has received little attention. This tutorial contributes to bridging this gap by providing a step-by-step guide to simulating study data for EVSI calculations. We discuss a general-purpose algorithm for simulating data and demonstrate its use to simulate 3 different outcome types. We then discuss how to induce correlations in the generated data, how to adjust for common issues in study implementation such as missingness and censoring, and how individual patient data from previous studies can be leveraged to undertake EVSI calculations. For all examples, we provide comprehensive code written in the R language and, where possible, Excel spreadsheets in the supplementary materials. This tutorial facilitates practical EVSI calculations and allows EVSI to be used to prioritize research and design studies.

A systematic review of the methodological quality of economic evaluations in genetic screening and testing for monogenic disorders

PURPOSE

Understanding the value of genetic screening and testing for monogenic disorders requires high-quality, methodologically robust economic evaluations. This systematic review sought to assess the methodological quality among such studies and examined opportunities for improvement.

METHODS

We searched PubMed, Cochrane, Embase, and Web of Science for economic evaluations of genetic screening/testing (2013-2019). Methodological rigor and adherence to best practices were systematically assessed using the British Medical Journal checklist.

RESULTS

Across the 47 identified studies, there were substantial variations in modeling approaches, reporting detail, and sophistication. Models ranged from simple decision trees to individual-level microsimulations that compared between 2 and >20 alternative interventions. Many studies failed to report sufficient detail to enable replication or did not justify modeling assumptions, especially for costing methods and utility values. Meta-analyses, systematic reviews, or calibration were rarely used to derive parameter estimates. Nearly all studies conducted some sensitivity analysis, and more sophisticated studies implemented probabilistic sensitivity/uncertainty analysis, threshold analysis, and value of information analysis.

CONCLUSION

We describe a heterogeneous body of work and present recommendations and exemplar studies across the methodological domains of (1) perspective, scope, and parameter selection; (2) use of uncertainty/sensitivity analyses; and (3) reporting transparency for improvement in the economic evaluation of genetic screening/testing.

Protocol for the economic evaluation of COVID-19 pandemic response policies

INTRODUCTION

Several treatment options are available for COVID-19 to date. However, the use of a combination of non-pharmaceutical interventions (NPIs) is necessary for jurisdictions to contain its spread. Although the implementation cost of NPIs may be low from the healthcare system perspective, it can be costly when considering the indirect costs from the societal perspective. COVID-19 vaccination campaigns have begun in several countries worldwide. Nonetheless, the quantity of vaccines available remain limited over the next 1 to 2 years. A tool for informing vaccine prioritisation that considers both cost and effectiveness will be highly useful. This study aims to identify the most cost-effective combination of COVID-19 response policies, using Singapore as an example.

METHODS AND ANALYSIS

An age-stratified Susceptible-Exposed-Infectious-Recovered model will be used to generate the number of infections stratified by disease severity under different intervention scenarios. Polices of interest include test-trace-isolate, travel restriction, compulsory face mask and hygiene practices, social distancing, dexamethasone/remdesivir therapy and vaccination. The latest phase 3 trial results and the WHO Target Product Profiles for COVID-19 vaccines will be used to model vaccine characteristics. A cost (expected resource utilisation and productivity losses) and quality-adjusted life years (QALYs) will be attached to these outputs for a cost-utility analysis. The primary outcome measure will be the incremental cost-effectiveness ratio generated from the incremental cost of policy alternatives expressed as a ratio of the incremental benefits (QALYs gained). Efficacy of policy options will be gathered from literature review and from its observed impacts in Singapore. Cost data will be gathered from healthcare institutions, Ministry of Health and published data. Sensitivity analysis such as threshold analysis and scenario analysis will be conducted.

ETHICS AND DISSEMINATION

Ethics approval was not required for this study. The study findings will be disseminated through peer-reviewed journals.

BayCANN: Streamlining Bayesian Calibration With Artificial Neural Network Metamodeling

Purpose: Bayesian calibration is generally superior to standard direct-search algorithms in that it estimates the full joint posterior distribution of the calibrated parameters. However, there are many barriers to using Bayesian calibration in health decision sciences stemming from the need to program complex models in probabilistic programming languages and the associated computational burden of applying Bayesian calibration. In this paper, we propose to use artificial neural networks (ANN) as one practical solution to these challenges. Methods: Bayesian Calibration using Artificial Neural Networks (BayCANN) involves (1) training an ANN metamodel on a sample of model inputs and outputs, and (2) then calibrating the trained ANN metamodel instead of the full model in a probabilistic programming language to obtain the posterior joint distribution of the calibrated parameters. We illustrate BayCANN using a colorectal cancer natural history model. We conduct a confirmatory simulation analysis by first obtaining parameter estimates from the literature and then using them to generate adenoma prevalence and cancer incidence targets. We compare the performance of BayCANN in recovering these "true" parameter values against performing a Bayesian calibration directly on the simulation model using an incremental mixture importance sampling (IMIS) algorithm. Results: We were able to apply BayCANN using only a dataset of the model inputs and outputs and minor modification of BayCANN's code. In this example, BayCANN was slightly more accurate in recovering the true posterior parameter estimates compared to IMIS. Obtaining the dataset of samples, and running BayCANN took 15 min compared to the IMIS which took 80 min. In applications involving computationally more expensive simulations (e.g., microsimulations), BayCANN may offer higher relative speed gains. Conclusions: BayCANN only uses a dataset of model inputs and outputs to obtain the calibrated joint parameter distributions. Thus, it can be adapted to models of various levels of complexity with minor or no change to its structure. In addition, BayCANN's efficiency can be especially useful in computationally expensive models. To facilitate BayCANN's wider adoption, we provide BayCANN's open-source implementation in R and Stan.

Bayesian versus Empirical Calibration of Microsimulation Models: A Comparative Analysis

Calibration of a microsimulation model (MSM) is a challenging but crucial step for the development of a valid model. Numerous calibration methods for MSMs have been suggested in the literature, most of which are usually adjusted to the specific needs of the model and based on subjective criteria for the selection of optimal parameter values. This article compares 2 general approaches for calibrating MSMs used in medical decision making, a Bayesian and an empirical approach. We use as a tool the MIcrosimulation Lung Cancer (MILC) model, a streamlined, continuous-time, dynamic MSM that describes the natural history of lung cancer and predicts individual trajectories accounting for age, sex, and smoking habits. We apply both methods to calibrate MILC to observed lung cancer incidence rates from the Surveillance, Epidemiology and End Results (SEER) database. We compare the results from the 2 methods in terms of the resulting parameter distributions, model predictions, and efficiency. Although the empirical method proves more practical, producing similar results with smaller computational effort, the Bayesian method resulted in a calibrated model that produced more accurate outputs for rare events and is based on a well-defined theoretical framework for the evaluation and interpretation of the calibration outcomes. A combination of the 2 approaches is an alternative worth considering for calibrating complex predictive models, such as microsimulation models.

Applied Methods for Estimating Transition Probabilities from Electronic Health Record Data

BACKGROUND

Electronic health record (EHR) data contain longitudinal patient information and standardized diagnostic codes. EHR data may be useful for estimating transition probabilities for state-transition models, but no guidelines exist on appropriate methods. We applied 3 potential methods to estimate transition probabilities from EHR data, using pediatric eating disorders (EDs) as a case study.

METHODS

We obtained EHR data from PEDsnet, which includes 8 US children's hospitals. Data included inpatient, outpatient, and emergency department visits for all patients with an ED. We mapped diagnoses to 3 ED health states: anorexia nervosa, bulimia nervosa, and other specified feeding or eating disorder. We estimated 1-y transition probabilities for males and females using 3 approaches: simple first-last proportions, a multistate Markov (MSM) model, and independent survival models.

RESULTS

Transition probability estimates varied widely between approaches. The first-last proportion approach estimated higher probabilities of remaining in the same health state, while the MSM and independent survival approaches estimated higher probabilities of transitioning to a different health state. All estimates differed substantially from published literature.

LIMITATIONS

As a source of health state information, EHR data are incomplete and sometimes inaccurate. EHR data were especially challenging for EDs, limiting the estimation and interpretation of transition probabilities.

CONCLUSIONS

The 3 approaches produced very different transition probability estimates. Estimates varied considerably from published literature and were rescaled and calibrated for use in a microsimulation model. Estimation of transition probabilities from EHR data may be more promising for diseases that are well documented in the EHR. Furthermore, clinicians and health systems should work to improve documentation of ED in the EHR. Further research is needed on methods for using EHR data to inform transition probabilities.

Modeling Obesity in Norway (The MOON Study): A Decision-Analytic Approach-Prevalence, Costs, and Years of Life Lost

BACKGROUND

Limited knowledge exists on the expected long-term effects and cost-effectiveness of initiatives aiming to reduce the burden of obesity.

AIM

To develop a Norwegian obesity-focused disease-simulation model: the MOON model.

MATERIAL AND METHODS

We developed a Markov model and simulated a Norwegian birth cohort's movement between the health states "normal weight,""overweight,""obese 1,""obese 2," and "dead" using a lifetime perspective. Model input was estimated using longitudinal data from health surveys and real-world data (RWD) from local and national registers (N = 99,348). The model is deterministic and probabilistic and stratified by gender. Model validity was assessed by estimating the cohort's expected prevalence, health care costs, and mortality related to overweight and obesity.

RESULTS

Throughout the cohort's life, the prevalence of overweight increased steadily and stabilized at 45% at 45 y of age. The number of obese 1 and 2 individuals peaked at age 75 y, when 44% of women and 35% of men were obese. The incremental costs per person associated with obesity was highest in older ages and, when accumulated over the lifetime, higher among women (€12,118, €9,495-€15,047) than men (€6,646, €5,252-€10,900). On average, obesity shortened the life expectancy of women/men in the whole cohort by 1.31/1.08 y. The life expectancy for normal-weight women/men at age 30 was 83.31/80.31. The life expectancy was reduced by 1.05/0.65 y if the individual was overweight, obese (2.87/2.71 y), or obese 2 (4.06/4.83 y).

CONCLUSION

The high expected prevalence of obesity in the future will lead to substantial health care costs and large losses in life-years. This underscores the need to implement interventions to reduce the burden of obesity; the MOON model will enable economic evaluations for a wide range of interventions.

Introducing cell-free DNA noninvasive testing in a Down syndrome public health screening program: a budget impact analysis

BACKGROUND

Non-invasive prenatal testing (NIPT) using cell-free fetal DNA in maternal plasma is a high accurate test for prenatal screening for Down syndrome. Although it has been reported to be cost effective as a contingent test, evidence about its budget impact is lacking.

OBJECTIVE

To evaluate, using computer simulations, the budget impact of implementing NIPT as a contingent test in the Quebec Program of screening for Trisomy 21.

METHODS

A semi-Markov analytic model built to simulate the budget impact of implementing NIPT into the current Quebec Trisomy 21 public Prenatal Screening, Serum Integrated prenatal screening (SIPS). Comparisons were made for a virtual population similar to that of expected Quebec pregnant women in 2015 in terms of size and age. Data input parameters were retrieved from a thorough literature search and in government databases, especially data from Quebec Program of screening for Trisomy 21. The 2015-2016 fiscal year budget impact was estimated from the Quebec healthcare system perspective and was expressed as the difference in the overall costs between the two alternatives (SIPS minus SPS + NIPT).

RESULTS

Our study found that, at a baseline cost for NIPT of CAD$ 795, NIPT as a second-tier test offered to high-risk women identified by current screening program (SIPS + NIPT) may be affordable for Quebec health care system. Compared to the current screening program, it would be implemented at a neutral cost, considering a modest annual savings of $ 80,432 (95% CI $ 79, $ 874-$ 81,462). Results were sensitive to the NIPT costs and the uptake-rate of invasive diagnostic tests.

CONCLUSION

Introducing NIPT as a contingent test in the Quebec Trisomy 21 screening program is an affordable strategy compared to the current practice. Further research is needed to confirm if our results can be reproduced in other healthcare jurisdictions.

Evidence-Based Medicine in Otolaryngology, Part XI: Modeling and Analysis to Support Decisions

OBJECTIVE

To provide a resource to educate clinical decision makers about the analyses and models that can be employed to support data-driven choices.

DATA SOURCES

Published studies and literature regarding decision analysis, decision trees, and models used to support clinical decisions.

REVIEW METHODS

Decision models provide insights into the evidence and its implications for those who make choices about clinical care and resource allocation. Decision models are designed to further our understanding and allow exploration of the common problems that we face, with parameters derived from the best available evidence. Analysis of these models demonstrates critical insights and uncertainties surrounding key problems via a readily interpretable yet quantitative format. This 11th installment of the Evidence-Based Medicine in Otolaryngology series thus provides a step-by-step introduction to decision models, their typical framework, and favored approaches to inform data-driven practice for patient-level decisions, as well as comparative assessments of proposed health interventions for larger populations.

CONCLUSIONS

Information to support decisions may arise from tools such as decision trees, Markov models, microsimulation models, and dynamic transmission models. These data can help guide choices about competing or alternative approaches to health care.

IMPLICATIONS FOR PRACTICE

Methods have been developed to support decisions based on data. Understanding the related techniques may help promote an evidence-based approach to clinical management and policy.

Estimating the impact of treatment and imaging modalities on 5-year net survival of 11 cancers in 200 countries: a simulation-based analysis

BACKGROUND

Accurate survival estimates are important for cancer control planning. Although observed survival estimates are unavailable for many countries, where they are available, wide variations are reported. Understanding the impact of specific treatment and imaging modalities can help decision makers to effectively allocate resources to improve cancer survival in their local context.

METHODS

We developed a microsimulation model of stage-specific cancer survival in 200 countries and territories for 11 cancers (oesophagus, stomach, colon, rectum, anus, liver, pancreas, lung, breast, cervix uteri, and prostate) comprising 60% of global diagnosed cancer cases. The model accounts for country-specific availability of treatment (chemotherapy, surgery, radiotherapy, and targeted therapy) and imaging modalities (ultrasound, x-ray, CT, MRI, PET, single-photon emission CT), as well as quality of care. We calibrated the model to reported survival estimates from CONCORD-3 (which reports global trends in cancer survival in 2000-14). We estimated 5-year net survival for diagnosed cancers in each country or territory and estimated potential survival gains from increasing the availability of individual treatment and imaging modalities, and more comprehensive packages of scale-up of these interventions. We report the mean and 95% uncertainty intervals (UIs) for all outcomes, calculated as the 2·5 and 97·5 percentiles of the simulation results.

FINDINGS

The estimated global 5-year net survival for all 11 cancers combined is 42·6% (95% uncertainty interval 40·3-44·3), with survival in high-income countries being an average of 12 times (range 4-17) higher than that in low-income countries. Expanding availability of surgery or radiotherapy or improving quality of care would yield the largest survival gains in low-income (2·5-3·4 percentage point increase in survival) and lower-middle-income countries (2·4-6·1 percentage point increase), whereas upper-middle-income and high-income countries are more likely to benefit from improved availability of targeted therapy (0·7 percentage point increase for upper-middle income and 0·4 percentage point increase for high income). Investing in medical imaging will also be necessary to achieve substantial survival gains, with traditional modalities estimated to provide the largest gains in low-income settings, while MRI and PET would yield the largest gains in higher-income countries. Simultaneous expansion of treatment, imaging, and quality of care could improve 5-year net survival by more than ten times in low-income countries (3·8% [95% UI 0·5-9·2] to 45·2% [40·2-52·1]) and could more than double 5-year net survival in lower-middle-income countries (20·1% [7·2-31·7] to 47·1% [42·8-50·8]).

INTERPRETATION

Scaling up both treatment and imaging availability could yield synergistic survival gains for patients with cancer. Expanding traditional modalities in lower-income settings might be a feasible pathway to improve survival before scaling up more modern technologies.

FUNDING

Harvard T H Chan School of Public Health.

The differential impacts of non-locally acquired infections and treatment interventions on heterosexual HIV transmission in Hong Kong

Introduction

Heterosexual infections have contributed to a high proportion of the HIV burden in Asia and Eastern Europe. Human mobility and non-local infections are important features in some cities/countries. An understanding of the determinants of the sustained growth of the heterosexual HIV epidemics would enable the potential impacts of treatment-based interventions to be assessed.

Methods

We developed a compartmental model for heterosexual HIV transmissions, parameterized by clinical and surveillance data (1984–2014) in Hong Kong. HIV sequence data were included for examining genetic linkages and clustering pattern. We performed sensitivity analyses to evaluate effects of high-risk sexual partnership and proportions of non-locally acquired infections. Four hypothetical interventions (a) immediate treatment, (b) enhancement of retention in care, (c) HIV testing campaigns, and (d) test-and-treat strategy, were examined.

Results

Data of 2174 patients (723 female and 1451 male) diagnosed with HIV between 1984 and 2012 in Hong Kong were collected for model parameterization. Among 1229 sequences of non-MSM (men who have sex with men) patients, 70% were isolates and 17% were either dyads or triads. In base-case scenario, the total estimated number of new infections in 2012–2023 would be 672 for male and 452 for female. Following 100% retention in care intervention, the total proportion of averted new infections in 2012–2023 would be 7% for male and 10% for female. HIV testing campaign in 2012 and 2017 followed by 100% immediate treatment strategy would avert 5% and 9% of male and female new infections, respectively. In the epidemic model, an increase of high-risk sexual partnership from 6% to 9% would increase the epidemic growth (annual number of newly diagnosed and newly infected cases) by about 10%. If no non-locally acquired infection occurred as from 2012, the epidemic growth would slump. To control the heterosexual epidemic, periodic HIV testing at 5-year intervals with immediate treatment would avert 5–13% of annual new infections in 2013–2023.

Conclusions

Enhanced HIV testing with immediate treatment is most effective in controlling the heterosexual epidemic, the impacts of which might however be attenuated by any increase of non-locally acquired infection, assuming little variations of high risk partnership over time.

Methods for Model Calibration under High Uncertainty: Modeling Cholera in Bangladesh

Background. Published data on a disease do not always correspond directly to the parameters needed to simulate natural history. Several calibration methods have been applied to computer-based disease models to extract needed parameters that make a model's output consistent with available data. Objective. To assess 3 calibration methods and evaluate their performance in a real-world application. Methods. We calibrated a model of cholera natural history in Bangladesh, where a lack of active surveillance biases available data. We built a cohort state-transition cholera natural history model that includes case hospitalization to reflect the passive surveillance data-generating process. We applied 3 calibration techniques: incremental mixture importance sampling, sampling importance resampling, and random search with rejection sampling. We adapted these techniques to the context of wide prior uncertainty and many degrees of freedom. We evaluated the resulting posterior parameter distributions using a range of metrics and compared predicted cholera burden estimates. Results. All 3 calibration techniques produced posterior distributions with a higher likelihood and better fit to calibration targets as compared with prior distributions. Incremental mixture importance sampling resulted in the highest likelihood and largest number of unique parameter sets to better inform joint parameter uncertainty. Compared with naïve uncalibrated parameter sets, calibrated models of cholera in Bangladesh project substantially more cases, many of which are not detected by passive surveillance, and fewer deaths. Limitations. Calibration cannot completely overcome poor data quality, which can leave some parameters less well informed than others. Calibration techniques may perform differently under different circumstances. Conclusions. Incremental mixture importance sampling, when adapted to the context of high uncertainty, performs well. By accounting for biases in data, calibration can improve model projections of disease burden.

Simulation models for transmission of health care-associated infection: A systematic review

BACKGROUND

Health care-associated infections (HAIs) are a global health burden because of their significant impact on patient health and health care systems. Mechanistic simulation modeling that captures the dynamics between patients, pathogens, and the environment is increasingly being used to improve understanding of epidemiological patterns of HAIs and to facilitate decisions on infection prevention and control (IPC). The purpose of this review is to present a systematic review to establish (1) how simulation models have been used to investigate HAIs and their mitigation and (2) how these models have evolved over time, as well as identify (3) gaps in their adoption and (4) useful directions for their future development.

METHODS

The review involved a systematic search and identification of studies using system dynamics, discrete event simulation, and agent-based model to study HAIs.

RESULTS

The complexity of simulation models developed for HAIs significantly increased but heavily concentrated on transmission dynamics of methicillin-resistant Staphylococcus aureus in the hospitals of high-income countries. Neither HAIs in other health care settings, the influence of contact networks within a health care facility, nor patient sharing and referring networks across health care settings were sufficiently understood.

CONCLUSIONS

This systematic review provides a broader overview of existing simulation models in HAIs to identify the gaps and to direct and facilitate further development of appropriate models in this emerging field.

The cost-effectiveness of conducting three versus two reverse transcription-polymerase chain reaction tests for diagnosing and discharging people with COVID-19: evidence from the epidemic in Wuhan, China

OBJECTIVES

The objectives were to evaluate the effectiveness of conducting three versus two reverse transcription-PCR (RT-PCR) tests for diagnosing and discharging people with COVID-19 with regard to public health and clinical impacts by incorporating asymptomatic and presymptomatic infection and to compare the medical costs associated with the two strategies.

METHODS

A model that consisted of six compartments was built. The compartments were the susceptible (S), the asymptomatic infective (A), the presymptomatic infective (L), the symptomatic infective (I), the recovered (R), and the deceased (D). The A, L and I classes were infective states. To construct the model, several parameters were set as fixed using existing evidence and the rest of the parameters were estimated by fitting the model to a smoothed curve of the cumulative confirmed cases in Wuhan from 24 January 2020 to 6 March 2020. Input data about the cost-effectiveness analysis were retrieved from the literature.

RESULTS

Conducting RT-PCR tests three times for diagnosing and discharging people with COVID-19 reduced the estimated total number of symptomatic cases to 45‌ 013 from 51 144 in the two-test strategy over 43 days. The former strategy also led to 850.1 quality-adjusted life years (QALYs) of health gain and a net healthcare expenditure saving of CN¥49.1 million. About 100.7 QALYs of the health gain were attributable to quality-adjusted life day difference between the strategies during the analytic period and 749.4 QALYs were attributable to years of life saved.

CONCLUSIONS

More accurate strategies and methods of testing for the control of COVID-19 may reduce both the number of infections and the total medical costs. Increasing the number of tests should be considered in regions with relatively severe epidemics when existing tests have moderate sensitivity.

Novel microsimulation model of tobacco use behaviours and outcomes: calibration and validation in a US population

BACKGROUND AND OBJECTIVE

Simulation models can project effects of tobacco use and cessation and inform tobacco control policies. Most existing tobacco models do not explicitly include relapse, a key component of the natural history of tobacco use. Our objective was to develop, calibrate and validate a novel individual-level microsimulation model that would explicitly include smoking relapse and project cigarette smoking behaviours and associated mortality risks.

METHODS

We developed the Simulation of Tobacco and Nicotine Outcomes and Policy (STOP) model, in which individuals transition monthly between tobacco use states (current/former/never) depending on rates of initiation, cessation and relapse. Simulated individuals face tobacco use-stratified mortality risks. For US women and men, we conducted cross-validation with a Cancer Intervention and Surveillance Modeling Network (CISNET) model. We then incorporated smoking relapse and calibrated cessation rates to reflect the difference between a transient quit attempt and sustained abstinence. We performed external validation with the National Health Interview Survey (NHIS) and the linked National Death Index. Comparisons were based on root-mean-square error (RMSE).

RESULTS

In cross-validation, STOP-generated projections of current/former/never smoking prevalence fit CISNET-projected data well (coefficient of variation (CV)-RMSE≤15%). After incorporating smoking relapse, multiplying the CISNET-reported cessation rates for women/men by 7.75/7.25, to reflect the ratio of quit attempts to sustained abstinence, resulted in the best approximation to CISNET-reported smoking prevalence (CV-RMSE 2%/3%). In external validation using these new multipliers, STOP-generated cumulative mortality curves for 20-year-old current smokers and never smokers each had CV-RMSE ≤1% compared with NHIS. In simulating those surveyed by NHIS in 1997, the STOP-projected prevalence of current/former/never smokers annually (1998-2009) was similar to that reported by NHIS (CV-RMSE 12%).

CONCLUSIONS

The STOP model, with relapse included, performed well when validated to US smoking prevalence and mortality. STOP provides a flexible framework for policy-relevant analysis of tobacco and nicotine product use.

Calibration of individual-based models to epidemiological data: A systematic review

Individual-based models (IBMs) informing public health policy should be calibrated to data and provide estimates of uncertainty. Two main components of model-calibration methods are the parameter-search strategy and the goodness-of-fit (GOF) measure; many options exist for each of these. This review provides an overview of calibration methods used in IBMs modelling infectious disease spread. We identified articles on PubMed employing simulation-based methods to calibrate IBMs informing public health policy in HIV, tuberculosis, and malaria epidemiology published between 1 January 2013 and 31 December 2018. Articles were included if models stored individual-specific information, and calibration involved comparing model output to population-level targets. We extracted information on parameter-search strategies, GOF measures, and model validation. The PubMed search identified 653 candidate articles, of which 84 met the review criteria. Of the included articles, 40 (48%) combined a quantitative GOF measure with an algorithmic parameter-search strategy–either an optimisation algorithm (14/40) or a sampling algorithm (26/40). These 40 articles varied widely in their choices of parameter-search strategies and GOF measures. For the remaining 44 (52%) articles, the parameter-search strategy could either not be identified (32/44) or was described as an informal, non-reproducible method (12/44). Of these 44 articles, the majority (25/44) were unclear about the GOF measure used; of the rest, only five quantitatively evaluated GOF. Only a minority of the included articles, 14 (17%) provided a rationale for their choice of model-calibration method. Model validation was reported in 31 (37%) articles. Reporting on calibration methods is far from optimal in epidemiological modelling studies of HIV, malaria and TB transmission dynamics. The adoption of better documented, algorithmic calibration methods could improve both reproducibility and the quality of inference in model-based epidemiology. There is a need for research comparing the performance of calibration methods to inform decisions about the parameter-search strategies and GOF measures.

Calibration—that is, “fitting” the model to data—is a crucial part of using mathematical models to better forecast and control the population-level spread of infectious diseases. Evidence that the mathematical model is well-calibrated improves confidence that the model provides a realistic picture of the consequences of health policy decisions. To make informed decisions, Policymakers need information about uncertainty: i.e., what is the range of likely outcomes (rather than just a single prediction). Thus, modellers should also strive to provide accurate measurements of uncertainty, both for their model parameters and for their predictions. This systematic review provides an overview of the methods used to calibrate individual-based models (IBMs) of the spread of HIV, malaria, and tuberculosis. We found that less than half of the reviewed articles used reproducible, non-subjective calibration methods. For the remaining articles, the method could either not be identified or was described as an informal, non-reproducible method. Only one-third of the articles obtained estimates of parameter uncertainty. We conclude that the adoption of better-documented, algorithmic calibration methods could improve both reproducibility and the quality of inference in model-based epidemiology.

Evaluation of survival extrapolation in immuno-oncology using multiple pre-planned data cuts: learnings to aid in model selection

BACKGROUND

Due to limited duration of follow up in clinical trials of cancer treatments, estimates of lifetime survival benefits are typically derived using statistical extrapolation methods. To justify the method used, a range of approaches have been proposed including statistical goodness-of-fit tests and comparing estimates against a previous data cut (i.e. interim data collected). In this study, we extend these approaches by presenting a range of extrapolations fitted to four pre-planned data cuts from the JAVELIN Merkel 200 (JM200) trial. By comparing different estimates of survival and goodness-of-fit as JM200 data mature, we undertook an iterative process of fitting and re-fitting survival models to retrospectively identify early indications of likely long-term survival.

METHODS

Standard and spline-based parametric models were fitted to overall survival data from each JM200 data cut. Goodness-of-fit was determined using an assessment of the estimated hazard function, information theory-based methods and objective comparisons of estimation accuracy. Best-fitting extrapolations were compared to establish which one provided the most accurate estimation, and how statistical goodness-of-fit differed.

RESULTS

Spline-based models provided the closest fit to the final JM200 data cut, though all extrapolation methods based on the earliest data cut underestimated the 'true' long-term survival (difference in restricted mean survival time [RMST] at 36 months: - 1.1 to - 0.5 months). Goodness-of-fit scores illustrated that an increasingly flexible model was favored as data matured. Given an early data cut, a more flexible model better aligned with clinical expectations could be reasonably justified using a range of metrics, including RMST and goodness-of-fit scores (which were typically within a 2-point range of the statistically 'best-fitting' model).

CONCLUSIONS

Survival estimates from the spline-based models are more aligned with clinical expectation and provided a better fit to the JM200 data, despite not exhibiting the definitively 'best' statistical goodness-of-fit. Longer-term data are required to further validate extrapolations, though this study illustrates the importance of clinical plausibility when selecting the most appropriate model. In addition, hazard-based plots and goodness-of-fit tests from multiple data cuts present useful approaches to identify when a more flexible model may be advantageous.

TRIAL REGISTRATION

JAVELIN Merkel 200 was registered with ClinicalTrials.gov as NCT02155647 on June 4, 2014.

Required efficacy for novel therapies in BCG-unresponsive non-muscle invasive bladder cancer: Do current recommendations really reflect clinically meaningful outcomes?

Background

Single‐arm trials are currently an accepted study design to investigate the efficacy of novel therapies (NT) in non‐muscle invasive bladder cancer (NMIBC) unresponsive to intravesical Bacillus Calmette‐Guérin (BCG) immunotherapy as randomized controlled trials are either unfeasible (comparator: early radical cystectomy; ERC), or unethical (comparator: placebo). To guide the design of such single‐arm trials, expert groups published recommendations for clinically meaningful outcomes. The aim of this study was to quantitatively verify the appropriateness of these recommendations.

Methods

We used a discrete event simulation framework in combination with a supercomputer to find the required efficacy at which a NT can compete with ERC when it comes to quality‐adjusted life expectancy (QALE). In total, 24 different efficacy thresholds (including the recommendations) were investigated.

Results

After ascertaining face validity with content experts, repeated verification, external validation, and calibration we considered our model valid. Both recommendations rarely showed an incremental benefit of the NT over ERC. In the most optimistic scenario, an increase in the IBCG recommendation by 10% and an increase in the FDA/AUA recommendation by 5% would yield results at which a NT could compete with ERC from a QALE perspective.

Conclusions

This simulation study demonstrated that the current recommendations regarding clinically meaningful outcomes for single‐arm trials evaluating the efficacy of NT in BCG‐unresponsive NMIBC may be too low. Based on our quantitative approach, we propose increasing these thresholds to at least 45%‐55% at 6 months and 35% at 18‐24 months (complete response rates/recurrence‐free survival) to promote the development of clinically truly meaningful NT.

Development and Calibration of a Dynamic HIV Transmission Model for 6 US Cities

Background. Heterogeneity in HIV microepidemics across US cities necessitates locally oriented, combination implementation strategies to prioritize resources. We calibrated and validated a dynamic, compartmental HIV transmission model to establish a status quo treatment scenario, holding constant current levels of care for 6 US cities. Methods. Built off a comprehensive evidence synthesis, we adapted and extended a previously published model to replicate the transmission, progression, and clinical care for each microepidemic. We identified a common set of 17 calibration targets between 2012 and 2015 and used the Morris method to select the most influential parameters for calibration. We then applied the Nelder-Mead algorithm to iteratively calibrate the model to generate 2000 best-fitting parameter sets. Finally, model projections were internally validated with a series of robustness checks and externally validated against published estimates of HIV incidence, while the face validity of 25-year projections was assessed by a Scientific Advisory Committee (SAC). Results. We documented our process for model development, calibration, and validation to maximize its transparency and reproducibility. The projected outcomes demonstrated a good fit to calibration targets, with a mean goodness-of-fit ranging from 0.0174 (New York City [NYC]) to 0.0861 (Atlanta). Most of the incidence predictions were within the uncertainty range for 5 of the 6 cities (ranging from 21% [Miami] to 100% [NYC]), demonstrating good external validity. The face validity of the long-term projections was confirmed by our SAC, showing that the incidence would decrease or remain stable in Atlanta, Los Angeles, NYC, and Seattle while increasing in Baltimore and Miami. Discussion. This exercise provides a basis for assessing the incremental value of further investments in HIV combination implementation strategies tailored to urban HIV microepidemics.

Quantifying Positive Health Externalities of Disease Control Interventions: Modeling Chikungunya and Dengue

Purpose. Health interventions can generate positive externalities not captured in traditional, single-disease cost-effectiveness analyses (CEAs), potentially biasing results. We illustrate this with the example of mosquito-borne diseases. When a particular mosquito species can transmit multiple diseases, a single-disease CEA comparing disease-specific interventions (e.g., vaccination) with interventions targeting the mosquito population (e.g., insecticide) would underestimate the insecticide's full benefits (i.e., preventing other diseases). Methods. We developed three dynamic transmission models: chikungunya, dengue, and combined chikungunya and dengue, each calibrated to disease-specific incidence and deaths in Colombia (June 2014 to December 2017). We compared the models' predictions of the incremental benefits and cost-effectiveness of an insecticide (10% efficacy), hypothetical chikungunya and dengue vaccines (40% coverage, 95% efficacy), and combinations of these interventions. Results. Model calibration yielded realistic parameters that produced close matches to disease-specific incidence and deaths. The chikungunya model predicted that vaccine would decrease the incidence of chikungunya and avert more total deaths than insecticide. The dengue model predicted that insecticide and the dengue vaccine would reduce dengue incidence and deaths, with no effect for the chikungunya vaccine. In the combined model, insecticide was more effective than either vaccine in reducing the incidence of and deaths from both diseases. In all models, the combined strategy was at least as effective as the most effective single strategy. In an illustrative CEA, the most frequently preferred strategy was vaccine in the chikungunya model, the status quo in the dengue model, and insecticide in the combined model. Limitations. There is uncertainty in the target calibration data. Conclusions. Failure to capture positive externalities can bias CEA results, especially when evaluating interventions that affect multiple diseases. Multidisease modeling is a reasonable alternative for addressing such biases.

Cost effectiveness of dengue vaccination following pre-vaccination serological screening in Sri Lanka

OBJECTIVE

This study sets an example of an economic evaluation of a model dengue vaccination strategy for Sri Lanka, following a mandatory pre-vaccination screening strategy.

METHODS

A decision analytic Markov model was developed to estimate the cost-effectiveness of a predicted dengue vaccination strategy over a time horizon of 10 years. The cost effectiveness of dengue vaccination strategy for seropositive individuals was estimated in terms of incremental cost effectiveness ratio (ICER) (cost per additional quality adjusted life-year [QALY]). District-specific ICER values and the budget impact for dengue vaccine were estimated with appropriate sensitivity analyses, also taking the variability of the pre-vaccination screening test performance into consideration.

RESULTS

The ICER for the predicted vaccination strategy following pre-vaccination screening was 4,382 USD/QALY for Sri Lanka. There was a significant regional variation in vaccine cost effectiveness. The disaggregated regional incidence of dengue and the need to perform pre-vaccination screening affects the cost effectiveness estimates significantly, where a safer version of the vaccine has the potential to become cost saving in high incidence districts.

CONCLUSIONS

The cost effectiveness of the predicted dengue vaccination strategy following pre-vaccination screening showed a significant regional variation across the districts of Sri Lanka. District-wise disease incidence and the need for pre-vaccination screening was found to be the most significant factors affecting the cost effectiveness of the vaccine.

Estimating Long-Term Drinking Patterns for People with Lifetime Alcohol Use Disorder

Background. There is a lack of data on alcohol consumption over time. This study characterizes the long-term drinking patterns of people with lifetime alcohol use disorders who have engaged in treatment or informal care. Methods. We developed multinomial logit models using the National Epidemiologic Survey on Alcohol and Related Conditions (NESARC) to estimate short-term transition probabilities (TPs) among the 4 World Health Organization drinking risk levels (low, medium, high, and very high risk) and abstinence by age, sex, and race/ethnicity. We applied an optimization algorithm to convert 3-year TPs from NESARC to 1-year TPs, then used simulated annealing to calibrate TPs to a propensity-scored matched set of participants derived from a separate 16-year study of alcohol consumption. We validated the resulting long-term TPs using NESARC-III, a cross-sectional study conducted on a different cohort. Results. Across 24 demographic groups, the 1-year probability of remaining in the same state averaged 0.93, 0.81, 0.49, 0.51, and 0.63 for abstinent, low, medium, high, and very high-risk states, respectively. After calibration to the 16-year study data (N = 420), resulting TPs produced state distributions that hit the calibration target. We find that the abstinent or low-risk states are very stable, and the annual probability of leaving the very high-risk state increases by about 20 percentage points beyond 8 years. Limitations. TPs for some demographic groups had small cell sizes. The data used to calibrate long-term TPs are based on a geographically narrow study. Conclusions. This study is the first to characterize long-term drinking patterns by combining short-term representative data with long-term data on drinking behaviors. Current research is using these patterns to estimate the long-term cost effectiveness of alcohol treatment.

Estimation of Eating Disorders Prevalence by Age and Associations With Mortality in a Simulated Nationally Representative US Cohort

IMPORTANCE

Eating disorders (EDs) are common psychiatric disorders associated with high mortality. However, data on ED disease dynamics and treatment coverage are sparse.

OBJECTIVES

To model the individual-level disease dynamics of ED from birth to age 40 years and to estimate the association of increased treatment coverage with ED-related mortality.

DESIGN, SETTING, AND PARTICIPANTS

In this decision analytical model study, an individual-level Markov state transition model was empirically calibrated in April 2019 using a Bayesian approach to synthesize available clinical and epidemiologic ED data. The simulation model was calibrated to nationally representative US survey data from 2007 and 2011. A virtual cohort of 100 000 individuals (50 000 [50%] male) was modeled from birth to age 40 years for 4 ED diagnoses: anorexia nervosa, bulimia nervosa, binge eating disorder, and other specified feeding and eating disorders.

EXPOSURES

Age-specific ED incidence and mortality rates and background (all-cause) mortality.

MAIN OUTCOMES AND MEASURES

The main outcomes were age-specific 12-month and lifetime ED prevalence and number of deaths per 100 000 general population individuals by age 40 years. The mean and 95% uncertainty intervals (UIs) of 1000 simulations, accounting for stochastic and parameter uncertainty, are reported.

RESULTS

The highest estimated mean annual prevalence of ED occurred at approximately age 21 years for both male individuals (7.4%; 95% UI, 3.5%-11.5%) and female individuals (10.3%; 95% UI, 7.0%-14.2%), with lifetime mean prevalence estimates increasing to 14.3% (95% UI, 9.7%-19.0%) for male individuals and 19.7% (95% UI, 15.8%-23.9%) for female individuals by age 40 years. Ninety-five percent of first-time cases occurred by age 25 years. Current treatment coverage averts an estimated mean of 41.7 deaths per 100 000 people (95% UI, 13.0-82.0 deaths per 100 000 people) by age 40 years, whereas increasing treatment coverage for all patients with ED could avert an estimated mean of 70.5 deaths per 100 000 people by age 40 years (95% UI, 26.0-143.0 deaths per 100 000 people).

CONCLUSIONS AND RELEVANCE

In this simulation modeling study, the estimated lifetime prevalence of ED was high, with approximately 1 in 7 male and 1 in 5 female individuals having an ED by age 40 years. The initial onset of EDs was highly concentrated during adolescence and young adulthood, suggesting that this is a critical period for prevention efforts. However, the high estimated prevalence of recurring ED later in life highlights the importance of identification and treatment of ED at older ages as well. These findings suggest that increasing treatment coverage could substantially reduce ED-related mortality.

Cost-effectiveness analysis of selective first-line use of biologics for unresectable RAS wild-type left-sided metastatic colorectal cancer

Background

Evidence from a retrospective analysis of multiple large phase iii trials suggested that primary tumour location (ptl) in RAS wild-type metastatic colorectal cancer (wtRAS mcrc) might have predictive value with respect to response to drug therapies. Recent studies also show a potential preferential benefit for epidermal growth factor inhibitors (egfris) for left-sided tumours. In the present study, we aimed to determine the incremental cost-effectiveness ratio (icer) for the first-line use of an egfri for patients with left-sided wtRAS mcrc.

Methods

We developed a state-transition model to determine the cost effectiveness of alternative treatment strategies in patients with left-sided mcrc:■ Standard of care■ Use of an egfri in first-line therapyThe cohort for the study consisted of patients diagnosed with unresectable wtRAS mcrc with an indication for chemotherapy and previously documented ptl. Model parameters were obtained from the published literature and calibration. The perspective was that of a provincial ministry of health in Canada. We used a 5-year time horizon and an annual discount rate of 1.5%.

Results

Selecting patients for first-line egfri treatment based on left-sided location of their colorectal primary tumour was more effective than the standard of care, resulting in an increase in quality-adjusted life-years (qalys) of 0.226 (or 0.644 life-years gained). However, the strategy was also more expensive, costing an average of $60,639 more per patient treated. The resulting icer was $268,094 per qaly. A 35% price reduction in the cost of egfri would be needed to make this strategy cost-effective at a willingness-to-pay threshold (wtp) of $100,000 per qaly.

Conclusions

Selective use of an egfri based on ptl was more cost-effective than unselected use of those agents; however, based on traditional wtp thresholds, it was still not cost-effective. While awaiting the elucidation of more precise predictive biomarkers that might improve cost-effectiveness, the price of egfris could be reduced to meet the wtp threshold.

Multiobjective Calibration of Disease Simulation Models Using Gaussian Processes

Background. Developing efficient procedures of model calibration, which entails matching model predictions to observed outcomes, has gained increasing attention. With faithful but complex simulation models established for cancer diseases, key parameters of cancer natural history can be investigated for possible fits, which can subsequently inform optimal prevention and treatment strategies. When multiple calibration targets exist, one approach to identifying optimal parameters relies on the Pareto frontier. However, computational burdens associated with higher-dimensional parameter spaces require a metamodeling approach. The goal of this work is to explore multiobjective calibration using Gaussian process regression (GPR) with an eye toward how multiple goodness-of-fit (GOF) criteria identify Pareto-optimal parameters. Methods. We applied GPR, a metamodeling technique, to estimate colorectal cancer (CRC)-related prevalence rates simulated from a microsimulation model of CRC natural history, known as the Colon Modeling Open Source Tool (CMOST). We embedded GPR metamodels within a Pareto optimization framework to identify best-fitting parameters for age-, adenoma-, and adenoma staging-dependent transition probabilities and risk factors. The Pareto frontier approach is demonstrated using genetic algorithms with both sum-of-squared errors (SSEs) and Poisson deviance GOF criteria. Results. The GPR metamodel is able to approximate CMOST outputs accurately on 2 separate parameter sets. Both GOF criteria are able to identify different best-fitting parameter sets on the Pareto frontier. The SSE criterion emphasizes the importance of age-specific adenoma progression parameters, while the Poisson criterion prioritizes adenoma-specific progression parameters. Conclusion. Different GOF criteria assert different components of the CRC natural history. The combination of multiobjective optimization and nonparametric regression, along with diverse GOF criteria, can advance the calibration process by identifying optimal regions of the underlying parameter landscape.

Combined estimation of disease progression and retention on antiretroviral therapy among treated individuals with HIV in the USA: a modelling study

BACKGROUND

Accurately estimating HIV disease progression and retention on antiretroviral therapy (ART) can help inform interventions to control HIV microepidemics and mathematical models used to inform health-resource allocation decisions. Our objective was to estimate the monthly probabilities of on-ART CD4 T-cell count progression, mortality, ART dropout, and ART reinitiation using a continuous-time multistate Markov model. We also aimed to validate health-state transition probability estimates to ensure they accurately reproduced the regional HIV microepidemics across the USA.

METHODS

In our modelling study, we considered a cohort of patients from the HIV Research Network, a consortium of 17 adult and paediatric HIV-care providers located in the northeastern (n=8), southern (n=5), and western (n=4) regions of the USA. Individuals aged 15 years or older who were in HIV care (defined as one CD4 test and one HIV-care visit in a calendar year period) with at least one ART prescription between Jan 1, 2010, and Dec 31, 2015, were included in the analysis. We used continuous-time multistate Markov models to estimate transitions between CD4 strata and between on-ART and off-ART states. We examined and adjusted for differences in probability of transition by region, race or ethnicity, sex, HIV risk group, and other baseline clinical indicators.

FINDINGS

The median age of the 32 242 individuals included in the analysis was 44 years (interquartile range 35-51). Over a median follow-up of 4·9 years (2·6-6·0), 8614 (26·7%) of 32 242 people interrupted ART and 1325 (4·1%) of 32 242 people died. Women, men who have sex with men, and individuals with no previous ART experience had greater increases in CD4 cell counts, whereas black people and people who inject drugs had increased probabilities of ART dropout and faster disease progression. Regardless of CD4 strata, individuals had increased hazard for ART dropout if they were from the south (adjusted hazard ratio [aHR] range from 1·91, 95% CI 1·71-2·13, to 2·45, 2·29-2·62) or the west (aHR range from 1·29, 1·10-1·51, to 1·66, 1·51-1·82) of the USA, compared with individuals from the northeast USA.

INTERPRETATION

Our results show heterogeneities in disease progression during ART and probability of ART retention across race and ethnicity, HIV risk groups, and regions. These differences should be viewed as targets for intervention and should be incorporated in mathematical models of regional HIV microepidemics in the USA.

FUNDING

US National Institutes of Health, Agency for Healthcare Research and Quality, and Health Resources and Services Administration.

The importance of accounting for the uncertainty around the preference-based health-related quality-of-life measures value sets: a systematic review

Preference-based measures of health-related quality-of-life including, but not limited to, the EQ-5D, HUI2 and the SF-6D have been increasingly used in calculations of quality-adjusted life years for cost effectiveness analyses. However, the uncertainty around the measures' value sets is commonly ignored in economic evaluation. There are several types of uncertainties, including methodological, structural, and parameter uncertainties, with the latter being the focus of this review paper. The objective is to highlight the gap in the literature regarding the existence of uncertainty in the value sets, focusing mainly on the EQ-5D and SF-6D. To the best of the authors' knowledge, this is the first systematic review revolving around uncertainty. After searching extensively for studies involving uncertainties in all preference-based measures, the results showed that uncertainty has been approached through different means, while parameter uncertainty has been ignored in most, if not all, cases. These findings suggest that uncertainty should be accounted for when using preference-based measures in economic evaluations. Ignoring this additional information could impact misleadingly on policy decisions.

Nonidentifiability in Model Calibration and Implications for Medical Decision Making

BACKGROUND

Calibration is the process of estimating parameters of a mathematical model by matching model outputs to calibration targets. In the presence of nonidentifiability, multiple parameter sets solve the calibration problem, which may have important implications for decision making. We evaluate the implications of nonidentifiability on the optimal strategy and provide methods to check for nonidentifiability.

METHODS

We illustrate nonidentifiability by calibrating a 3-state Markov model of cancer relative survival (RS). We performed 2 different calibration exercises: 1) only including RS as a calibration target and 2) adding the ratio between the 2 nondeath states over time as an additional target. We used the Nelder-Mead (NM) algorithm to identify parameter sets that best matched the calibration targets. We used collinearity and likelihood profile analyses to check for nonidentifiability. We then estimated the benefit of a hypothetical treatment in terms of life expectancy gains using different, but equally good-fitting, parameter sets. We also applied collinearity analysis to a realistic model of the natural history of colorectal cancer.

RESULTS

When only RS is used as the calibration target, 2 different parameter sets yield similar maximum likelihood values. The high collinearity index and the bimodal likelihood profile on both parameters demonstrated the presence of nonidentifiability. These different, equally good-fitting parameter sets produce different estimates of the treatment effectiveness (0.67 v. 0.31 years), which could influence the optimal decision. By incorporating the additional target, the model becomes identifiable with a collinearity index of 3.5 and a unimodal likelihood profile.

CONCLUSIONS

In the presence of nonidentifiability, equally likely parameter estimates might yield different conclusions. Checking for the existence of nonidentifiability and its implications should be incorporated into standard model calibration procedures.