A Call for Open-Source Cost-Effectiveness Analysis

Transparency, openness, and reproducible research practices are frequently underused in health economic evaluations

OBJECTIVES

To investigate the extent to which articles of economic evaluations of healthcare interventions indexed in MEDLINE incorporate research practices that promote transparency, openness, and reproducibility.

STUDY DESIGN AND SETTING

We evaluated a random sample of health economic evaluations indexed in MEDLINE during 2019. We included articles written in English reporting an incremental cost-effectiveness ratio in terms of costs per life years gained, quality-adjusted life years, and/or disability-adjusted life years. Reproducible research practices, openness, and transparency in each article were extracted in duplicate. We explored whether reproducible research practices were associated with self-report use of a guideline.

RESULTS

We included 200 studies published in 147 journals. Almost half were published as open access articles (n = 93; 47%). Most studies (n = 150; 75%) were model-based economic evaluations. In 109 (55%) studies, authors self-reported use a guideline (e.g., for study conduct or reporting). Few studies (n = 31; 16%) reported working from a protocol. In 112 (56%) studies, authors reported the data needed to recreate the incremental cost-effectiveness ratio for the base case analysis. This percentage was higher in studies using a guideline than studies not using a guideline (72/109 [66%] with guideline vs. 40/91 [44%] without guideline; risk ratio 1.50, 95% confidence interval 1.15-1.97). Only 10 (5%) studies mentioned access to raw data and analytic code for reanalyses.

CONCLUSION

Transparency, openness, and reproducible research practices are frequently underused in health economic evaluations. This study provides baseline data to compare future progress in the field.

Developing Criteria for Health Economic Quality Evaluation Tool

OBJECTIVES

Because existing publication guidelines and checklists have limitations when used to assess the quality of cost-effectiveness analysis, we developed a novel quality assessment tool for cost-effectiveness analyses, differentiating methods and reporting quality and incorporating the relative importance of different quality attributes.

METHODS

We defined 15 quality domains from a scoping review and identified 72 methods and reporting quality attributes (36 each). After designing a best-worst scaling survey, we fielded an online survey to researchers and practitioners to estimate the relative importance of the attributes in February 2021. We analyzed the survey data using a sequential conditional logit model. The final tool included 48 quality attributes deemed most important for assessing methods and reporting quality (24 each), accompanied by a free and web-based scoring system.

RESULTS

A total of 524 participants completed the methodology section, and 372 completed both methodology and reporting sections. Quality attributes pertaining to the "modeling" and "data inputs and evidence synthesis" domains were deemed most important for methods quality, including "structure of the model reflects the underlying condition and intervention's impact" and "model validation is conducted." Quality attributes pertaining to "modeling" and "Intervention/comparator(s)" domains were considered most important for reporting quality, including "model descriptions are detailed enough for replication." Despite its growing prominence, "equity considerations" were not deemed as important as other quality attributes.

CONCLUSIONS

The Criteria for Health Economic Quality Evaluation tool allows users to differentiate methods and reporting as well as quantifies the relative importance of quality attributes. Alongside other considerations, it could help assess and improve the quality of cost-effectiveness evidence to inform value-based decisions.

Consolidated Health Economic Evaluation Reporting Standards 2022 (CHEERS 2022) statement: updated reporting guidance for health economic evaluations

Health economic evaluations are comparative analyses of alternative courses of action in terms of their costs and consequences. The Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement, published in 2013, was created to ensure health economic evaluations are identifiable, interpretable, and useful for decision making. It was intended as guidance to help authors report accurately which health interventions were being compared and in what context, how the evaluation was undertaken, what the findings were, and other details that may aid readers and reviewers in interpretation and use of the study. The new CHEERS 2022 statement replaces previous CHEERS reporting guidance. It reflects the need for guidance that can be more easily applied to all types of health economic evaluation, new methods and developments in the field, as well as the increased role of stakeholder involvement including patients and the public. It is also broadly applicable to any form of intervention intended to improve the health of individuals or the population, whether simple or complex, and without regard to context (such as health care, public health, education, social care, etc.). This summary article presents the new CHEERS 2022 28-item checklist and recommendations for each item. The CHEERS 2022 statement is primarily intended for researchers reporting economic evaluations for peer reviewed journals as well as the peer reviewers and editors assessing them for publication. However, we anticipate familiarity with reporting requirements will be useful for analysts when planning studies. It may also be useful for health technology assessment bodies seeking guidance on reporting, as there is an increasing emphasis on transparency in decision making.

Value-based evidence across health care sectors: a push for transparent real-world studies, data, and evidence dissemination

There is currently a heightened need for transparency in pharmaceutical sectors. The inclusion of real-world (RW) evidence, in addition to clinical trial evidence, in decision-making processes, was an important step forward toward a more inclusive established value proposition. This advance has introduced new transparency challenges. Increasing transparency is a critical step toward accelerating improvement in type, quality, and access to data, regardless of whether these originate from clinical trials or from RW studies. However, so far, advances in transparency have been relatively restricted to clinical trials, and there remains a lack of similar expectations or standards of transparency concerning the generation and reporting of RW data. This perspective paper aims to highlight the need for transparency concerning RW studies, data, and evidence across health care sectors, to identify areas for improvement, and provide concrete recommendations and practices for the future. Specific issues are discussed from different stakeholder perspectives, culminating in recommended actions, from individual stakeholder perspectives, for improved RW study, data, and evidence transparency. Furthermore, a list of potential guidelines for consideration by stakeholders is proposed. While recommendations from different stakeholder perspectives are made, true transparency in the processes involved in the generation, reporting, and use of RW evidence will require a concerted effort from all stakeholders across health care sectors.

Consolidated Health Economic Evaluation Reporting Standards 2022 (CHEERS 2022) Statement: Updated Reporting Guidance for Health Economic Evaluations

Health economic evaluations are comparative analyses of alternative courses of action in terms of their costs and consequences. The Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement, published in 2013, was created to ensure health economic evaluations are identifiable, interpretable, and useful for decision making. It was intended as guidance to help authors report accurately which health interventions were being compared and in what context, how the evaluation was undertaken, what the findings were, and other details that may aid readers and reviewers in interpretation and use of the study. The new CHEERS 2022 statement replaces previous CHEERS reporting guidance. It reflects the need for guidance that can be more easily applied to all types of health economic evaluation, new methods and developments in the field, and the increased role of stakeholder involvement, including patients and the public. It is also broadly applicable to any form of intervention intended to improve the health of individuals or the population, whether simple or complex, and without regard to context (such as healthcare, public health, education, social care, etc.). This summary article presents the new CHEERS 2022 28-item checklist and recommendations for each item. The CHEERS 2022 statement is primarily intended for researchers reporting economic evaluations for peer-reviewed journals as well as the peer reviewers and editors assessing them for publication. However, we anticipate familiarity with reporting requirements will be useful for analysts when planning studies. It may also be useful for health technology assessment bodies seeking guidance on reporting, as there is an increasing emphasis on transparency in decision making.

Opportunities and Barriers to the Development and Use of Open Source Health Economic Models: A Survey

OBJECTIVES

Health economic (HE) models are routinely used to support health policy and resource allocation decisions but are often considered "black boxes" that may be prone to error and bias. Open source models (OSMs) have been advocated to increase the transparency, credibility, and reuse of HE models. Previous studies have demonstrated interest in OSMs among the health economics and outcomes research community, but the number of OSMs remains low.

METHODS

We conducted an online survey of ISPOR (the leading professional society for health economics and outcomes research) members' perspectives on the usefulness of OSMs and barriers to their development and implementation.

RESULTS

Respondents (N = 230) included academics (27%), pharmaceutical (or related) industry representatives (23%), health research or consulting representatives (21%), governmental or nonprofit agency representatives (10%), and others (19%). Respondents were generally not familiar with barriers to the development and adoption of OSMs. Most agreed that OSMs would improve transparency (92%), efficiency (76%), and HE model reuse (86%) and promote confidence in using HE models (75%). The use of OSMs by health technology assessment authorities was considered a very important indicator of the usefulness of OSMs by 49% of respondents. Three-quarters of respondents perceived legal concerns and the ability to transfer data as important barriers to the development and use of OSMs.

CONCLUSIONS

Respondents believe that OSMs could increase the transparency, efficiency, and credibility of HE models, but that several barriers hamper their widespread adoption. Our results suggest that fundamental changes may be needed across the health economics and outcomes research community if OSMs are to become widely adopted.

Consolidated Health Economic Evaluation Reporting Standards 2022 (CHEERS 2022) Statement: Updated Reporting Guidance for Health Economic Evaluations

Health economic evaluations are comparative analyses of alternative courses of action in terms of their costs and consequences. The Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement, published in 2013, was created to ensure health economic evaluations are identifiable, interpretable, and useful for decision making. It was intended as guidance to help authors report accurately which health interventions were being compared and in what context, how the evaluation was undertaken, what the findings were, and other details that may aid readers and reviewers in interpretation and use of the study. The new CHEERS 2022 statement replaces previous CHEERS reporting guidance. It reflects the need for guidance that can be more easily applied to all types of health economic evaluation, new methods and developments in the field, as well as the increased role of stakeholder involvement including patients and the public. It is also broadly applicable to any form of intervention intended to improve the health of individuals or the population, whether simple or complex, and without regard to context (such as health care, public health, education, social care, etc). This summary article presents the new CHEERS 2022 28-item checklist and recommendations for each item. The CHEERS 2022 statement is primarily intended for researchers reporting economic evaluations for peer reviewed journals as well as the peer reviewers and editors assessing them for publication. However, we anticipate familiarity with reporting requirements will be useful for analysts when planning studies. It may also be useful for health technology assessment bodies seeking guidance on reporting, as there is an increasing emphasis on transparency in decision making.

Consolidated Health Economic Evaluation Reporting Standards 2022 (CHEERS 2022) statement: updated reporting guidance for health economic evaluations

Health economic evaluations are comparative analyses of alternative courses of action in terms of their costs and consequences. The Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement, published in 2013, was created to ensure health economic evaluations are identifiable, interpretable, and useful for decision making. It was intended as guidance to help authors report accurately which health interventions were being compared and in what context, how the evaluation was undertaken, what the findings were, and other details that may aid readers and reviewers in interpretation and use of the study. The new CHEERS 2022 statement replaces previous CHEERS reporting guidance. It reflects the need for guidance that can be more easily applied to all types of health economic evaluation, new methods and developments in the field, as well as the increased role of stakeholder involvement including patients and the public. It is also broadly applicable to any form of intervention intended to improve the health of individuals or the population, whether simple or complex, and without regard to context (such as health care, public health, education, social care, etc). This summary article presents the new CHEERS 2022 28-item checklist and recommendations for each item. The CHEERS 2022 statement is primarily intended for researchers reporting economic evaluations for peer reviewed journals as well as the peer reviewers and editors assessing them for publication. However, we anticipate familiarity with reporting requirements will be useful for analysts when planning studies. It may also be useful for health technology assessment bodies seeking guidance on reporting, as there is an increasing emphasis on transparency in decision making.

Consolidated Health Economic Evaluation Reporting Standards 2022 (CHEERS 2022) statement: updated reporting guidance for health economic evaluations

Health economic evaluations are comparative analyses of alternative courses of action in terms of their costs and consequences. The Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement, published in 2013, was created to ensure health economic evaluations are identifiable, interpretable, and useful for decision making. It was intended as guidance to help authors report accurately which health interventions were being compared and in what context, how the evaluation was undertaken, what the findings were, and other details that may aid readers and reviewers in interpretation and use of the study. The new CHEERS 2022 statement replaces previous CHEERS reporting guidance. It reflects the need for guidance that can be more easily applied to all types of health economic evaluation, new methods and developments in the field, as well as the increased role of stakeholder involvement including patients and the public. It is also broadly applicable to any form of intervention intended to improve the health of individuals or the population, whether simple or complex, and without regard to context (such as health care, public health, education, social care, etc). This summary article presents the new CHEERS 2022 28-item checklist and recommendations for each item. The CHEERS 2022 statement is primarily intended for researchers reporting economic evaluations for peer reviewed journals as well as the peer reviewers and editors assessing them for publication. However, we anticipate familiarity with reporting requirements will be useful for analysts when planning studies. It may also be useful for health technology assessment bodies seeking guidance on reporting, as there is an increasing emphasis on transparency in decision making.

Consolidated Health Economic Evaluation Reporting Standards 2022 (CHEERS 2022) statement: updated reporting guidance for health economic evaluations

Health economic evaluations are comparative analyses of alternative courses of action in terms of their costs and consequences. The Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement, published in 2013, was created to ensure health economic evaluations are identifiable, interpretable, and useful for decision making. It was intended as guidance to help authors report accurately which health interventions were being compared and in what context, how the evaluation was undertaken, what the findings were, and other details that may aid readers and reviewers in interpretation and use of the study. The new CHEERS 2022 statement replaces previous CHEERS reporting guidance. It reflects the need for guidance that can be more easily applied to all types of health economic evaluation, new methods and developments in the field, as well as the increased role of stakeholder involvement including patients and the public. It is also broadly applicable to any form of intervention intended to improve the health of individuals or the population, whether simple or complex, and without regard to context (such as health care, public health, education, social care, etc). This summary article presents the new CHEERS 2022 28-item checklist and recommendations for each item. The CHEERS 2022 statement is primarily intended for researchers reporting economic evaluations for peer reviewed journals as well as the peer reviewers and editors assessing them for publication. However, we anticipate familiarity with reporting requirements will be useful for analysts when planning studies. It may also be useful for health technology assessment bodies seeking guidance on reporting, as there is an increasing emphasis on transparency in decision making.

Consolidated Health Economic Evaluation Reporting Standards 2022 (CHEERS 2022) statement: updated reporting guidance for health economic evaluations

Health economic evaluations are comparative analyses of alternative courses of action in terms of their costs and consequences. The Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement, published in 2013, was created to ensure health economic evaluations are identifiable, interpretable, and useful for decision making. It was intended as guidance to help authors report accurately which health interventions were being compared and in what context, how the evaluation was undertaken, what the findings were, and other details that may aid readers and reviewers in interpretation and use of the study. The new CHEERS 2022 statement replaces previous CHEERS reporting guidance. It reflects the need for guidance that can be more easily applied to all types of health economic evaluation, new methods and developments in the field, as well as the increased role of stakeholder involvement including patients and the public. It is also broadly applicable to any form of intervention intended to improve the health of individuals or the population, whether simple or complex, and without regard to context (such as health care, public health, education, social care, etc). This summary article presents the new CHEERS 2022 28-item checklist and recommendations for each item. The CHEERS 2022 statement is primarily intended for researchers reporting economic evaluations for peer reviewed journals as well as the peer reviewers and editors assessing them for publication. However, we anticipate familiarity with reporting requirements will be useful for analysts when planning studies. It may also be useful for health technology assessment bodies seeking guidance on reporting, as there is an increasing emphasis on transparency in decision making.

Consolidated Health Economic Evaluation Reporting Standards 2022 (CHEERS 2022) statement: updated reporting guidance for health economic evaluations

Health economic evaluations are comparative analyses of alternative courses of action in terms of their costs and consequences. The Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement, published in 2013, was created to ensure health economic evaluations are identifiable, interpretable, and useful for decision making. It was intended as guidance to help authors report accurately which health interventions were being compared and in what context, how the evaluation was undertaken, what the findings were, and other details that may aid readers and reviewers in interpretation and use of the study. The new CHEERS 2022 statement replaces previous CHEERS reporting guidance. It reflects the need for guidance that can be more easily applied to all types of health economic evaluation, new methods and developments in the field, as well as the increased role of stakeholder involvement including patients and the public. It is also broadly applicable to any form of intervention intended to improve the health of individuals or the population, whether simple or complex, and without regard to context (such as health care, public health, education, social care, etc). This summary article presents the new CHEERS 2022 28-item checklist and recommendations for each item. The CHEERS 2022 statement is primarily intended for researchers reporting economic evaluations for peer reviewed journals as well as the peer reviewers and editors assessing them for publication. However, we anticipate familiarity with reporting requirements will be useful for analysts when planning studies. It may also be useful for health technology assessment bodies seeking guidance on reporting, as there is an increasing emphasis on transparency in decision making.

Consolidated Health Economic Evaluation Reporting Standards 2022 (CHEERS 2022) statement: updated reporting guidance for health economic evaluations

Health economic evaluations are comparative analyses of alternative courses of action in terms of their costs and consequences. The Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement, published in 2013, was created to ensure health economic evaluations are identifiable, interpretable, and useful for decision making. It was intended as guidance to help authors report accurately which health interventions were being compared and in what context, how the evaluation was undertaken, what the findings were, and other details that may aid readers and reviewers in interpretation and use of the study. The new CHEERS 2022 statement replaces previous CHEERS reporting guidance. It reflects the need for guidance that can be more easily applied to all types of health economic evaluation, new methods and developments in the field, as well as the increased role of stakeholder involvement including patients and the public. It is also broadly applicable to any form of intervention intended to improve the health of individuals or the population, whether simple or complex, and without regard to context (such as health care, public health, education, social care, etc). This summary article presents the new CHEERS 2022 28-item checklist and recommendations for each item. The CHEERS 2022 statement is primarily intended for researchers reporting economic evaluations for peer reviewed journals as well as the peer reviewers and editors assessing them for publication. However, we anticipate familiarity with reporting requirements will be useful for analysts when planning studies. It may also be useful for health technology assessment bodies seeking guidance on reporting, as there is an increasing emphasis on transparency in decision making.

Consolidated Health Economic Evaluation Reporting Standards 2022 (CHEERS 2022) statement: updated reporting guidance for health economic evaluations

Health economic evaluations are comparative analyses of alternative courses of action in terms of their costs and consequences. The Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement, published in 2013, was created to ensure health economic evaluations are identifiable, interpretable, and useful for decision making. It was intended as guidance to help authors report accurately which health interventions were being compared and in what context, how the evaluation was undertaken, what the findings were, and other details that may aid readers and reviewers in interpretation and use of the study. The new CHEERS 2022 statement replaces previous CHEERS reporting guidance. It reflects the need for guidance that can be more easily applied to all types of health economic evaluation, new methods and developments in the field, as well as the increased role of stakeholder involvement including patients and the public. It is also broadly applicable to any form of intervention intended to improve the health of individuals or the population, whether simple or complex, and without regard to context (such as health care, public health, education, social care, etc). This summary article presents the new CHEERS 2022 28-item checklist and recommendations for each item. The CHEERS 2022 statement is primarily intended for researchers reporting economic evaluations for peer reviewed journals as well as the peer reviewers and editors assessing them for publication. However, we anticipate familiarity with reporting requirements will be useful for analysts when planning studies. It may also be useful for health technology assessment bodies seeking guidance on reporting, as there is an increasing emphasis on transparency in decision making.

Consolidated health economic evaluation reporting standards 2022 (CHEERS 2022) statement: updated reporting guidance for health economic evaluations

Health economic evaluations are comparative analyses of alternative courses of action in terms of their costs and consequences. The Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement, published in 2013, was created to ensure health economic evaluations are identifiable, interpretable, and useful for decision making. It was intended as guidance to help authors report accurately which health interventions were being compared and in what context, how the evaluation was undertaken, what the findings were, and other details that may aid readers and reviewers in interpretation and use of the study. The new CHEERS 2022 statement replaces previous CHEERS reporting guidance. It reflects the need for guidance that can be more easily applied to all types of health economic evaluation, new methods and developments in the field, as well as the increased role of stakeholder involvement including patients and the public. It is also broadly applicable to any form of intervention intended to improve the health of individuals or the population, whether simple or complex, and without regard to context (such as health care, public health, education, social care, etc.). This summary article presents the new CHEERS 2022 28-item checklist and recommendations for each item. The CHEERS 2022 statement is primarily intended for researchers reporting economic evaluations for peer-reviewed journals, as well as the peer reviewers and editors assessing them for publication. However, we anticipate familiarity with reporting requirements will be useful for analysts when planning studies. It may also be useful for health technology assessment bodies seeking guidance on reporting, as there is an increasing emphasis on transparency in decision making.

Consolidated Health Economic Evaluation Reporting Standards 2022 (CHEERS 2022) Statement: Updated Reporting Guidance for Health Economic Evaluations

Health economic evaluations are comparative analyses of alternative courses of action in terms of their costs and consequences. The Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement, published in 2013, was created to ensure health economic evaluations are identifiable, interpretable, and useful for decision making. It was intended as guidance to help authors report accurately which health interventions were being compared and in what context, how the evaluation was undertaken, what the findings were, and other details that may aid readers and reviewers in interpretation and use of the study. The new CHEERS 2022 statement replaces previous CHEERS reporting guidance. It reflects the need for guidance that can be more easily applied to all types of health economic evaluation, new methods and developments in the field, as well as the increased role of stakeholder involvement including patients and the public. It is also broadly applicable to any form of intervention intended to improve the health of individuals or the population, whether simple or complex, and without regard to context (such as health care, public health, education, social care, etc). This summary article presents the new CHEERS 2022 28-item checklist and recommendations for each item. The CHEERS 2022 statement is primarily intended for researchers reporting economic evaluations for peer reviewed journals as well as the peer reviewers and editors assessing them for publication. However, we anticipate familiarity with reporting requirements will be useful for analysts when planning studies. It may also be useful for health technology assessment bodies seeking guidance on reporting, as there is an increasing emphasis on transparency in decision making.

Consolidated Health Economic Evaluation Reporting Standards 2022 (CHEERS 2022) Statement: Updated Reporting Guidance for Health Economic Evaluations

Health economic evaluations are comparative analyses of alternative courses of action in terms of their costs and consequences. The Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement, published in 2013, was created to ensure health economic evaluations are identifiable, interpretable, and useful for decision making. It was intended as guidance to help authors report accurately which health interventions were being compared and in what context, how the evaluation was undertaken, what the findings were, and other details that may aid readers and reviewers in interpretation and use of the study. The new CHEERS 2022 statement replaces previous CHEERS reporting guidance. It reflects the need for guidance that can be more easily applied to all types of health economic evaluation, new methods and developments in the field, as well as the increased role of stakeholder involvement including patients and the public. It is also broadly applicable to any form of intervention intended to improve the health of individuals or the population, whether simple or complex, and without regard to context (such as health care, public health, education, social care, etc). This summary article presents the new CHEERS 2022 28-item checklist and recommendations for each item. The CHEERS 2022 statement is primarily intended for researchers reporting economic evaluations for peer reviewed journals as well as the peer reviewers and editors assessing them for publication. However, we anticipate familiarity with reporting requirements will be useful for analysts when planning studies. It may also be useful for health technology assessment bodies seeking guidance on reporting, as there is an increasing emphasis on transparency in decision making.

Designing and testing of a health-economic Markov model to assess the cost-effectiveness of treatments for Bipolar disorder: TiBipoMod

BACKGROUND

Bipolar disorder is an often recurrent mood disorder that is associated with a significant economic and health-related burden. Increasing the availability of health-economic evidence may aid in reducing this burden. The aim of this study is to describe the design of an open-source health-economic Markov model for assessing the cost-effectiveness of interventions in the treatment of Bipolar Disorders type I and II, TiBipoMod.

METHODS

TiBipoMod is a decision-analytic Markov model that allows for user-defined incorporation of both pharmacological and non-pharmacological interventions for the treatment of BD. TiBipoMod includes the health states remission, depression, (hypo)mania and death. Costs and effects are modeled over a lifetime horizon from a societal and healthcare perspective, and results are presented as the total costs, Quality-Adjusted Life Years (QALY), Life Years (LY), and incremental costs per QALYs and LYs gained.

RESULTS

Functionalities of TiBipoMod are demonstrated by performing a cost-utility analysis of mindfulness-based cognitive therapy (MBCT) compared to the standard of care. Treatment with MBCT resulted in an increase of 0.18 QALYs per patient, and a dominant incremental cost-effectiveness ratio per QALY gained for MBCT at a probability of being cost-effective of 71% when assuming a €50,000 willingness-to-pay threshold.

CONCLUSION

TiBipoMod can easily be adapted and used to determine the cost-effectiveness of interventions in the treatment in Bipolar Disorder type I and II, and is freely available for academic purposes upon request at the authors.

Consolidated Health Economic Evaluation Reporting Standards (CHEERS) 2022 Explanation and Elaboration: A Report of the ISPOR CHEERS II Good Practices Task Force

Health economic evaluations are comparative analyses of alternative courses of action in terms of their costs and consequences. The Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement, published in 2013, was created to ensure health economic evaluations are identifiable, interpretable, and useful for decision making. It was intended as guidance to help authors report accurately which health interventions were being compared and in what context, how the evaluation was undertaken, what the findings were, and other details that may aid readers and reviewers in interpretation and use of the study. The new CHEERS 2022 statement replaces the previous CHEERS reporting guidance. It reflects the need for guidance that can be more easily applied to all types of health economic evaluation, new methods and developments in the field, and the increased role of stakeholder involvement including patients and the public. It is also broadly applicable to any form of intervention intended to improve the health of individuals or the population, whether simple or complex, and without regard to context (such as healthcare, public health, education, and social care). This Explanation and Elaboration Report presents the new CHEERS 2022 28-item checklist with recommendations and explanation and examples for each item. The CHEERS 2022 statement is primarily intended for researchers reporting economic evaluations for peer-reviewed journals and the peer reviewers and editors assessing them for publication. Nevertheless, we anticipate familiarity with reporting requirements will be useful for analysts when planning studies. It may also be useful for health technology assessment bodies seeking guidance on reporting, given that there is an increasing emphasis on transparency in decision making.

Reproducible research practices, openness and transparency in health economic evaluations: study protocol for a cross-sectional comparative analysis

INTRODUCTION

There has been a growing awareness of the need for rigorously and transparent reported health research, to ensure the reproducibility of studies by future researchers. Health economic evaluations, the comparative analysis of alternative interventions in terms of their costs and consequences, have been promoted as an important tool to inform decision-making. The objective of this study will be to investigate the extent to which articles of economic evaluations of healthcare interventions indexed in MEDLINE incorporate research practices that promote transparency, openness and reproducibility.

METHODS AND ANALYSIS

This is the study protocol for a cross-sectional comparative analysis. We registered the study protocol within the Open Science Framework (osf.io/gzaxr). We will evaluate a random sample of 600 cost-effectiveness analysis publications, a specific form of health economic evaluations, indexed in MEDLINE during 2012 (n=200), 2019 (n=200) and 2022 (n=200). We will include published papers written in English reporting an incremental cost-effectiveness ratio in terms of costs per life years gained, quality-adjusted life years and/or disability-adjusted life years. Screening and selection of articles will be conducted by at least two researchers. Reproducible research practices, openness and transparency in each article will be extracted using a standardised data extraction form by multiple researchers, with a 33% random sample (n=200) extracted in duplicate. Information on general, methodological and reproducibility items will be reported, stratified by year, citation of the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement and journal. Risk ratios with 95% CIs will be calculated to represent changes in reporting between 2012-2019 and 2019-2022.

ETHICS AND DISSEMINATION

Due to the nature of the proposed study, no ethical approval will be required. All data will be deposited in a cross-disciplinary public repository. It is anticipated the study findings could be relevant to a variety of audiences. Study findings will be disseminated at scientific conferences and published in peer-reviewed journals.

Does the Institute for Clinical and Economic Review Revise Its Findings in Response to Industry Comments?

BACKGROUND

The Institute for Clinical and Economic Review (ICER) has gained prominance through its work conducting health technology assessments of pharmaceuticals in the United States.

OBJECTIVE

To understand the influence of industry comments on pharmaceutical value assessments conducted by ICER.

METHODS

We reviewed 15 ICER reports issued from 2017 through 2019. We quantified ICER's revisions to its cost-effectiveness analysis (CEA) estimates between release of its draft and revised evidence reports and whether ratios shifted across ICER-specified categories of high, intermediate, or low value. We also reviewed industry-submitted comments recommending revision to ICER's CEAs, noting ICER's response as no change, text revised, assumption(s) revised, or conclusion revised. We evaluated each comment in terms of clarity, whether it offered an alternative to ICER's approach, and whether it characterized the expected impact of revision on ICER's analysis.

RESULTS

We identified 53 ICER-reported ratios. Of these, 45 (84.9%) changed between the draft and revised report, but 26 changes (57.8%) were small (<10%). Six ratios shifted across value categories. We identified 256 industry comments recommending that ICER revise its CEA. Of these, 159 (62%) lacked clarity, 145 (57%) offered no alternative, and 243 (95%) did not characterize their impact on ICER's estimated ratio. Ninety-one comments (35.5%) caused ICER to revise its assumptions, but only 5 (2.0%) caused ICER to revise its conclusions. Four of these 5 comments characterized their impact on ICER's findings.

CONCLUSIONS

Changes in ICER's estimates of cost-effectiveness between its draft and revised evidence reports are generally modest. Greater precision in industry comments could increase the influence of industry critiques, thus enhancing the dialogue around pharmaceutical value.

Achieving Appropriate Model Transparency: Challenges and Potential Solutions for Making Value-Based Decisions in the United States

Transparency in decision modeling remains a topic of rigorous debate among healthcare stakeholders, given tensions between the potential benefits of external access during model development and the need to protect intellectual property and reward research investments. Strategies to increase decision model transparency by allowing direct external access to a model's structure, source code, and data can take on many forms but are bounded between the status quo and free publicly available open-source models. Importantly, some level of transparency already exists in terms of methods and other technical specifications for published models. The purpose of this paper is to delineate pertinent issues surrounding efforts to increase transparency via direct access to models and to offer key considerations for the field of health economics and outcomes research moving forward from a US academic perspective. Given the current environment faced by modelers in academic settings, expected benefits and challenges of allowing direct model access are discussed. The paper also includes suggestions for pathways toward increased transparency as well as an illustrative real-world example used in work with the Institute for Clinical and Economic Review to support assessments of the value of new health interventions. Potential options to increase transparency via direct model access during model development include adequate funding to support the additional effort required and mechanisms to maintain security of the underlying intellectual property. Ultimately, the appropriate level of transparency requires balancing the interests of several groups but, if done right, has the potential to improve models and better integrate them into healthcare priority setting and decision making in the US context.

A Need for Change! A Coding Framework for Improving Transparency in Decision Modeling

The use of open-source programming languages, such as R, in health decision sciences is growing and has the potential to facilitate model transparency, reproducibility, and shareability. However, realizing this potential can be challenging. Models are complex and primarily built to answer a research question, with model sharing and transparency relegated to being secondary goals. Consequently, code is often neither well documented nor systematically organized in a comprehensible and shareable approach. Moreover, many decision modelers are not formally trained in computer programming and may lack good coding practices, further compounding the problem of model transparency. To address these challenges, we propose a high-level framework for model-based decision and cost-effectiveness analyses (CEA) in R. The proposed framework consists of a conceptual, modular structure and coding recommendations for the implementation of model-based decision analyses in R. This framework defines a set of common decision model elements divided into five components: (1) model inputs, (2) decision model implementation, (3) model calibration, (4) model validation, and (5) analysis. The first four components form the model development phase. The analysis component is the application of the fully developed decision model to answer the policy or the research question of interest, assess decision uncertainty, and/or to determine the value of future research through value of information (VOI) analysis. In this framework, we also make recommendations for good coding practices specific to decision modeling, such as file organization and variable naming conventions. We showcase the framework through a fully functional, testbed decision model, which is hosted on GitHub for free download and easy adaptation to other applications. The use of this framework in decision modeling will improve code readability and model sharing, paving the way to an ideal, open-source world.

Transparency in Health Economic Modeling: Options, Issues and Potential Solutions

Economic models are increasingly being used by health economists to assess the value of health technologies and inform healthcare decision making. However, most published economic models represent a kind of black box, with known inputs and outputs but undisclosed internal calculations and assumptions. This lack of transparency makes the evaluation of the model results challenging, complicates comparisons between models, and limits the reproducibility of the models. Here, we aim to provide an overview of the possible steps that could be undertaken to make economic models more transparent and encourage model developers to share more detailed calculations and assumptions with their peers. Scenarios with different levels of transparency (i.e., how much information is disclosed) and reach of transparency (i.e., who has access to the disclosed information) are discussed, and five key concerns (copyrights, model misuse, confidential data, software, and time/resources) pertaining to model transparency are presented, along with possible solutions. While a shift toward open-source models is underway in health economics, as has happened before in other research fields, the challenges ahead should not be underestimated. Importantly, there is a pressing need to find an acceptable trade-off between the added value of model transparency and the time and resources needed to achieve such transparency. To this end, it will be crucial to set incentives at different stakeholder levels. Despite the many challenges, the many benefits of publicly sharing economic models make increased transparency a goal worth pursuing.

Transparency in Health Economic Modeling: Options, Issues and Potential Solutions

Economic models are increasingly being used by health economists to assess the value of health technologies and inform healthcare decision making. However, most published economic models represent a kind of black box, with known inputs and outputs but undisclosed internal calculations and assumptions. This lack of transparency makes the evaluation of the model results challenging, complicates comparisons between models, and limits the reproducibility of the models. Here, we aim to provide an overview of the possible steps that could be undertaken to make economic models more transparent and encourage model developers to share more detailed calculations and assumptions with their peers. Scenarios with different levels of transparency (i.e., how much information is disclosed) and reach of transparency (i.e., who has access to the disclosed information) are discussed, and five key concerns (copyrights, model misuse, confidential data, software, and time/resources) pertaining to model transparency are presented, along with possible solutions. While a shift toward open-source models is underway in health economics, as has happened before in other research fields, the challenges ahead should not be underestimated. Importantly, there is a pressing need to find an acceptable trade-off between the added value of model transparency and the time and resources needed to achieve such transparency. To this end, it will be crucial to set incentives at different stakeholder levels. Despite the many challenges, the many benefits of publicly sharing economic models make increased transparency a goal worth pursuing.

Simulation Modeling and Metamodeling to Inform National and International HIV Policies for Children and Adolescents

Objective and Approach:

Computer-based simulation models serve an important purpose in informing HIV care for children and adolescents. We review current model-based approaches to informing pediatric and adolescent HIV estimates and guidelines.

Findings:

Clinical disease simulation models and epidemiologic models are used to inform global and regional estimates of numbers of children and adolescents living with HIV and in need of antiretroviral therapy, to develop normative guidelines addressing strategies for diagnosis and treatment of HIV in children, and to forecast future need for pediatric and adolescent antiretroviral therapy formulations and commodities. To improve current model-generated estimates and policy recommendations, better country-level and regional-level data are needed about children living with HIV, as are improved data about survival and treatment outcomes for children with perinatal HIV infection as they age into adolescence and adulthood. In addition, novel metamodeling and value of information methods are being developed to improve the transparency of model methods and results, as well as to allow users to more easily tailor model-based analyses to their own settings.

Conclusions:

Substantial progress has been made in using models to estimate the size of the pediatric and adolescent HIV epidemic, to inform the development of guidelines for children and adolescents affected by HIV, and to support targeted implementation of policy recommendations to maximize impact. Ongoing work will address key limitations and further improve these model-based projections.

Can We Develop Sustainable and Sharable Cost-Effectiveness Models for Value Assessment in the U.S. Health Care System?

There is currently a movement to make economic models more transparent, with some arguing for completely open-source models. However, increasing model transparency raises several logistical challenges, including ownership and funding. This article outlines recent experience and approaches to some of the logistical hurdles that must be overcome in pursuit of more transparent economic models. The Institute for Clinical and Economic Review (ICER) has recently completed a pilot transparency initiative that made drafts of executable economic models available to involved drug manufacturers during the review process. By directly viewing and interacting with models' structures, inputs, key assumptions, and results, stakeholders were better equipped to provide valuable feedback as part of the health technology assessments. This experience, along with feedback received from the modeling collaborators and relevant manufacturers during this pilot, have helped determine ICER's approach to sharing economic models associated with ongoing and future evidence reviews. This process has been expanded to all assessments going forward, making it the standard approach to model transparency. DISCLOSURES: No outside funding supported the writing of this article. Both authors are employees of the Institute for Clinical and Economic Review (ICER), which is an an independent organization that evaluates the evidence on the value of health care interventions. ICER receives grant funding from the California Healthcare Foundation, Laura and John Arnold Foundation, and New England States Consortium Systems Organization. ICER's annual policy summit is supported by dues from Aetna, AHIP, Allergan, Alnylam, Anthem, AstraZeneca, Biogen, Blue Shield of California, Cambia Health Services, CVS Caremark, Editas, Express Scripts, Genentech, GlaxoSmithKline, Harvard Pilgrim Health Care, Health Care Service Corporation, Health Partners, Johnson & Johnson, Kaiser Permanente, LEO, Mallinckrodt, Merck, National Pharmaceutical Council, Novartis, Premera, Prime Therapeutics, Regeneron, Sanofi, Spark Therapeutics, and United Healthcare.

Now Is the Time for Transparency in Value-Based Healthcare Decision Modeling

In contrast to many other countries, during the 20 years since the founding of Value in Health, the United States has moved further away from using value-based healthcare decision modeling (VHDM) for drugs and other medical care choices. US public and private health plans can be typically characterized as using "budget impact" decision making rather than VHDM, with drugs having low per-member per-month spending likely to be covered and reimbursed regardless of value. Orphan drugs and specialty drugs with relatively few patients (eg, end-stage cancer drugs) are often covered, whether cost-effective or not, because health plans want to avoid negative publicity. Although there are many explanations for the poor US uptake of VHDM, a key reason is that VHDM models and data often lack transparency and are not generally made available to researchers for independent verification and reproducibility. This violates the scientific method, and is counter to the stated position of the National Academy of Sciences and the top journals in the sciences and social sciences. Value in Health and related peer-reviewed journals could make a key contribution to improving scientific rigor and real-world healthcare decision-maker acceptability by requiring that VHDM models, source code, and data used in published articles be made freely available to interested readers.

Toward Relevant and Credible Cost-Effectiveness Analyses for Value Assessment in the Decentralized U.S. Health Care System

In the United States, there is an increased interest to understand the value of health technologies. Cost-effectiveness analysis is arguably the most appropriate framework to quantify value and to inform reimbursement decision making regarding medical interventions; however, a thorough analysis is resource intensive and complex. In many countries, the cost-effectiveness of medical interventions is evaluated by expert agencies at the national level, but in the United States, reimbursement decision making occurs at the local level. This raises the question of how we can provide a means to transparent cost-effectiveness analysis that reflects the local context and patient population and is based on the latest evidence and scientific insights. In other words, how can we maximize the relevance and credibility of cost-effectiveness evaluations in the context of a decentralized decision-making environment? Published cost-effectiveness analyses typically fail on these dimensions. Access to transparent open-source models that can be adapted to reflect the local setting in a relatively straightforward manner is an essential step toward such a goal. However, no model for cost-effectiveness analysis is ever truly "right" or "complete," and it must evolve along with clinical evidence and improvements in scientific methodology to ensure that its credibility remains. We propose a transparent approach of iterative development and collaboration between content and methodology experts to produce up-to-date, open-source consensus-based cost-effectiveness models that account for parameter and structural uncertainty to help local decision makers understand the confidence with which they might make a decision. Our proposed approach provides a way to adapt formal assessments of value-long the province of centralized health care systems-into the decentralized U.S. health care landscape. DISCLOSURES: This research was funded through the Innovation and Value Initiative, a nonprofit multistakeholder research organization. The Innovation and Value Initiative contracted with Precision Medicine Group for research activities related to this article. Jansen and Incerti are salaried employees and shareholders of Precision Medicine Group. Curtis is a paid consultant for the Innovation and Value Initiative. Curtis also reports consulting fees and grants from Amgen, AbbVie, BMS, Corrona, Janssen, Lilly, Myriad, Pfizer, Roche/Genentech, Radius, and UCB, unrelated to this article.

Future Directions for Cost-effectiveness Analyses in Health and Medicine

Objectives. In 2016, the Second Panel on Cost-effectiveness in Health and Medicine updated the seminal work of the original panel from 2 decades earlier. The Second Panel had an opportunity to reflect on the evolution of cost-effectiveness analysis (CEA) and to provide guidance for the next generation of practitioners and consumers. In this article, we present key topics for future research and policy. Methods. During the course of its deliberations, the Second Panel discussed numerous topics for advancing methods and for improving the use of CEA in decision making. We identify and consider 7 areas for which the panel believes that future research would be particularly fruitful. In each of these areas, we highlight outstanding research needs. The list is not intended as an exhaustive inventory but rather a set of key items that surfaced repeatedly in the panel’s discussions. In the online Appendix , we also list and expound briefly on 8 other important topics. Results. We highlight 7 key areas: CEA and perspectives (determining, valuing, and summarizing elements for the analysis), modeling (comparative modeling and model transparency), health outcomes (valuing temporary health and path states, as well as health effects on caregivers), costing (a cost catalogue, valuing household production, and productivity effects), evidence synthesis (developing theory on learning across studies and combining data from clinical trials and observational studies), estimating and using cost-effectiveness thresholds (empirically representing 2 broad concepts: opportunity costs and public willingness to pay), and reporting and communicating CEAs (written protocols and a quality scoring system). Conclusions. Cost-effectiveness analysis remains a flourishing and evolving field with many opportunities for research. More work is needed on many fronts to understand how best to incorporate CEA into policy and practice.

Cost-effectiveness modeling for neuropathic pain treatments: investigating the relative importance of parameters using an open-source model

AIMS

The study objective was to develop an open-source replicate of a cost-effectiveness model developed by National Institute for Health and Care (NICE), in order to explore uncertainties in health economic modeling of novel pharmacological neuropathic pain treatments.

MATERIALS AND METHODS

The NICE model, consisting of a decision tree with branches for discrete levels of pain relief and adverse event (AE) severities, was replicated using R, and used to compare a hypothetical neuropathic pain drug to pregabalin. Model parameters were sourced from NICE's clinical guidelines and associated with probability distributions to account for underlying uncertainty. A simulation-based scenario analysis was conducted to assess how uncertainty in efficacy and AEs affected the net monetary benefit (NMB) for the hypothetical treatment at a cost-effectiveness threshold of £20,000 per QALY.

RESULTS

Relative to pregabalin, an increase in efficacy was associated with greater NMB than an improvement in tolerability. A greater NMB was observed when efficacy was marginally higher than that of pregabalin, while maintaining the same level of AEs than when efficacy was equivalent to pregabalin, but with a more substantial reduction in AEs. In the latter scenario, the NMB was only positive at a low cost-effectiveness threshold.

LIMITATIONS

The replicate model shares the limitations described in the NICE guidelines. There is a lack of support in scientific literature for the assumption that increased efficacy is associated with a greater reduction in tolerability. The replicate model also included a single comparator, unlike the NICE model.

CONCLUSIONS

Pain relief is a stronger driver of NMB than tolerability, at a cost-effectiveness threshold of £20,000 per QALY. Health technology assessment decisions which are influenced by NICE's model may reward efficacy gains, even if they are associated with more severe AEs. This contrasts with recommendations from clinical guidelines for neuropathic pain, which place more equal weighting on improvements in efficacy and tolerability as value drivers.

Can Economic Model Transparency Improve Provider Interpretation of Cost-Effectiveness Analysis? A Response

To enhance the credibility and the value of health economic analyses, we argue that the computer model source code underlying these analyses should be made publicly available. Only with open publication is it possible for others to assess whether alternative assumptions, beyond those examined by the model authors, alter the model's findings. Because reproducibility is critical for scientific acceptance and because computation increasingly permeates scientific inquiry, other fields have moved toward open publication of computer models, and health economics should avoid falling behind. Making source code available shines a light on these otherwise black boxes and facilitates their complete evaluation and understandability. The preceding commentary makes 2 arguments against open publication. It claims first that open publication would undermine intellectual property rights and discourage work in this field. We respond that the impact on intellectual property would be minimal, and that open publication could even increase model value. The second argument against open publication is the possibility of model misuse. If anything, however, open publication would reduce this risk by making the model implementation completely transparent. We argue finally that open publication of models would have ancillary benefits by making the research more amenable for adaptation and innovation. Moving toward open publication will present challenges, but we believe that the benefits of increased scientific credibility and utility, particularly for health policy and clinical practice decisions, will certainly outweigh the harms.