Economic evaluation in the US: what is the missing link?

Challenges of Health Technology Assessment in Pluralistic Healthcare Systems: An ISPOR Council Report

Health technology assessment (HTA) has been growing in use over the past 40 years, especially in its impact on decisions regarding the reimbursement, adoption, and use of new drugs, devices, and procedures. In countries or jurisdictions with "pluralistic" healthcare systems, there are multiple payers or sectors, each of which could potentially benefit from HTA. Nevertheless, a single HTA, conducted centrally, may not meet the needs of these different actors, who may have different budgets, current standards of care, populations to serve, or decision-making processes. This article reports on the research conducted by an ISPOR Health Technology Assessment Council Working Group established to examine the specific challenges of conducting and using HTA in countries with pluralistic healthcare systems. The Group used its own knowledge and expertise, supplemented by a narrative literature review and survey of US payers, to identify existing challenges and any initiatives taken to address them. We recommend that countries with pluralistic healthcare systems establish a national focus for HTA, develop a uniform set of HTA methods guidelines, ensure that HTAs are produced in a timely fashion, facilitate the use of HTA in the local setting, and develop a framework to encourage transparency in HTA. These efforts can be enhanced by the development of good practice guidance from ISPOR or similar groups and increased training to facilitate local use of HTA.

The evolution of ICER's review process for new medical interventions and a critical review of economic evaluations (2018-2019): how stakeholders can collaborate with ICER to improve the quality of evidence in ICER's reports

BACKGROUND: Since its inception in 2006, the Institute for Clinical and Economic Review (ICER) has rapidly gained influence on drug pricing and reimbursement decisions despite historical resistance to the use of cost-effectiveness thresholds in the US health care system. Although patient groups, physicians, and pharmaceutical manufacturers voiced their concerns about the potential negative effects of increased use of ICER's assessments on patient access to innovative medications, there is little guidance and consensus on how the stakeholders should collaborate with ICER to ensure that its reviews reflect the best clinical and economic evidence. OBJECTIVES: To (1) summarize the evolution of ICER's evaluation procedure, scope, and topics; (2) evaluate the effectiveness of stakeholder engagement approaches; and (3) inform stakeholders of their potential role in collaborating with ICER in estimating the cost-effectiveness of new interventions. METHODS: Publicly available ICER evaluations from 2008 to 2019 were systematically reviewed. Changes in evaluation procedures, scope, and topics were summarized. For evaluations that occurred in 2018 (n = 12) and 2019 (n = 8), key characteristics were extracted from 172 letters documenting interactions between ICER and all stakeholders who provided comments to draft reports. Stakeholder suggestions were analyzed in terms of their effectiveness indicated by ICER's reconsideration of its original cost-effectiveness analysis approach. RESULTS: The number of ICER evaluations increased consistently from 2 to 12 per year between 2008 and 2018 but declined to 8 in 2019. Stakeholder opportunity to engage with ICER increased from 1 to 3 per evaluation between 2008 and 2015. ICER initially focused on reviewing general treatment strategies but shifted its focus to specific pharmaceuticals and medical devices in 2014. In 2018 and 2019, 30% of 172 stakeholder letters resulted in a revision in the base-case analysis (49 comments in 2018, 23 in 2019); nearly half of comments in these letters included specific alternative data or a published article to rationalize recommendations. Other common types of suggestions that resulted in ICER's base-case analysis revisions included comments relating to inconsistent methods used to derive model inputs across different treatments (12/49 in 2018, 5/23 in 2019); clinical justifications (12/49 in 2018, 0/23 in 2019); and justifications based on patient perspectives (1/49 in 2018, 5/23 in 2019). These revisions rarely affected ICER's conclusions on the cost-effectiveness of evaluated interventions. Among the 20 assessments that involved 172 stakeholder engagements in 2018 and 2019, only 2% (n = 3) of the engagements (all from 2018) were associated with a change in the cost-effectiveness conclusion. CONCLUSIONS: Between 2018 and 2019, stakeholders leveraged ICER evaluations as opportunities to promote dialogue for better understanding of the value of technologies. Actionable, evidence-based recommendations were accepted more often than other recommendations. DISCLOSURES: No outside funding supported this study. The authors have no conflicts of interest to disclose. Findings from this study were presented as a poster at Virtual ISPOR, May 17-20, 2021.

Peer Review and Transparency in Evidence-Source Selection in Value and Health Technology Assessment

OBJECTIVES

Value and health technology assessment (V/HTA) is often used in clinical, access, and reimbursement decisions. V/HTA data-source selection may not be transparent, which is a necessary element for stakeholder understanding and trust and for fostering accountability among decision makers. Peer review is considered one mechanism for judging data trustworthiness. Our objective was (1) to use publicly available documentation of V/HTA methods to identify requirements for inclusion of peer-reviewed evidence sources, (2) to compare and contrast US and non-US approaches, and (3) to assess evidence sources used in published V/HTA reports.

METHODS

Publicly available methods documentation from 11 V/HTA organizations in North America and Europe were manually searched and abstracted for descriptions of requirements and recommendations regarding search strategy and evidence-source selection. The bibliographies of a subset of V/HTA reports published in 2018 were manually abstracted for evidence-source types used in each.

RESULTS

Heterogeneity in evidence-source retrieval and selection was observed across all V/HTA organizations, with more pronounced differences between US and non-US organizations. Not all documentation of organizations' methods address the evidence-source selection processes (7 of 11), and few explicitly reference peer-reviewed sources (3 of 11). Documentation of the evidence-source selection strategy was inconsistent across reports (6 of 13), and the level of detail provided varied across organizations. Some information on evidence-source selection was often included in confidential documentation and was not publicly available.

CONCLUSIONS

Disparities exist among V/HTA organizations in requirements and guidance regarding evidence-source selection. Standardization of evidence-source selection strategies and documentation could help improve V/HTA transparency and has implications for decision making based on report findings.

MOST IMPORTANT BARRIERS AND FACILITATORS REGARDING THE USE OF HEALTH TECHNOLOGY ASSESSMENT

Objectives: Several studies have reported multiple barriers to and facilitators for the uptake of health technology assessment (HTA) information by policy makers. This study elicited, using best-worst scaling (BWS), the most important barriers and facilitators and their relative weight in the use of HTA by policy makers.

Methods: Two BWS object case surveys (one for barriers, one for facilitators) were conducted among sixteen policy makers and thirty-three HTA experts in the Netherlands. A list of twenty-two barriers and nineteen facilitators was included. In each choice task, participants were asked to choose the most important and the least important barrier/facilitator from a set of five. We used Hierarchical Bayes modeling to generate the mean relative importance score (RIS) for each factor and a subgroup analysis was conducted to assess differences between policy makers and HTA experts.

Results: The five most important barriers (RIS > 6.00) were “no explicit framework for decision-making process,” “insufficient support by stakeholders,” “lack of support,” “limited generalizability,” and “absence of appropriate incentives.” The six most important facilitators were: “availability of explicit framework for decision making,” “sufficient support by stakeholders,” “appropriate incentives,” “sufficient quality,” “sufficient awareness,” and “sufficient support within the organization.” Overall, perceptions did not differ markedly between policy makers and HTA experts.

Conclusions: Our study suggests that barriers and facilitators related to “policy characteristics” and “organization and resources” were particularly important. It is important to stimulate a pulse at the national level to create an explicit framework for including HTA in the decision-making context.

"Can we declare victory and move on?" The case against funding burden-of-disease studies

Resources devoted to the development of burden-of-disease studies detract from much needed cost-effectiveness and cost-benefit studies. Practitioners need to help funders of burden-of-disease projects recognize the potential of all tools of decision analysis and economic evaluation in improving the efficiency and equity for the health care system.

Cost benefit analysis of mesh reinforcement of stapled left pancreatectomy

OBJECTIVES

Pancreatic leak is a morbid complication following left pancreatectomy, which results in prolonged hospitalization, additional diagnostic testing and invasive procedures. The present authors have previously demonstrated that mesh reinforcement of stapled left pancreatectomy results in fewer pancreatic leaks. This study was conducted to investigate whether mesh reinforcement also results in cost benefits for the health care system.

METHODS

A cost benefit model was developed to estimate net cost savings from the payer's perspective. The model is based on the results of a randomized, single-blinded trial of mesh versus no mesh reinforcement of the pancreatic remnant after left pancreatectomy. A two-way sensitivity analysis was conducted to determine the model's sensitivity to fluctuations in the cost of mesh and the effectiveness of the mesh in reducing clinically significant leaks.

RESULTS

Average total costs for an episode of care were US$13 337 and US$15 505 for patients who did and did not receive mesh, respectively, which indicates savings of US$2168. Two-way sensitivity analysis showed that, given a probability of 1.9% for developing a clinically significant leak in patients in whom mesh reinforcement was used, the strategy would continue to save costs if mesh were priced at ≤US$1804.

CONCLUSIONS

Mesh reinforcement decreases clinically significant pancreatic leaks. Despite the additional cost of mesh reinforcement, the use of mesh reinforcement results in overall cost savings for the health care system because of the resultant decrease in the occurrence of clinically significant leaks.

Physical therapist practice and the role of diagnostic imaging

For healthcare providers involved in the management of patients with musculoskeletal disorders, the ability to order diagnostic imaging is a beneficial adjunct to screening for medical referral and differential diagnosis. A trial of conservative treatment, such as physical therapy, is often recommended prior to the use of imaging in many treatment guidelines for the management of musculoskeletal conditions. In the United States, physical therapists are becoming more autonomous and can practice some degree of direct access in 48 states and Washington, DC. Referral for imaging privileges could increase the effectiveness and efficiency of healthcare delivery, particularly in combination with direct access management. This clinical commentary proposes that, given the American Physical Therapy Association's goal to have physical therapists as primary care musculoskeletal specialists of choice, it would be beneficial for physical therapists to have imaging privileges in their practice. The purpose of this commentary is 3-fold: (1) to make a case for the use of imaging privileges by physical therapists, using a historical perspective; (2) to discuss the barriers preventing physical therapists from having this privilege; and (3) to offer suggestions on strategies and guidelines to facilitate the appropriate inclusion of referral for imaging privileges in physical therapist practice. J Orthop Sports Phys Ther 2011;41(11):829-837. doi:10.2519/jospt.2011.3556.

Are Key Principles for improved health technology assessment supported and used by health technology assessment organizations?

Previously, our group-the International Working Group for HTA Advancement-proposed a set of fifteen Key Principles that could be applied to health technology assessment (HTA) programs in different jurisdictions and across a range of organizations and perspectives. In this commentary, we investigate the extent to which these principles are supported and used by fourteen selected HTA organizations worldwide. We find that some principles are broadly supported: examples include being explicit about HTA goals and scope; considering a wide range of evidence and outcomes; and being unbiased and transparent. Other principles receive less widespread support: examples are addressing issues of generalizability and transferability; being transparent on the link between HTA findings and decision-making processes; considering a full societal perspective; and monitoring the implementation of HTA findings. The analysis also suggests a lack of consensus in the field about some principles--for example, considering a societal perspective. Our study highlights differences in the uptake of key principles for HTA and indicates considerable room for improvement for HTA organizations to adopt principles identified to reflect good HTA practices. Most HTA organizations espouse certain general concepts of good practice--for example, assessments should be unbiased and transparent. However, principles that require more intensive follow-up--for example, monitoring the implementation of HTA findings--have received little support and execution.

Health technology assessment: a review of international activity and examples of approaches with computed tomographic colonography

The growth of health technology assessment (HTA) internationally is currently reflected in the growing membership of the International Network of Agencies for Health Technology Assessment. Many national and regional HTA institutions emerged in the 1980s and 1990s, and more recently, HTA has emerged in newly industrialized countries and in European Union member states in transition. Health technology assessment activities are becoming an increasingly important part of health care culture, with the appearance of HTA units in hospitals and hospital departments. This article provides a brief overview of who conducts HTA internationally and looks at how HTA is conducted and how this information is used. To highlight the different structures, processes, and methods available, a portion of this article is dedicated to describing different approaches that have been observed with respect to the assessment of computed tomographic colonography in North America for population-based colorectal cancer screening.

Assessing comparative effectiveness research in the US

With the inclusion of $US1.1 billion earmarked for comparative effectiveness research (CER) in the recently enacted stimulus package, the US government indicated it will play an important role in informing prescribing and reimbursement decisions. However, this sizable investment does beg four important questions: what is the nature of CER data; what methods are suitable for collecting CER data; who is (should be) responsible for collecting CER data; and how will (should) CER data be used by the federal government? Using three recent high-profile cases of drugs and drug classes, we assess the current state of federal- and state-funded CER in the US. From these cases we observe that evidence is gradually emerging as a filter for certain prescribing and coverage decisions. The first case indicates evidence should not be gathered and applied in a post hoc fashion after a reimbursement decision has already been reached. Case 2 suggests limitations associated with making inferences from systematic reviews when applying the evidence to the treatment of individual patients. Case 3 points to a comprehensive, but more costly and time-consuming, way of gathering data to inform prescribing and reimbursement decisions. Despite caveats, we argue that there is room for building a more systematic and better coordinated evidence base in the US, so that all stakeholders are better equipped to understand variation in clinical outcomes while promoting appropriate prescribing patterns. Accordingly, CER could help close the gap between what we know and what we do in pharmaceutical care. For the majority of cases in which CER is carried out, we favour a pluralistic system of CER analyses with a clearing-house for systematic reviews conducted by multiple evidence-based practice centres, each uniquely suited to its constituency.

A forensic evaluation of the National Emphysema Treatment Trial using the expected value of information approach

BACKGROUND/RATIONALE

Expected value of information (EVI) analyses allow researchers to estimate the returns to conducting research. We used EVI techniques to estimate the value of the National Emphysema Treatment Trial (NETT), a multicenter randomized trial of lung-volume-reduction surgery (LVRS) versus medical therapy (MT) for patients with severe emphysema, then compared that result to the trial cost.

METHODS

We gathered information on costs and benefits of LVRS and MT before the trial and the costs of conducting the NETT, and compared these data with the results of the cost-effectiveness analysis conducted alongside the trial. We used 2 thresholds to represent the societal value of a quality-adjusted life year (QALY): USD 50,000 and USD100,000.

RESULTS

The cost effectiveness of LVRS versus MT using historical (nontrial) information was USD 305,000/QALY. Based on these data and the threshold incremental cost-effectiveness ratio values, the expected value of perfect information was USD 46 million and USD 670 million for thresholds USD 50,000 and USD 100,000 per QALY, respectively. The NETT was powered for 1,250 patients in each arm; ultimately approximately 600 patients in each arm were recruited. With 1,250 patients per arm, the expected value of sample information was USD 660 million for the threshold of USD100,000. The actual cost of the NETT was approximately USD 60 million. The expected net benefit of sampling was USD 600 million.

CONCLUSIONS

Given the difference between the cost of the trial and the economic benefits of the information, the EVI analyses suggest that federal investment in the NETT trial represented good value for money.

Key principles for the improved conduct of health technology assessments for resource allocation decisions

Health technology assessment (HTA) is a dynamic, rapidly evolving process, embracing different types of assessments that inform real-world decisions about the value (i.e., benefits, risks, and costs) of new and existing technologies. Historically, most HTA agencies have focused on producing high quality assessment reports that can be used by a range of decision makers. However, increasingly organizations are undertaking or commissioning HTAs to inform a particular resource allocation decision, such as listing a drug on a national or local formulary, defining the range of coverage under insurance plans, or issuing mandatory guidance on the use of health technologies in a particular healthcare system. A set of fifteen principles that can be used in assessing existing or establishing new HTA activities is proposed, providing examples from existing HTA programs. The principal focus is on those HTA activities that are linked to, or include, a particular resource allocation decision. In these HTAs, the consideration of both costs and benefits, in an economic evaluation, is critical. It is also important to consider the link between the HTA and the decision that will follow. The principles are organized into four sections: (i) “Structure” of HTA programs; (ii) “Methods” of HTA; (iii) “Processes for Conduct” of HTA; and (iv) “Use of HTAs in Decision Making.”

Economic evaluations of medical care interventions for cancer patients: how, why, and what does it mean?

While the past decade has seen the development of multiple new interventions to diagnose and treat cancer, as well as to improve the quality of life for cancer patients, many of these interventions have substantial costs. This has resulted in increased scrutiny of the costs of care for cancer, as well as the costs relative to the benefits for cancer treatments. It is important for oncologists and other members of the cancer community to consider and understand how economic evaluations of cancer interventions are performed and to be able to use and critique these evaluations. This review discusses the components, main types, and analytic issues of health economic evaluations using studies of cancer interventions as examples. We also highlight limitations of these economic evaluations and discuss why members of the cancer community should care about economic analyses.

'Lost in translation': accounting for between-country differences in the analysis of multinational cost-effectiveness data

Cost-effectiveness analysis has gained status over the last 15 years as an important tool for assisting resource allocation decisions in a budget-limited environment such as healthcare. Randomised (multicentre) multinational controlled trials are often the main vehicle for collecting primary patient-level information on resource use, cost and clinical effectiveness associated with alternative treatment strategies. However, trial-wide cost effectiveness results may not be directly applicable to any one of the countries that participate in a multinational trial, requiring some form of additional modelling to customise the results to the country of interest. This article proposes an algorithm to assist with the choice of the appropriate analytical strategy when facing the task of adapting the study results from one country to another. The algorithm considers different scenarios characterised by: (a) whether the country of interest participated in the trial; and (b) whether individual patient-level data (IPD) from the trial are available. The analytical options available range from the use of regression-based techniques to the application of decision-analytic models. Decision models are typically used when the evidence base is available exclusively in summary format whereas regression-based methods are used mainly when the country of interest actively recruited patients into the trial and there is access to IPD (or at least country-specific summary data). Whichever method is used to reflect between-country variability in cost-effectiveness data, it is important to be transparent regarding the assumptions made in the analysis and (where possible) assess their impact on the study results.