Considerations for transferability of health technology assessments: a scoping review of tools, methods, and practices

How do the Institute for Clinical and Economic Review's Assessments of Comparative Effectiveness Compare With the German Federal Joint Committee's Assessments of Added Benefit? A Qualitative Study

OBJECTIVES

We compared the Institute for Clinical and Economic Review's (ICER) ratings of comparative clinical effectiveness with the German Federal Joint Committee's (G-BA) added benefit ratings, and explored what factors may explain the disagreement between the 2 organizations.

METHODS

We included drugs if they were assessed by ICER under its 2020 to 2023 Value Assessment Framework and had a corresponding assessment by G-BA as of January 2024 for the same indication, patient population, and comparator drug. To compare assessments, we modified ICER's proposed crosswalk between G-BA and ICER benefit ratings to account for G-BA's certainty ratings. We also determined whether each pair was based on similar evidence. Assessment pairs exhibiting disagreement based on the modified crosswalk despite a similar evidence base were qualitatively analyzed to identify reasons for disagreement.

RESULTS

Out of 15 drug assessment pairs matched on indication, patient subgroup, and comparator, none showed agreement in their assessments when based on similar evidence. Disagreement was attributed to differences in evidence evaluation, including evaluations of safety, generalizability, and study design, as well as G-BA's rejection of the available evidence in 4 cases as unsuitable.

CONCLUSIONS

The findings demonstrate that even under conditions where populations and comparators are identical and the evidence base is consistent, different assessors may arrive at divergent conclusions about comparative effectiveness, thus underscoring the presence of value judgments within assessments of clinical effectiveness. To support initiatives that seek to facilitate the exchange of value assessments between countries, these value judgments should always be transparently presented and justified in assessment summaries.

Application of natural language processing to predict final recommendation of Brazilian health technology assessment reports

INTRODUCTION

Health technology assessment (HTA) plays a vital role in healthcare decision-making globally, necessitating the identification of key factors impacting evaluation outcomes due to the significant workload faced by HTA agencies.

OBJECTIVES

The aim of this study was to predict the approval status of evaluations conducted by the Brazilian Committee for Health Technology Incorporation (CONITEC) using natural language processing (NLP).

METHODS

Data encompassing CONITEC's official report summaries from 2012 to 2022. Textual data was tokenized for NLP analysis. Least Absolute Shrinkage and Selection Operator, logistic regression, support vector machine, random forest, neural network, and extreme gradient boosting (XGBoost), were evaluated for accuracy, area under the receiver operating characteristic curve (ROC AUC) score, precision, and recall. Cluster analysis using the k-modes algorithm categorized entries into two clusters (approved, rejected).

RESULTS

The neural network model exhibited the highest accuracy metrics (precision at 0.815, accuracy at 0.769, ROC AUC at 0.871, and recall at 0.746), followed by XGBoost model. The lexical analysis uncovered linguistic markers, like references to international HTA agencies' experiences and government as demandant, potentially influencing CONITEC's decisions. Cluster and XGBoost analyses emphasized that approved evaluations mainly concerned drug assessments, often government-initiated, while non-approved ones frequently evaluated drugs, with the industry as the requester.

CONCLUSIONS

NLP model can predict health technology incorporation outcomes, opening avenues for future research using HTA reports from other agencies. This model has the potential to enhance HTA system efficiency by offering initial insights and decision-making criteria, thereby benefiting healthcare experts.

Addressing the gap in health economics data to support national cancer control plans in low- and middle-income countries: The Childhood Cancers Budgeting Rapidly to Incorporate Disadvantaged Groups for Equity (CC-BRIDGE) tool

BACKGROUND

National cancer control plans (NCCPs) are complex public health programs that incorporate evidence-based cancer control strategies to improve health outcomes for all individuals in a country. Given the scope of NCCPs, small and vulnerable populations, such as patients with childhood cancer, are often missed. To support planning efforts, a rapid, modifiable tool was developed that estimates a context-specific national budget to fund pediatric cancer programs, provides 5-year scale-up scenarios, and calculates annual cost-effectiveness.

METHODS

The tool was codeveloped by teams of policymakers, clinicians, and public health advocates in Zimbabwe, Zambia, and Uganda. The 11 costing categories included real-world data, modeled data, and data from the literature. A base-case and three 5-year scale-up scenarios were created using modifiable inputs. The cost-effectiveness of the disability-adjusted life years averted was calculated. Results were compared with each country's projected gross domestic product per capita for 2022 through 2026.

RESULTS

The number of patients/total budget for year 1 was 250/$1,109,366 for Zimbabwe, 280/$1,207,555 for Zambia, and 1000/$2,277,397 for Uganda. In year 5, these values were assumed to increase to 398/$5,545,445, 446/$4,926,150, and 1594/$9,059,331, respectively. Base-case cost per disability-adjusted life year averted/ratio to gross domestic product per capita for year 1, assuming 20% survival, was: $807/0.5 for Zimbabwe, $785/0.7 for Zambia, and $420/0.5 for Uganda.

CONCLUSIONS

This costing tool provided a framework to forecast a budget for childhood-specific cancer services. By leveraging minimal primary data collection with existing secondary data, local teams obtained rapid results, ensuring that childhood cancer budgeting is not neglected once in every 5 to 6 years of planning processes.

Scoping Meta-Review of Methods Used to Assess Artificial Intelligence-Based Medical Devices for Heart Failure

Artificial intelligence and machine learning (AI/ML) are playing increasingly important roles, permeating the field of medical devices (MDs). This rapid progress has not yet been matched by the Health Technology Assessment (HTA) process, which still needs to define a common methodology for assessing AI/ML-based MDs. To collect existing evidence from the literature about the methods used to assess AI-based MDs, with a specific focus on those used for the management of heart failure (HF), the International Federation of Medical and Biological Engineering (IFMBE) conducted a scoping meta-review. This manuscript presents the results of this search, which covered the period from January 1974 to October 2022. After careful independent screening, 21 reviews, mainly conducted in North America and Europe, were retained and included. Among the findings were that deep learning is the most commonly utilised method and that electronic health records and registries are among the most prevalent sources of data for AI/ML algorithms. Out of the 21 included reviews, 19 focused on risk prediction and/or the early diagnosis of HF. Furthermore, 10 reviews provided evidence of the impact on the incidence/progression of HF, and 13 on the length of stay. From an HTA perspective, the main areas requiring improvement are the quality assessment of studies on AI/ML (included in 11 out of 21 reviews) and their data sources, as well as the definition of the criteria used to assess the selection of the most appropriate AI/ML algorithm.

Economic evaluations of non-communicable diseases conducted in Sub-Saharan Africa: a critical review of data sources

BACKGROUND

Policymakers in sub-Saharan Africa (SSA) face challenging decisions regarding the allocation of health resources. Economic evaluations can help decision makers to determine which health interventions should be funded and or included in their benefits package. A major problem is whether the evaluations incorporated data from sources that are reliable and relevant to the country of interest. We aimed to review the quality of the data sources used in all published economic evaluations for cardiovascular disease and diabetes in SSA.

METHODS

We systematically searched selected databases for all published economic evaluations for CVD and diabetes in SSA. We modified a hierarchy of data sources and used a reference case to measure the adherence to reporting and methodological characteristics, and descriptively analysed author statements.

RESULTS

From 7,297 articles retrieved from the search, we selected 35 for study inclusion. Most were modelled evaluations and almost all focused on pharmacological interventions. The studies adhered to the reporting standards but were less adherent to the methodological standards. The quality of data sources varied. The quality level of evidence in the data domains of resource use and costs were generally considered of high quality, with studies often sourcing information from reliable databases within the same jurisdiction. The authors of most studies referred to data sources in the discussion section of the publications highlighting the challenges of obtaining good quality and locally relevant data.

CONCLUSIONS

The data sources in some domains are considered high quality but there remains a need to make substantial improvements in the methodological adherence and overall quality of data sources to provide evidence that is sufficiently robust to support decision making in SSA within the context of UHC and health benefits plans. Many SSA governments will need to strengthen and build their capacity to conduct economic evaluations of interventions and health technology assessment for improved priority setting. This capacity building includes enhancing local infrastructures for routine data production and management. If many of the policy makers are using economic evaluations to guide resource allocation, it is imperative that the evidence used is of the feasibly highest quality.