Economic evaluation of medical tests at the early phases of development: a systematic review of empirical studies

Perceptions and Attitudes Regarding Medical Device Development in Canada Among Canadian Innovators: A Qualitative Study

OBJECTIVES

The Canadian medical device industry presents unique challenges to innovators. However, little attention has been paid to exploring the distinct experience of Canadian medical device innovators in the literature. The objective of this study is to explore the experience of Canadian innovators in navigating this industry, with a focus on their perceptions and attitudes towards the use of health economic evaluation.

METHODS

Semi-structured interviews were conducted using virtual conferencing technology. All participants were C-level employees of small- and medium-sized enterprises (SMEs) with adequate knowledge of their company's overall strategy. Qualitative data were analyzed to reveal emerging themes.

RESULTS

Interviews were performed with ten participants. Forty percent of participants rated themselves as having either minimal or basic knowledge of health economics. Thirty percent of participants had not pursued early economic evaluation of their device, while 90% rated health economics as being either "Quite important" or "Very important" to their company. The perception of increased barriers to successful device adoption in Canada relative to the USA was a prominent sentiment among participants, with 50% expressing discontentment with either the device approval process or health technology assessment process in Canada. Twenty percent stated that their primary target market involved the USA and/or other international jurisdictions.

CONCLUSION

Canadian medical device innovators appear to understand the importance of health economic evaluation in the innovation process. However, they report difficulty with device approval and adoption, with some innovators focusing their efforts outside of Canada altogether. Further research should be directed toward understanding how to better support SMEs, given that they are a tremendous source of growth for the Canadian medical device industry.

Frameworks for Health Technology Assessment at an Early Stage of Product Development: A Review and Roadmap to Guide Applications

OBJECTIVES

Early health technology assessment (eHTA) can be used to evaluate and optimize a medical product's value proposition and to inform go/no-go decisions by using health economic modeling, literature scanning, and stakeholder preference studies at an early stage of development. eHTA frameworks offer high-level guidance on conducting this complex, iterative, and multidisciplinary process. The objective of this study was to review and summarize existing eHTA frameworks, understood as systematic approaches to guide early evidence generation and decision making.

METHODS

Using a rapid review methodology, we identified all relevant studies published in English, French, and Spanish from PubMed/MEDLINE and Embase until February 2022. We only included frameworks relevant to the preclinical and early clinical (phase I) stages of medical product development.

RESULTS

From 737 reviewed abstracts, 53 publications describing 46 frameworks were selected for inclusion and classified into categories based on their scope: (1) criteria frameworks, which provide an overview of eHTA; (2) process frameworks, which offer stepwise guidance for conducting eHTA, including preferred methods; and (3) methods frameworks, which provide detailed descriptions of specific eHTA methods. Most of the frameworks did not specify their target users or the specific stage of technology development.

CONCLUSIONS

Despite some variability and gaps found across existing frameworks, the structure provided by this review helps inform eHTA applications. Remaining challenges are the frameworks' limited accessibility to users without a background in health economics, poor distinctions being made among early lifecycle stages and technology types, and the inconsistent terminology used to describe eHTA in different contexts.

Headroom Analysis for Early Economic Evaluation: A Systematic Review

OBJECTIVES

The headroom analysis is an early economic evaluation that quantifies the highest price at which an intervention may still be cost effective. Currently, there is no comprehensive review on how it is applied. This study investigated the application of the headroom analysis, specifically (1) how the headroom analysis is framed (2) the analytical approach and sources of evidence used, and (3) how expert judgement is used and reported.

METHODS

A systematic search was conducted in PubMed, Embase, Web of Science, EconLit, and Google Scholar on 28 April 2022. Studies were eligible if they reported an application of the headroom analysis. Data were presented in tabular form and summarised descriptively.

RESULTS

We identified 42 relevant papers. The headroom analysis was applied to medicines (29%), diagnostic or screening tests (29%), procedures, programmes and systems (21%), medical devices (19%), and a combined test and device (2%). All studies used model-based analyses, with 40% using simple models and 60% using more comprehensive models. Thirty-three percent of the studies assumed perfect effectiveness of the health technology, while 67% adopted realistic assumptions. Ten percent of the studies calculated an effectiveness-seeking headroom instead of a cost-seeking headroom. Expert judgement was used in 71% of the studies; 23 studies (55%) used expert opinion, 6 studies (14%) used expert elicitation, and 1 study (2%) used both.

CONCLUSIONS

Because the application of the headroom analysis varies considerably, we recommend its appropriate use and clear reporting of analytical approaches, level of evidence available, and the use of expert judgement.

Exploring the approach to parameter uncertainty in early economic evaluations of surgical technology - a systematic review

INTRODUCTION

The role of early economic evaluation (EEE) in the development of medical technology has been increasingly recognized; however, data on the use of EEE in surgical technology are sparse. The objective of this review was to explore the use of EEE in the development of surgical technologies, with emphasis on how uncertainty has been addressed.

AREAS COVERED

A systematic review was conducted, and original articles employing any form of EEE of surgical technology were selected for review, with 10 studies included in the analysis. These studies demonstrated significant variation in the approach to managing parameter uncertainty, specifically regarding the type of analysis used and the inclusion of effectiveness parameters in sensitivity analysis. The conclusions drawn did not appear to factor in uncertainty in the models.

EXPERT OPINION

Approaches to handling parameter uncertainty in previous EEEs of surgical technology have been limited, with some studies failing to address parameter uncertainty. In addition, EEEs do not appear to follow established guidelines with respect to the use of sensitivity analyses. It is important that EEEs of surgical technology address parameter uncertainty in order to draw more robust conclusions from the analysis and allow investors to consider this uncertainty when making investment decisions.

Decision-analytic modeling for early health technology assessment of medical devices - a scoping review

Objective

The goal of this review was to identify decision-analytic modeling studies in early health technology assessments (HTA) of high-risk medical devices, published over the last three years, and to provide a systematic overview of model purposes and characteristics. Additionally, the aim was to describe recent developments in modeling techniques.

Methods

For this scoping review, we performed a systematic literature search in PubMed and Embase including studies published in English or German. The search code consisted of terms describing early health technology assessment and terms for decision-analytic models. In abstract and full-text screening, studies were excluded that were not modeling studies for a high-risk medical device or an in-vitro diagnostic test. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram was used to report on the search and exclusion of studies. For all included studies, study purpose, framework and model characteristics were extracted and reported in systematic evidence tables and a narrative summary.

Results

Out of 206 identified studies, 19 studies were included in the review. Studies were either conducted for hypothetical devices or for existing devices after they were already available on the market. No study extrapolated technical data from early development stages to estimate potential value of devices in development. All studies except one included cost as an outcome. Two studies were budget impact analyses. Most studies aimed at adoption and reimbursement decisions. The majority of studies were on in-vitro diagnostic tests for personalized and targeted medicine. A timed automata model, to our knowledge a model type new to HTA, was tested by one study. It describes the agents in a clinical pathway in separate models and, by allowing for interaction between the models, can reflect complex individual clinical pathways and dynamic system interactions. Not all sources of uncertainty for in-vitro tests were explicitly modeled. Elicitation of expert knowledge and judgement was used for substitution of missing empirical data. Analysis of uncertainty was the most valuable strength of decision-analytic models in early HTA, but no model applied sensitivity analysis to optimize the test positivity cutoff with regard to the benefit-harm balance or cost-effectiveness. Value-of-information analysis was rarely performed. No information was found on the use of causal inference methods for estimation of effect parameters from observational data.

Conclusion

Our review provides an overview of the purposes and model characteristics of nineteen recent early evaluation studies on medical devices. The review shows the growing importance of personalized interventions and confirms previously published recommendations for careful modeling of uncertainties surrounding diagnostic devices and for increased use of value-of-information analysis. Timed automata may be a model type worth exploring further in HTA. In addition, we recommend to extend the application of sensitivity analysis to optimize positivity criteria for in-vitro tests with regard to benefit-harm or cost-effectiveness. We emphasize the importance of causal inference methods when estimating effect parameters from observational data.

Methods for Early Assessment of the Societal Value of Health Technologies: A Scoping Review and Proposal for Classification

OBJECTIVES

Early assessments of health technologies help to better align and integrate their development and assessment. Such assessments can take many forms and serve different purposes, hampering users in their selection of the most appropriate method for a specific goal. The aim of this scoping review was to structure the large set of methods according to their specific goal.

METHODS

A scoping review was conducted using PubMed and reference lists of retrieved articles, to identify review studies with a methodological focus. From the included reviews, all individual methods were listed. Based on additional literature and examples, we extracted the specific goal of each method. All goals were clustered to derive a set of subclasses and methods were grouped into these subclasses.

RESULTS

Of the 404 screened, 5 reviews were included, and 1 was added when searching reference lists. The reviews described 56 methods, of which 43 (77%) were included and classified as methods to (1) explore the nature and magnitude of the problem, (2) estimate the nature and magnitude of the expected (societal) value, (3) identify conditions for the potential value to materialize, and (4) help develop and design the type of research that is needed.

CONCLUSIONS

The wide range of methods for exploring the societal value of health technologies at an early stage of development can be subdivided into a limited number of classes, distinguishing methods according to their specific objective. This facilitates selection of appropriate methods, depending on the specific needs and aims.

Integrating Early Economic Evaluation into Target Product Profile development for medical tests: advantages and potential applications

Target Product Profiles (TPPs) outline the characteristics that new health technologies require to address an unmet clinical need. To date, published TPPs for medical tests have focused on infectious diseases, mostly in the context of low- and middle-income countries. Recently, there have been calls for a broader use of TPPs as a mechanism to ensure that diagnostic innovation is aligned with clinical needs, yet the methodology underpinning TPP development remains suboptimal. Here, we propose that early economic evaluation (EEE) should be integrated within the TPP methodology to create a more rigorous framework for the development of "fit-for-purpose" tests. We discuss the potential benefits that EEE could bring to the core activities underpinning TPP development-scoping, drafting, consensus building, and updating-and argue that using EEE to help inform TPPs provides a more objective, evidence-based, and transparent approach to defining test specifications.