Quantitative Benefit-Risk Assessment in Medical Product Decision Making: A Good Practices Report of an ISPOR Task Force

Key Opinion Leaders' Interviews to Inform the Future of Benefit-Risk Planning in the Medical Total Product Life Cycle of Global Pharmaceutical and Medical Device Organizations

BACKGROUND AND OBJECTIVES

Key opinion leader (KOL) interviews were conducted by the Benefit-Risk Assessment Planning (BRAP) Taskforce to seek expert opinion mainly from industry and regulatory bodies, about the current status and future direction of benefit-risk assessment (BRA) planning in the lifecycle of medical product development. The findings from these interviews are intended to help communication concerning planning for BRA between industry and regulators and shape future guidance.

METHODS

Key opinion leader interviews consisted of 5 questions related to BRA planning, which were administered to volunteers (mainly clinicians and statisticians) within a pool of experienced pharmaceutical and medical device professionals representing academia, industry, regulatory agencies and a patient group. The interviewees' responses to the 5 questions were summarized. To analyze the qualitative data, a Coding System was developed to label themes arising from the interviews. The key findings from the interviews were summarized into a Master Template. A quantitative analysis based on descriptive statistics was also conducted.

RESULTS

Of the 27 interviewees, there were 11 professionals from regulatory agencies, 11 from industry, 4 from academia and 1 from a patient advocacy group. Key findings based on the comments provided by 48% of the interviewees indicated the need of incorporating BRA into other (e.g., existing) processes with the importance of alignment between processes being stressed in the comments provided by 59% of the interviewees. Commencing BRA early in the product lifecycle was emphasized in comments provided by 44% of the interviewees. Among other needs identified were an appropriate contextualization of benefits and risks (based on comments provided by 41% of interviewees) through adoption of an integrated approach with structured support by regulatory agencies and a need for understanding the audience with better communication of benefit-risk (BR) among all stakeholders (based on comments provided by 44% of the interviewees). Almost all comments provided by interviewees (96%) highlighted the importance of utilizing patient experience/preference to guide new product development and BRA. Comments provided by 74% of the interviewees expressed the need to understand patient tolerance for risk and trade-offs, with a majority (78%) of interviewees highlighting how to gather information, and 59% stressing the need for the selection and development of appropriate methodologies as important considerations for enhancing the quality and relevance of the data collected from patients.

CONCLUSIONS

Interviewees indicated that BRA should commence early in the medical product development and inform decision-making throughout the product lifecycle. Better planning and integration of BRA into existing processes within industry would be valuable. The importance of incorporating the patient voice into BRA and medical product development was emphasized. Other key findings from the KOL interviews included a need for improved communication of BR information, and establishment of methodologies for performing BRA and soliciting patient input.

A Survey to Assess the Current Status of Structured Benefit-Risk Assessment in the Global Drug and Medical Device Industry

BACKGROUND

This industry survey was conducted to gain insight into the ways structured Benefit-Risk assessment (sBRA) of medical products is approached across drug or medical device developing companies, including frameworks and methods that are currently used and areas where future work is being planned.

METHODS

A survey containing 28 questions covering five key areas of sBRA was set-up and shared with representatives from the participating companies. Each company was asked to complete a single survey response including inputs across the company's multidisciplinary key representatives involved in benefit-risk assessment.

RESULTS

Of the 26 participating companies, 21 (81%) are conducting sBRA. Considering these 21 qualitative frameworks were used by almost every company (19, 90%), while only 12 (57%) have used a quantitative method. Many companies have sBRA training (17, 81%), document templates (16,76%), Standard Operating Procedures (SOPs)/checklists (13, 62%), and /or best practice manuals/examples (12,57%) available. Considering all 26 companies Software tools (15, 58%) and BR planning documents (11,42%) were identified as areas into which many companies intend to put effort.

CONCLUSIONS

The industry survey confirmed a wide usage of sBRA by many companies involved in research and development. Nevertheless, sBRA is evolving and several future opportunities like the implementation of visualization tools were identified by the representatives of the pharmaceutical companies. Finally, challenges like the cross-functional comprehension of the added value of sBRA are still seen.

Multidimensional Thresholding for Individual-Level Preference Elicitation

OBJECTIVES

Multiple methods are available for collecting health preference information. However, information on the design and analysis of novel methods is limited. This article aims to provide the first introduction into the design and analysis of multidimensional thresholding (MDT).

METHODS

We introduce MDT as a 2-step approach: First, participants rank the largest possible improvements in all considered attributes by their importance. Second, participants complete a series of systematically combined trade-off questions. Hit-and-Run sampling is used for obtaining preference weights. We also use a computational experiment to compare different MDT designs.

RESULTS

The outlined MDT can generate preference information suitable for specifying a multiattribute utility function at the individual level. The computational experiment demonstrates the method's ability to recover preference weights at a high level of precision. While all designs in the computation experiment perform comparably well on average, the design outlined in the paper stands out with a high level of precision even if differences in relative attribute importance are large.

CONCLUSION

MDT is suitable for preference elicitation, in particular if sample sizes are small. Future research should help improve the methods (e.g., remove the need for an initial ranking) to increase the potential reach of MDT.

Two Methods, One Story? Comparing Results of a Choice Experiment and Multidimensional Thresholding From a Clinician Preference Study in Aneurysmal Subarachnoid Hemorrhage

OBJECTIVES

An increasing number of methods are used to elicit health preference information. It is unclear whether different elicitation methods produce similar results and policy advice. Here, we compared the results from a discrete choice experiment (DCE) and multidimensional thresholding (MDT) that were conducted in the same sample.

METHODS

Clinicians (N = 350) completed a DCE and MDT to elicit their preferences for 4 attributes related to the medical management of subarachnoid hemorrhage after aneurysm repair. Preference weights were compared between the DCE and MDT using a complete combinatorial convolution test. Additionally, data from the DCE and MDT were used to compute preference-based net treatment values for 16 hypothetical treatment profiles versus 1000 simulated comparators. The implied treatment recommendations were compared between the DCE and MDT.

RESULTS

Preference weight distributions and median weights did not differ significantly between the DCE and MDT for any attribute: likelihood of delayed cerebral ischemia (medians 0.48 vs 0.40; P = .41), risk of lung complications (medians 0.27 vs 0.30; P = .52), risk of hypotension (medians 0.10 vs 0.11; P = .55), and risk of anemia (medians 0.07 vs 0.07; P = .50). The DCE and MDT produced similar treatment net value distributions (P > .05) and implied the same treatment recommendations in 82.3% of cases.

CONCLUSIONS

The DCE and MDT elicited similar preference distributions and produced the same treatment recommendations for most tested cases. However, the share of people supporting the average treatment recommendation differed. More research is needed to determine how these findings would compare with those in other populations (in particular, patients) and applications.

Decision tool of medical endoscope maintenance service in Chinese hospitals: a conjoint analysis

BACKGROUND

Medical devices are instruments, apparatus, appliances, software, implants, reagents, materials or other articles that are intended for use in the treatment or diagnosis of disease or injury in humans. Concerning medical endoscope devices, which enable doctors to observe and manipulate the area under examination through a puncture hole in the body cavity or organ, hospitals predominantly consider the quality and cost of maintenance services when making their selection. The effective and efficient provision of maintenance services plays a crucial role in ensuring cost-effective and high-quality management of medical devices. In this study, we have developed an innovative decision tool that analyzed key factors impacting the choice of medical devices' maintenance service. This tool assists hospitals in evaluating and selecting appropriate maintenance services for medical device, specifically endoscopy devices. Moreover, it also serves as a valuable resource for manufacturers and suppliers to enhance their after-sales service offerings.

METHODS

A cross-sectional survey was undertaken in 50 Chinese hospitals, including primary and tertiary hospitals. Moreover, 56 medical staff and 65 medical engineers were recruited from 50 Chinese hospitals to participate the survey. A comprehensive set of factors were defined and investigated. Conjoint analysis and orthogonal design were used for survey design and statistical analysis.

RESULTS

Factors importance and utility values of decision-making factors were analyzed at the aggregate, occupation, and medical institution levels. (1) At the aggregate level, the most critical factor is "maintenance response" and the least important one is "maintenance efficiency". (2) At the occupation level, medical staff paid more attention to "maintenance response" while medical engineers paid more attention to "maintenance quality". (3) At the medical institution level, Primary hospitals paid more attention to "maintenance price", while tertiary hospitals paid more attention to "maintenance quality".

CONCLUSIONS

In general, this study provides a more scientific decision-making tool to both hospitals in choosing maintenance service for medical device such as endoscopy, and it also helps manufacturers and suppliers improve the after-sales service.

Illustrating Emerging Good Practices for Quantitative Benefit-Risk Assessment: A Hypothetical Case Study of Systemic Biologic Treatments for Plaque Psoriasis

OBJECTIVES

Quantitative benefit-risk assessment (qBRA) is a structured process to evaluate the benefit-risk balance of treatment options to support decision making. The ISPOR qBRA Task Force was recently established to provide recommendations for the design, conduct, and reporting of qBRA. This report presents a hypothetical case study illustrating how to apply the Task Force's recommendations toward a qBRA to inform the benefit-risk assessment of brodalumab at the time of initial marketing approval. The qBRA evaluated 2 dosing regimens of brodalumab (210 mg or 140 mg twice weekly) compared with weight-based dosing of ustekinumab and placebo.

METHODS

We followed the 5 steps recommended by the Task Force. Attributes included treatment response (≥75% improvement in Psoriasis Area and Severity Index), suicidal ideation and behavior, and infections. Performance data were drawn from pivotal clinical trials of brodalumab. The qBRA used multicriteria decision analysis and preference weights from a hypothetical discrete choice experiment. Sensitivity analyses examined the robustness of benefit-risk ranking to uncertainty in clinical effect and preference estimates, consideration of a subgroup (nail psoriasis), and the maintenance phase of treatment (52 weeks instead of 12).

RESULTS

Results from this hypothetical qBRA suggest that brodalumab 210 mg had a more favorable benefit-risk profile compared with ustekinumab and placebo. Ranking of brodalumab compared with ustekinumab was dependent on brodalumab's dose. Sensitivity analyses demonstrated robustness of benefit-risk ranking to uncertainty in clinical effect and preference estimates, as well as choice of attributes and length of follow-up.

CONCLUSION

This case study demonstrates how to implement the ISPOR Task Force's good practice recommendations on qBRA.