Structured Frameworks to Increase the Transparency of the Assessment of Benefits and Risks of Medicines: Current Status and Possible Future Directions

Expedited Approval in Oncology: A Study of European Regulators' Perspectives and Trade-Offs

The purpose of this study was to describe the perspectives and trade-offs of EU regulators about expedited approvals for treatments of patients with advanced cancer. The study consisted of a cross-sectional survey with assessors and experts from the network of the European Medicines Agency (EMA). First, general attitudes were collected using a 5-point Likert scale. Second, a conjoint analysis was used to assess the compatibility of "conditional approval" for different scenarios of response rate, response duration, and toxicity from a single-arm trial, and timeliness of confirmatory data. The survey was completed by 60 participants out of 351 invited participants. The majority agreed that outstanding activity is indicative of efficacy, that toxicity is a concern, and that confirmatory trials should be ongoing at the time of approval. The majority also agreed that conditional approval is not well understood by patients and doctors. Opinions varied on the usefulness of external comparator studies. Higher antitumour activity could outweigh higher toxicity or longer time to confirmation. Based on confidence with expedited pathways and evidence generation, participants could be categorized as "supportive," "pragmatic," and "cautious." The importance of toxicity and timeliness of confirmatory studies was similar between groups but varied for antitumour activity. This study highlighted different attitudes when balancing speed of access with incomplete evidence in situations of high unmet medical needs. Confidence in expedited pathways and early evidence varied. There is an opportunity for improving communication and better alignment on evidentiary standards for expedited approvals.

Benefit-risk assessment of vaccines

Benefit-risk assessment (BRA) is critical for decision-making throughout the vaccine life cycle. It requires scientific assessment of evidence to make an informed judgment on whether the vaccine has a favourable benefit-risk profile i.e. the benefits of the vaccine outweigh its risks for use in its intended indication. The assessment must also consider data gaps and uncertainties, using sensitivity analyses to show the impact of these uncertainties in the assessment. The BRA field has advanced considerably over the past years, including the use of structured BRA frameworks, quantitative BRA models and use of the patient experience data. Analytical tools and procedures to standardize BRA implementation have become increasingly important. A Benefit-Risk Assessment Module has been prepared to enable the planning, assessment, and communication of relevant BRA information via a structured B-R framework. The module can help facilitate the conduct and communication of defensible BRAs by vaccine developers, funders, regulators and policy makers in high, middle or low-income countries, both for regulatory submissions and in public health responses to infectious diseases, including for epidemics.

The Brighton collaboration standardized module for vaccine benefit-risk assessment

Vaccine Benefit-Risk (B-R) assessment consists of evaluating the benefits and risks of a vaccine and making a judgment whether the expected key benefits outweigh the potential key risks associated with its expected use. B-R supports regulatory and public health decision-making throughout the vaccine's lifecycle. In August 2021, the Brighton Collaboration's Benefit-Risk Assessment of VAccines by TechnolOgy (BRAVATO) Benefit-Risk Assessment Module working group was established to develop a standard module to support the planning, conduct and evaluation of structured B-R assessments for vaccines from different platforms, based on data from clinical trials, post-marketing studies and real-world evidence. It enables sharing of relevant information via value trees, effects tables and graphical depictions of B-R trade-offs. It is intended to support vaccine developers, funders, regulators and policy makers in high-, middle- or low-income countries to help inform decision-making and facilitate transparent communication concerning development, licensure, deployment and other lifecycle decisions.

Methodological guidelines and publications of benefit-risk assessment for health technology assessment: a scoping review protocol

BACKGROUND

Benefit-risk assessment (BRA) is used in multiple phases along the health technology's life-cycle to evaluate the balance between the benefits and risks, as it is fundamental to all stakeholders. BRA and its methodological approaches have been applied primarily in the context of regulatory agencies. However, BRA's application and extent in the context of health technology assessment (HTA) bodies remain less clear. Our goal is to perform a scoping review to identify and map methodological guidelines and publications on methods of BRA. This will be done considering the different phases of the life-cycle of health technologies to underline both the depth and extent of research concerning BRA, especially in the context of HTA.

METHODS AND ANALYSIS

This scoping review protocol was developed following the framework proposed by Arksey and O'Malley, and the updated guidelines by the Joanna Briggs Institute. We will include methodological publications that provide recommendations or guidelines on methods for BRA. We will conduct electronic searches on Medline (PubMed) and EMBASE (Ovid) databases; manual searches on the main websites of HTA bodies and drug regulatory organisations; and contact experts in the field. Systematic extraction forms will be used to screen and assess the identified publications by independent assessors. We will provide a qualitative synthesis using descriptive statistics and visual tools. Results will be summarised in systematic evidence tables and comparative evidence scoping charts.

ETHICS AND DISSEMINATION

This review will use data publicly available and does not require ethics approval. The results of this scoping review will contribute to scientific knowledge and act as a basis for methodologists, guideline developers and researchers for the development of BRA to inform regulatory decisions, reimbursement and coverage decision making. The results will be disseminated through peer-reviewed articles, conferences, policy briefs and workshops.

TRIAL REGISTRATION NUMBER

Open Science Framework (https://doi.org/10.17605/OSF.IO/69T3V).

Regulatory benefit-risk assessment of oncology drugs: A systematic review of FDA and EMA approvals

The European Medicines Agency (EMA) and FDA have policy goals of strengthening benefit-risk (B-R) capabilities; but how this has been translating into regulatory practice is unclear. A systematic review of oncology drug approvals between 2015 and 2020 was conducted with approvals identified through review of FDA and EMA annual reports, with extraction of information on submission, clinical program and B-R assessment from publicly available review documents. Data were extracted from 236 reviews (EMA: 66 new submissions, 100 label extensions; FDA: 70 new submissions). The standard of evidence for B-R assessments seems to have diversified over time; yet, despite policy targets to extend their use, these assessments rarely include patient experience or real-world data.

Appropriate design and reporting of superiority, equivalence and non-inferiority clinical trials incorporating a benefit-risk assessment: the BRAINS study including expert workshop

Background

Randomised controlled trials are designed to assess the superiority, equivalence or non-inferiority of a new health technology, but which trial design should be used is not always obvious in practice. In particular, when using equivalence or non-inferiority designs, multiple outcomes of interest may be important for the success of a trial, despite the fact that usually only a single primary outcome is used to design the trial. Benefit-risk methods are used in the regulatory clinical trial setting to assess multiple outcomes and consider the trade-off of the benefits against the risks, but are not regularly implemented in publicly funded trials.

Objectives

The aim of the project is to aid the design of clinical trials with multiple outcomes of interest by defining when each trial design is appropriate to use and identifying when to use benefit-risk methods to assess outcome trade-offs (qualitatively or quantitatively) in a publicly funded trial setting.

Methods

A range of methods was used to elicit expert opinion to answer the project objectives, including a web-based survey of relevant researchers, a rapid review of current literature and a 2-day consensus workshop of experts (in 2019).

Results

We created a list of 19 factors to aid researchers in selecting the most appropriate trial design, containing the following overarching sections: population, intervention, comparator, outcomes, feasibility and perspectives. Six key reasons that indicate a benefit-risk method should be considered within a trial were identified: (1) when the success of the trial depends on more than one outcome; (2) when important outcomes within the trial are in competing directions (i.e. a health technology is better for one outcome, but worse for another); (3) to allow patient preferences to be included and directly influence trial results; (4) to provide transparency on subjective recommendations from a trial; (5) to provide consistency in the approach to presenting results from a trial; and (6) to synthesise multiple outcomes into a single metric. Further information was provided to support the use of benefit-risk methods in appropriate circumstances, including the following: methods identified from the review were collated into different groupings and described to aid the selection of a method; potential implementation of methods throughout the trial process were provided and discussed (with examples); and general considerations were described for those using benefit-risk methods. Finally, a checklist of five pieces of information that should be present when reporting benefit-risk methods was defined, with two additional items specifically for reporting the results.

Conclusions

These recommendations will assist research teams in selecting which trial design to use and deciding whether or not a benefit-risk method could be included to ensure research questions are answered appropriately. Additional information is provided to support consistent use and clear reporting of benefit-risk methods in the future. The recommendations can also be used by funding committees to confirm that appropriate considerations of the trial design have been made.

Limitations

This research was limited in scope and should be considered in conjunction with other trial design methodologies to assess appropriateness. In addition, further research is needed to provide concrete information about which benefit-risk methods are best to use in publicly funded trials, along with recommendations that are specific to each method.

Study registration

The rapid review is registered as PROSPERO CRD42019144882.

Funding

Funded by the Medical Research Council UK and the National Institute for Health and Care Research as part of the Medical Research Council-National Institute for Health and Care Research Methodology Research programme.

A Structured Benefit-Risk Assessment Operating Model for Investigational Medicinal Products in the Pharmaceutical Industry

Robust and transparent formal benefit-risk (BR) analyses for medicinal products represent a means to better understand the appropriate use of medicinal products, and to maximize their value to prescribers and patients. Despite regulatory and social imperatives to conduct structured BR (sBR) assessments, and the availability of a plethora of methodological tools, there exists large variability in the uptake and execution of sBR assessments among pharmaceutical companies. As such, in this paper we present an sBR assessment framework developed and implemented within a large global pharmaceutical company that aims to guide the systematic assessment of BR across the continuum of drug development activities, from first-time-in-human studies through to regulatory submission. We define and emphasize the concepts of Key Clinical Benefits and Key Safety Risks as the foundation for BR analysis. Furthermore, we define and foundationally employ the concepts of sBR and a Core Company BR position as the key elements for our BR framework. We outline 3 simple stages for how to perform the fundamentals of an sBR analysis, along with an emphasis on the weighting of Key Clinical Benefits and Key Safety Risks, and a focus on any surrounding uncertainties. Additionally, we clarify existing definitions to differentiate descriptive, semi-quantitative, and fully quantitative BR methodologies. By presenting our framework, we wish to stimulate productive conversation between industry peers and health authorities regarding best practice in the BR field. This paper may also help facilitate the pragmatic implementation of sBR methodologies for organizations without an established framework for such assessments.

Quantitative Benefit-Risk Assessment in Medical Product Decision Making: A Good Practices Report of an ISPOR Task Force

Benefit-risk assessment is commonly conducted by drug and medical device developers and regulators, to evaluate and communicate issues around benefit-risk balance of medical products. Quantitative benefit-risk assessment (qBRA) is a set of techniques that incorporate explicit outcome weighting within a formal analysis to evaluate the benefit-risk balance. This report describes emerging good practices for the 5 main steps of developing qBRAs based on the multicriteria decision analysis process. First, research question formulation needs to identify the needs of decision makers and requirements for preference data and specify the role of external experts. Second, the formal analysis model should be developed by selecting benefit and safety endpoints while eliminating double counting and considering attribute value dependence. Third, preference elicitation method needs to be chosen, attributes framed appropriately within the elicitation instrument, and quality of the data should be evaluated. Fourth, analysis may need to normalize the preference weights, base-case and sensitivity analyses should be conducted, and the effect of preference heterogeneity analyzed. Finally, results should be communicated efficiently to decision makers and other stakeholders. In addition to detailed recommendations, we provide a checklist for reporting qBRAs developed through a Delphi process conducted with 34 experts.

Global Landscape of Benefit-Risk Considerations for Medicinal Products: Current State and Future Directions

In the last decade there has been a significant increase in the literature discussing the use of benefit-risk methods in medical product (including devices) development. Government agencies, medical product industry groups, academia, and collaborative consortia have extensively discussed the advantages of structured benefit-risk assessments. However, the abundance of information has not resulted in a consistent way to utilize these findings in medical product development. Guidelines and papers on methods, even though well structured, have not led to a firm consensus on a clear and consistent approach. This paper summarizes the global landscape of benefit-risk considerations for product- or program-level decisions from available literature and regulatory guidance, providing the perspectives of three stakeholder groups-regulators, collaborative groups and consortia, and patients. The paper identifies key themes, potential impact on benefit-risk assessments, and significant future trends.

The value of anticancer drugs - a regulatory view

The high prices of new anticancer drugs and the marginal added benefit perceived by some stakeholders have fuelled a debate on the value of anticancer drugs in the European Union, even though an agreed definition of what constitutes a drug's value does not exist. In this Perspective, we discuss the value of drugs from different viewpoints and objectives of decision makers: for regulators, assessment of the benefit-risk balance of a drug is a cornerstone for approval; payers rely on cost-effectiveness analyses carried out by health technology assessment agencies for reimbursement decisions; for patients, treatment choices are based on personal preferences and attitudes to risk; and clinicians can use several scales (such as the ESMO Magnitude of Clinical Benefit Scale (ESMO-MCBS)) that have been developed as an attempt to measure value objectively. Although a unique definition that fully captures the concept of value is unlikely to emerge, herein we discuss the importance of understanding different perspectives, and how regulators can help to inform different decision makers.

Associations between uncertainties identified by the European Medicines Agency and national decision making on reimbursement by HTA agencies

We aimed to determine whether uncertainties identified by the European Medicines Agency (EMA) were associated with negative relative effectiveness assessments (REAs) and negative overall reimbursement recommendations by national health technology assessment (HTA) agencies. Therefore, we identified all HTA reports from Haute Autorité de Santé (HAS; France), National Institute for Health and Care Excellence (NICE; England/Wales), Scottish Medicine Consortium (SMC; Scotland), and Zorginstituut Nederland (ZIN; The Netherlands) for a cohort of innovative medicines that the EMA had approved in 2009 to 2010 (excluding vaccines). Uncertainty regarding pivotal trial methodology, clinical outcomes, and their clinical relevance were combined to reflect a low, medium, or high level of uncertainty. We assessed associations by calculating risk ratios (RRs) and 95% confidence intervals (CIs), and agreement between REA and overall reimbursement recommendation outcomes. We identified 36 medicines for which 121 reimbursement recommendations had been issued by the HTA agencies between September 2009 and July 2018. High versus low uncertainty was associated with an increased risk for negative REAs and negative overall reimbursement recommendations: RRs 1.9 (95% CI 0.9-3.9) and 1.6 (95% CI 0.7-3.5), respectively, which was supported by further sensitivity analyses. We identified a lack of agreement between 33 (27%) REA and overall reimbursement recommendation outcomes, which were mostly restricted recommendations that followed on negative REAs in case of low or medium uncertainty. In conclusion, high uncertainty identified by the EMA was associated with negative REAs and negative overall reimbursement recommendations. To reduce uncertainty and ultimately facilitate efficient patient access, regulators, HTA agencies, and other stakeholders should discuss how uncertainties should be weighed and addressed early in the drug life cycle of innovative treatments.

Risk-benefit in food safety and nutrition - Outcome of the 2019 Parma Summer School

Risk-benefit assessment is the comparison of the risk of a situation to its related benefits, i.e. a comparison of scenarios estimating the overall health impact. The risk-benefit analysis paradigm mirrors the classical risk analysis one: risk-benefit assessment goes hand-in-hand with risk-benefit management and risk-benefit communication. The various health effects associated with food consumption, together with the increasing demand for advice on healthy and safe diets, have led to the development of different research disciplines in food safety and nutrition. In this sense, there is a clear need for a holistic approach, including and comparing all of the relevant health risks and benefits. The risk-benefit assessment of foods is a valuable approach to estimate the overall impact of food on health. It aims to assess together the negative and positive health effects associated with food intake by integrating chemical and microbiological risk assessment with risk and benefit assessment in food safety and nutrition. The 2019 Parma Summer School on risk-benefit in food safety and nutrition had the objective was to provide an opportunity to learn from experts in the field of risk-benefit approach in food safety and nutrition, including theory, case studies, and communication of risk-benefit assessments plus identify challenges for the future. It was evident that whereas tools and approaches have been developed, more and more case studies have been performed which can form an inherent validation of the risk-benefit approach. Executed risk-benefit assessment case studies apply the steps and characteristics developed: a problem formulation (with at least 2 scenarios), a tiered approach until a decision can be made, one common currency to describe both beneficial and adverse effects (DALYs in most instances). It was concluded that risk-benefit assessment in food safety and nutrition is gaining more and more momentum, while also many challenges remain for the future. Risk-benefit is on the verge of really enrolling into the risk assessment and risk analysis paradigm. The interaction between risk-benefit assessors and risk-benefit managers is pivotal in this, as is the interaction with risk-benefit communicators.

A WHO tool for risk-based decision making on blood safety interventions

Background

Risk‐based decision making is increasingly recognized as key to support national blood policy makers and blood operators concerning the implementation of safety interventions, especially to address emerging infectious threats and new technology opportunities. There is an urgent need for practical decision support tools, especially for low‐ and middle‐income countries that may not have the financial or technical capability to develop risk models. WHO supported the development of such a tool for blood safety. The tool enables users to perform both a quantitative Multi‐Criteria Decision Assessment and a novel step‐by‐step qualitative assessment.

Study Design and Methods

This paper summarizes the content, functionalities, and added value of the new WHO tool. A fictitious case study of a safety intervention to reduce the risk of HIV transmission by transfusion was used to demonstrate the use and usefulness of the tool.

Results

Application of the tool highlighted strengths and weaknesses of both the quantitative and qualitative approaches. The quantitative approach facilitates assessment of the robustness of the decision but lacks nuances and interpretability especially when multiple constraints are taken into consideration. Conversely, while unable to provide an assessment of robustness, the step‐by‐step qualitative approach helps structuring the thought process and argumentation for a preferred intervention in a systematic manner.

Conclusion

The relative strengths and weaknesses of the quantitative and step‐by‐step qualitative approach to risk‐based decision making are complementary and mutually enhancing. A combination of the two approaches is therefore advisable to support the selection of appropriate blood safety interventions for a particular setting.

The utility of real-world evidence for benefit-risk assessment, communication, and evaluation of pharmaceuticals: Case studies

PURPOSE

In recent years, novel types of real-world evidence (RWE) have played a role in various decision-making processes relating to medicinal products, including regulatory approval, patient access, health technology assessment, safety monitoring, clinical use, and post-approval lifecycle management. We therefore reviewed the potential utility of RWE in the cycle of medicinal product benefit-risk (BR) assessment, communication/risk minimization and evaluation ("BRACE").

METHODS

A convenience sample of illustrative studies was drawn from the published literature and examined. Specifically, we examined the purpose for using RWE, the type of RWE used, its novelty and how it might be integrated with other data and activities of the BRACE cycle, and how it contributed to regulatory decision-making.

RESULTS

Eight studies were selected with each illustrating a different activity in the BRACE cycle ranging from BR assessment in the preapproval setting, post-approval assessment of safety or effectiveness, communicating BR information to patients and healthcare professionals, and evaluating the effectiveness of risk minimization initiatives to support a positive BR balance.

CONCLUSIONS

RWE has an important role in informing regulatory decision-making regarding the BR management of medicines. With increasing digitalization, facilitating data collection and stakeholder engagement in health, this role is only expected to expand in the future. To reach the full potential of RWE, both regulators and sponsors will need to be familiar with a range of existing and emerging methods for generating and analyzing such evidence appropriately and achieve convergence regarding how different types of RWE can best be used to inform BR management and decision-making.

Quantitative Benefit-Risk Assessment: State of the Practice Within Industry

BACKGROUND

Benefit-risk assessments for medicinal products and devices have advanced significantly over the past decade. The purpose of this study was to characterize the extent to which the life sciences industry is utilizing quantitative benefit-risk assessment (qBRA) methods.

METHODS

Semi-structured interviews were conducted with a sample of industry professionals working in drug and/or medical device benefit-risk assessments (n = 20). Questions focused on the use, timing, and impact of qBRA; implementation challenges; and future plans. Interviews were recorded, transcribed, and coded for thematic analysis.

RESULTS

While most surveyed companies had applied qBRA, application was limited to a small number of assets-primarily to support internal decision-making and regulatory submissions. Positive impacts associated with use included improved team decision-making and communication. Multi-criteria decision analysis and discrete choice experiment were the most frequently utilized qBRA methods. A key challenge of qBRA use was the lack of clarity regarding its value proposition. Championing by senior company leadership and receptivity of regulators to such analyses were cited as important catalysts for successful adoption of qBRA. Investment in qBRA methods, via capability building and pilot studies, was also under way in some instances.

CONCLUSION

qBRA application within this sample of life sciences companies was widespread, but concentrated in a small fraction of assets. Its use was primarily for internal decision-making or regulatory submissions. While some companies had plans to build further capacity in this area, others were waiting for further regulatory guidance before doing so.

Postmarket Safety Communication for Protection of Public Health: A Comparison of Regulatory Policy in Australia, Canada, the European Union, and the United States

In the wake of the withdrawal of the nonsteroidal anti-inflammatory drug rofecoxib, regulators worldwide reconsidered their approach to postmarket safety. Many have since adopted a "life cycle" approach to regulation of medicines, facilitating faster approval of new medicines while planning for potential postmarket safety issues. A crucial aspect of postmarket safety is the effective and timely communication of emerging risk information using postmarket safety advisories, commonly issued as letters to healthcare professionals, drug safety bulletins, media alerts, and website announcements. Yet regulators differ in their use of postmarket safety advisories. We examined the capacity of regulators in the United States, Europe, Canada, and Australia to warn about postmarket safety issues through safety advisories by assessing their governance, legislative authority, risk communication capabilities, and transparency.

Benefit-Risk Assessment of Esketamine Nasal Spray vs. Placebo in Treatment-Resistant Depression

This post hoc analysis assessed the benefit–risk profile of esketamine nasal spray + oral antidepressant (AD) induction and maintenance treatment in patients with treatment‐resistant depression (TRD). The Benefit–Risk Action Team framework was utilized to assess the benefit–risk profile using data from three induction studies and one maintenance study. Benefits were proportion of remitters or responders in induction studies and proportion of stable remitters or stable responders who remained relapse‐free in the maintenance study. Risks were death, suicidal ideation, most common adverse events (AEs), and potential long‐term risks. Per 100 patients on esketamine + AD vs. AD + placebo in induction therapy, 5–21 additional patients would remit and 14–17 additional patients would respond. In maintenance therapy, 19–32 fewer relapses would occur with esketamine. In both cases, there was little difference in serious or severe common AEs (primarily dissociation, vertigo, and dizziness). These findings support a positive benefit–risk balance for esketamine + AD as induction and maintenance treatment in patients with TRD.

Early Health Technology Assessment during Nonalcoholic Steatohepatitis Drug Development: A Two-Round, Cross-Country, Multicriteria Decision Analysis

Background. The assessment of value along the clinical development of new biopharmaceutical compounds is a challenging task. Complex and uncertain evidence has to be analyzed, considering a multitude of value preferences from different stakeholders. Objective. To investigate the use of multicriteria decision analysis (MCDA) to support decision making during drug development while considering payer and health technology assessment (HTA) value concerns, by applying the Advance Value Framework in nonalcoholic steatohepatitis (NASH) and testing for the consistency of the results. Design. A multiattribute value theory methodology was applied and 2 rounds of decision conferences (DCs) were organized in 3 countries (England, France, and Germany), with the participation of national key experts and stakeholders using the MACBETH questioning protocol and algorithm. A total of 51 health care professionals, patient advocates, and methodologists, including (ex-) committee members or assessors from national HTA bodies, participated in 6 DCs in the study countries. Target Population. NASH patients in fibrosis stages F2 to 3 were considered. Interventions. The value of a hypothetical product profile was assessed against 3 compounds under development using their phase 2 results. Outcome Measures. DC participants' value preferences were elicited involving criteria selection, options scoring, and criteria weighting. Results. Highly consistent valuation rankings were observed in all DCs, always favoring the same compound. Highly consistent rankings of criteria clusters were observed, favoring therapeutic benefit criteria, followed by safety profile and innovation level criteria. Limitations. There was a lack of comparative treatment effects, early evidence on surrogate endpoints was used, and stakeholder representativeness was limited in some DCs. Conclusions. The use of MCDA is promising in supporting early HTA, illustrating high consistency in results across countries and between study rounds.

Can Standardisation of the Public Assessment Report Improve Benefit-Risk Communication?

Background

National regulatory authorities (NRAs) make the decision to register a medicine based on an assessment of its benefits and risks and publicly available assessment reports are used as a tool to communicate the basis for the decision. The Universal Methodology for Benefit-Risk Assessment (UMBRA) has also been used to effectively communicate the basis of regulatory decisions. Many NRAs in emerging markets place reliance on the public assessment reports (PARs) of reference agencies to inform about their own regulatory decisions. However, PAR users often criticise the redacted nature of PARs and may be challenged in identifying key benefits and risks, value judgements, and benefit-risk (BR) trade-offs.

Methods

PARs for ertugliflozin l-pyroglutamic acid, erenumab, and durvalumab published by regulatory bodies in Australia, Europe, Canada, and the United States were compared with the validated UMBRA Benefit-Risk Template to evaluate the BR decision documentation. Published validation of UMBRA included report of a consortium of four regulatory authorities in Australia, Canada, Switzerland, and Singapore indicating that their clinical assessment templates were modified to align with the UMBRA approach. A focus group discussed the use of PARs as potential knowledge management tools for stakeholder understanding of regulatory decision making. The South African Health Product Regulatory Authority (SAHPRA) approach to document and communicate the BR decisions was evaluated.

Results

Results indicate key elements to include in the PARs including regulatory history, an effects table and a record of the strengths and uncertainties for each benefit and risk. Focus group participants agreed that a harmonised PAR template would support improved regulatory decision-making transparency. SAHPRA communication of BR decisions could be improved through the use of the UMBRA BR Template as a guidance for BR assessment and the basis of the South Africa public assessment report format.

Conclusion

SAHPRA's use of a structured template that supports transparent and quality decision making could have a major impact in ensuring consistency in the BR assessment of new medicines. The implementation of this effective approach for communicating BR decisions will advance agency goals of being a trusted, responsive, accountable regulatory body in which all healthcare stakeholders may rely on with confidence.

Health Technology Agency insights: informing modification of a qualitative benefit risk framework for Health Technology Reassessment of prescription medications

OBJECTIVES

This study's intent was to determine if a qualitative benefit risk framework could be used or modified to further enable Health Technology Reassessment (HTR) of prescription medicine recommendations. The purpose of this research was to understand Canadian Health Technology Agency assessors past experiences and insights to inform any modifications to the Universal Methodology for Benefit-Risk Assessment (UMBRA) qualitative framework. The UMBRA framework consists of an eight-step process, used during the assessment phase, to aid in decision making and dissemination.

METHODS

A qualitative descriptive study was conducted and included a purposeful, criterion-based sample of eight assessors who had participated in Health Technology Assessment (HTA) or HTR for prescription medicines or in qualitative decision-making frameworks.

RESULTS

Participant interviews lead to four common themes: "adoption of a qualitative benefit risk framework," "data (either too much or not enough)," "importance of incorporating stakeholder values," and "feasibility of the UMBRA framework." Methodological challenges with HTR were highlighted including the lack of clinical outcome data and the ability to compare clinically relevant meaningful differences. The implementation of a ranking or weighing process found within the UMBRA framework was not favored by half of the participants.

CONCLUSIONS

Research participants did not consider all steps of the UMBRA framework to be transferable to the assessment phase of HTR given the need for simplicity, resource efficiency, and stakeholder input throughout the process. The assessor experiences and insights and the resultant key themes can be used in future research to aid in the development of a qualitative recommendation framework for HTR.

Quantifying Preferences in Drug Benefit-Risk Decisions

Benefit-risk assessment is used in various phases along the drug lifecycle, such as marketing authorization and surveillance, health technology assessment (HTA), and clinical decisions, to understand whether, and for which patients, a drug has a favorable or more valuable profile with reference to one or more comparators. Such assessments are inherently preference-based as several clinical and nonclinical outcomes of varying importance might act as evaluation criteria, and decision makers must establish acceptable trade-offs between these outcomes. Different healthcare stakeholder perspectives, such as those from patients and healthcare professionals, are key for informing benefit-risk trade-offs. However, the degree to which such preferences inform the decision is often unclear as formal preference-based evaluation frameworks are generally not used for regulatory decisions, and, if used, rarely communicated in HTA decisions. We argue that for better decisions, as well as for reasons of transparency, preferences in benefit-risk decisions should more often be quantified and communicated explicitly.

Regulatory and Evidence Requirements and the Changing Landscape in Regulation for Marketing Authorisation

In this chapter, we describe the changing landscape of the EU pharmaceutical legislation concerning regulation and evidence requirements for marketing authorisation. First, we describe the legal requirements for marketing authorisation and the development of EU pharmaceutical legislation and the concept of risk-benefit balance. Second, we describe special types of authorisation, such as conditional approval and approval under exceptional circumstances, and special provisions such as incentives for orphan medicinal products and paediatric investigational plans. Lastly, we describe the available methodological guidelines focussing on choice of endpoints.

A decade of marketing approval of gene and cell-based therapies in the United States, European Union and Japan: An evaluation of regulatory decision-making

There is a widely held expectation of clinical advance with the development of gene and cell-based therapies (GCTs). Yet, establishing benefits and risks is highly uncertain. We examine differences in decision-making for GCT approval between jurisdictions by comparing regulatory assessment procedures in the United States (US), European Union (EU) and Japan. A cohort of 18 assessment procedures was analyzed by comparing product characteristics, evidentiary and non-evidentiary factors considered for approval and post-marketing risk management. Product characteristics are very heterogeneous and only three products are marketed in multiple jurisdictions. Almost half of all approved GCTs received an orphan designation. Overall, confirmatory evidence or indications of clinical benefit were evident in US and EU applications, whereas in Japan approval was solely granted based on non-confirmatory evidence. Due to scientific uncertainties and safety risks, substantial post-marketing risk management activities were requested in the EU and Japan. EU and Japanese authorities often took unmet medical needs into consideration in decision-making for approval. These observations underline the effects of implemented legislation in these two jurisdictions that facilitate an adaptive approach to licensing. In the US, the recent assessments of two chimeric antigen receptor-T cell (CAR-T) products are suggestive of a trend toward a more permissive approach for GCT approval under recent reforms, in contrast to a more binary decision-making approach for previous approvals. It indicates that all three regulatory agencies are currently willing to take risks by approving GCTs with scientific uncertainties and safety risks, urging them to pay accurate attention to post-marketing risk management.

Adverse Drug Event Discovery Using Biomedical Literature: A Big Data Neural Network Adventure

BACKGROUND

The study of adverse drug events (ADEs) is a tenured topic in medical literature. In recent years, increasing numbers of scientific articles and health-related social media posts have been generated and shared daily, albeit with very limited use for ADE study and with little known about the content with respect to ADEs.

OBJECTIVE

The aim of this study was to develop a big data analytics strategy that mines the content of scientific articles and health-related Web-based social media to detect and identify ADEs.

METHODS

We analyzed the following two data sources: (1) biomedical articles and (2) health-related social media blog posts. We developed an intelligent and scalable text mining solution on big data infrastructures composed of Apache Spark, natural language processing, and machine learning. This was combined with an Elasticsearch No-SQL distributed database to explore and visualize ADEs.

RESULTS

The accuracy, precision, recall, and area under receiver operating characteristic of the system were 92.7%, 93.6%, 93.0%, and 0.905, respectively, and showed better results in comparison with traditional approaches in the literature. This work not only detected and classified ADE sentences from big data biomedical literature but also scientifically visualized ADE interactions.

CONCLUSIONS

To the best of our knowledge, this work is the first to investigate a big data machine learning strategy for ADE discovery on massive datasets downloaded from PubMed Central and social media. This contribution illustrates possible capacities in big data biomedical text analysis using advanced computational methods with real-time update from new data published on a daily basis.

Evaluating Quality of Decision-Making Processes in Medicines' Development, Regulatory Review, and Health Technology Assessment: A Systematic Review of the Literature

Introduction: Although pharmaceutical companies, regulatory authorities, and health technology assessment (HTA) agencies have been increasingly using decision-making frameworks, it is not certain whether these enable better quality decision making. This could be addressed by formally evaluating the quality of decision-making process within those organizations. The aim of this literature review was to identify current techniques (tools, questionnaires, surveys, and studies) for measuring the quality of the decision-making process across the three stakeholders.

Methods: Using MEDLINE, Web of Knowledge, and other Internet-based search engines, a literature review was performed to systematically identify techniques for assessing quality of decision making in medicines development, regulatory review, and HTA. A structured search was applied using key words and a secondary review was carried out. In addition, the measurement properties of each technique were assessed and compared. Ten Quality Decision-Making Practices (QDMPs) developed previously were then used as a framework for the evaluation of techniques identified in the review. Due to the variation in studies identified, meta-analysis was inappropriate.

Results: This review identified 13 techniques, where 7 were developed specifically to assess decision making in medicines' development, regulatory review, or HTA; 2 examined corporate decision making, and 4 general decision making. Regarding how closely each technique conformed to the 10 QDMPs, the 13 techniques assessed a median of 6 QDMPs, with a mode of 3 QDMPs. Only 2 techniques evaluated all 10 QDMPs, namely the Organizational IQ and the Quality of Decision Making Orientation Scheme (QoDoS), of which only one technique, QoDoS could be applied to assess decision making of both individuals and organizations, and it possessed generalizability to capture issues relevant to companies as well as regulatory authorities.

Conclusion: This review confirmed a general paucity of research in this area, particularly regarding the development and systematic application of techniques for evaluating quality decision making, with no consensus around a gold standard. This review has identified QoDoS as the most promising available technique for assessing decision making in the lifecycle of medicines and the next steps would be to further test its validity, sensitivity, and reliability.

Factors related to drug approvals: predictors of outcome?

There is growing interest in characterising factors associated with positive regulatory outcomes for drug marketing authorisations. We assessed empirical studies published over the past 15 years seeking to identify predictive factors. Factors were classified to one of four 'factor clusters': evidentiary support; product or indication characteristics; company experience or strategy; social and regulatory factors. We observed a heterogeneous mix of technical factors (e.g., study designs, clinical evidence of efficacy) and less studied social factors (e.g., company-regulator interactions). We confirmed factors known to be of relevance to drug approval decisions (imperative) and a cohort of less understood (compensatory) social factors. Having robust supportive clinical evidence, addressing rare or serious illness, following scientific advice and prior company experience were associated with positive outcomes, which illustrated the multifactorial nature of regulatory decision making and factors need to be considered holistically while having varying, context-dependent importance.

Assessing the Quality of Decision Making in the Development and Regulatory Review of Medicines: Identifying Biases and Best Practices

BACKGROUND

Although the quality of decision making (QDM) in the development and regulatory review of medicines influences the delivery of new products, there appears to be no suitable instrument to assess QDM in this area. The aim of this study was to assess differences in QDM using a validated instrument, the Quality of Decision-Making Orientation Scheme (QoDoS), to identify best practices and biases affecting individuals and their organization, as well as to assess differences in decision-making behaviors between pharmaceutical companies and regulatory agencies. QoDoS also enables the measurement against 10 quality decision-making practices (QDMPs) that underpin a quality process.

METHODS

QoDoS, consisting of 47 items that assess individual and organizational decision-making approaches and influences, was completed by 76 participants from regulatory agencies and pharmaceutical companies.

RESULTS

Having a systematic, structured approach to aid in decision making is achieved to a greater extent at an individual level (72%) compared with that of the organization (38%). Key differences between company and agency decision making were uncovered. While it was recognized that both stakeholders felt that their decision making could be improved (100% agencies; 92% companies), training in the science of decision making was rarely provided.

CONCLUSIONS

QoDoS has the ability to measure differences in QDM between individuals and organizations within companies and agencies. The benefits of assessing QDMPs with QoDoS include enabling an increased awareness of biases and best practices that should be incorporated into a decision-making framework; increasing productivity and reducing uncertainty around decision making, thereby resulting in more predictable outcomes for organizations. In addition, it provides a basis for discussion of the issues in decision making within an organization as well as between stakeholders to encourage a level of partnership. Finally, measurements of QDM will enable trust, consistency, transparency, and timeliness to be built into critical decisions that affect medicines' availability.

Discrepancies between multicriteria decision analysis-based ranking and intuitive ranking for pharmaceutical benefit-risk profiles in a hypothetical setting

WHAT IS KNOWN AND OBJECTIVE

Multicriteria decision analysis (MCDA) has been generally considered a promising decision-making methodology for the assessment of drug benefit-risk profiles. There have been many discussions in both public and private sectors on its feasibility and applicability, but it has not been employed in official decision-makings. For the purpose of examining to what extent MCDA would reflect the first-hand, intuitive preference of evaluators in practical pharmaceutical assessments, we conducted a questionnaire survey involving the participation of employees of pharmaceutical companies.

METHODS

Showing profiles of the efficacy and safety of four hypothetical drugs, each respondent was asked to rank them following the standard MCDA process and then to rank them intuitively (i.e. without applying any analytical framework).

RESULTS AND DISCUSSION

These two approaches resulted in substantially different ranking patterns from the same individuals, and the concordance rate was surprisingly low (17%). Although many respondents intuitively showed a preference for mild, balanced risk-benefit profiles over profiles with a conspicuous advantage in either risk or benefit, the ranking orders based on MCDA scores did not reflect the intuitive preference.

WHAT IS NEW AND CONCLUSION

Observed discrepancies between the rankings seemed to be primarily attributed to the structural characteristics of MCDA, which assumes that evaluation on each benefit and risk component should have monotonic impact on final scores. It would be difficult for MCDA to reflect commonly observed non-monotonic preferences for risk and benefit profiles. Possible drawbacks of MCDA should be further investigated prior to the real-world application of its benefit-risk assessment.

Quality of Regulatory Decision-Making Practices: Issues Facing Companies and Agencies

BACKGROUND

The science of decision making is well established, although in reality it is a mixture of science and art. What is currently lacking is research into decision making in medicines research and development. The aims of this study were to determine the current decision-making practices and methodologies for measuring the quality of the decision making and the barriers and solutions for making quality decisions within pharmaceutical companies and regulatory agencies.

METHODS

Two analogous questionnaires were developed for use in this study. Fourteen agencies and 25 companies were asked to complete the questionnaire, assessing their decision making for submitting or approving a new drug application.

RESULTS

The 68% and 71% response rate from companies and agencies, respectively, suggests interest in this topic, but the area is largely unexplored within the pharmaceutical environment. Moreover, all companies and 90% of the agencies believed that decision making at their organizations could be improved. Although both stakeholders have, to some extent, already implemented frameworks and various methodologies, these are often informal and unsystematic. Challenges remain and there is a need to change the organizational culture by increasing the awareness of the quality aspect in decision making.

CONCLUSIONS

The findings of this study support the need to further characterize and assess the practices and behaviors of individuals and organizations. Furthermore, the barriers, mainly relating to the influence of biases, should be addressed by developing the general principles of a formal quality decision framework and identifying quality decision-making practices in order to ensure that structured decisions are made throughout the life cycle of medicines.