Postmarket Safety Outcomes for New Molecular Entity (NME) Drugs Approved by the Food and Drug Administration Between 2002 and 2014

Are Causal Statements Reported in Pharmacovigilance Disproportionality Analyses Using Individual Case Safety Reports Exaggerated in Related Citations? A Meta-epidemiological Study

BACKGROUND

Previous meta-epidemiological surveys have found considerable misinterpretation of results of disproportionality analyses. We aim to explore the relationship between the strength of causal statements used in title and abstract conclusions of pharmacovigilance disproportionality analyses and the strength of causal language used in citing studies.

METHODS

On March 30, 2022, we selected the 30 disproportionality studies with the highest Altmetric Attention Scores. For each article, we extracted all citing studies using the Dimension database (n = 1434). In parallel, two authors assessed the strength of causal statements in the title and abstract conclusions of source articles and in the paragraph of citing studies. Based on previous studies, the strength of causal language was quantified based on a four-level scale (1-appropriate interpretation; 2-ambiguous interpretation; 3-conditionally causal; 4-unconditionally causal). Discrepancies were solved by discussion until consensus among the team. We assessed the association between the strength of causal statements in source articles and citing studies, separately for the title and abstract conclusions, through multinomial regression models.

RESULTS

Overall, 27% (n = 8) of source studies used unconditionally causal statements in their title, 30% (n = 9) in their abstract conclusion, and 17% (n = 5) in both. Only 20% (n = 6) used appropriate statements in their title and in their abstract's conclusions. Among the 622 citing studies analyzed, 285 (45.8%) used unconditionally causal statements when referring to the findings from disproportionality analysis, and only 164 (26.4%) used appropriate language. Multinomial models found that the strength of causal statements in citing studies was positively associated with the strength of causal language used in abstract conclusions of source articles (Likelihood Ratio Test (LogLRT) p < 0.00001) but not in the titles. In particular, among studies citing source articles with appropriate interpretation, 30.2% (95% confidence interval [CI] 22.8-37.6) contained unconditionally causal statements in their abstract conclusions, versus 56.4% (95% CI 48.7-64.2) for studies citing source articles with unconditionally causal statements.

CONCLUSIONS

Nearly half of the studies citing pharmacovigilance disproportionality analyses results used causal claims, particularly when the causal language used in the source article was stronger. There is a need for higher caution when writing, interpreting, and citing disproportionality studies.

FDA Adverse Event Reporting System (FAERS) Essentials: A Guide to Understanding, Applying, and Interpreting Adverse Event Data Reported to FAERS

The US Food and Drug Administration (FDA) performs safety assessments throughout the life cycle of a drug. Postmarketing safety surveillance promotes the identification of adverse events not known at the time of approval. Adverse event reports, also called individual case safety reports (ICSRs), submitted to FDA, are collected and stored in the FDA Adverse Event Reporting System (FAERS). ICSRs stored in FAERS may be reviewed-along with multiple other data sources-to detect potential safety signals and to perform a thorough evaluation to determine if a causal association exists between a drug and an adverse event. Although FAERS is a powerful tool for drug safety surveillance and assessment, understanding the content, application, and proper interpretation of the data contained in FAERS is necessary to reach scientifically and medically accurate conclusions and contextualize findings. This article aims to highlight considerations and explain fundamental concepts of FAERS to promote accurate analyses and appropriate interpretation of the data available.

Deduplicating the FDA adverse event reporting system with a novel application of network-based grouping

OBJECTIVE

To improve the reliability of data mining for product safety concerns in the Food and Drug Administration's (FDA) Adverse Event Reporting System (FAERS) by robustly identifying duplicate reports describing the same patient experience.

MATERIALS AND METHODS

A duplicate detection algorithm based on a probabilistic record linkage algorithm, including features extracted from report narratives, and designed to support FAERS case safety review as part of the Information Visualization Platform (InfoViP) has been upgraded into a full deduplication pipeline for the entire FAERS database. The pipeline contains several new and updated components, including a network analysis-based community detection routine for breaking up sparsely connected groups of duplicates constructed from chains of pairwise comparisons. The pipeline was applied to all 29 million FAERS reports to assemble groups of duplicate cases.

RESULTS

The pipeline was evaluated on 12 human expert adjudicated data sets with a total of 2300 reports and was found to have better overall performance than the current tool used at the FDA for labeling duplicates on 10 of them, with F1 scores ranging from 0.36 to 0.93, with half above 0.75. Because minimizing false discovery increases human expert review efficiency, the improved deduplication pipeline was applied to all historic and daily incoming FAERS reports at FDA and identified about 5 million reports as duplicates.

CONCLUSIONS

The InfoViP deduplication pipeline is operating at FDA to identify duplicate case reports in FAERS and provide deduplicated input for improved efficiency and accuracy of safety review operations like adverse event data mining calculations.

First-In-Class Drugs Experienced Different Regulatory Treatment In The US And Europe

First-in-class drugs can be highly innovative because of their novel mechanisms of action, but they also carry uncertainty in the absence of clinical experience. To understand how such drugs advance through development to enter the market, we investigated Food and Drug Administration (FDA) approval data for 186 first-in-class drugs (2013-23) and data for 121 drugs approved by both the FDA and the European Medicines Agency (EMA; 2013-22), focusing on review durations, expedited program use, and characteristics of pivotal efficacy trials. The FDA applied substantial regulatory flexibility to first-in-class drugs, with 50 percent lacking clinical endpoints and 30 percent lacking blinding and comparator drugs in the pivotal trials. This flexibility was particularly evident in cancer drugs, for which up to 90 percent lacked clinical endpoints and blinding. The FDA designated 81 percent of first-in-class drugs for expedited programs compared with 30 percent designated by the EMA. Review durations varied by therapeutic area, ranging from 7.7 months to 14.5 months at the FDA, and were slightly slower at the EMA. Regulators need to carefully balance flexibilities with rigorous assessments of evidence for first-in-class drugs.

Description and Validation of a Novel AI Tool, LabelComp, for the Identification of Adverse Event Changes in FDA Labeling

INTRODUCTION

The accurate identification and timely updating of adverse reactions in drug labeling are crucial for patient safety and effective drug use. Postmarketing surveillance plays a pivotal role in identifying previously undetected adverse events (AEs) that emerge when a drug is used in broader and more diverse patient populations. However, traditional methods of updating drug labeling with new AE information have been manual, time consuming, and error prone. This paper introduces the LabelComp tool, an innovative artificial intelligence (AI) tool designed to enhance the efficiency and accuracy of postmarketing drug safety surveillance. Utilizing a combination of text analytics and a trained Bidirectional Encoder Representations from Transformers (BERT) model, the LabelComp tool automatically identifies changes in AE terms from updated drug labeling documents.

OBJECTIVE

Our objective was to create and validate an AI tool with high accuracy that could enable researchers and FDA reviewers to efficiently identify safety-related drug labeling changes.

RESULTS

Our validation study of 87 drug labeling PDF pairs demonstrates the tool's high accuracy, with F1 scores of overall performance ranging from 0.795 to 0.936 across different evaluation tiers and a recall of at least 0.997 with only one missed AE out of 483 total AEs detected, indicating the tool's efficacy in identifying new AEs.

CONCLUSION

The LabelComp tool can support drug safety surveillance and inform regulatory decision-making. The publication of this tool also aims to encourage further community-driven enhancements, aligning with broader interests in applying AI to advance regulatory science and public health.

Trust but Verify: Lessons Learned for the Application of AI to Case-Based Clinical Decision-Making From Postmarketing Drug Safety Assessment at the US Food and Drug Administration

Adverse drug reactions are a common cause of morbidity in health care. The US Food and Drug Administration (FDA) evaluates individual case safety reports of adverse events (AEs) after submission to the FDA Adverse Event Reporting System as part of its surveillance activities. Over the past decade, the FDA has explored the application of artificial intelligence (AI) to evaluate these reports to improve the efficiency and scientific rigor of the process. However, a gap remains between AI algorithm development and deployment. This viewpoint aims to describe the lessons learned from our experience and research needed to address both general issues in case-based reasoning using AI and specific needs for individual case safety report assessment. Beginning with the recognition that the trustworthiness of the AI algorithm is the main determinant of its acceptance by human experts, we apply the Diffusion of Innovations theory to help explain why certain algorithms for evaluating AEs at the FDA were accepted by safety reviewers and others were not. This analysis reveals that the process by which clinicians decide from case reports whether a drug is likely to cause an AE is not well defined beyond general principles. This makes the development of high performing, transparent, and explainable AI algorithms challenging, leading to a lack of trust by the safety reviewers. Even accounting for the introduction of large language models, the pharmacovigilance community needs an improved understanding of causal inference and of the cognitive framework for determining the causal relationship between a drug and an AE. We describe specific future research directions that underpin facilitating implementation and trust in AI for drug safety applications, including improved methods for measuring and controlling of algorithmic uncertainty, computational reproducibility, and clear articulation of a cognitive framework for causal inference in case-based reasoning.

A Comparison of Signals of Designated Medical Events and Non-designated Medical Events: Results from a Scoping Review

INTRODUCTION AND OBJECTIVE

The European Medicines Agency (EMA) maintains a list of designated medical events (DMEs), events that are inherently serious and are prioritized for signal detection, irrespective of statistical criteria. We have analysed the results of our previously published scoping review to determine whether DME signals differ from those of other adverse events in terms of time to communication and characteristics of supporting reports of suspected adverse drug reactions.

METHODS

For all signals, we obtained the launch year of medicinal products from textbooks or regulatory agencies, extracted the year of the first report in VigiBase and calculated the interval between the first report and communication (time to communication, TTC). We further retrieved the average completeness (via vigiGrade) of the reports in each case series in the years before the communication. We categorised as DME signals those concerning an event in the EMA's list. We described the two groups of signals using medians and interquartile ranges (IQR) and compared them using the Brunner-Munzel test, calculating 95% confidence intervals (95% CI) and P values.

RESULTS

Of 4520 signals, 919 concerned DMEs and 3601 concerned non-DMEs. Signals of DMEs were supported by a median of 15 reports (IQR 6-38 reports) with a completeness score of 0.52 (IQR 0.43-0.62) and signals of non-DMEs by 20 reports (IQR 6-84 reports) with a completeness score of 0.46 (IQR 0.38-0.56). The probability that a random DME signal was supported by fewer reports than non-DME signals was 0.56 (95% CI 0.54-0.58, P < 0.001) and that of one having lower average completeness was 0.39 (95% CI 0.36-0.41, P < 0.001). The median TTCs of DME and non-DME signals did not differ (10 years), but the TTC was as low as 2 years when signals (irrespective of classification) were supported by reports whose average completeness was > 0.80.

CONCLUSIONS

Signals of designated medical events were supported by fewer reports and higher completeness scores than signals of other adverse events. Although statistically significant, the differences in effect sizes between the two groups were small. This suggests that listing certain adverse events as DMEs is not having the expected effect of encouraging a focus on reports of the types of suspected adverse reactions that deserve special attention. Further enhancing the completeness of the reports of suspected adverse drug reactions supporting signals of designated medical events might shorten their time to communication and reduce the number of reports required to support them.

Postmarket safety communications on drugs approved in Japan: A 25-year analysis

Drug safety communications (DSCs) are essential tools for communicating important postmarket serious drug safety information to healthcare professionals and patients. Previous studies characterized DSCs issued by the U.S. Food and Drug Administration (FDA) and the European Medicines Agency (EMA); however, knowledge about the activities of the Pharmaceuticals and Medical Devices Agency (PMDA)/the Ministry of Health, Labor and Welfare (MHLW) is limited. This study characterized DSCs by the PMDA/MHLW in comparison with previously reported DSCs by the FDA and the EMA. We retrospectively analyzed 37 DSCs of 41 adverse drug reactions (ADRs) for 33 drugs in Japan from 1997 to 2022. Most DSCs were related to non-oncology drugs (30/37, 81.1%), and the median (interquartile range) time from approval to DSC issuance was 19 (10-51) months. Notably, the regulatory review reports and the latest labels before DSC issuance did not describe 16/28 (57.1%) and 12/37 (32.4%) of the ADRs related to DSCs, respectively. Most DSCs resulted in label revisions (36/37, 97.3%) and seven drugs were eventually withdrawn. Some DSC characteristics are similar among the PMDA/MHLW, the FDA, and the EMA; however, the number, contents, and range of new safety issues addressed by DSCs differ among the three jurisdictions. Our study emphasized the importance of continuous efforts to gather postmarket drug safety information because substantial ADRs that led to DSCs were recognized after approval and were associated with critical label revisions and withdrawals. Future studies are required to address global challenges for regulatory harmonization of safety-related regulatory actions.

An Expedition on Synthetic Methodology of FDA-approved Anticancer Drugs (2018-2021)

New drugs being established in the market every year produce specified structures for selective biological targeting. With medicinal insights into molecular recognition, these begot molecules open new rooms for designing potential new drug molecules. In this review, we report the compilation and analysis of a total of 56 drugs including 33 organic small molecules (Mobocertinib, Infigratinib, Sotorasib, Trilaciclib, Umbralisib, Tepotinib, Relugolix, Pralsetinib, Decitabine, Ripretinib, Selpercatinib, Capmatinib, Pemigatinib, Tucatinib, Selumetinib, Tazemetostat, Avapritinib, Zanubrutinib, Entrectinib, Pexidartinib, Darolutamide, Selinexor, Alpelisib, Erdafitinib, Gilteritinib, Larotrectinib, Glasdegib, Lorlatinib, Talazoparib, Dacomitinib, Duvelisib, Ivosidenib, Apalutamide), 6 metal complexes (Edotreotide Gallium Ga-68, fluoroestradiol F-18, Cu 64 dotatate, Gallium 68 PSMA-11, Piflufolastat F-18, 177Lu (lutetium)), 16 macromolecules as monoclonal antibody conjugates (Brentuximabvedotin, Amivantamab-vmjw, Loncastuximabtesirine, Dostarlimab, Margetuximab, Naxitamab, Belantamabmafodotin, Tafasitamab, Inebilizumab, SacituzumabGovitecan, Isatuximab, Trastuzumab, Enfortumabvedotin, Polatuzumab, Cemiplimab, Mogamulizumab) and 1 peptide enzyme (Erwiniachrysanthemi-derived asparaginase) approved by the U.S. FDA between 2018 to 2021. These drugs act as anticancer agents against various cancer types, especially non-small cell lung, lymphoma, breast, prostate, multiple myeloma, neuroendocrine tumor, cervical, bladder, cholangiocarcinoma, myeloid leukemia, gastrointestinal, neuroblastoma, thyroid, epithelioid and cutaneous squamous cell carcinoma. The review comprises the key structural features, approval times, target selectivity, mechanisms of action, therapeutic indication, formulations, and possible synthetic approaches of these approved drugs. These crucial details will benefit the scientific community for futuristic new developments in this arena.

An overview of methodological flaws of real-world studies investigating drug safety in the post-marketing setting

INTRODUCTION

The evaluation of the post-marketing safety profile of drugs is a continuous monitoring process for approved and marketed medicines and it is crucial for detecting new adverse drug reactions. As such, real-world studies are essential to complement pre-marketing evidence with information concerning drug risk-benefit profile and use in wider patient populations and they have a great potential to support post-marketing drug safety evaluations.

AREAS COVERED

A detailed description of the main limitations of real-world data sources (i.e. claims databases, electronic healthcare records, drug/disease registers and spontaneous reporting system databases) and of the main methodological challenges of real-world studies in generating real-world evidence is provided.

EXPERT OPINION

Real-world evidence biases can be ascribed to both the methodological approach and the specific limitations of the different real-world data sources used to carry out the study. As such, it is crucial to characterize the quality of real-world data, by establishing guidelines and best practices for the assessment of data fitness for purpose. On the other hand, it is important that real-world studies are conducted using a rigorous methodology, aimed at minimizing the risk of bias.

Trends in the Quality of Evidence Supporting FDA Drug Approvals: Results from a Literature Review

CONTEXT

New drug approvals in the United States must be supported by substantial evidence from "adequate and well-controlled" trials. The Food and Drug Administration (FDA) has flexibility in how it applies this standard.

METHODS

The authors conducted a systematic literature review of studies evaluating the design and outcomes of the key trials supporting new drug approvals in the United States. They extracted data on the trial characteristics, endpoint types, and expedited regulatory pathways.

FINDINGS

Among 48 publications eligible for inclusion, 30 covered trial characteristics, 23 covered surrogate measures, and 30 covered regulatory pathways. Trends point toward less frequent randomization, double-blinding, and active controls, with variation by drug type and indication. Surrogate measures are becoming more common but are not consistently well correlated with clinical outcomes. Drugs approved through expedited regulatory pathways often have less rigorous trial design characteristics.

CONCLUSIONS

The characteristics of trials used to approve new drugs have evolved over the past two decades along with greater use of expedited regulatory pathways and changes in the nature of drugs being evaluated. While flexibility in regulatory standards is important, policy changes can emphasize high-quality data collection before or after FDA approval.

How do safety warnings on medicines affect prescribing?

INTRODUCTION

Many adverse effects of medicines only become known after approval, prompting regulatory agencies to issue post-market safety advisories to inform clinicians and support safer care. Our team evaluated advisories issued by national regulators in Australia, Canada, Denmark, the United Kingdom, and the United States from 2007 to 2016 inclusive, comparing regulators' decisions to warn, effects on prescribing, doctors' awareness and responses to warnings, relevant regulatory policies, and specific case studies.

AREAS COVERED

Based mainly on our research program and a narrative review, this commentary describes how often regulators issue safety advisories and effects on clinical practice. We found extensive differences in decisions to warn, timing and content of warnings. Monitoring advice is often inadequate. The most systematic estimate suggests an average reduction in prescribing of around 6% compared with settings with no advisory. Interviews with doctors suggest limited awareness, uptake, and at times belief in these warnings.

EXPERT OPINION

Post-market safety advisories are an important intervention aiming to improve prescribing and use of medicines. However, differing warnings mean that some patients may be exposed to riskier prescribing than others. Better integration of new safety information into clinical practice is needed, as well as improved transparency, independence, and public engagement in regulatory decision-making.

On the use of the likelihood ratio test methodology in pharmacovigilance

The safety of medical products due to adverse events (AE) from drugs, therapeutic biologics, and medical devices is a major public health concern worldwide. Likelihood ratio test (LRT) approaches to pharmacovigilance constitute a class of rigorous statistical tools that permit objective identification of AEs of a specific drug and/or a class of drugs cataloged in spontaneous reporting system databases. However, the existing LRT approaches encounter certain theoretical and computational challenges when an underlying Poisson model assumption is violated, including in cases of zero-inflated data. We briefly review existing LRT approaches and propose a novel class of (pseudo-) LRT methods to address these challenges. Our approach uses an alternative parametrization to formulate a unified framework with a common test statistic that can handle both Poisson and zero-inflated Poisson (ZIP) models. The proposed framework is computationally efficient, and it reveals deeper insights into the comparative behaviors of the Poisson and the ZIP models for handling AE data. Our extensive simulation studies document notably superior performances of the proposed methods over existing approaches particularly under zero-inflation, both in terms of statistical (eg, much better control of the nominal level and false discovery rate with substantially enhanced power) and computational ( ∼ $$ \sim $$ 100-500-fold gains in average running times) performance metrics. An application of our method on the statin drug class from the FDA FAERS database reveals interesting insights on potential AEs. An R package, pvLRT, implementing our methods has been released in the public domain.

Association of expedited review programmes with postmarketing safety events of new drugs approved by the US food and drug administration between 2007 and 2017

Objective

To explore the associations between the risks of postmarketing safety events of new drugs and the four expedited programmes of priority review, accelerated approval, fast track and breakthrough therapy established by the US Food and Drug Administration (FDA); and to investigate whether multiple uses of expedited programmes, and the combinations of expedited programmes with orphan designation, were relevant to different safety profiles.

Design

Cohort study.

Setting

USA.

Participants

All new drugs approved by the FDA between 1 January 2007 and 31 December 2017, followed up until 10 April 2021.

Outcome measures

Safety events included safety-related withdrawal, new boxed warning, drug safety communication, postapproval risk evaluation mitigation strategy and safety-related labelling changes. The duration from marketing approval to the occurrence of a safety event was measured.

Method

Cox models were performed to determine the factors related to the time-to-safety event.

Results

The FDA approved 338 new drugs between 2007 and 2017, among which 53.6% (181) were under expedited review and 32.2% (109) received two or more expedited programmes. It took median time of 1.75 years (IQR 1.10–2.93) and 2.31 years (IQR 1.33–4.21), respectively, for new drugs to be observed of their first event and first serious event. The raised risk for first safety event was found to associate with breakthrough therapy (adjusted HR 1.83; 95% CI 1.21 to 2.77; p=0.004), and with the combination of accelerated approval with orphan designation (adjusted HR 2.84; 95% CI 1.12 to 7.23; p=0.028). Triple or more use of expedited programmes correlated with higher risk for first serious event (adjusted HR 4.16; 95% CI 1.69 to 10.22; p=0.002).

Conclusions

The increased risks of the breakthrough therapies, accelerated orphan drugs and triple or more use of expedited programmes indicated the necessity for intensive postmarketing risk surveillance.

Sources of Evidence Triggering and Supporting Safety-Related Labeling Changes: A 10-Year Longitudinal Assessment of 22 New Molecular Entities Approved in 2008 by the US Food and Drug Administration

INTRODUCTION

New safety issues concerning US FDA-approved drugs are commonly communicated through safety-related labeling changes. Therefore, to optimize and refine postmarket safety surveillance strategies, it is important to comprehensively characterize the sources of data giving rise to safety-related labeling changes.

OBJECTIVES

Our objective was to characterize the sources of data triggering and supporting the identification of new safety risks of FDA-approved drugs communicated through safety-related labeling changes.

METHODS

We conducted a retrospective study with a 10-year observation period using FDA's internal electronic data repositories for all prescription new molecular entities (NME) approved in 2008. We collected and analyzed information on new safety issues, the section of the full prescribing information updated, initiators (FDA, drug manufacturer), and triggering and supporting sources of evidence.

RESULTS

Among 22 NMEs approved in 2008, 189 new safety issues for 18 NMEs were identified. Compared to drug manufacturer, FDA initiated safety-related labeling changes in nine of the ten changes to the Boxed Warnings, 28 of the 52 changes to the Warnings and Precautions, and 43 of the 134 changes to the Adverse Reactions sections of the full prescribing information. The most frequent triggering sources of evidence included the drug manufacturer safety database (32.3%) and FDA Adverse Event Reporting System (FAERS) safety reports (15.3%) for all relevant sections of the full prescribing information, and class-labeling changes (17.5%) for Boxed Warnings and the Warnings and Precautions sections. The most frequent triggering source of evidence was FAERS safety reports (69%) in the first year after drug approval and the drug manufacturer safety database in subsequent years.

CONCLUSIONS

Our findings emphasize the continued importance of safety reports from FAERS and drug manufacturer safety databases and a comprehensive drug safety surveillance program throughout a drug's lifecycle.

Postmarketing safety of orphan drugs: a longitudinal analysis of the US Food and Drug Administration database between 1999 and 2018

Background

Information about the specific regulatory environment of orphan drugs is scarce and inconsistent. Uncertainties surrounding the postmarketing long-term safety of orphan drugs remain. This study aimed to evaluate the labelling changes of orphan drugs and to identify postmarketing safety-associated approval factors.

Methods

This retrospective cohort study includes all drugs with orphan drug designation approved by the Center for Drug Evaluation and Research of the US Food and Drug Administration between 1999 and 2018. Main outcomes are safety-related labelling changes up to 31 December 2019. We defined any safety-related labelling changes as postmarketing safety events (PMSE). Safety-related withdrawals, suspensions, and boxed warnings were further categorised as severe postmarketing safety events (SPSE). Outcome measurements include frequencies of PMSE, SPSE, and association between approval factors and the occurrence of safety events.

Results

Amongst the 214 drugs identified with orphan drug designation (25.7% biologics), 83.6% were approved through at least one expedited programme, and 29.4% were approved with boxed warnings. During a median follow-up of 6.74 years since approval, 69.2% and 14.5% of the analysed orphan drugs had PMSE and SPSE, respectively. Safety-related withdrawal (0%, 0/214), suspended marketing (0.46%, 1/214) and new boxed warnings are uncommon (3.7%, 8/214). The safety-related labelling changes were more frequent in the drugs approved with boxed warnings [Incidence rate ratio (IRR): 1.95 (1.02–3.73)] and approved for long-term use [IRR: 2.76 (1.52–5.00)].

Conclusions and Relevance

In this long-term postmarketing analysis, approximately 70% of FDA-approved orphan drugs had safety-related labelling changes although severe safety events were rare. While maintaining early access to orphan drugs, the drug regulatory body has taken timely regulatory action with postmarketing surveillance to ensure patient safety.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13023-021-02166-9.

"Artificial Intelligence" for Pharmacovigilance: Ready for Prime Time?

There is great interest in the application of 'artificial intelligence' (AI) to pharmacovigilance (PV). Although US FDA is broadly exploring the use of AI for PV, we focus on the application of AI to the processing and evaluation of Individual Case Safety Reports (ICSRs) submitted to the FDA Adverse Event Reporting System (FAERS). We describe a general framework for considering the readiness of AI for PV, followed by some examples of the application of AI to ICSR processing and evaluation in industry and FDA. We conclude that AI can usefully be applied to some aspects of ICSR processing and evaluation, but the performance of current AI algorithms requires a 'human-in-the-loop' to ensure good quality. We identify outstanding scientific and policy issues to be addressed before the full potential of AI can be exploited for ICSR processing and evaluation, including approaches to quality assurance of 'human-in-the-loop' AI systems, large-scale, publicly available training datasets, a well-defined and computable 'cognitive framework', a formal sociotechnical framework for applying AI to PV, and development of best practices for applying AI to PV. Practical experience with stepwise implementation of AI for ICSR processing and evaluation will likely provide important lessons that will inform the necessary policy and regulatory framework to facilitate widespread adoption and provide a foundation for further development of AI approaches to other aspects of PV.

The Use of Real-World Data to Assess the Impact of Safety-Related Regulatory Interventions

The regulation of medicines seeks to ensure the efficacy, safety, and quality of prescription and non-prescription medicines. Given that the conditions under which a medicine's benefits outweigh its risks are complex, it is essential that communications about the safe and effective use of medicines be clear and actionable. Assessing the impact of interventions to improve the safe and effective use of medicines is a developing area, and one in which real-world data are playing an increasingly important role. Although real-world data are commonly used to assess the impact of regulatory interventions, there are several areas where their use could be improved. Specific areas for improvement include assessing regulatory interventions across a wider range of medicines, rather than concentrating on a relatively few therapeutic areas; assessing more clinically relevant outcomes rather than relying on measures such as changes in the number of prescriptions, which may not always correlate with the desired impact; assessing the potential unintended or negative consequences of regulatory interventions; applying methods to address potential confounders; assessing long-term, rather than just short-term, impacts of an intervention; increasing the use of comparator groups, when feasible; and evaluating the impact of regulatory interventions from multiple dimensions, rather than from a single dimension. Expanded use of real-world data could inform some of these efforts, although data sources beyond administrative claims data will likely be necessary to achieve all these goals.

Quantifying the Severity of Adverse Drug Reactions Using Social Media: Network Analysis

BACKGROUND

Adverse drug reactions (ADRs) affect the health of hundreds of thousands of individuals annually in the United States, with associated costs of hundreds of billions of dollars. The monitoring and analysis of the severity of ADRs is limited by the current qualitative and categorical systems of severity classification. Previous efforts have generated quantitative estimates for a subset of ADRs but were limited in scope because of the time and costs associated with the efforts.

OBJECTIVE

The aim of this study is to increase the number of ADRs for which there are quantitative severity estimates while improving the quality of these severity estimates.

METHODS

We present a semisupervised approach that estimates ADR severity by using social media word embeddings to construct a lexical network of ADRs and perform label propagation. We used this method to estimate the severity of 28,113 ADRs, representing 12,198 unique ADR concepts from the Medical Dictionary for Regulatory Activities.

RESULTS

Our Severity of Adverse Events Derived from Reddit (SAEDR) scores have good correlations with real-world outcomes. The SAEDR scores had Spearman correlations of 0.595, 0.633, and -0.748 for death, serious outcome, and no outcome, respectively, with ADR case outcomes in the Food and Drug Administration Adverse Event Reporting System. We investigated different methods for defining initial seed term sets and evaluated their impact on the severity estimates. We analyzed severity distributions for ADRs based on their appearance in boxed warning drug label sections, as well as for ADRs with sex-specific associations. We found that ADRs discovered in the postmarketing period had significantly greater severity than those discovered during the clinical trial (P<.001). We created quantitative drug-risk profile (DRIP) scores for 968 drugs that had a Spearman correlation of 0.377 with drugs ranked by the Food and Drug Administration Adverse Event Reporting System cases resulting in death, where the given drug was the primary suspect.

CONCLUSIONS

Our SAEDR and DRIP scores are well correlated with the real-world outcomes of the entities they represent and have demonstrated utility in pharmacovigilance research. We make the SAEDR scores for 12,198 ADRs and the DRIP scores for 968 drugs publicly available to enable more quantitative analysis of pharmacovigilance data.

Comprehensive evaluation of post-approval regulatory actions during the drug lifecycle - a focus on benefits and risks

Background: Prior studies investigated regulatory actions that reflected a negative impact on drug risks. We aimed to evaluate occurrence of regulatory actions that reflected a negative or positive impact on benefits or risks, as well as relations between them.Research design and methods: We followed EMA-approved innovative drugs from approval (2009-2010) until July 2020 or withdrawal to identify regulatory actions. We assessed these for impact on benefits or risks and relations between actions. Additionally, we scrutinized drug lifecycles for time-variant characteristics that may contribute to specific patterns of regulatory actions.Results: We identified 14 letters and 361 label updates for 40 drugs. Of the label updates, 85 (24%) reflected a positive impact, mostly concerning indications, and 276 (76%) a negative impact, mostly adverse drug reactions. Many updates (54%) occurred simultaneously with other updates, also if these reflected a different impact. Furthermore, levels of patient exposure, innovativeness, needs for regulatory learning and unexpected risks may contribute to patterns of regulatory actions.Conclusions: Almost a quarter of regulatory actions reflected a positive impact on benefits and risks. Also, simultaneous learning about benefits and risks suggests an important role for drug development in risk characterization. These findings may impact regulatory analyses and decision-making.

The Impact of Variability in Patient Exposure During Premarket Clinical Development on Postmarket Safety Outcomes

We characterized the size of the premarket safety population for 278 small-molecule new molecular entities (NMEs) and 61 new therapeutic biologics (NTBs) approved by the US Food and Drug Administration (FDA) between October 1, 2002, and December 31, 2014, evaluating the relationship of premarket safety population size to regulatory characteristics and postmarket safety outcomes. The median size of the safety population was 1,044, and was lower for NTBs than NMEs (median: 920 vs. 1,138, P = 0.04), orphan products than nonorphan products (393 vs. 1,606, P < 0.001), and for products with fast-track designation (617 vs. 1,455, P < 0.001), priority review (630 vs. 1,735, P < 0.001), and accelerated approval (475 vs. 1,164, P < 0.001), than products without that designation. The median number of postmarket safety label updates and issues added to the label were higher with larger premarket exposure among nonorphan products, but not among orphan products. Products with accelerated approval using a surrogate end point had a higher median number of safety issues added to the label than those with full approval, but this did not vary with the size of the safety population; fast-track and priority review were not associated with the number of safety issues added to the label. A smaller safety population size was associated with a longer time to first safety outcome for nonorphan products but not orphan products. For orphan and nonorphan products combined, smaller premarket safety population size is not associated with the number or timing of postmarket safety outcomes, regardless of expedited program participation.

A New Era in Pharmacovigilance: Toward Real-World Data and Digital Monitoring

Adverse drug reactions (ADRs) are a major concern for patients, clinicians, and regulatory agencies. The discovery of serious ADRs leading to substantial morbidity and mortality has resulted in mandatory phase IV clinical trials, black box warnings, and withdrawal of drugs from the market. Real‐world data, data collected during routine clinical care, is being adopted by innovators, regulators, payors, and providers to inform decision making throughout the product life cycle. We outline several different approaches to modern pharmacovigilance, including spontaneous reporting databases, electronic health record monitoring and research frameworks, social media surveillance, and the use of digital devices. Some of these platforms are well‐established while others are still emerging or experimental. We highlight both the potential opportunity, as well as the existing challenges within these pharmacovigilance systems that have already begun to impact the drug development process, as well as the landscape of postmarket drug safety monitoring. Further research and investment into different and complementary pharmacovigilance systems is needed to ensure the continued safety of pharmacotherapy.

Towards Automating Adverse Event Review: A Prediction Model for Case Report Utility

INTRODUCTION

The rapidly expanding size of the Food and Drug Administration's (FDA) Adverse Event Reporting System database requires modernized pharmacovigilance practices. Techniques to systematically identify high utility individual case safety reports (ICSRs) will support safety signal management.

OBJECTIVES

The aim of this study was to develop and validate a model predictive of an ICSR's pharmacovigilance utility (PVU).

METHODS

PVU was operationalized as an ICSR's inclusion in an FDA-authored pharmacovigilance review's case series supporting a recommendation to modify product labeling. Multivariable logistic regression models were used to examine the association between PVU and ICSR features. The best performing model was selected for bootstrapping validation. As a sensitivity analysis, we evaluated the model's performance across subgroups of safety issues.

RESULTS

We identified 10,381 ICSRs evaluated in 69 pharmacovigilance reviews, of which 2115 ICSRs were included in a case series. The strongest predictors of ICSR inclusion were reporting of a designated medical event (odds ratio (OR) 1.93, 95% CI 1.54-2.43) and positive dechallenge (OR 1.67, 95% CI 1.50-1.87). The strongest predictors of ICSR exclusion were death reported as the only outcome (OR 2.72, 95% CI 1.76-4.35), more than three suspect products (OR 2.69, 95% CI 2.23-3.24), and > 15 preferred terms reported (OR 2.69, 95% CI 1.90-3.82). The validated model showed modest discriminative ability (C-statistic of 0.71). Our sensitivity analysis demonstrated heterogeneity in model performance by safety issue (C-statistic range 0.58-0.74).

CONCLUSIONS

Our model demonstrated the feasibility of developing a tool predictive of ICSR utility. The model's modest discriminative ability highlights opportunities for further enhancement and suggests algorithms tailored to safety issues may be beneficial.

Postmarketing Safety-Related Regulatory Actions for New Therapeutic Biologics Approved in the United States 2002-2014: Similarities and Differences With New Molecular Entities

We examined the relationship of regulatory and review characteristics to postmarketing safety-related regulatory actions for 61 new therapeutic biologics (NTBs) approved between October 1, 2002 and December 31, 2014. We also compared NTBs with small-molecule new molecular entities (NMEs) on these measures. Postmarketing safety-related regulatory actions were defined as a safety-related withdrawal or a safety-related update to a safety section of the label through June 30, 2018. Four NTBs were withdrawn, two for safety reasons. At least one safety-related update was added to the labels of 54 (88.5%) NTBs. Label updates occurred throughout the follow-up period. Time to the first safety-related regulatory action was shorter for NTBs approved under accelerated approval. The occurrence of safety events was more likely to occur with NTBs than with NMEs. This may be explained in part by the higher proportion of NTBs in the anatomical therapeutic chemical classification categories with higher frequency of safety-related updates. NTBs also had shorter time to safety events than NMEs. These findings underscore the importance of continued development of the life cycle safety surveillance system for both drugs and biologics with consideration for product type and its characteristics, including pharmacologic action.

FDA postmarketing safety labeling changes: What have we learned since 2010 about impacts on prescribing rates, drug utilization, and treatment outcomes

PURPOSE

Prior literature reviews have identified gaps in understanding of how postmarketing safety labeling changes and related FDA communications impact key clinical and behavioral outcomes. We conducted a review of newly published studies on this topic to determine what new evidence exists and to identify which gaps may still remain. We believe that this information can support FDA as it develops and implements future risk communication approaches.

METHODS

We searched PubMed and Embase for studies published between January 1, 2010, and August 7, 2017 that examined the impact of labeling changes or associated FDA safety-related communications. For each study, we extracted information on research design and findings for key clinical outcomes and behaviors. We also conducted a ROBINS-I review to identify potential for bias in the research design of each study.

RESULTS

We found that the estimated impacts of FDA labeling changes on several key outcomes-including adverse events-varied. Labeling changes also yielded unintended consequences on drug prescribing in some cases, despite low provider adherence. Finally, some studies we reviewed exhibited potential for bias due to confounding, among other factors.

CONCLUSIONS

The new studies we reviewed contain many of the same limitations identified in previously published reviews. While there are several challenges to conducting this research there is substantial room for improvement in the quality of the evidence base. More information, particularly with respect to the types of populations and medications affected by labeling changes, is needed to support the development of more effective and targeted safety communications.

Characteristics on Drug Safety Measures in Japan Stratified by System Organ Classes and Therapeutic Categories in Relation to the Approval Date

Revisions of drug package inserts (PIs) may be made immediately after approval or after considerable clinical experience; however, it is unclear whether there is a relationship between the characteristics of these safety measures and the period since drug approval. Here, we analyzed 209 cases of safety measures (revisions of the PIs) taken in Japan over 5 years (FY2014 to FY2018). The median, minimum, and maximum period from approval date in Japan to PI revision date was 6.29 years (interquartile range 2.68-15.53 years), 0.16 years, and 59.69 years, respectively. The cases were classified into four groups depending on types of adverse reaction and therapeutic category in relation to the national approval date and international birth date, resulting in the grouping together of particular adverse reactions and therapeutic drugs. For example, "Hepatobiliary disorders", "Blood and lymphatic system disorders", "Respiratory, thoracic and mediastinal disorders", "Antineoplastics", "Chemotherapeutics", and "Other agents affecting metabolism" were associated with the group of safety measures taken early after approval of a drug soon after the international birth date, suggesting that careful attention at an earlier stage after approval is necessary for these adverse reactions and drugs. Understanding such features of PI revisions makes pharmacovigilance planning more appropriate, contributing to the implementation of rapid and proper safety measures after drug approval.

An evaluation of statistical approaches to postmarketing surveillance

Safety of medical products presents a serious concern worldwide. Surveillance systems of postmarket medical products have been established for continual monitoring of adverse events (AEs) in many countries, and the proliferation of electronic health record systems further facilitates continual monitoring for AEs. We review existing statistical methods for signal detection that are mostly in use in postmarketing safety surveillance of spontaneously reported AEs and we study their performance characteristics by simulation. We compare those with the likelihood ratio test (LRT) method (appropriately modified for use in pharmacovigilance) and use three different methods to generate data (AE based, drug based, and a modification of the method of Ahmed et al). Performance metrics include type I error, power, sensitivity, and false discovery rate, among others. The results show superior performance of the LRT method in almost all simulation experiments. An application to the FDA Adverse Event Reporting System database is illustrated using rhabdomyolysis-related preferred terms reported to FDA during the third-quarter of 2014 to the first-quarter of 2017 for statin drugs. We present a critical discussion and recommendations for use of these methods.