Decision making in drug safety-a literature review of criteria used to prioritize newly detected safety issues

Mapping Strategies to Assess and Increase the Validity of Published Disproportionality Signals: A Meta-Research Study

BACKGROUND AND AIM

Disproportionality analysis is traditionally used in spontaneous reporting systems to generate working hypotheses about potential adverse drug reactions: the so-called disproportionality signals. We aim to map the methods used by researchers to assess and increase the validity of their published disproportionality signals.

METHODS

From a systematic literature search of published disproportionality analyses up until 1 January 2020, we randomly selected and analyzed 100 studies. We considered five domains: (1) rationale for the study, (2) design of disproportionality analyses, (3) case-by-case assessment, (4) use of complementary data sources, and (5) contextualization of the results within existing evidence.

RESULTS

Among the articles, multiple strategies were adopted to assess and enhance the results validity. The rationale, in 95 articles, was explicitly referred to the accrued evidence, mostly observational data (n = 46) and regulatory documents (n = 45). A statistical adjustment was performed in 34 studies, and specific strategies to correct for biases were implemented in 33 studies. A case-by-case assessment was complementarily performed in 35 studies, most often by investigating temporal plausibility (n = 26). Complementary data sources were used in 25 articles. In 78 articles, results were contextualized using accrued evidence from the literature and regulatory documents, the most important sources being observational (n = 45), other disproportionalities (n = 37), and case reports (n = 36).

CONCLUSIONS

This meta-research study highlighted the heterogeneity in methods and strategies used by researchers to assess the validity of disproportionality signals. Mapping these strategies is a first step towards testing their utility in different scenarios and developing guidelines for designing future disproportionality analysis.

Pneumocystis jirovecii pneumonia associated with immune checkpoint inhibitors: A systematic literature review of published case reports and disproportionality analysis based on the FAERS database

Background: Pneumocystis jirovecii pneumonia (PJP) has been reported with ICIs but limited to case reports. The clinical features of PJP with ICIs remain mostly unknown. This study aims to investigate the association of PJP with ICIs and describe clinical features.

Methods: Reports of PJP recorded in FAERS (January 2004–December 2022) were identified through the preferred term “Pneumocystis jirovecii pneumonia”. Demographic and clinical features were described, and disproportionality signals were assessed through the Reporting Odds Ratio (ROR) and Information Component (IC), using traditional chemotherapy and targeted therapy as comparators, and adjusting signals by excluding contaminant immunosuppressive drugs and pre-existing diseases. A systematic literature review was conducted to describe clinical features of published PJP reports with ICIs. Bradford Hill criteria was adopted for global assessment of the evidence.

Results: We identified 677 reports of PJP associated with ICIs, in which 300 (44.3%) PJP cases with fatal outcome. Nivolumab (IC025 2.05), pembrolizumab (IC025 1.88), ipilimumab (IC025 1.43), atezolizumab (IC025 0.36), durvalumab (IC025 1.65), nivolumab plus ipilimumab (IC025 1.59) have significant signals compared to other drugs in FAERS database. After excluding pre-existing diseases and immunosuppressive agents which may increase susceptibility of PJP, the signals for PJP associated with nivolumab, pembrolizumab, durvalumab, nivolumab plus ipilimumab remained robust (IC025 > 0). When compared to other anticancer regimens, although all ICIs showed a lower disproportionate signal for PJP than chemotherapy, nivolumab (IC025 0.33, p < 0.001), pembrolizumab (IC025 0.16, p < 0.001), both PD-1 inhibitors, presented a higher signal for PJP than targeted therapy. Male gender (IC025 0.26, p < 0.001) and age >65 years (IC025 0.38, p < 0.001) were predominant in PJP cases associated with across all ICIs. In literature, 15 PJP cases associated with ICIs were reported in 10 published case reports. 12 of 15 (80.0%) of cases received PD-1 inhibitors before PJP was diagnosed.

Conclusion: By the combined analysis of post-marketing data from FAERS and published case reports, we identified ICIs may be associated with PJP, especially in males aged >65years. After accounting for confounders, PD-1 inhibitors emerged with a robust disproportionality signal when compared to PD-L1/CTLA-4 inhibitors as well as targeted therapy. Further research is warranted to validate our findings.

Signals of Adverse Drug Reactions Communicated by Pharmacovigilance Stakeholders: A Scoping Review of the Global Literature

INTRODUCTION AND OBJECTIVE

Signals of adverse drug reactions (ADRs) can be supported by reports of ADRs and by interventional and non-interventional studies. The evidence base and features of ADR reports that are used to support signals remain to be comprehensively described. To this end, we have undertaken a scoping review.

METHODS

We searched the following databases: PubMed, EMBASE, PsycINFO, Web of Science, and Google Scholar, without language or time restrictions. We also hand searched the bibliographies of relevant studies. We included studies of any design if the results were described as signals. We assessed the levels of evidence using the Oxford Centre for Evidence-Based Medicine (OCEBM) criteria and coded features of reports of ADRs using the Bradford Hill guidelines.

RESULTS

Overall, 1974 publications reported 2421 studies of signals; 1683/2421 were clinical assessments of anecdotal reports of ADRs, but only 225 (13%) of these included explicit judgments on which features of the ADR reports were supportive of a signal. These 225 studies yielded 228 signals; these were supported by features, which were: 'experimental evidence' (i.e., positive dechallenge or rechallenge, 154 instances [68%]), 'temporality' (i.e., time to onset, 130 [57%]), 'exclusion of competing causes' (49 [21%]), and others (40 [17%]). Positive dechallenge/rechallenge often co-occurred with temporality (77/228). OCEBM 4 (i.e., case series and case-control studies) was the most frequent level of evidence (2078 studies). Between 2013 and 2019, there was a three-fold increase in clinical assessments of reports of ADRs compared with a less than two-fold increase in studies supported by higher levels of evidence (i.e., OCEBM 1-3). We identified an increased rate between 2013 and 2019 in disproportionality analyses (about 15 studies per year), mostly from academia.

CONCLUSIONS

Most signals were supported by temporality and dechallenge/rechallenge, but clear reporting of judgments on causality remains infrequent. The number of studies supported only by anecdotal reports of ADRs increased from year to year. The impact of a growing number of signals of disproportionate reporting communicated without an accompanying clinical assessment should be evaluated.

Factors Influencing Regulatory Decision-Making in Signal Management: Analysis Based on the Signals Identified from the FAERS

PURPOSE

This study aimed to identify factors that influence the decision to take safety regulatory actions in routine signal management based on spontaneous reports. For this purpose, we analyzed the safety signals identified from the Food and Drug Administration (FDA) Adverse Event Reporting System (FAERS) and related information.

METHOD

From the signals that the FDA identified in the FAERS between 2008 1Q and 2014 4Q, we selected 216 signals for which regulatory action was or was not taken. Characteristics of the signals were extracted from the FAERS quarterly reports that give information about what signals were identified from the FAERS and what actions were taken for them, and the FAERS data released in the same quarter when the signal was published. Univariate and multivariable logistic regression analysis was used to assess the relationship between the characteristics of each of the signals and the decision on regulatory action.

RESULT

As a result of the univariate logistic regression analysis, we selected 5 factors (positive rechallenge, number of cases accumulated in the last one-year period before the signal indication, previous awareness, serious outcome, risk for special populations) to include in the multivariable logistic regression model (p < 0.2). The multivariate logistic regression analysis showed that the number of cases accumulated in the last one-year period before the signal indication and previous awareness were associated with the regulatory action (p < 0.05).

CONCLUSION

The present study showed that number of cases accumulated in the last one-year period before the signal indication and previous awareness potentially associated with the United States regulatory action. When assessing safety signals, we should be careful of the adverse events with a large number of cases accumulated rapidly in a short period. In addition, we should pay attention to new information on not only unknown risks but also previously identified and potential risks.

Machine learning model combining features from algorithms with different analytical methodologies to detect laboratory-event-related adverse drug reaction signals

Background

The importance of identifying and evaluating adverse drug reactions (ADRs) has been widely recognized. Many studies have developed algorithms for ADR signal detection using electronic health record (EHR) data. In this study, we propose a machine learning (ML) model that enables accurate ADR signal detection by integrating features from existing algorithms based on inpatient EHR laboratory results.

Materials and methods

To construct an ADR reference dataset, we extracted known drug–laboratory event pairs represented by a laboratory test from the EU-SPC and SIDER databases. All possible drug–laboratory event pairs, except known ones, are considered unknown. To detect a known drug–laboratory event pair, three existing algorithms—CERT, CLEAR, and PACE—were applied to 21-year inpatient EHR data. We also constructed ML models (based on random forest, L1 regularized logistic regression, support vector machine, and a neural network) that use the intermediate products of the CERT, CLEAR, and PACE algorithms as inputs and determine whether a drug–laboratory event pair is associated. For performance comparison, we evaluated the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), F1-measure, and area under receiver operating characteristic (AUROC).

Results

All measures of ML models outperformed those of existing algorithms with sensitivity of 0.593–0.793, specificity of 0.619–0.796, NPV of 0.645–0.727, PPV of 0.680–0.777, F1-measure of 0.629–0.709, and AUROC of 0.737–0.816. Features related to change or distribution of shape were considered important for detecting ADR signals.

Conclusions

Improved performance of ML models indicated that applying our model to EHR data is feasible and promising for detecting more accurate and comprehensive ADR signals.

Characteristics of drugs safety signals that predict safety related product information update

Purpose

Investigation of drug safety signals is one of the major tasks in pharmacovigilance. Among many potential signals identified, only a few reflect adverse drug reactions requiring regulatory actions, such as product information (PI) update. Limited information is available regarding the signal characteristics that might predict PI update following signal evaluation. The objective of this study was to identify signal characteristics associated with PI updates following signal evaluation by the European Medicines Agency Pharmacovigilance Risk Assessment Committee during 2012 to 2016.

Methods

A comparative study was performed based on data from 172 safety signals. Characteristics of signals were extracted from the European Pharmacovigilance Issues Tracking Tool database. Multivariable logistic regression analysis was used to assess the relationship between signal characteristics and the decision to update the PI.

Results

Multivariable logistic regression analysis showed that the presence of evidence in multiple types of data sources (adjusted odds ratio [OR] 7.8 95% CI [1.5, 40.1]); mechanistic plausibility of the drug‐event association (adjusted OR 3.9 95% CI [1.9, 8.0]); seriousness of the event (adjusted OR 4.2 95% CI [1.3, 13.9]); and age of drugs ≤5 years (adjusted OR 3.9 95% CI [1.2, 12.7]) were associated with the decision to change the PI (P < 0.05).

Conclusions

This study identified 4 characteristics of drug safety signals that have shown to be associated with PI changes as outcome of signal evaluation. These characteristics may be used as criteria for selection and prioritization of potential signals that are more likely to necessitate product information updates.