The value of time-to-onset in statistical signal detection of adverse drug reactions: a comparison with disproportionality analysis in spontaneous reports from the Netherlands

Interplay of Spontaneous Reporting and Longitudinal Healthcare Databases for Signal Management: Position Statement from the Real-World Evidence and Big Data Special Interest Group of the International Society of Pharmacovigilance

Signal management, defined as the set of activities from signal detection to recommendations for action, is conducted using different data sources and leveraging data from spontaneous reporting databases (SRDs), which represent the cornerstone of pharmacovigilance. However, the exponentially increasing generation and availability of real-world data collected in longitudinal healthcare databases (LHDs), along with the rapid evolution of artificial intelligence-based algorithms and other advanced analytical methods, offers a wide range of opportunities to complement SRDs throughout all stages of signal management, especially signal detection. Integrating information derived from SRDs and LHDs may reduce their respective limitations, thus potentially enhancing post-marketing surveillance. The aim of this position statement is to critically evaluate the complementary role of SRDs and LHDs in signal management, exploring the potential benefits and challenges in integrating information coming from these two data sources. Furthermore, we presented successful cases of the interplay between SRDs and LHDs for signal management, along with future opportunities and directions to improve such interplay.

Medication Hazards and Outcome Patterns of Pediatric Drug-Associated Liver Injury in Taiwan: An Analysis of 1998-2017 Spontaneous Adverse Drug Reaction Reports

BACKGROUND AND OBJECTIVES

Accumulating pediatric efficacy and safety data on drug use is inherently challenging yet essential. This study aimed to analyze the frequency and compute the odds of pediatric drug-associated liver injury across age groups (early childhood, middle childhood, and adolescence) and therapeutic categories using adverse drug reactions (ADRs) reporting data spanning nearly two decades.

METHODS

We analyzed the reports of suspected ADRs occurring in children and adolescents in the Taiwan National Adverse Drug Reaction Reporting System during the period from May 1998 until July 2017. Standardized Medical Dictionary for Regulatory Activities Queries were utilized to identify suspected hepatic ADRs. Outcome patterns across age groups were compared using the chi-squared test, and disproportionality analysis was employed to calculate reporting odds ratios (RORs) of hepatic versus nonhepatic reports.

RESULTS

Among 16,673 reports, 484 (2.9%) were identified as suspected hepatic ADRs, involving 193 distinct drugs. The mean age of affected individuals was 8.2 years. Outcome types in adolescents were predominantly serious (91.8%). Antibacterials for systemic use (18.8%) and antiepileptics (8.7%) were the most frequently implicated therapeutic categories. Drugs with high ADR occurrence rates and significant RORs included oxacillin (5.2%; ROR: 12.07), methotrexate (4.1%; ROR: 9.07), and phenobarbital (2.7%; ROR: 5.04). Some medications exhibited higher ratios of used-versus-recommended doses, suggesting inappropriate dosing.

CONCLUSIONS

Pediatric drug-associated liver injury was not uncommon and may result in serious outcomes. This study underscores the need for heightened vigilance in administering certain high-risk drugs and attentiveness in proper dosing for children, including adolescents.

Advanced AI-Driven Prediction of Pregnancy-Related Adverse Drug Reactions

Ensuring drug safety during pregnancy is critical due to the potential risks to both the mother and fetus. However, the exclusion of pregnant women from clinical trials complicates the assessment of adverse drug reactions (ADRs) in this population. This study aimed to develop and validate risk prediction models for pregnancy-related ADRs of drugs using advanced Machine Learning (ML) and Deep Learning (DL) techniques, leveraging real-world data from the FDA Adverse Event Reporting System. We explored three methods─Information Component, Reporting Odds Ratio, and 95% confidence interval of ROR─for classifying drugs into high-risk and low-risk categories. DL models, including Directed Message Passing Neural Networks (DMPNN), Graph Neural Networks, and Graph Convolutional Networks, were developed and compared to traditional ML models like Random Forest, Support Vector Machines, and XGBoost. Among these, the DMPNN model, which integrated molecular graph information and molecular descriptors, exhibited the highest predictive performance, particularly at the preferred term level. The model was validated against external data sets from SIDER and DailyMed, demonstrating strong generalizability. Additionally, the model was applied to assess the risk of 22 oral hypoglycemic drugs, and potential substructure alerts for pregnancy-related ADRs were identified. These findings suggest that the DMPNN model is a valuable tool for predicting ADRs in pregnant women, offering significant advancement in drug safety assessment and providing crucial insights for safer medication use during pregnancy.

The evolving role of disproportionality analysis in pharmacovigilance

INTRODUCTION

From 2009 to 2015, the IMI PROTECT conducted rigorous studies addressing questions about optimal implementation and significance of disproportionality analyses, leading to the development of Good Signal Detection Practices. The ensuing period witnessed the independent exploration of research paths proposed by IMI PROTECT, accumulating valuable experience and insights that have yet to be seamlessly integrated.

AREAS COVERED

This state-of-the-art review integrates IMI PROTECT recommendations with recent acquisitions and evolving challenges. It deals with defining the object of study, disproportionality methods, subgrouping, masking, drug-drug interaction, duplication, expectedness, the debated use of disproportionality results as risk measures, integration with other types of data.

EXPERT OPINION

Despite the ongoing skepticism regarding the usefulness of disproportionality analyses and individual case safety reports, their ability to timely detect safety signals regarding rare and unpredictable adverse reactions remains unparalleled. Moreover, recent exploration into their potential for characterizing safety signals revealed valuable insights concerning potential risk factors and the patient's perspective. To fully realize their potential beyond hypothesis generation and achieve a comprehensive evidence synthesis with other kinds of data and studies, each with their unique limitations and contributions, we need to investigate methods for more transparently communicating disproportionality results and mapping and addressing pharmacovigilance biases.

Ocular adverse events of cenegermin used in neurotrophic keratopathy: an analysis of the FDA adverse event reporting system database

BACKGROUND

Cenegermin, a recombinant human nerve growth factor, is an orphan drug approved for neurotrophic keratitis. The safety information on the label is incomplete, and the adverse reactions noted are mostly mild and tolerable. However, the occurrence of painful epithelial plagues and irreversible corneal deposits after cenegermin usage have been reported. Real-world data on long-term ocular safety are lacking. We aimed to assess the cenegermin-associated eye safety profile in the FDA pharmacovigilance database.

METHODS

The signals of cenegermin-related ocular adverse events (AEs) from 2018 to 2022 were quantified using the reporting odds ratio (ROR) and information component (IC). The grading system was used to prioritize the signals.

RESULTS

We identified 3288 cases of cenegermin-related ocular AEs and 56 positive ocular-related signals. Fifty unexpected signals of ocular AE were identified. Eye ulcer was classified as a designated medical event. Twenty AEs, including corneal perforation, eye infection, corneal deposits, and eye inflammation, were recognized as important medical event. The median onset time for ocular AEs was 6 days (interquartile range [IQR]: 1-29 days).

CONCLUSION

This study revealed new cenegermin-related ocular AE signals. Clinical practice requires close monitoring to early identify and manage adverse reactions that may cause occurrence of serious irreversible consequences.

Adverse drug reaction signal detection methods in spontaneous reporting system: A systematic review

BACKGROUND

A series of signal detection methods have been developed to detect adverse drug reaction (ADR) signals in spontaneous reporting system. However, different signal detection methods yield quite different signal detection results, and we do not know which method has the best detection performance. How to choose the most suitable signal detection method is an urgent problem to be solved. In this study, we systematically reviewed the characteristics and application scopes of current signal detection methods, with the goal of providing references for the optimization selection of signal detection methods in spontaneous reporting system.

METHODS

We searched six databases from inception to January 2023. The search strategy targeted literatures regarding signal detection methods in spontaneous reporting system. We used thematic analysis approach to summarize the advantages, disadvantages, and application scope of each signal detection method.

RESULTS

A total of 93 literatures were included, including 27 reviews and 66 methodological studies. Moreover, 31 signal detection methods were identified in these literatures. Each signal detection method has its inherent advantages and disadvantages, resulting in different application scopes of these methods.

CONCLUSION

Our systematic review finds that there are variabilities in the advantages, disadvantages, and application scopes of different signal detection methods. This finding indicates that the most suitable signal detection method varies across different drug safety scenarios. Moreover, when selecting signal detection method in a particular drug safety scenario, the following factors need to be considered: purpose of research, database size, drug characteristics, adverse event characteristics, and characteristics of the relations between drugs and adverse events.

Drug Repurposing Using FDA Adverse Event Reporting System (FAERS) Database

Drug repurposing is an emerging approach to reassigning existing pre-approved therapies for new indications. The FDA Adverse Event Reporting System (FAERS) is a large database of over 28 million adverse event reports submitted by medical providers, patients, and drug manufacturers and provides extensive drug safety signal data. In this review, four common drug repurposing strategies using FAERS are described, including inverse signal detection for a single disease, drug-drug interactions that mitigate a target ADE, identifying drug-ADE pairs with opposing gene perturbation signatures and identifying drug-drug pairs with congruent gene perturbation signatures. The purpose of this review is to provide an overview of these different approaches using existing successful applications in the literature. With the fast expansion of adverse drug event reports, FAERS-based drug repurposing represents a promising strategy for discovering new uses for existing therapies.

Pharmacovigilance of triazole antifungal agents: Analysis of the FDA adverse event reporting system (FAERS) database

Triazole antifungal drugs (TAD) are widely used to treat invasive fungal infections due to their broad antifungal spectrum and low toxicity. Despite their preference in the clinic, multiple Adverse Events (AE) are still reported each year.

OBJECTIVE

We aimed to characterize the distribution of Adverse Events associated with Triazole antifungal drugs in different systems and to identify Important Medical Events (IME) signals for Triazole antifungal drugs.

METHODS

The U.S. Food and Drug Administration Adverse Event Reporting System (FAERS) was queried for Adverse Events related to Triazole antifungal drugs from 2012 to 2022. The Adverse Events caused by all other drugs and non-TAD antifungal drugs were analyzed as references. Reporting odds ratio and Bayesian confidence propagation neural network of information components were used to evaluate the association between Triazole antifungal drugs and Important Medical Events. Visual signal spectrum is mapped to identify potential adverse reaction signals.

RESULTS

Overall, 10,262 Adverse Events were reported to be associated with Triazole antifungal drugs, of which 5,563 cases were defined as Important Medical Events. Common adverse drug reactions (ADR) mentioned in the instructions such as delirium and hypokalemia were detected, as well as unlabeled ADRs such as rhabdomyolysis and hepatitis fulminant. Cholestasis, drug-induced liver injury, QT interval prolongation and renal impairment have notable signals in all Triazole antifungal drugs, with 50 percent of patients developing a severe clinical outcome. Isavuconazole had the lowest signal intensity and demonstrated a superior safety profile.

CONCLUSION

Most results are generally consistent with previous studies and are documented in the prescribing instructions, but some IMEs are not included, such as hepatitis fulminant. Additional pharmaco-epidemiological or experimental studies are required to validate the small number of unlabeled ADRs. TAD-related Important Medical Eventshave a considerable potential to cause clinically serious outcomes. Clinical use of Triazole antifungal drugs requires more attention.

Development of a multivariate prediction model to identify individual case safety reports which require clinical review

BACKGROUND

The number of Individual Case Safety Reports (ICSRs) in pharmacovigilance databases are rapidly increasing world-wide. The majority of ICSRs at the Netherlands Pharmacovigilance Centre Lareb is reviewed manually to identify potential signal triggering reports (PSTR) or ICSRs which need further clinical assessment for other reasons.

OBJECTIVES

To develop a prediction model to identify ICSRs that require clinical review, including PSTRs. Secondly, to identify the most important features of these reports.

METHODS

All ICSRs (n = 30 424) received by Lareb between October 1, 2017 and February 26, 2021 were included. ICSRs originating from marketing authorisation holders and ICSRs reported on vaccines were excluded. The outcome was defined as PSTR (yes/no), where PSTR 'yes' was defined as an ICSR discussed at a signal detection meeting. Nineteen features were included, concerning structured information on: patients, adverse drug reactions (ADR) or drugs. Data were divided into a training (70%) and test set (30%) using a stratified split to maintain the PSTR/no PSTR ratio. Logistic regression, elastic net logistic regression and eXtreme Gradient Boosting models were trained and tuned on a training set. Random down-sampling of negative controls was applied on the training set to adjust for the imbalanced dataset. Final models were evaluated on the test set. Model performances were assessed using the area under the curve (AUC) with 95% confidence interval of a receiver operating characteristic (ROC), and specificity and precision were assessed at a threshold for perfect sensitivity (100%, to not miss any PSTRs). Feature importance plots were inspected and a selection of features was used to re-train and test model performances with fewer features.

RESULTS

1439 (4.7%) of reports were PSTR. All three models performed equally with a highest AUC of 0.75 (0.73-0.77). Despite moderate model performances, specificity (5%) and precision (5%) were low. Most important features were: 'absence of ADR in the Summary of product characteristics', 'ADR reported as serious', 'ADR labelled as an important medical event', 'ADR reported by physician' and 'positive rechallenge'. Model performances were similar when using only nine of the most important features.

CONCLUSIONS

We developed a prediction model with moderate performances to identify PSTRs with nine commonly available features. Optimisation of the model using more ICSR information (e.g., free text fields) to increase model precision is required before implementation.

Detection algorithms and attentive points of safety signal using spontaneous reporting systems as a clinical data source

Continuous evaluation of drug safety is needed following approval to determine adverse events (AEs) in patient populations with diverse backgrounds. Spontaneous reporting systems are an important source of information for the detection of AEs not identified in clinical trials and for safety assessments that reflect the real-world use of drugs in specific populations and clinical settings. The use of spontaneous reporting systems is expected to detect drug-related AEs early after the launch of a new drug. Spontaneous reporting systems do not contain data on the total number of patients that use a drug; therefore, signal detection by disproportionality analysis, focusing on differences in the ratio of AE reports, is frequently used. In recent years, new analyses have been devised, including signal detection methods focused on the difference in the time to onset of an AE, methods that consider the patient background and those that identify drug-drug interactions. However, unlike commonly used statistics, the results of these analyses are open to misinterpretation if the method and the characteristics of the spontaneous reporting system cannot be evaluated properly. Therefore, this review describes signal detection using data mining, considering traditional methods and the latest knowledge, and their limitations.

Numbers of spontaneous reports: How to use and interpret?

Due to the high intensity of the COVID-19 vaccination campaigns and heightened attention for safety issues, the number of spontaneous reports has surged. In the Netherlands, pharmacovigilance centre Lareb has received more than 100 000 reports on adverse events following immunization (AEFI) associated with Covid-19 vaccination. It is tempting to interpret absolute numbers of reports of AEFIs in signal detection. Signal detection of spontaneously reported adverse drug reactions has its origin in case-by-case analysis, where all case reports are assessed by clinically qualified assessors. The concept of clinical review of cases-even if only a few per country-followed by sharing concerns of suspicions of potential adverse reactions again proved the strength of the system. Disproportionality analysis can be useful in signal identification, and comparing reported cases with expected based on background incidence can be useful to support signal detection. However, they cannot be used without an in-depth analysis of the underlying clinical data and pharmacological mechanism. This in-depth analysis has been performed, and is ongoing, for the signal of vaccine-induced immune thrombotic thrombocytopenia (VITT) in relation to the AstraZeneca and Janssen Covid-19 vaccines. Although not frequency or incidence rates, reporting rates can provide an impression of the occurrence of the event. But the unknown underreporting should also be part of this context. To quantify the incidence rates, follow-up epidemiological studies are needed.

Review of Statistical Methodologies for Detecting Drug-Drug Interactions Using Spontaneous Reporting Systems

Concomitant use of multiple drugs for therapeutic purposes is known as “polypharmacy situations,” which has been recognized as an important social problem recently. In polypharmacy situations, each drug not only induces adverse events (AEs) but also increases the risk of AEs due to drug–drug interactions (DDIs). The proportion of AEs caused by DDIs is estimated to be around 30% of unexpected AEs. The randomized clinical trials in pre-marketing typically focus emphasis on the verification of single drug safety and efficacy rather than the surveys of DDI, and therefore, patients on multiple drugs are usually excluded. However, unlike pre-marketing randomized clinical trials, in clinical practice (= post marketing), many patients use multiple drugs. The spontaneous reporting system is one of the significant sources drug safety surveillance in post-marketing. Commonly, signals of potential drug-induced AEs detected from this source are validated in real-world settings. Recently, not only methodological studies on signal detection of “single” drug, but also on several methodological studies on signal detection of DDIs have been conducted. On the other hand, there are few articles that systematically summarize the statistical methodology for signal detection of DDIs. Therefore, this article reviews the studies on the latest statistical methodologies from classical methodologies for signal detection of DDIs using spontaneous reporting system. This article describes how to calculate for each detection method and the major findings from the published literatures about DDIs. Finally, this article presented several limitations related to the currently used methodologies for signal detection of DDIs and suggestions for further studies.

Time to onset in statistical signal detection revisited: A follow-up study in long-term onset adverse drug reactions

Purpose

In a previous study, we developed a signal detection method using the time to onset (TTO) of adverse drug reactions (ADRs). The aim of the current study was to investigate this method in a subset of ADRs with a longer TTO and to compare its performance with disproportionality analysis.

Methods

Using The Netherlands's spontaneous reporting database, TTO distributions for drug—ADR associations with a median TTO of 7 days or more were compared with other drugs with the same ADR using the two‐sample Anderson–Darling (AD) test. Presence in the Summary of Product Characteristics (SPC) was used as the gold standard for identification of a true ADR. Twelve combinations with different values for the number of reports and median TTO were tested. Performance in terms of sensitivity and positive predictive value (PPV) was compared with disproportionality analysis. A sensitivity analysis was performed to compare the results with those from the previous study.

Results

A total of 38 017 case reports, containing 32 478 unique drug—ADR associations. Sensitivity was lower for the TTO method (range 0.08‐0.34) compared with disproportionality analysis (range 0.60‐0.87), whereas PPV was similar for both methods (range 0.93‐1.0). The results from the sensitivity analysis were similar to the original analysis.

Conclusions

Because of its low sensitivity, the developed TTO method cannot replace disproportionality analysis as a signal detection tool. It may be useful in combination with other methods.

A prediction model-based algorithm for computer-assisted database screening of adverse drug reactions in the Netherlands

Purpose

The statistical screening of pharmacovigilance databases containing spontaneously reported adverse drug reactions (ADRs) is mainly based on disproportionality analysis. The aim of this study was to improve the efficiency of full database screening using a prediction model‐based approach.

Methods

A logistic regression‐based prediction model containing 5 candidate predictors was developed and internally validated using the Summary of Product Characteristics as the gold standard for the outcome. All drug‐ADR associations, with the exception of those related to vaccines, with a minimum of 3 reports formed the training data for the model. Performance was based on the area under the receiver operating characteristic curve (AUC). Results were compared with the current method of database screening based on the number of previously analyzed associations.

Results

A total of 25 026 unique drug‐ADR associations formed the training data for the model. The final model contained all 5 candidate predictors (number of reports, disproportionality, reports from healthcare professionals, reports from marketing authorization holders, Naranjo score). The AUC for the full model was 0.740 (95% CI; 0.734–0.747). The internal validity was good based on the calibration curve and bootstrapping analysis (AUC after bootstrapping = 0.739). Compared with the old method, the AUC increased from 0.649 to 0.740, and the proportion of potential signals increased by approximately 50% (from 12.3% to 19.4%).

Conclusions

A prediction model‐based approach can be a useful tool to create priority‐based listings for signal detection in databases consisting of spontaneous ADRs.