Sources of Safety Data and Statistical Strategies for Design and Analysis: Clinical Trials

Development of a Framework Structuring Themes in the Course of Adverse Drug Reactions from a Patient's Perspective

INTRODUCTION

There is a need for more extensive information about adverse drug reactions (ADRs) for patients than currently available, including information on the course of ADRs. Aspects characterising the course of ADRs from the patient perspective have not been identified before.

OBJECTIVE

We aimed to develop a framework based on common themes in the course of ADRs identified from patient descriptions in patient-reported ADRs.

METHODS

In this qualitative study, patient descriptions of the course of patient-reported ADRs were analysed by a thematic analysis with an inductive approach using three different existing datasets containing patient-reported ADRs. Two datasets included patient-reported ADRs from cohort event monitoring of biologics and direct oral anticoagulants and one dataset included spontaneous reports from patients concerning medication for lower urinary tract symptoms. A conceptual framework was developed from the identified main themes and subthemes.

RESULTS

Patient-reported data concerning 3888 ADRs were analysed. Six main themes with multiple subthemes were identified from patient descriptions of the course of ADRs. Four themes were descriptive: frequency of an ADR episode, duration of an ADR episode, moment or period of ADR occurrence, and development in the intensity of the ADR. Two themes concerned factors influencing the course of ADRs: triggering factors and improving factors.

CONCLUSIONS

The presented framework illustrates that patients describe extensive details on the course and timeframe of ADRs. The identified themes provide a basis for improving the systematic data collection of more extensive details about ADRs from patients as a first step towards the provision of more comprehensive ADR information to patients.

Application of Artificial Intelligence or machine learning in risk sharing agreements for pharmacotherapy risk management

Applications of Artificial Intelligence in medical informatics solutions risk sharing have social value. At a time of ever-increasing cost for the provision of medicines to citizens, there is a need to restrain the growth of health care costs. The search for computer technologies to stop or slow down the growth of costs acquires a new very important and significant meaning. We discussed the two information technologies in pharmacotherapy and the possibility of combining and sharing them, namely the combination of risk-sharing agreements and Machine Learning, which was made possible by the development of Artificial Intelligence (AI). Neural networks could be used to predict the outcome to reduce the risk factors for treatment. AI-based data processing automation technologies could be also used for risk-sharing agreements automation.

A Science-Based Methodology Framework for the Assessment of Combination Safety Risks in Clinical Trials

Multiple components factor into the assessment of combination safety risks when two or more novel individual products are used in combination in clinical trials. These include, but are not limited to, biology, biochemistry, pharmacology, class effects, and preclinical and clinical findings (such as adverse drug reactions, drug target and mechanism of action, target expression, signaling, and drug-drug interactions). This paper presents a science-based methodology framework for the assessment of combination safety risks when two or more investigational products are used in clinical trials. The aim of this methodology framework is to improve prediction of the risks, to enable the appropriate safety risk mitigation and management to be put in place for the combination, and the development of the project combination safety strategy.

Important Considerations for Signal Detection and Evaluation

Safety clinicians have a wealth of resources describing how to perform signal detection. Nevertheless, there are some nuances concerning approaches taken by regulatory authorities and statistical considerations that should be appreciated. New approaches, such as the FDA Medical Queries, illustrate the value of considering medical concepts over individual adverse events. One area which would benefit from further clarity is how safety signals may be evaluated for evidence of a causal relationship to the drug of interest. Just as such safety signals can take many forms, the types of tools and methods required to interrogate these signals are equally as diverse. An understanding of the complexity of this process can aid the safety reviewer in successfully characterizing the emerging safety profile of a drug during the pre-marketing phase of development.

Pharmacovigilance: reporting requirements throughout a product's lifecycle

Comprehensive methods for evaluating safety are needed to objectively assess the full risk profile of a medication. The confidence of the prescribing provider in the safety and effectiveness of pharmaceuticals is extremely important. Pharmacovigilance is a key component of drug safety regulatory processes and is paramount for ensuring the safety profile of medications used to treat patients. All participants in the healthcare system, including healthcare providers and consumers, should understand and meaningfully engage in the pharmacovigilance process; healthcare providers should integrate pharmacovigilance into everyday practice, inviting feedback from patients. This narrative review aims to give an overview of the main topics underlying pharmacovigilance and drug safety in pharmaceutical research phase after the authorization of a drug in the United States. The US Food and Drug Administration guidance and post-approval regulatory actions are considered from an industry perspective.

Plain language summary

Regulatory processes that ensure the safety of drugs is monitored Government agencies regulate the safe use of medicinal products. By determining and enforcing pharmacovigilance, the monitoring of drugs for potential risks, they safeguard the welfare of consumers of medicines. Comprehensive, documented methods for evaluating the safety of a drug during its development and its subsequent use allow identification of any risks associated with the drug's use throughout its lifetime. The comprehensive identification of safety issues associated with a drug is improved when all parties involved in the development and use of drugs participate in the pharmacovigilance process. For example, clinicians should regularly ask their patients if they are experiencing any issues with their treatment, and patients should be encouraged to report problems they encounter with a particular medication to their healthcare provider. This narrative review provides an overview of the main topics underlying pharmacovigilance and drug safety after approval of a drug in the United States. Guidelines and actions from the US Food and Drug Administration are considered from an industry perspective.

Comparing the Value of Data Visualization Methods for Communicating Harms in Clinical Trials

In clinical trials, harms (i.e., adverse events) are often reported by simply counting the number of people who experienced each event. Reporting only frequencies ignores other dimensions of the data that are important for stakeholders, including severity, seriousness, rate (recurrence), timing, and groups of related harms. Additionally, application of selection criteria to harms prevents most from being reported. Visualization of data could improve communication of multidimensional data. We replicated and compared the characteristics of 6 different approaches for visualizing harms: dot plot, stacked bar chart, volcano plot, heat map, treemap, and tendril plot. We considered binary events using individual participant data from a randomized trial of gabapentin for neuropathic pain. We assessed their value using a heuristic approach and a group of content experts. We produced all figures using R and share the open-source code on GitHub. Most original visualizations propose presenting individual harms (e.g., dizziness, somnolence) alone or alongside higher level (e.g., by body systems) summaries of harms, although they could be applied at either level. Visualizations can present different dimensions of all harms observed in trials. Except for the tendril plot, all other plots do not require individual participant data. The dot plot and volcano plot are favored as visualization approaches to present an overall summary of harms data. Our value assessment found the dot plot and volcano plot were favored by content experts. Using visualizations to report harms could improve communication. Trialists can use our provided code to easily implement these approaches.

How Can We Quantify and Compare Harm in Surgical Trials?

PURPOSE

To describe methods that can be used to quantify and compare harm caused by surgical interventions in ophthalmology trials.

DESIGN

Perspective study.

METHODS

A published landmark glaucoma trial was used as an exemplar. A consensus-derived classification system of severity of complications was applied to published data of the Tube Versus Trabeculectomy glaucoma trial. The severity grade of each complication was multiplied to the number of patients who incurred that complication to estimate a total harm score for each intervention. Graphical tools were also used to display the differences in complications between trial arms. A review of literature on best practice for reporting harm data was also conducted.

RESULTS

Analyzing treatment harm is challenging with the relatively small number of events and sample sizes used in randomized controlled trials. However, quantification and graphical representation of harm after surgery is possible. Reframing the research question to one for detecting signals of adverse reactions and use of Bayesian analyses can be useful.

CONCLUSIONS

Analysis of harm data in clinical trials needs further attention. A severity classification system and a total harm score can be used to quantify harm after glaucoma surgery. Graphical tools can also help interpret complication data.

Harms in Systematic Reviews Paper 1: An introduction to research on harms

OBJECTIVE

Most systematic reviews of interventions focus on potential benefits. Common methods and assumptions that are appropriate for assessing benefits can be inappropriate for harms. This paper provides a primer on researching harms, particularly in systematic reviews.

STUDY DESIGN AND SETTING

Commentary describing challenges with assessing harm.

RESULTS

Investigators should be familiar with various terminologies used to describe, classify, and group harms. Published reports of clinical trials include limited information about harms, so systematic reviewers should not depend on these studies and journal articles to reach conclusions about harms. Visualizations might improve communication of multiple dimensions of harms such as severity, relatedness, and timing.

CONCLUSION

The terminology, classification, detection, collection, and reporting of harms create unique challenges that take time, expertise, and resources to navigate in both primary studies and evidence syntheses. Systematic reviewers might reach incorrect conclusions if they focus on evidence about harms found in published reports of randomized trials of a particular health problem. Systematic reviews could be improved through better identification and reporting of harms in primary studies and through better training and uptake of appropriate methods for synthesizing evidence about harms.

Joint modelling of multivariate longitudinal clinical laboratory safety outcomes, concomitant medication and clinical adverse events: application to artemisinin-based treatment during pregnancy clinical trial

Background

In drug trials, clinical adverse events (AEs), concomitant medication and laboratory safety outcomes are repeatedly collected to support drug safety evidence. Despite the potential correlation of these outcomes, they are typically analysed separately, potentially leading to misinformation and inefficient estimates due to partial assessment of safety data. Using joint modelling, we investigated whether clinical AEs vary by treatment and how laboratory outcomes (alanine amino-transferase, total bilirubin) and concomitant medication are associated with clinical AEs over time following artemisinin-based antimalarial therapy.

Methods

We used data from a trial of artemisinin-based treatments for malaria during pregnancy that randomized 870 women to receive artemether–lumefantrine (AL), amodiaquine–artesunate (ASAQ) and dihydroartemisinin–piperaquine (DHAPQ). We fitted a joint model containing four sub-models from four outcomes: longitudinal sub-model for alanine aminotransferase, longitudinal sub-model for total bilirubin, Poisson sub-model for concomitant medication and Poisson sub-model for clinical AEs. Since the clinical AEs was our primary outcome, the longitudinal sub-models and concomitant medication sub-model were linked to the clinical AEs sub-model via current value and random effects association structures respectively. We fitted a conventional Poisson model for clinical AEs to assess if the effect of treatment on clinical AEs (i.e. incidence rate ratio (IRR)) estimates differed between the conventional Poisson and the joint models, where AL was reference treatment.

Results

Out of the 870 women, 564 (65%) experienced at least one AE. Using joint model, AEs were associated with the concomitant medication (log IRR 1.7487; 95% CI: 1.5471, 1.9503; p < 0.001) but not the total bilirubin (log IRR: -0.0288; 95% CI: − 0.5045, 0.4469; p = 0.906) and alanine aminotransferase (log IRR: 0.1153; 95% CI: − 0.0889, 0.3194; p = 0.269). The Poisson model underestimated the effects of treatment on AE incidence such that log IRR for ASAQ was 0.2118 (95% CI: 0.0082, 0.4154; p = 0.041) for joint model compared to 0.1838 (95% CI: 0.0574, 0.3102; p = 0.004) for Poisson model.

Conclusion

We demonstrated that although the AEs did not vary across the treatments, the joint model yielded efficient AE incidence estimates compared to the Poisson model. The joint model showed a positive relationship between the AEs and concomitant medication but not with laboratory outcomes.

Trial registration

ClinicalTrials.gov: NCT00852423

Advantages of visualisations to evaluate and communicate adverse event information in randomised controlled trials

BACKGROUND

Randomised controlled trials (RCTs) provide valuable information and inform the development of harm profiles of new treatments. Harms are typically assessed through the collection of adverse events (AEs). Despite AEs being routine outcomes collected in trials, analysis and reporting of AEs in journal articles are continually shown to be suboptimal. One key challenge is the large volume of AEs, which can make evaluation and communication problematic. Prominent practice is to report frequency tables of AEs by arm. Visual displays offer an effective solution to assess and communicate complex information; however, they are rarely used and there is a lack of practical guidance on what and how to visually display complex AE data.

METHODS

In this article, we demonstrate the use of two plots identified to be beneficial for wide use in RCTs, since both can display multiple AEs and are suitable to display point estimates for binary, count, or time-to-event AE data: the volcano and dot plots. We compare and contrast the use of data visualisations against traditional frequency table reporting, using published AE information in two placebo-controlled trials, of remdesivir for COVID-19 and GDNF for Parkinson disease. We introduce statistical programmes for implementation in Stata.

RESULTS/CASE STUDY

Visualisations of AEs in the COVID-19 trial communicated a risk profile for remdesivir which differed from the main message in the published authors' conclusion. In the Parkinson's disease trial of GDNF, the visualisation provided immediate communication of harm signals, which had otherwise been contained within lengthy descriptive text and tables. Asymmetry in the volcano plot helped flag extreme events that were less obvious from review of the frequency table and dot plot. The dot plot allowed a more comprehensive representation by means of a more detailed summary.

CONCLUSIONS

Visualisations can better support investigators to assimilate large volumes of data and enable improved informal between-arm comparisons compared to tables. We endorse increased uptake for use in trial publications. Care in construction of visual displays needs to be taken as there can be potential to overemphasise treatment effects in some circumstances.

Statistical methods for the analysis of adverse event data in randomised controlled trials: a scoping review and taxonomy

BACKGROUND

Statistical methods for the analysis of harm outcomes in randomised controlled trials (RCTs) are rarely used, and there is a reliance on simple approaches to display information such as in frequency tables. We aimed to identify whether any statistical methods had been specifically developed to analyse prespecified secondary harm outcomes and non-specific emerging adverse events (AEs).

METHODS

A scoping review was undertaken to identify articles that proposed original methods or the original application of existing methods for the analysis of AEs that aimed to detect potential adverse drug reactions (ADRs) in phase II-IV parallel controlled group trials. Methods where harm outcomes were the (co)-primary outcome were excluded. Information was extracted on methodological characteristics such as: whether the method required the event to be prespecified or could be used to screen emerging events; and whether it was applied to individual events or the overall AE profile. Each statistical method was appraised and a taxonomy was developed for classification.

RESULTS

Forty-four eligible articles proposing 73 individual methods were included. A taxonomy was developed and articles were categorised as: visual summary methods (8 articles proposing 20 methods); hypothesis testing methods (11 articles proposing 16 methods); estimation methods (15 articles proposing 24 methods); or methods that provide decision-making probabilities (10 articles proposing 13 methods). Methods were further classified according to whether they required a prespecified event (9 articles proposing 12 methods), or could be applied to emerging events (35 articles proposing 61 methods); and if they were (group) sequential methods (10 articles proposing 12 methods) or methods to perform final/one analyses (34 articles proposing 61 methods).

CONCLUSIONS

This review highlighted that a broad range of methods exist for AE analysis. Immediate implementation of some of these could lead to improved inference for AE data in RCTs. For example, a well-designed graphic can be an effective means to communicate complex AE data and methods appropriate for counts, time-to-event data and that avoid dichotomising continuous outcomes can improve efficiencies in analysis. Previous research has shown that adoption of such methods in the scientific press is limited and that strategies to support change are needed.

TRIAL REGISTRATION

PROSPERO registration: https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=97442.

Systematic review of statistical methods for safety data in malaria chemoprevention in pregnancy trials

Background

Drug safety assessments in clinical trials present unique analytical challenges. Some of these include adjusting for individual follow-up time, repeated measurements of multiple outcomes and missing data among others. Furthermore, pre-specifying appropriate analysis becomes difficult as some safety endpoints are unexpected. Although existing guidelines such as CONSORT encourage thorough reporting of adverse events (AEs) in clinical trials, they provide limited details for safety data analysis. The limited guidelines may influence suboptimal analysis by failing to account for some analysis challenges above. A typical example where such challenges exist are trials of anti-malarial drugs for malaria prevention during pregnancy. Lack of proper standardized evaluation of the safety of antimalarial drugs has limited the ability to draw conclusions about safety. Therefore, a systematic review was conducted to establish the current practice in statistical analysis for preventive antimalarial drug safety in pregnancy.

Methods

The search included five databases (PubMed, Embase, Scopus, Malaria in Pregnancy Library and Cochrane Central Register of Controlled Trials) to identify original English articles reporting Phase III randomized controlled trials (RCTs) on anti-malarial drugs for malaria prevention in pregnancy published from January 2010 to July 2019.

Results

Eighteen trials were included in this review that collected multiple longitudinal safety outcomes including AEs. Statistical analysis and reporting of the safety outcomes in all the trials used descriptive statistics; proportions/counts (n = 18, 100%) and mean/median (n = 2, 11.1%). Results presentation included tabular (n = 16, 88.9%) and text description (n = 2, 11.1%). Univariate inferential methods were reported in most trials (n = 16, 88.9%); including Chi square/Fisher’s exact test (n = 12, 66.7%), t test (n = 2, 11.1%) and Mann–Whitney/Wilcoxon test (n = 1, 5.6%). Multivariable methods, including Poisson and negative binomial were reported in few trials (n = 3, 16.7%). Assessment of a potential link between missing efficacy data and safety outcomes was not reported in any of the trials that reported efficacy missing data (n = 7, 38.9%).

Conclusion

The review demonstrated that statistical analysis of safety data in anti-malarial drugs for malarial chemoprevention in pregnancy RCTs is inadequate. The analyses insufficiently account for multiple safety outcomes potential dependence, follow-up time and informative missing data which can compromise anti-malarial drug safety evidence development, based on the available data.

Next-Generation DILI Biomarkers: Prioritization of Biomarkers for Qualification and Best Practices for Biospecimen Collection in Drug Development

The diagnosis and management of drug-induced liver injury (DILI) remains a challenge in clinical trials in drug development. The qualification of emerging biomarkers capable of predicting DILI soon after the initiation of treatment, differentiating DILI from underlying liver disease, identifying the causal entity, and assigning appropriate treatment options after DILI is diagnosed are needed. Qualification efforts have been hindered by lack of properly stored and consented biospecimens that are linked to clinical data relevant to a specific context of use. Recommendations are made for biospecimen collection procedures, with the focus on clinical trials, and for specific emerging biomarkers to focus qualification efforts.

An evaluation of statistical approaches to postmarketing surveillance

Safety of medical products presents a serious concern worldwide. Surveillance systems of postmarket medical products have been established for continual monitoring of adverse events (AEs) in many countries, and the proliferation of electronic health record systems further facilitates continual monitoring for AEs. We review existing statistical methods for signal detection that are mostly in use in postmarketing safety surveillance of spontaneously reported AEs and we study their performance characteristics by simulation. We compare those with the likelihood ratio test (LRT) method (appropriately modified for use in pharmacovigilance) and use three different methods to generate data (AE based, drug based, and a modification of the method of Ahmed et al). Performance metrics include type I error, power, sensitivity, and false discovery rate, among others. The results show superior performance of the LRT method in almost all simulation experiments. An application to the FDA Adverse Event Reporting System database is illustrated using rhabdomyolysis-related preferred terms reported to FDA during the third-quarter of 2014 to the first-quarter of 2017 for statin drugs. We present a critical discussion and recommendations for use of these methods.

Limitations of the incidence density ratio as approximation of the hazard ratio

Background

Incidence density ratios (IDRs) are frequently used to account for varying follow-up times when comparing the risks of adverse events in two treatment groups. The validity of the IDR as approximation of the hazard ratio (HR) is unknown in the situation of differential average follow up by treatment group and non-constant hazard functions. Thus, the use of the IDR when individual patient data are not available might be questionable.

Methods

A simulation study was performed using various survival-time distributions with increasing and decreasing hazard functions and various situations of differential follow up by treatment group. HRs and IDRs were estimated from the simulated survival times and compared with the true HR. A rule of thumb was derived to decide in which data situations the IDR can be used as approximation of the HR.

Results

The results show that the validity of the IDR depends on the survival-time distribution, the difference between the average follow-up durations, the baseline risk, and the sample size. For non-constant hazard functions, the IDR is only an adequate approximation of the HR if the average follow-up durations of the groups are equal and the baseline risk is not larger than 25%. In the case of large differences in the average follow-up durations between the groups and non-constant hazard functions, the IDR represents no valid approximation of the HR.

Conclusions

The proposed rule of thumb allows the use of the IDR as approximation of the HR in specific data situations, when it is not possible to estimate the HR by means of adequate survival-time methods because the required individual patient data are not available. However, in general, adequate survival-time methods should be used to analyze adverse events rather than the simple IDR.

Interdisciplinary Safety Evaluation and Quantitative Safety Monitoring: Introduction to a Series of Papers

The American Statistical Association and DIA have created an interdisciplinary working group of drug safety experts from academia, industry and regulatory backgrounds to explore the future direction for safety monitoring. This introduction to the series explains the background and rationale for this special section.

Sources of Safety Data and Statistical Strategies for Design and Analysis: Transforming Data Into Evidence

BACKGROUND

Safety evaluation is a key aspect of medical product development. It is a continual and iterative process requiring thorough thinking, and dedicated time and resources.

METHODS

In this article, we discuss how safety data are transformed into evidence to establish and refine the safety profile of a medical product, and how the focus of safety evaluation, data sources, and statistical methods change throughout a medical product's life cycle.

RESULTS

Some challenges and statistical strategies for medical product safety evaluation are discussed. Examples of safety issues identified in different periods, that is, premarketing and postmarketing, are discussed to illustrate how different sources are used in the safety signal identification and the iterative process of safety assessment. The examples highlighted range from commonly used pediatric vaccine given to healthy children to medical products primarily used to treat a medical condition in adults. These case studies illustrate that different products may require different approaches, and once a signal is discovered, it could impact future safety assessments.

CONCLUSIONS

Many challenges still remain in this area despite advances in methodologies, infrastructure, public awareness, international harmonization, and regulatory enforcement. Innovations in safety assessment methodologies are pressing in order to make the medical product development process more efficient and effective, and the assessment of medical product marketing approval more streamlined and structured. Health care payers, providers, and patients may have different perspectives when weighing in on clinical, financial and personal needs when therapies are being evaluated.