Electronic Health Data for Postmarket Surveillance: A Vision Not Realized

Network Analysis and Machine Learning for Signal Detection and Prioritization Using Electronic Healthcare Records and Administrative Databases: A Proof of Concept in Drug-Induced Acute Myocardial Infarction

BACKGROUND

Safety signals for potential drug-induced adverse events (AEs) typically emerge from multiple data sources, primarily spontaneous reporting systems, despite known limitations. Increasingly, real-world data from sources such as electronic health records (EHRs) and administrative databases are leveraged for signal detection. Although network analysis has shown promise in mapping relationships between clinical attributes for signal detection in spontaneous reporting system databases, its application in real-world data from EHRs and administrative databases remains limited.

OBJECTIVE

This study aimed to evaluate the performance of network analysis in detecting safety signals within Italian administrative databases, using drug-induced acute myocardial infarction (AMI) as a proof of concept.

METHODS

We employed a case-crossover design to explore the association between drug exposure and AMI using the Healthcare Administrative Database of Mantova, Italy, from 2014 to 2018. Patients with their first AMI hospitalization were identified after a 365-day washout period to exclude prior hospitalizations. We constructed a network to analyse the relationships between prescribed drugs and diagnoses, represented as nodes, with undirected edges illustrating their interactions. For each patient with AMI, we identified all diagnoses and drugs recorded or redeemed within 365 days of the first AMI episode and generated various drug-diagnosis, drug-drug, and diagnosis-diagnosis pairs. We calculated the frequency of these pairs, and three types of edge weights quantified the strength of connections. We identified outlier drug-AMI pairs using a predictive score (F) based on frequency (C) and full edge weights (WF), with validation for known AMI associations. We prioritized signals using the F score, C of AMI, and WF, analysed through k-means clustering to identify patterns in the data.

RESULTS

From 2014 to 2018, a total of 3918 patients had an AMI, with 4686 AMI diagnoses. Of those, 2866 had prescriptions in the previous year, totalling 498,591 prescriptions. A network analysis identified 2968 unique nodes, revealing 529,935 diagnosis-diagnosis connections, 235,380 drug-diagnosis connections, and 102,831 drug-drug connections. The median number of connections (C) was 404 (Q1-Q3: 194-671) for drug nodes and 380 (Q1-Q3: 216-664) for diagnosis nodes. The median WF was 11.8 (Q1-Q3: 9-14), and the median F score across pairs was 0.1 (Q1-Q3: 0.1-0.3). A total of 249 potential safety signals were detected, with 63.4% aligning with known AEs. Among the remaining signals, 80 were prioritized, and five emerged as the highest priority: terazosin, tamsulosin, allopurinol, esomeprazole, and omeprazole.

CONCLUSIONS

Overall, our novel method demonstrates that network analysis is a valuable tool for signal detection and prioritization in drug-induced AEs based on EHRs and administrative databases.

Optimal Significance Levels and Sample Sizes for Signal Detection Methods Based on Non-constant Hazards

BACKGROUND AND OBJECTIVES

Statistical methods for signal detection of adverse drug reactions (ADRs) in electronic health records (EHRs) need information about optimal significance levels and sample sizes to achieve sufficient power. Sauzet and Cornelius proposed tests for signal detection based on the hazard functions of Weibull type distributions (WSP tests) which use the time-to-event information available in EHRs. Optimal significance levels and sample sizes for the application of the WPS tests are derived.

METHOD

A simulation study was performed with a range of scenarios for sample size, rate of event due (ADRs), and not due to the drug and random time to ADR occurrence. Based on the area under the curve of the receiver operating characteristic graph, we obtain optimal significance levels of the different WSP tests for the implementation in a hypothesis free signal detection setting and approximate sample sizes required to reach a power of 80% or 90%.

RESULTS

The dWSP-pPWSP (combination of double WSP and power WSP) test with a significance level of 0.004 was recommended. Sample sizes needed for a power of 80% were found to start at 60 events for an ADR rate equal to the background rate of 0.1. The number of events required for a background rate of 0.05 and an ADR rate equal to a 20% increase of the background rate was 900.

CONCLUSION

Based on this study, it is recommended to use the dWSP-pWSP test combination for signal detection with a significance level of 0.004 when the same test is applied to all adverse events not depending on rates.

The evolving role of disproportionality analysis in pharmacovigilance

INTRODUCTION

From 2009 to 2015, the IMI PROTECT conducted rigorous studies addressing questions about optimal implementation and significance of disproportionality analyses, leading to the development of Good Signal Detection Practices. The ensuing period witnessed the independent exploration of research paths proposed by IMI PROTECT, accumulating valuable experience and insights that have yet to be seamlessly integrated.

AREAS COVERED

This state-of-the-art review integrates IMI PROTECT recommendations with recent acquisitions and evolving challenges. It deals with defining the object of study, disproportionality methods, subgrouping, masking, drug-drug interaction, duplication, expectedness, the debated use of disproportionality results as risk measures, integration with other types of data.

EXPERT OPINION

Despite the ongoing skepticism regarding the usefulness of disproportionality analyses and individual case safety reports, their ability to timely detect safety signals regarding rare and unpredictable adverse reactions remains unparalleled. Moreover, recent exploration into their potential for characterizing safety signals revealed valuable insights concerning potential risk factors and the patient's perspective. To fully realize their potential beyond hypothesis generation and achieve a comprehensive evidence synthesis with other kinds of data and studies, each with their unique limitations and contributions, we need to investigate methods for more transparently communicating disproportionality results and mapping and addressing pharmacovigilance biases.

Continuous Improvement of Digital Health Applications Linked to Real-World Performance Monitoring: Safe Moving Targets?

Real-time high-quality data on the performance of digital health applications is needed for feedback-led optimization and to ensure safety and performance, particularly if they will have on-market updates. Developers must verify that applications accurately and consistently fulfill their intended purpose in real-world use. In particular, new thinking from regulators recognizes the importance of monitoring real-world performance. It is acknowledged that real-world data can deliver information from wider patient populations than are generally included in controlled studies, and in certain circumstances, this can enable extensions of the application's intended purpose. Proactive postmarket surveillance surveys are an important source of real-world data that are distinct from clinical investigations but may vary in quality and, if inappropriately designed, can be subject to uncontrolled bias. We aimed to describe the practice of real-world data gathering through patient-reported and clinician-reported outcomes and high-quality surveys and identify challenges, uncertainties, and health policy gaps.

Use of Electronic Health Record Data for Drug Safety Signal Identification: A Scoping Review

INTRODUCTION

Pharmacovigilance programs protect patient health and safety by identifying adverse event signals through postmarketing surveillance of claims data and spontaneous reports. Electronic health records (EHRs) provide new opportunities to address limitations of traditional approaches and promote discovery-oriented pharmacovigilance.

METHODS

To evaluate the current state of EHR-based medication safety signal identification, we conducted a scoping literature review of studies aimed at identifying safety signals from routinely collected patient-level EHR data. We extracted information on study design, EHR data elements utilized, analytic methods employed, drugs and outcomes evaluated, and key statistical and data analysis choices.

RESULTS

We identified 81 eligible studies. Disproportionality methods were the predominant analytic approach, followed by data mining and regression. Variability in study design makes direct comparisons difficult. Studies varied widely in terms of data, confounding adjustment, and statistical considerations.

CONCLUSION

Despite broad interest in utilizing EHRs for safety signal identification, current efforts fail to leverage the full breadth and depth of available data or to rigorously control for confounding. The development of best practices and application of common data models would promote the expansion of EHR-based pharmacovigilance.

Advancing the Regulation of Traditional and Complementary Medicine Products: A Comparison of Five Regulatory Systems on Traditional Medicines with a Long History of Use

Background

An appropriate regulatory system to ensure and promote the quality, safety, and efficacy of the products of traditional medicine (TM) and complementary medicine (CM) is critical to not only public health but also economic growth. The regulatory approach and evaluation standards for TM/CM products featured with a long history of use are yet to be developed. This study aims to investigate and compare the existing regulatory approaches for TM/CM products with a long history of use.

Method

A mixed approach of documentary analysis involving official and legal documents from official websites, as well as a scoping review of scholarly work in scientific databases about regulatory systems of TM/CM products in China, Hong Kong, Taiwan, Japan, and Korea, was employed in this study and used for comparison.

Results

For registration purposes, all five regulatory systems recognized the history of use as part of the totality of evidence when evaluating the safety and efficacy of TM/CM products with a long history of use. Generally, the list of classic formulas is predefined and bound to the formulas recommended in the prescribed list of ancient medical textbooks. Expedited pathways are usually in place and scientific data of nonclinical and clinical studies may be exempted. At the same time, additional restrictions with the scope of products constitute a comprehensive approach in the regulation. Quality assurance and postmarketing safety surveillance were found to be the major focus across the regulatory schemes investigated in this study.

Conclusion

The regulatory systems investigated in this study allow less stringent registration requirements for TM/CM products featured with a long history of use, assuming safety and efficacy to be plausible based on historic use. Considering the safety and efficacy of these products, regulatory standards should emphasize the technical requirements for quality control and postmarket surveillance.

A statistical quality assessment method for longitudinal observations in electronic health record data with an application to the VA million veteran program

Background

To describe an automated method for assessment of the plausibility of continuous variables collected in the electronic health record (EHR) data for real world evidence research use.

Methods

The most widely used approach in quality assessment (QA) for continuous variables is to detect the implausible numbers using prespecified thresholds. In augmentation to the thresholding method, we developed a score-based method that leverages the longitudinal characteristics of EHR data for detection of the observations inconsistent with the history of a patient. The method was applied to the height and weight data in the EHR from the Million Veteran Program Data from the Veteran’s Healthcare Administration (VHA). A validation study was also conducted.

Results

The receiver operating characteristic (ROC) metrics of the developed method outperforms the widely used thresholding method. It is also demonstrated that different quality assessment methods have a non-ignorable impact on the body mass index (BMI) classification calculated from height and weight data in the VHA’s database.

Conclusions

The score-based method enables automated and scaled detection of the problematic data points in health care big data while allowing the investigators to select the high-quality data based on their need. Leveraging the longitudinal characteristics in EHR will significantly improve the QA performance.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-021-01643-2.

US Food and Drug Administration utilization of postmarketing requirements and postmarketing commitments, 2009-2018

BACKGROUND/AIMS

The US Food and Drug Administration outlines clinical studies as postmarketing requirements and commitments to be fulfilled following approval of new drugs and biologics ("therapeutics"). Regulators have increasingly emphasized lifecycle evaluation of approved therapeutics, and postmarketing studies are intended to advance our understanding of therapeutic safety and efficacy. However, little is known about the indications that clinical studies outlined in postmarketing requirements and commitments investigate, including whether they are intended to generate evidence for approved or other clinical indications. Therefore, we characterized US Food and Drug Administration postmarketing requirements and commitments for new therapeutics approved from 2009 to 2018.

METHODS

We conducted a cross-sectional study of all novel therapeutics, including small-molecule drugs and biologics, receiving original US Food and Drug Administration approval from 2009 to 2018, using approval letters accessed through the Drug@FDA database. Outcomes included the number and characteristics of US Food and Drug Administration postmarketing requirements and commitments for new therapeutics at original approval, including the types of studies outlined, the indications to be investigated, and the clinical evidence to be generated.

RESULTS

From 2009 to 2018, the US Food and Drug Administration approved 343 new therapeutics with 1978 postmarketing requirements and commitments. Overall, 750 (37.9%) postmarketing requirements and commitments outlined clinical studies. For 71 of 343 (20.7%) therapeutics, no postmarketing requirements or commitments for clinical studies were outlined, while at least 1 was outlined for 272 (79.3%; median 2 (interquartile range: 1-4)). Among these 272 therapeutics, the number of postmarketing requirements and commitments for clinical studies per therapeutic did not change from 2009 (median: 2 (interquartile range: 1-4)) to 2018 (median: 2 (interquartile range: 1-3)). Among the 750 postmarketing requirements and commitments for clinical studies, 448 (59.7%) outlined new prospective cohort studies, registries, or clinical trials, while the remainder outlined retrospective studies, secondary analyses, or completion of ongoing studies. Although 455 (60.7%) clinical studies investigated only original approved therapeutic indications, 123 (16.4%) enrolled from an expansion of the approved disease population and 61 (8.1%) investigated diseases unrelated to approved indications.

CONCLUSIONS

The US Food and Drug Administration approves most new therapeutics with at least 1 postmarketing requirement or commitment for a clinical study, and outlines investigations of safety or efficacy for both approved and unapproved indications. The median number of 2 clinical studies outlined has remained relatively constant over the last decade. Given increasing emphasis by the US Food and Drug Administration on faster approval and lifecycle evaluation of therapeutics, these findings suggest that more postmarketing requirements and commitments may be necessary to address gaps in the clinical evidence available for therapeutics at approval.

The concept of justifiable healthcare and how big data can help us to achieve it

Over the last decades, the face of health care has changed dramatically, with big improvements in what is technically feasible. However, there are indicators that the current approach to evaluating evidence in health care is not holistic and hence in the long run, health care will not be sustainable. New conceptual and normative frameworks for the evaluation of health care need to be developed and investigated. The current paper presents a novel framework of justifiable health care and explores how the use of artificial intelligence and big data can contribute to achieving the goals of this framework.

Applying regulatory science in traditional chinese medicines for improving public safety and facilitating innovation in China: a scoping review and regulatory implications

BACKGROUND

The National Medical Products Administration (NMPA) in China has set to advance the regulatory capacity of traditional Chinese medicines (TCMs) with the adoption of regulatory science (RS). However, the priority of actions at the interface of RS and TCMs were yet to be defined. This research aims to identify the priority areas and summarize core actions for advancing RS for traditional medicines in China.

METHODS

A mixed approach of documentary analysis of government policies, regulations and official information about TCMs regulation in China, and a scoping review of literature using 4 databases (PubMed, ScienceDirect, Scopus and CNKI) on major concerns in TCMs regulation was employed.

RESULTS

Ten priority areas in the development of TCM-related regulatory science in China have been identified, including: (1) modernizing the regulatory system with a holistic approach; (2) advancing the methodology for the quality control of TCMs; (3) fostering the control mechanism of TCMs manufacturing process; (4) improving clinical evaluation of TCMs and leveraging real world data; (5) re-evaluation of TCMs injection; (6) developing evaluation standards for classic TCMs formula; (7) harnessing diverse data to improve pharmacovigilance of TCMs; (8) evaluating the value of integrative medicine in clinical practice with scientific research; (9) advancing the regulatory capacity to encourage innovation in TCMs; and (10) advancing a vision of collaboration for RS development in TCMs.

CONCLUSIONS

RS for TCMs in China encompasses revolution of operational procedures, advancement in science and technology, and cross-disciplinary collaborations. Such experiences could be integrated in the communications among drug regulatory authorities to promote standardized and scientific regulation of traditional medicines.

Bias in pharmacoepidemiologic studies using secondary health care databases: a scoping review

BACKGROUND

The availability of clinical and therapeutic data drawn from medical records and administrative databases has entailed new opportunities for clinical and epidemiologic research. However, these databases present inherent limitations which may render them prone to new biases. We aimed to conduct a structured review of biases specific to observational clinical studies based on secondary databases, and to propose strategies for the mitigation of those biases.

METHODS

Scoping review of the scientific literature published during the period 2000-2018 through an automated search of MEDLINE, EMBASE and Web of Science, supplemented with manually cross-checking of reference lists. We included opinion essays, methodological reviews, analyses or simulation studies, as well as letters to the editor or retractions, the principal objective of which was to highlight the existence of some type of bias in pharmacoepidemiologic studies using secondary databases.

RESULTS

A total of 117 articles were included. An increasing trend in the number of publications concerning the potential limitations of secondary databases was observed over time and across medical research disciplines. Confounding was the most reported category of bias (63.2% of articles), followed by selection and measurement biases (47.0% and 46.2% respectively). Confounding by indication (32.5%), unmeasured/residual confounding (28.2%), outcome misclassification (28.2%) and "immortal time" bias (25.6%) were the subcategories most frequently mentioned.

CONCLUSIONS

Suboptimal use of secondary databases in pharmacoepidemiologic studies has introduced biases in the studies, which may have led to erroneous conclusions. Methods to mitigate biases are available and must be considered in the design, analysis and interpretation phases of studies using these data sources.

The Role of European Healthcare Databases for Post-Marketing Drug Effectiveness, Safety and Value Evaluation: Where Does Italy Stand?

Enormous progress has been made globally in the use of evidence derived from patients' clinical information as they access their routine medical care. The value of real-world data lies in their complementary nature compared with data from randomised controlled trials: less detailed information on drug efficacy but longer observational periods and larger, more heterogeneous study populations reflecting clinical practice because individuals are included who would not usually be recruited in trials. Real-world data can be collected in various types of electronic sources, such as electronic health records, claims databases and drug or disease registries. These data sources vary in nature from country to country, according to national healthcare system structures and national policies. In Italy, a growing number of healthcare databases have been used to evaluate post-marketing drug utilisation and safety in the last two decades. The aim of this narrative review is to describe the available Italian sources of real-world data and their contribution to generating post-marketing evidence on drug use and safety. We also discuss the strengths and limitations of the most commonly used Italian healthcare databases in addressing various research questions concerning drug utilisation, comparative effectiveness and safety studies, as well as health technology assessment and other areas.

Building a framework for the evaluation of knowledge translation for the Canadian Network for Observational Drug Effect Studies

Purpose

The Canadian Network for Observational Drug Effect Studies (CNODES), a network of pharmacoepidemiologists and other researchers from seven provincial sites, provides evidence on the benefits and risks of drugs used by Canadians. The Knowledge Translation Team, one of CNODES' four main teams, evaluates the impact of its efforts using an iterative and emergent approach. This article shares key lessons from early evaluation phases, including identifying stakeholders and their evaluation needs, choosing evaluation theories and approaches, and developing evaluation questions, designs, and methods appropriate for the CNODES context.

Methods

Stakeholder analysis was conducted using documentary analysis to determine key contextual factors and research evidence needs of decision maker partners and other stakeholders. Selected theories and frameworks from the evaluation and knowledge translation literature informed decisions about evaluation design and implementation. A developmental approach to evaluation was deemed appropriate due to the innovative, complex, and ever‐changing context.

Results

A theory of change, logic model, and potential evaluation questions were developed, informed by the stakeholder analysis. Early indicators of program impact (citation metrics, alternative metrics) have been documented; efforts to collect data on additional indicators are ongoing.

Conclusion

A flexible, iterative, and emergent evaluation approach allows the Knowledge Translation Team to apply lessons learned from completed projects to ongoing research projects, adapt its approaches based on stakeholder needs, document successes, and be accountable to funders/stakeholders. This evaluation approach may be useful for other international pharmacoepidemiology research networks planning and implementing evaluations of similarly complex, multistakeholder initiatives that are subject to constant change.

Cross-national health care database utilization between Spain and France: results from the EPICHRONIC study assessing the prevalence of type 2 diabetes mellitus

AIM

The EPICHRONIC (EPIdemiology of CHRONIC diseases) project investigated the possibility of developing common procedures for French and Spanish electronic health care databases to enable large-scale pharmacoepidemiological studies on chronic diseases. A feasibility study assessed the prevalence of type 2 diabetes mellitus (T2DM) in Navarre and the Basque Country (Spain) and the Midi-Pyrénées region (France).

PATIENTS AND METHODS

We described and compared database structures and the availability of hospital, outpatient, and drug-dispensing data from 5.9 million inhabitants. Due to differences in database structures and recorded data, we could not develop a common procedure to estimate T2DM prevalence, but identified an algorithm specific to each database. Patients were identified using primary care diagnosis codes previously validated in Spanish databases and a combination of primary care diagnosis codes, hospital diagnosis codes, and data on exposure to oral antidiabetic drugs from the French database.

RESULTS

Spanish and French databases (the latter termed Système National d'Information Inter-Régimes de l'Assurance Maladie [SNIIRAM]) included demographic, primary care diagnoses, hospital diagnoses, and outpatient drug-dispensing data. Diagnoses were encoded using the International Classification of Primary Care (version 2) and the International Classification of Diseases, version 9 and version 10 (ICD-9 and ICD-10) in the Spanish databases, whereas the SNIIRAM contained ICD-10 codes. All data were anonymized before transferring to researchers. T2DM prevalence in the population over 20 years was estimated to be 6.6-7.0% in the Spanish regions and 6.3% in the Midi-Pyrénées region with ~2% higher estimates for males in the three regions.

CONCLUSION

Tailored procedures can be designed to estimate the prevalence of T2DM in population-based studies from Spanish and French electronic health care records.

Sources of Safety Data and Statistical Strategies for Design and Analysis: Transforming Data Into Evidence

BACKGROUND

Safety evaluation is a key aspect of medical product development. It is a continual and iterative process requiring thorough thinking, and dedicated time and resources.

METHODS

In this article, we discuss how safety data are transformed into evidence to establish and refine the safety profile of a medical product, and how the focus of safety evaluation, data sources, and statistical methods change throughout a medical product's life cycle.

RESULTS

Some challenges and statistical strategies for medical product safety evaluation are discussed. Examples of safety issues identified in different periods, that is, premarketing and postmarketing, are discussed to illustrate how different sources are used in the safety signal identification and the iterative process of safety assessment. The examples highlighted range from commonly used pediatric vaccine given to healthy children to medical products primarily used to treat a medical condition in adults. These case studies illustrate that different products may require different approaches, and once a signal is discovered, it could impact future safety assessments.

CONCLUSIONS

Many challenges still remain in this area despite advances in methodologies, infrastructure, public awareness, international harmonization, and regulatory enforcement. Innovations in safety assessment methodologies are pressing in order to make the medical product development process more efficient and effective, and the assessment of medical product marketing approval more streamlined and structured. Health care payers, providers, and patients may have different perspectives when weighing in on clinical, financial and personal needs when therapies are being evaluated.

Preliminary Results of a Novel Algorithmic Method Aiming to Support Initial Causality Assessment of Routine Pharmacovigilance Case Reports for Medication-Induced Liver Injury: The PV-RUCAM

INTRODUCTION

Data incompleteness in pharmacovigilance (PV) health records limits the use of current causality assessment methods for drug-induced liver injury (DILI). In addition to the inherent complexity of this adverse event, identifying cases of high causal probability is difficult.

OBJECTIVE

The aim was to evaluate the performance of an improved, algorithmic and standardised method called the Pharmacovigilance-Roussel Uclaf Causality Assessment Method (PV-RUCAM), to support assessment of suspected DILI. Performance was compared in different settings with regard to applicability and differentiation capacity.

METHODS

A PV-RUCAM score was developed based on the seven sections contained in the original RUCAM. The score provides cut-off values for or against DILI causality, and was applied on two datasets of bona fide individual case safety reports (ICSRs) extracted randomly from clinical trial reports and a third dataset of electronic health records from a global PV database. The performance of PV-RUCAM adjudication was compared against two standards: a validated causality assessment method (original RUCAM) and global introspection.

RESULTS

The findings showed moderate agreement against standards. The overall error margin of no false negatives was satisfactory, with 100% sensitivity, 91% specificity, a 25% positive predictive value and a 100% negative predictive value. The Spearman's rank correlation coefficient illustrated a statistically significant monotonic association between expert adjudication and PV-RUCAM outputs (R = 0.93). Finally, there was high inter-rater agreement (K _w = 0.79) between two PV-RUCAM assessors.

CONCLUSION

Within the PV setting of a pharmaceutical company, the PV-RUCAM has the potential to facilitate and improve the assessment done by non-expert PV professionals compared with other methods when incomplete reports must be evaluated for suspected DILI. Prospective validation of the algorithmic tool is necessary prior to implementation for routine use.

Sources of Safety Data and Statistical Strategies for Design and Analysis: Postmarket Surveillance

BACKGROUND

Safety data are continuously evaluated throughout the life cycle of a medical product to accurately assess and characterize the risks associated with the product. The knowledge about a medical product's safety profile continually evolves as safety data accumulate.

METHODS

This paper discusses data sources and analysis considerations for safety signal detection after a medical product is approved for marketing. This manuscript is the second in a series of papers from the American Statistical Association Biopharmaceutical Section Safety Working Group.

RESULTS

We share our recommendations for the statistical and graphical methodologies necessary to appropriately analyze, report, and interpret safety outcomes, and we discuss the advantages and disadvantages of safety data obtained from passive postmarketing surveillance systems compared to other sources.

CONCLUSIONS

Signal detection has traditionally relied on spontaneous reporting databases that have been available worldwide for decades. However, current regulatory guidelines and ease of reporting have increased the size of these databases exponentially over the last few years. With such large databases, data-mining tools using disproportionality analysis and helpful graphics are often used to detect potential signals. Although the data sources have many limitations, analyses of these data have been successful at identifying safety signals postmarketing. Experience analyzing these dynamic data is useful in understanding the potential and limitations of analyses with new data sources such as social media, claims, or electronic medical records data.

Statistical Power for Postlicensure Medical Product Safety Data Mining

OBJECTIVE

To perform sample size calculations when using tree-based scan statistics in longitudinal observational databases.

METHODS

Tree-based scan statistics enable data mining on epidemiologic datasets where thousands of disease outcomes are organized into hierarchical tree structures with automatic adjustment for multiple testing. We show how to evaluate the statistical power of the unconditional and conditional Poisson versions. The null hypothesis is that there is no increase in the risk for any of the outcomes. The alternative is that one or more outcomes have an excess risk. We varied the excess risk, total sample size, frequency of the underlying event rate, and the level of across-the-board health care utilization. We also quantified the reduction in statistical power resulting from specifying a risk window that was too long or too short.

RESULTS

For 500,000 exposed people, we had at least 98 percent power to detect an excess risk of 1 event per 10,000 exposed for all outcomes. In the presence of potential temporal confounding due to across-the-board elevations of health care utilization in the risk window, the conditional tree-based scan statistic controlled type I error well, while the unconditional version did not.

DISCUSSION

Data mining analyses using tree-based scan statistics expand the pharmacovigilance toolbox, ensuring adequate monitoring of thousands of outcomes of interest while controlling for multiple hypothesis testing. These power evaluations enable investigators to design and optimize implementation of retrospective data mining analyses.

Evaluating the effectiveness of risk minimisation measures: the application of a conceptual framework to Danish real-world dabigatran data

PURPOSE

For both marketing authorization holders and regulatory authorities, evaluating the effectiveness of risk minimization measures is now an integral part of pharmacovigilance in the European Union. The overall aim of activities in this area is to assess the performance of risk minimization measures implemented in order to ensure a positive benefit-risk balance in patients treated with a medicinal product.

METHODS

Following a review of the relevant literature, we developed a conceptual framework consisting of four domains (data, knowledge, behaviour and outcomes) intended for the evaluation of risk minimization measures put into practice in the Danish health-care system. For the implementation of the framework, four classes of monitoring variables can be named and defined: patient descriptors, performance-related indicators of knowledge, behaviour and outcomes.

RESULTS

We reviewed the features of the framework when applied to historical, real-world data following the introduction of dabigatran in Denmark for the prophylactic treatment of patients with non-valvular atrial fibrillation.

CONCLUSIONS

The application of the framework provided useful graphical displays and an opportunity for a statistical evaluation (interrupted time series analysis) of a regulatory intervention. © 2017 Commonwealth of Australia. Pharmacoepidemiology & Drug Safety © 2017 John Wiley & Sons, Ltd.

Good Signal Detection Practices: Evidence from IMI PROTECT

Over a period of 5 years, the Innovative Medicines Initiative PROTECT (Pharmacoepidemiological Research on Outcomes of Therapeutics by a European ConsorTium) project has addressed key research questions relevant to the science of safety signal detection. The results of studies conducted into quantitative signal detection in spontaneous reporting, clinical trial and electronic health records databases are summarised and 39 recommendations have been formulated, many based on comparative analyses across a range of databases (e.g. regulatory, pharmaceutical company). The recommendations point to pragmatic steps that those working in the pharmacovigilance community can take to improve signal detection practices, whether in a national or international agency or in a pharmaceutical company setting. PROTECT has also pointed to areas of potentially fruitful future research and some areas where further effort is likely to yield less.

Characteristics of efficacy evidence supporting approval of supplemental indications for prescription drugs in United States, 2005-14: systematic review

OBJECTIVE

To characterize the types of comparators and endpoints used in efficacy trials for approvals of supplemental indications, compared with the data supporting these drugs' originally approved indications.

DESIGN

Systematic review.

SETTING

Publicly accessible data on supplemental indications approved by the US Food and Drug Administration from 2005 to 2014.

MAIN OUTCOME MEASURES

Types of comparators (active, placebo, historical, none) and endpoints (clinical outcomes, clinical scales, surrogate) in the efficacy trials for these drugs' supplemental and original indication approvals.

RESULTS

The cohort included 295 supplemental indications. Thirty per cent (41/136) of supplemental approvals for new indications were supported by efficacy trials with active comparators, compared with 51% (47/93) of modified use approvals and 11% (7/65) of approvals expanding the patient population (P<0.001), almost all of which related to pediatric patients (61/65; 94%). Trials using clinical outcome endpoints led to approval for 32% (44/137) of supplemental approvals for new indications, 30% (28/93) of modified indication approvals, and 22% (14/65) of expanded population approvals (P=0.29). Orphan drugs had supplemental approvals for 40 non-orphan indications, which were supported by similar proportions of trials using active comparators (28% (11/40) for non-orphan supplemental indications versus 24% (10/42) for original orphan indications; P=0.70) and clinical outcome endpoints (25% (10/40) versus 31% (13/42); P=0.55).

CONCLUSIONS

Wide variations were seen in the evidence supporting approval of supplemental indications, with the fewest active comparators and clinical outcome endpoints used in trials leading to supplemental approvals that expanded the patient population.