Electronic Health Data for Postmarket Surveillance: A Vision Not Realized

Use of Electronic Health Record Data for Drug Safety Signal Identification: A Scoping Review

INTRODUCTION

Pharmacovigilance programs protect patient health and safety by identifying adverse event signals through postmarketing surveillance of claims data and spontaneous reports. Electronic health records (EHRs) provide new opportunities to address limitations of traditional approaches and promote discovery-oriented pharmacovigilance.

METHODS

To evaluate the current state of EHR-based medication safety signal identification, we conducted a scoping literature review of studies aimed at identifying safety signals from routinely collected patient-level EHR data. We extracted information on study design, EHR data elements utilized, analytic methods employed, drugs and outcomes evaluated, and key statistical and data analysis choices.

RESULTS

We identified 81 eligible studies. Disproportionality methods were the predominant analytic approach, followed by data mining and regression. Variability in study design makes direct comparisons difficult. Studies varied widely in terms of data, confounding adjustment, and statistical considerations.

CONCLUSION

Despite broad interest in utilizing EHRs for safety signal identification, current efforts fail to leverage the full breadth and depth of available data or to rigorously control for confounding. The development of best practices and application of common data models would promote the expansion of EHR-based pharmacovigilance.

Advancing the Regulation of Traditional and Complementary Medicine Products: A Comparison of Five Regulatory Systems on Traditional Medicines with a Long History of Use

Background

An appropriate regulatory system to ensure and promote the quality, safety, and efficacy of the products of traditional medicine (TM) and complementary medicine (CM) is critical to not only public health but also economic growth. The regulatory approach and evaluation standards for TM/CM products featured with a long history of use are yet to be developed. This study aims to investigate and compare the existing regulatory approaches for TM/CM products with a long history of use.

Method

A mixed approach of documentary analysis involving official and legal documents from official websites, as well as a scoping review of scholarly work in scientific databases about regulatory systems of TM/CM products in China, Hong Kong, Taiwan, Japan, and Korea, was employed in this study and used for comparison.

Results

For registration purposes, all five regulatory systems recognized the history of use as part of the totality of evidence when evaluating the safety and efficacy of TM/CM products with a long history of use. Generally, the list of classic formulas is predefined and bound to the formulas recommended in the prescribed list of ancient medical textbooks. Expedited pathways are usually in place and scientific data of nonclinical and clinical studies may be exempted. At the same time, additional restrictions with the scope of products constitute a comprehensive approach in the regulation. Quality assurance and postmarketing safety surveillance were found to be the major focus across the regulatory schemes investigated in this study.

Conclusion

The regulatory systems investigated in this study allow less stringent registration requirements for TM/CM products featured with a long history of use, assuming safety and efficacy to be plausible based on historic use. Considering the safety and efficacy of these products, regulatory standards should emphasize the technical requirements for quality control and postmarket surveillance.

A statistical quality assessment method for longitudinal observations in electronic health record data with an application to the VA million veteran program

Background

To describe an automated method for assessment of the plausibility of continuous variables collected in the electronic health record (EHR) data for real world evidence research use.

Methods

The most widely used approach in quality assessment (QA) for continuous variables is to detect the implausible numbers using prespecified thresholds. In augmentation to the thresholding method, we developed a score-based method that leverages the longitudinal characteristics of EHR data for detection of the observations inconsistent with the history of a patient. The method was applied to the height and weight data in the EHR from the Million Veteran Program Data from the Veteran’s Healthcare Administration (VHA). A validation study was also conducted.

Results

The receiver operating characteristic (ROC) metrics of the developed method outperforms the widely used thresholding method. It is also demonstrated that different quality assessment methods have a non-ignorable impact on the body mass index (BMI) classification calculated from height and weight data in the VHA’s database.

Conclusions

The score-based method enables automated and scaled detection of the problematic data points in health care big data while allowing the investigators to select the high-quality data based on their need. Leveraging the longitudinal characteristics in EHR will significantly improve the QA performance.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-021-01643-2.

US Food and Drug Administration utilization of postmarketing requirements and postmarketing commitments, 2009-2018

BACKGROUND/AIMS

The US Food and Drug Administration outlines clinical studies as postmarketing requirements and commitments to be fulfilled following approval of new drugs and biologics ("therapeutics"). Regulators have increasingly emphasized lifecycle evaluation of approved therapeutics, and postmarketing studies are intended to advance our understanding of therapeutic safety and efficacy. However, little is known about the indications that clinical studies outlined in postmarketing requirements and commitments investigate, including whether they are intended to generate evidence for approved or other clinical indications. Therefore, we characterized US Food and Drug Administration postmarketing requirements and commitments for new therapeutics approved from 2009 to 2018.

METHODS

We conducted a cross-sectional study of all novel therapeutics, including small-molecule drugs and biologics, receiving original US Food and Drug Administration approval from 2009 to 2018, using approval letters accessed through the Drug@FDA database. Outcomes included the number and characteristics of US Food and Drug Administration postmarketing requirements and commitments for new therapeutics at original approval, including the types of studies outlined, the indications to be investigated, and the clinical evidence to be generated.

RESULTS

From 2009 to 2018, the US Food and Drug Administration approved 343 new therapeutics with 1978 postmarketing requirements and commitments. Overall, 750 (37.9%) postmarketing requirements and commitments outlined clinical studies. For 71 of 343 (20.7%) therapeutics, no postmarketing requirements or commitments for clinical studies were outlined, while at least 1 was outlined for 272 (79.3%; median 2 (interquartile range: 1-4)). Among these 272 therapeutics, the number of postmarketing requirements and commitments for clinical studies per therapeutic did not change from 2009 (median: 2 (interquartile range: 1-4)) to 2018 (median: 2 (interquartile range: 1-3)). Among the 750 postmarketing requirements and commitments for clinical studies, 448 (59.7%) outlined new prospective cohort studies, registries, or clinical trials, while the remainder outlined retrospective studies, secondary analyses, or completion of ongoing studies. Although 455 (60.7%) clinical studies investigated only original approved therapeutic indications, 123 (16.4%) enrolled from an expansion of the approved disease population and 61 (8.1%) investigated diseases unrelated to approved indications.

CONCLUSIONS

The US Food and Drug Administration approves most new therapeutics with at least 1 postmarketing requirement or commitment for a clinical study, and outlines investigations of safety or efficacy for both approved and unapproved indications. The median number of 2 clinical studies outlined has remained relatively constant over the last decade. Given increasing emphasis by the US Food and Drug Administration on faster approval and lifecycle evaluation of therapeutics, these findings suggest that more postmarketing requirements and commitments may be necessary to address gaps in the clinical evidence available for therapeutics at approval.

The concept of justifiable healthcare and how big data can help us to achieve it

Over the last decades, the face of health care has changed dramatically, with big improvements in what is technically feasible. However, there are indicators that the current approach to evaluating evidence in health care is not holistic and hence in the long run, health care will not be sustainable. New conceptual and normative frameworks for the evaluation of health care need to be developed and investigated. The current paper presents a novel framework of justifiable health care and explores how the use of artificial intelligence and big data can contribute to achieving the goals of this framework.

Applying regulatory science in traditional chinese medicines for improving public safety and facilitating innovation in China: a scoping review and regulatory implications

BACKGROUND

The National Medical Products Administration (NMPA) in China has set to advance the regulatory capacity of traditional Chinese medicines (TCMs) with the adoption of regulatory science (RS). However, the priority of actions at the interface of RS and TCMs were yet to be defined. This research aims to identify the priority areas and summarize core actions for advancing RS for traditional medicines in China.

METHODS

A mixed approach of documentary analysis of government policies, regulations and official information about TCMs regulation in China, and a scoping review of literature using 4 databases (PubMed, ScienceDirect, Scopus and CNKI) on major concerns in TCMs regulation was employed.

RESULTS

Ten priority areas in the development of TCM-related regulatory science in China have been identified, including: (1) modernizing the regulatory system with a holistic approach; (2) advancing the methodology for the quality control of TCMs; (3) fostering the control mechanism of TCMs manufacturing process; (4) improving clinical evaluation of TCMs and leveraging real world data; (5) re-evaluation of TCMs injection; (6) developing evaluation standards for classic TCMs formula; (7) harnessing diverse data to improve pharmacovigilance of TCMs; (8) evaluating the value of integrative medicine in clinical practice with scientific research; (9) advancing the regulatory capacity to encourage innovation in TCMs; and (10) advancing a vision of collaboration for RS development in TCMs.

CONCLUSIONS

RS for TCMs in China encompasses revolution of operational procedures, advancement in science and technology, and cross-disciplinary collaborations. Such experiences could be integrated in the communications among drug regulatory authorities to promote standardized and scientific regulation of traditional medicines.

Bias in pharmacoepidemiologic studies using secondary health care databases: a scoping review

BACKGROUND

The availability of clinical and therapeutic data drawn from medical records and administrative databases has entailed new opportunities for clinical and epidemiologic research. However, these databases present inherent limitations which may render them prone to new biases. We aimed to conduct a structured review of biases specific to observational clinical studies based on secondary databases, and to propose strategies for the mitigation of those biases.

METHODS

Scoping review of the scientific literature published during the period 2000-2018 through an automated search of MEDLINE, EMBASE and Web of Science, supplemented with manually cross-checking of reference lists. We included opinion essays, methodological reviews, analyses or simulation studies, as well as letters to the editor or retractions, the principal objective of which was to highlight the existence of some type of bias in pharmacoepidemiologic studies using secondary databases.

RESULTS

A total of 117 articles were included. An increasing trend in the number of publications concerning the potential limitations of secondary databases was observed over time and across medical research disciplines. Confounding was the most reported category of bias (63.2% of articles), followed by selection and measurement biases (47.0% and 46.2% respectively). Confounding by indication (32.5%), unmeasured/residual confounding (28.2%), outcome misclassification (28.2%) and "immortal time" bias (25.6%) were the subcategories most frequently mentioned.

CONCLUSIONS

Suboptimal use of secondary databases in pharmacoepidemiologic studies has introduced biases in the studies, which may have led to erroneous conclusions. Methods to mitigate biases are available and must be considered in the design, analysis and interpretation phases of studies using these data sources.

Building a framework for the evaluation of knowledge translation for the Canadian Network for Observational Drug Effect Studies

Purpose

The Canadian Network for Observational Drug Effect Studies (CNODES), a network of pharmacoepidemiologists and other researchers from seven provincial sites, provides evidence on the benefits and risks of drugs used by Canadians. The Knowledge Translation Team, one of CNODES' four main teams, evaluates the impact of its efforts using an iterative and emergent approach. This article shares key lessons from early evaluation phases, including identifying stakeholders and their evaluation needs, choosing evaluation theories and approaches, and developing evaluation questions, designs, and methods appropriate for the CNODES context.

Methods

Stakeholder analysis was conducted using documentary analysis to determine key contextual factors and research evidence needs of decision maker partners and other stakeholders. Selected theories and frameworks from the evaluation and knowledge translation literature informed decisions about evaluation design and implementation. A developmental approach to evaluation was deemed appropriate due to the innovative, complex, and ever‐changing context.

Results

A theory of change, logic model, and potential evaluation questions were developed, informed by the stakeholder analysis. Early indicators of program impact (citation metrics, alternative metrics) have been documented; efforts to collect data on additional indicators are ongoing.

Conclusion

A flexible, iterative, and emergent evaluation approach allows the Knowledge Translation Team to apply lessons learned from completed projects to ongoing research projects, adapt its approaches based on stakeholder needs, document successes, and be accountable to funders/stakeholders. This evaluation approach may be useful for other international pharmacoepidemiology research networks planning and implementing evaluations of similarly complex, multistakeholder initiatives that are subject to constant change.

Cross-national health care database utilization between Spain and France: results from the EPICHRONIC study assessing the prevalence of type 2 diabetes mellitus

AIM

The EPICHRONIC (EPIdemiology of CHRONIC diseases) project investigated the possibility of developing common procedures for French and Spanish electronic health care databases to enable large-scale pharmacoepidemiological studies on chronic diseases. A feasibility study assessed the prevalence of type 2 diabetes mellitus (T2DM) in Navarre and the Basque Country (Spain) and the Midi-Pyrénées region (France).

PATIENTS AND METHODS

We described and compared database structures and the availability of hospital, outpatient, and drug-dispensing data from 5.9 million inhabitants. Due to differences in database structures and recorded data, we could not develop a common procedure to estimate T2DM prevalence, but identified an algorithm specific to each database. Patients were identified using primary care diagnosis codes previously validated in Spanish databases and a combination of primary care diagnosis codes, hospital diagnosis codes, and data on exposure to oral antidiabetic drugs from the French database.

RESULTS

Spanish and French databases (the latter termed Système National d'Information Inter-Régimes de l'Assurance Maladie [SNIIRAM]) included demographic, primary care diagnoses, hospital diagnoses, and outpatient drug-dispensing data. Diagnoses were encoded using the International Classification of Primary Care (version 2) and the International Classification of Diseases, version 9 and version 10 (ICD-9 and ICD-10) in the Spanish databases, whereas the SNIIRAM contained ICD-10 codes. All data were anonymized before transferring to researchers. T2DM prevalence in the population over 20 years was estimated to be 6.6-7.0% in the Spanish regions and 6.3% in the Midi-Pyrénées region with ~2% higher estimates for males in the three regions.

CONCLUSION

Tailored procedures can be designed to estimate the prevalence of T2DM in population-based studies from Spanish and French electronic health care records.

Sources of Safety Data and Statistical Strategies for Design and Analysis: Transforming Data Into Evidence

BACKGROUND

Safety evaluation is a key aspect of medical product development. It is a continual and iterative process requiring thorough thinking, and dedicated time and resources.

METHODS

In this article, we discuss how safety data are transformed into evidence to establish and refine the safety profile of a medical product, and how the focus of safety evaluation, data sources, and statistical methods change throughout a medical product's life cycle.

RESULTS

Some challenges and statistical strategies for medical product safety evaluation are discussed. Examples of safety issues identified in different periods, that is, premarketing and postmarketing, are discussed to illustrate how different sources are used in the safety signal identification and the iterative process of safety assessment. The examples highlighted range from commonly used pediatric vaccine given to healthy children to medical products primarily used to treat a medical condition in adults. These case studies illustrate that different products may require different approaches, and once a signal is discovered, it could impact future safety assessments.

CONCLUSIONS

Many challenges still remain in this area despite advances in methodologies, infrastructure, public awareness, international harmonization, and regulatory enforcement. Innovations in safety assessment methodologies are pressing in order to make the medical product development process more efficient and effective, and the assessment of medical product marketing approval more streamlined and structured. Health care payers, providers, and patients may have different perspectives when weighing in on clinical, financial and personal needs when therapies are being evaluated.

Preliminary Results of a Novel Algorithmic Method Aiming to Support Initial Causality Assessment of Routine Pharmacovigilance Case Reports for Medication-Induced Liver Injury: The PV-RUCAM

INTRODUCTION

Data incompleteness in pharmacovigilance (PV) health records limits the use of current causality assessment methods for drug-induced liver injury (DILI). In addition to the inherent complexity of this adverse event, identifying cases of high causal probability is difficult.

OBJECTIVE

The aim was to evaluate the performance of an improved, algorithmic and standardised method called the Pharmacovigilance-Roussel Uclaf Causality Assessment Method (PV-RUCAM), to support assessment of suspected DILI. Performance was compared in different settings with regard to applicability and differentiation capacity.

METHODS

A PV-RUCAM score was developed based on the seven sections contained in the original RUCAM. The score provides cut-off values for or against DILI causality, and was applied on two datasets of bona fide individual case safety reports (ICSRs) extracted randomly from clinical trial reports and a third dataset of electronic health records from a global PV database. The performance of PV-RUCAM adjudication was compared against two standards: a validated causality assessment method (original RUCAM) and global introspection.

RESULTS

The findings showed moderate agreement against standards. The overall error margin of no false negatives was satisfactory, with 100% sensitivity, 91% specificity, a 25% positive predictive value and a 100% negative predictive value. The Spearman's rank correlation coefficient illustrated a statistically significant monotonic association between expert adjudication and PV-RUCAM outputs (R = 0.93). Finally, there was high inter-rater agreement (K _w = 0.79) between two PV-RUCAM assessors.

CONCLUSION

Within the PV setting of a pharmaceutical company, the PV-RUCAM has the potential to facilitate and improve the assessment done by non-expert PV professionals compared with other methods when incomplete reports must be evaluated for suspected DILI. Prospective validation of the algorithmic tool is necessary prior to implementation for routine use.

Sources of Safety Data and Statistical Strategies for Design and Analysis: Postmarket Surveillance

BACKGROUND

Safety data are continuously evaluated throughout the life cycle of a medical product to accurately assess and characterize the risks associated with the product. The knowledge about a medical product's safety profile continually evolves as safety data accumulate.

METHODS

This paper discusses data sources and analysis considerations for safety signal detection after a medical product is approved for marketing. This manuscript is the second in a series of papers from the American Statistical Association Biopharmaceutical Section Safety Working Group.

RESULTS

We share our recommendations for the statistical and graphical methodologies necessary to appropriately analyze, report, and interpret safety outcomes, and we discuss the advantages and disadvantages of safety data obtained from passive postmarketing surveillance systems compared to other sources.

CONCLUSIONS

Signal detection has traditionally relied on spontaneous reporting databases that have been available worldwide for decades. However, current regulatory guidelines and ease of reporting have increased the size of these databases exponentially over the last few years. With such large databases, data-mining tools using disproportionality analysis and helpful graphics are often used to detect potential signals. Although the data sources have many limitations, analyses of these data have been successful at identifying safety signals postmarketing. Experience analyzing these dynamic data is useful in understanding the potential and limitations of analyses with new data sources such as social media, claims, or electronic medical records data.

Statistical Power for Postlicensure Medical Product Safety Data Mining

OBJECTIVE

To perform sample size calculations when using tree-based scan statistics in longitudinal observational databases.

METHODS

Tree-based scan statistics enable data mining on epidemiologic datasets where thousands of disease outcomes are organized into hierarchical tree structures with automatic adjustment for multiple testing. We show how to evaluate the statistical power of the unconditional and conditional Poisson versions. The null hypothesis is that there is no increase in the risk for any of the outcomes. The alternative is that one or more outcomes have an excess risk. We varied the excess risk, total sample size, frequency of the underlying event rate, and the level of across-the-board health care utilization. We also quantified the reduction in statistical power resulting from specifying a risk window that was too long or too short.

RESULTS

For 500,000 exposed people, we had at least 98 percent power to detect an excess risk of 1 event per 10,000 exposed for all outcomes. In the presence of potential temporal confounding due to across-the-board elevations of health care utilization in the risk window, the conditional tree-based scan statistic controlled type I error well, while the unconditional version did not.

DISCUSSION

Data mining analyses using tree-based scan statistics expand the pharmacovigilance toolbox, ensuring adequate monitoring of thousands of outcomes of interest while controlling for multiple hypothesis testing. These power evaluations enable investigators to design and optimize implementation of retrospective data mining analyses.

Evaluating the effectiveness of risk minimisation measures: the application of a conceptual framework to Danish real-world dabigatran data

PURPOSE

For both marketing authorization holders and regulatory authorities, evaluating the effectiveness of risk minimization measures is now an integral part of pharmacovigilance in the European Union. The overall aim of activities in this area is to assess the performance of risk minimization measures implemented in order to ensure a positive benefit-risk balance in patients treated with a medicinal product.

METHODS

Following a review of the relevant literature, we developed a conceptual framework consisting of four domains (data, knowledge, behaviour and outcomes) intended for the evaluation of risk minimization measures put into practice in the Danish health-care system. For the implementation of the framework, four classes of monitoring variables can be named and defined: patient descriptors, performance-related indicators of knowledge, behaviour and outcomes.

RESULTS

We reviewed the features of the framework when applied to historical, real-world data following the introduction of dabigatran in Denmark for the prophylactic treatment of patients with non-valvular atrial fibrillation.

CONCLUSIONS

The application of the framework provided useful graphical displays and an opportunity for a statistical evaluation (interrupted time series analysis) of a regulatory intervention. © 2017 Commonwealth of Australia. Pharmacoepidemiology & Drug Safety © 2017 John Wiley & Sons, Ltd.

Good Signal Detection Practices: Evidence from IMI PROTECT

Over a period of 5 years, the Innovative Medicines Initiative PROTECT (Pharmacoepidemiological Research on Outcomes of Therapeutics by a European ConsorTium) project has addressed key research questions relevant to the science of safety signal detection. The results of studies conducted into quantitative signal detection in spontaneous reporting, clinical trial and electronic health records databases are summarised and 39 recommendations have been formulated, many based on comparative analyses across a range of databases (e.g. regulatory, pharmaceutical company). The recommendations point to pragmatic steps that those working in the pharmacovigilance community can take to improve signal detection practices, whether in a national or international agency or in a pharmaceutical company setting. PROTECT has also pointed to areas of potentially fruitful future research and some areas where further effort is likely to yield less.

Characteristics of efficacy evidence supporting approval of supplemental indications for prescription drugs in United States, 2005-14: systematic review

OBJECTIVE

To characterize the types of comparators and endpoints used in efficacy trials for approvals of supplemental indications, compared with the data supporting these drugs' originally approved indications.

DESIGN

Systematic review.

SETTING

Publicly accessible data on supplemental indications approved by the US Food and Drug Administration from 2005 to 2014.

MAIN OUTCOME MEASURES

Types of comparators (active, placebo, historical, none) and endpoints (clinical outcomes, clinical scales, surrogate) in the efficacy trials for these drugs' supplemental and original indication approvals.

RESULTS

The cohort included 295 supplemental indications. Thirty per cent (41/136) of supplemental approvals for new indications were supported by efficacy trials with active comparators, compared with 51% (47/93) of modified use approvals and 11% (7/65) of approvals expanding the patient population (P<0.001), almost all of which related to pediatric patients (61/65; 94%). Trials using clinical outcome endpoints led to approval for 32% (44/137) of supplemental approvals for new indications, 30% (28/93) of modified indication approvals, and 22% (14/65) of expanded population approvals (P=0.29). Orphan drugs had supplemental approvals for 40 non-orphan indications, which were supported by similar proportions of trials using active comparators (28% (11/40) for non-orphan supplemental indications versus 24% (10/42) for original orphan indications; P=0.70) and clinical outcome endpoints (25% (10/40) versus 31% (13/42); P=0.55).

CONCLUSIONS

Wide variations were seen in the evidence supporting approval of supplemental indications, with the fewest active comparators and clinical outcome endpoints used in trials leading to supplemental approvals that expanded the patient population.