A Comparative Assessment of Observational Medical Outcomes Partnership and Mini-Sentinel Common Data Models and Analytics: Implications for Active Drug Safety Surveillance

MetaLAB-HOI: Template standardization of health outcomes enable massive and accurate detection of adverse drug reactions from electronic health records

PURPOSE

This study aimed to advance the MetaLAB algorithm and verify its performance with multicenter data to effectively detect major adverse drug reactions (ADRs), including drug-induced liver injury.

METHODS

Based on MetaLAB, we created an optimal scenario for detecting ADRs by considering demographic and clinical records. MetaLAB-HOI was developed to identify ADR signals using common model-based multicenter electronic health record (EHR) data from the clinical health outcomes of interest (HOI) template and design for drug-exposed and nonexposed groups. In this study, we calculated the odds ratio of 101 drugs for HOI in Konyang University Hospital, Seoul National University Hospital, Chungbuk National University Hospital, and Seoul National University Bundang Hospital.

RESULTS

The overlapping drugs in four medical centers are amlodipine, aspirin, bisoprolol, carvedilol, clopidogrel, clozapine, digoxin, diltiazem, methotrexate, and rosuvastatin. We developed MetaLAB-HOI, an algorithm that can detect ADRs more efficiently using EHR. We compared the detection results of four medical centers, with drug-induced liver injuries as representative ADRs.

CONCLUSIONS

MetaLAB-HOI's strength lies in fully utilizing the patient's clinical information, such as prescription, procedure, and laboratory results, to detect ADR signals. Considering changes in the patient's condition over time, we created an algorithm based on a scenario that accounted for each drug exposure and onset period supervised by specialists for HOI. We determined that when a template capable of detecting ADR based on clinical evidence is developed and manualized, it can be applied in medical centers for new drugs with insufficient data.

Conversion of CPRD AURUM Data into the OMOP Common Data Model

Introduction

Efforts to standardize clinical data using Common Data Models (CDMS) has grown in recent years. Use of CDMs allows for quicker understanding of data structure and reuse of existing tools. One CDM is the Observational Medical Outcomes Partnership (OMOP) CDM. Clinical Practice Research Datalink (CPRD) is a data collection program collecting general practitioner data in the UK.

Objective

Our objective was to convert a static copy of CPRD AURUM data into the OMOP CDM and run existing tools on the converted data.

Methods

Two methods were used to convert each CPRD file into the OMOP CDM. The first was direct mapping used when converting CPRD files that had comparable tables in the OMOP CDM. The original names were changed to the OMOP equivalent and source values converted to standardized OMOP concepts. CPRD files: Patient (to OMOP Person), Staff (to Provider), Drug Issue (to Drug Exposure) and Practice (to Care Site) were directly mapped. The second method was indirect where for the CPRD Observation file the domain of each data row was used to assign data to proper OMOP tables or columns done by converting all source values to standard concepts.

Results

The OMOP CDM conversion populated 12 tables and 20,240,453,339 rows, with the largest table being the Measurement table (5,202,579,174 data row). Mapping source values to OMOP standard concepts, we found 60.2% (46,413 of 77,149) of source concepts were also standard concepts. The Drug Exposure table had the fewest source values already in the standard form as only 4.7% (1,433 of 30,194) of the source concepts were standard concepts. On a data retention level, only 2.00% of all data rows were excluded as they did not have a clear fit in the developed CDM and were not able to stand alone without additional information which was not present.

Conclusion

CPRD AURUM was successfully converted into the OMOP CDM with minimal data loss. Existing OHDSI tools were used with the converted data to show efficacy of the converted data. The existence of a standardized version of CPRD AURUM data vastly increases its reusability in future research due to increased understanding and tools available.

The US Food and Drug Administration Sentinel System: a national resource for a learning health system

The US Food and Drug Administration (FDA) created the Sentinel System in response to a requirement in the FDA Amendments Act of 2007 that the agency establish a system for monitoring risks associated with drug and biologic products using data from disparate sources. The Sentinel System has completed hundreds of analyses, including many that have directly informed regulatory decisions. The Sentinel System also was designed to support a national infrastructure for a learning health system. Sentinel governance and guiding principles were designed to facilitate Sentinel's role as a national resource. The Sentinel System infrastructure now supports multiple non-FDA projects for stakeholders ranging from regulated industry to other federal agencies, international regulators, and academics. The Sentinel System is a working example of a learning health system that is expanding with the potential to create a global learning health system that can support medical product safety assessments and other research.

Machine Learning in Causal Inference: Application in Pharmacovigilance

Monitoring adverse drug events or pharmacovigilance has been promoted by the World Health Organization to assure the safety of medicines through a timely and reliable information exchange regarding drug safety issues. We aim to discuss the application of machine learning methods as well as causal inference paradigms in pharmacovigilance. We first reviewed data sources for pharmacovigilance. Then, we examined traditional causal inference paradigms, their applications in pharmacovigilance, and how machine learning methods and causal inference paradigms were integrated to enhance the performance of traditional causal inference paradigms. Finally, we summarized issues with currently mainstream correlation-based machine learning models and how the machine learning community has tried to address these issues by incorporating causal inference paradigms. Our literature search revealed that most existing data sources and tasks for pharmacovigilance were not designed for causal inference. Additionally, pharmacovigilance was lagging in adopting machine learning-causal inference integrated models. We highlight several currently trending directions or gaps to integrate causal inference with machine learning in pharmacovigilance research. Finally, our literature search revealed that the adoption of causal paradigms can mitigate known issues with machine learning models. We foresee that the pharmacovigilance domain can benefit from the progress in the machine learning field.

Transformation and Evaluation of the MIMIC Database in the OMOP Common Data Model: Development and Usability Study

Background

In the era of big data, the intensive care unit (ICU) is likely to benefit from real-time computer analysis and modeling based on close patient monitoring and electronic health record data. The Medical Information Mart for Intensive Care (MIMIC) is the first open access database in the ICU domain. Many studies have shown that common data models (CDMs) improve database searching by allowing code, tools, and experience to be shared. The Observational Medical Outcomes Partnership (OMOP) CDM is spreading all over the world.

Objective

The objective was to transform MIMIC into an OMOP database and to evaluate the benefits of this transformation for analysts.

Methods

We transformed MIMIC (version 1.4.21) into OMOP format (version 5.3.3.1) through semantic and structural mapping. The structural mapping aimed at moving the MIMIC data into the right place in OMOP, with some data transformations. The mapping was divided into 3 phases: conception, implementation, and evaluation. The conceptual mapping aimed at aligning the MIMIC local terminologies to OMOP's standard ones. It consisted of 3 phases: integration, alignment, and evaluation. A documented, tested, versioned, exemplified, and open repository was set up to support the transformation and improvement of the MIMIC community's source code. The resulting data set was evaluated over a 48-hour datathon.

Results

With an investment of 2 people for 500 hours, 64% of the data items of the 26 MIMIC tables were standardized into the OMOP CDM and 78% of the source concepts mapped to reference terminologies. The model proved its ability to support community contributions and was well received during the datathon, with 160 participants and 15,000 requests executed with a maximum duration of 1 minute.

Conclusions

The resulting MIMIC-OMOP data set is the first MIMIC-OMOP data set available free of charge with real disidentified data ready for replicable intensive care research. This approach can be generalized to any medical field.

An OMOP-CDM based pharmacovigilance data-processing pipeline (PDP) providing active surveillance for ADR signal detection from real-world data sources

Background

Adverse drug reactions (ADRs) are regarded as a major cause of death and a major contributor to public health costs. For the active surveillance of drug safety, the use of real-world data and real-world evidence as part of the overall pharmacovigilance process is important. In this regard, many studies apply the data-driven approaches to support pharmacovigilance. We developed a pharmacovigilance data-processing pipeline (PDP) that utilized electronic health records (EHR) and spontaneous reporting system (SRS) data to explore pharmacovigilance signals.

Methods

To this end, we integrated two medical data sources: Konyang University Hospital (KYUH) EHR and the United States Food and Drug Administration (FDA) Adverse Event Reporting System (FAERS). As part of the presented PDP, we converted EHR data on the Observation Medical Outcomes Partnership (OMOP) data model. To evaluate the ability of using the proposed PDP for pharmacovigilance purposes, we performed a statistical validation using drugs that induce ear disorders.

Results

To validate the presented PDP, we extracted six drugs from the EHR that were significantly involved in ADRs causing ear disorders: nortriptyline, (hazard ratio [HR] 8.06, 95% CI 2.41–26.91); metoclopramide (HR 3.35, 95% CI 3.01–3.74); doxycycline (HR 1.73, 95% CI 1.14–2.62); digoxin (HR 1.60, 95% CI 1.08–2.38); acetaminophen (HR 1.59, 95% CI 1.47–1.72); and sucralfate (HR 1.21, 95% CI 1.06–1.38). In FAERS, the strongest associations were found for nortriptyline (reporting odds ratio [ROR] 1.94, 95% CI 1.73–2.16), sucralfate (ROR 1.22, 95% CI 1.01–1.45), doxycycline (ROR 1.30, 95% CI 1.20–1.40), and hydroxyzine (ROR 1.17, 95% CI 1.06–1.29). We confirmed the results in a meta-analysis using random and fixed models for doxycycline, hydroxyzine, metoclopramide, nortriptyline, and sucralfate.

Conclusions

The proposed PDP could support active surveillance and the strengthening of potential ADR signals via real-world data sources. In addition, the PDP was able to generate real-world evidence for drug safety.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-021-01520-y.

Using and improving distributed data networks to generate actionable evidence: the case of real-world outcomes in the Food and Drug Administration's Sentinel system

The US Food and Drug Administration (FDA) Sentinel System uses a distributed data network, a common data model, curated real-world data, and distributed analytic tools to generate evidence for FDA decision-making. Sentinel system needs include analytic flexibility, transparency, and reproducibility while protecting patient privacy. Based on over a decade of experience, a critical system limitation is the inability to identify enough medical conditions of interest in observational data to a satisfactory level of accuracy. Improving the system's ability to use computable phenotypes will require an "all of the above" approach that improves use of electronic health data while incorporating the growing array of complementary electronic health record data sources. FDA recently funded a Sentinel System Innovation Center and a Community Building and Outreach Center that will provide a platform for collaboration across disciplines to promote better use of real-world data for decision-making.

Suitability of databases in the Asia-Pacific for collaborative monitoring of vaccine safety

INTRODUCTION

Information regarding availability of electronic healthcare databases in the Asia-Pacific region is critical for planning vaccine safety assessments particularly, as COVID-19 vaccines are introduced. This study aimed to identify data sources in the region, potentially suitable for vaccine safety surveillance. This manuscript is endorsed by the International Society for Pharmacoepidemiology (ISPE).

METHODS

Nineteen countries targeted for database reporting were identified using published country lists and review articles. Surveillance capacity was assessed using two surveys: a 9-item introductory survey and a 51-item full survey. Survey questions related to database characteristics, covariate and health outcome variables, vaccine exposure characteristics, access and governance, and dataset linkage capability. Other questions collated research/regulatory applications of the data and local publications detailing database use for research.

RESULTS

Eleven databases containing vaccine-specific information were identified across 8 countries. Databases were largely national in coverage (8/11, 73%), encompassed all ages (9/11, 82%) with population size from 1.4 to 52 million persons. Vaccine exposure information varied particularly for standardized vaccine codes (5/11, 46%), brand (7/11, 64%) and manufacturer (5/11, 46%). Outcome data were integrated with vaccine data in 6 (55%) databases and available via linkage in 5 (46%) databases. Data approval processes varied, impacting on timeliness of data access.

CONCLUSIONS

Variation in vaccine data availability, complexities in data access including, governance and data release approval procedures, together with requirement for data linkage for outcome information, all contribute to the challenges in building a distributed network for vaccine safety assessment in the Asia-Pacific and globally. Common data models (CDMs) may help expedite vaccine safety research across the region.

Can We Rely on Results From IQVIA Medical Research Data UK Converted to the Observational Medical Outcome Partnership Common Data Model?: A Validation Study Based on Prescribing Codeine in Children

Exploring and combining results from more than one real‐world data (RWD) source might be necessary in order to explore variability and demonstrate generalizability of the results or for regulatory requirements. However, the heterogeneous nature of RWD poses challenges when working with more than one source, some of which can be solved by analyzing databases converted into a common data model (CDM). The main objective of the study was to evaluate the implementation of the Observational Medical Outcome Partnership (OMOP) CDM on IQVIA Medical Research Data (IMRD)‐UK data. A drug utilization study describing the prescribing of codeine for pain in children was used as a case study to be replicated in IMRD‐UK and its corresponding OMOP CDM transformation. Differences between IMRD‐UK source and OMOP CDM were identified and investigated. In IMRD‐UK updated to May 2017, results were similar between source and transformed data with few discrepancies. These were the result of different conventions applied during the transformation regarding the date of birth for children younger than 15 years and the start of the observation period, and of a misclassification of two drug treatments. After the initial analysis and feedback provided, a rerun of the analysis in IMRD‐UK updated to September 2018 showed almost identical results for all the measures analyzed. For this study, the conversion to OMOP CDM was adequate. Although some decisions and mapping could be improved, these impacted on the absolute results but not on the study inferences. This validation study supports six recommendations for good practice in transforming to CDMs.

Transforming French Electronic Health Records into the Observational Medical Outcome Partnership's Common Data Model: A Feasibility Study

BACKGROUND

Common data models (CDMs) enable data to be standardized, and facilitate data exchange, sharing, and storage, particularly when the data have been collected via distinct, heterogeneous systems. Moreover, CDMs provide tools for data quality assessment, integration into models, visualization, and analysis. The observational medical outcome partnership (OMOP) provides a CDM for organizing and standardizing databases. Common data models not only facilitate data integration but also (and especially for the OMOP model) extends the range of available statistical analyses.

OBJECTIVE

This study aimed to evaluate the feasibility of implementing French national electronic health records in the OMOP CDM.

METHODS

The OMOP's specifications were used to audit the source data, specify the transformation into the OMOP CDM, implement an extract-transform-load process to feed data from the French health care system into the OMOP CDM, and evaluate the final database.

RESULTS

Seventeen vocabularies corresponding to the French context were added to the OMOP CDM's concepts. Three French terminologies were automatically mapped to standardized vocabularies. We loaded nine tables from the OMOP CDM's "standardized clinical data" section, and three tables from the "standardized health system data" section. Outpatient and inpatient data from 38,730 individuals were integrated. The median (interquartile range) number of outpatient and inpatient stays per patient was 160 (19-364).

CONCLUSION

Our results demonstrated that data from the French national health care system can be integrated into the OMOP CDM. One of the main challenges was the use of international OMOP concepts to annotate data recorded in a French context. The use of local terminologies was an obstacle to conceptual mapping; with the exception of an adaptation of the International Classification of Diseases 10th Revision, the French health care system does not use international terminologies. It would be interesting to extend our present findings to the 65 million people registered in the French health care system.

Feasibility of Mapping Austrian Health Claims Data to the OMOP Common Data Model

The Main Association of Austrian Social Security Institutions collects pseudonymized claims data from Austrian social security institutions and information about hospital stays in a database for research purposes. For new studies the same data are repeatedly reprocessed and it is difficult to compare different study results even though the data is already preprocessed and prepared in a proprietary data model. Based on a study on adverse drug events in relation to inappropriate medication in geriatric patients the suitability of the Observational Medical Outcomes Partnership (OMOP) common data model (CDM) is analyzed and data is transformed into the OMOP CDM. 1,023 (99.7%) of drug codes and 3,812 (99.2%) of diagnoses codes coincide with the OMOP vocabularies. The biggest obstacles are missing mappings for the Local Vocabularies like the Austrian pharmaceutical registration numbers and the Socio-Economic Index to the OMOP vocabularies. OMOP CDM is a promising approach for the standardization of Austrian claims data. In the long run, the benefits of standardization and reproducibility of research should outweigh this initial drawback.

Choosing Among Common Data Models for Real-World Data Analyses Fit for Making Decisions About the Effectiveness of Medical Products

Many real-world data analyses use common data models (CDMs) to standardize terminologies for medication use, medical events and procedures, data structures, and interpretations of data to facilitate analyses across data sources. For decision makers, key aspects that influence the choice of a CDM may include (i) adaptability to a specific question; (ii) transparency to reproduce findings, assess validity, and instill confidence in findings; and (iii) ease and speed of use. Organizing CDMs preserve the original information from a data source and have maximum adaptability. Full mapping data models, or preconfigured rules systems, are easy to use, since all raw codes are mapped to medical constructs. Adaptive rule systems grow libraries of reusable measures that can easily adjust to preserve adaptability, expedite analyses, and ensure study-specific transparency.

The Generalized Data Model for clinical research

BACKGROUND

Most healthcare data sources store information within their own unique schemas, making reliable and reproducible research challenging. Consequently, researchers have adopted various data models to improve the efficiency of research. Transforming and loading data into these models is a labor-intensive process that can alter the semantics of the original data. Therefore, we created a data model with a hierarchical structure that simplifies the transformation process and minimizes data alteration.

METHODS

There were two design goals in constructing the tables and table relationships for the Generalized Data Model (GDM). The first was to focus on clinical codes in their original vocabularies to retain the original semantic representation of the data. The second was to retain hierarchical information present in the original data while retaining provenance. The model was tested by transforming synthetic Medicare data; Surveillance, Epidemiology, and End Results data linked to Medicare claims; and electronic health records from the Clinical Practice Research Datalink. We also tested a subsequent transformation from the GDM into the Sentinel data model.

RESULTS

The resulting data model contains 19 tables, with the Clinical Codes, Contexts, and Collections tables serving as the core of the model, and containing most of the clinical, provenance, and hierarchical information. In addition, a Mapping table allows users to apply an arbitrarily complex set of relationships among vocabulary elements to facilitate automated analyses.

CONCLUSIONS

The GDM offers researchers a simpler process for transforming data, clear data provenance, and a path for users to transform their data into other data models. The GDM is designed to retain hierarchical relationships among data elements as well as the original semantic representation of the data, ensuring consistency in protocol implementation as part of a complete data pipeline for researchers.

Creation of a Multicenter Pediatric Inpatient Data Repository Derived from Electronic Health Records

BACKGROUND

Integration of electronic health records (EHRs) data across sites and access to that data remain limited.

OBJECTIVE

We developed an EHR-based pediatric inpatient repository using nine U.S. centers from the National Institute of Child Health and Human Development Pediatric Trials Network.

METHODS

A data model encompassing 147 mandatory and 99 optional elements was developed to provide an EHR data extract of all inpatient encounters from patients <17 years of age discharged between January 6, 2013 and June 30, 2017. Sites received instructions on extractions, transformation, testing, and transmission to the coordinating center.

RESULTS

We generated 177 staging reports to process all nine sites' 147 mandatory and 99 optional data elements to the repository. Based on 520 prespecified criteria, all sites achieved 0% errors and <2% warnings. The repository includes 386,159 inpatient encounters from 264,709 children to support study design and conduct of future trials in children.

CONCLUSION

Our EHR-based data repository of pediatric inpatient encounters utilized a customized data model heavily influenced by the PCORnet format, site-based data mapping, a comprehensive set of data testing rules, and an iterative process of data submission. The common data model, site-based extraction, and technical expertise were key to our success. Data from this repository will be used in support of Pediatric Trials Network studies and the labeling of drugs and devices for children.

Predicting Future Cardiovascular Events in Patients With Peripheral Artery Disease Using Electronic Health Record Data

BACKGROUND

Patients with peripheral artery disease (PAD) are at risk of major adverse cardiac and cerebrovascular events. There are no readily available risk scores that can accurately identify which patients are most likely to sustain an event, making it difficult to identify those who might benefit from more aggressive intervention. Thus, we aimed to develop a novel predictive model-using machine learning methods on electronic health record data-to identify which PAD patients are most likely to develop major adverse cardiac and cerebrovascular events.

METHODS AND RESULTS

Data were derived from patients diagnosed with PAD at 2 tertiary care institutions. Predictive models were built using a common data model that allowed for utilization of both structured (coded) and unstructured (text) data. Only data from time of entry into the health system up to PAD diagnosis were used for modeling. Models were developed and tested using nested cross-validation. A total of 7686 patients were included in learning our predictive models. Utilizing almost 1000 variables, our best predictive model accurately determined which PAD patients would go on to develop major adverse cardiac and cerebrovascular events with an area under the curve of 0.81 (95% CI, 0.80-0.83).

CONCLUSIONS

Machine learning algorithms applied to data in the electronic health record can learn models that accurately identify PAD patients at risk of future major adverse cardiac and cerebrovascular events, highlighting the great potential of electronic health records to provide automated risk stratification for cardiovascular diseases. Common data models that can enable cross-institution research and technology development could potentially be an important aspect of widespread adoption of newer risk-stratification models.

Lessons from meta-analyses of randomized clinical trials for analysis of distributed networks of observational databases

Networks of constellations of longitudinal observational databases, often electronic medical records or transactional insurance claims or both, are increasingly being used for studying the effects of medicinal products in real-world use. Such databases are frequently configured as distributed networks. That is, patient-level data are kept behind firewalls and not communicated outside of the data vendor other than in aggregate form. Instead, data are standardized across the network, and queries of the network are executed locally by data partners, and summary results provided to a central research partner(s) for amalgamation, aggregation, and summarization. Such networks can be huge covering years of data on upwards of 100 million patients. Examples of such networks include the FDA Sentinel Network, ASPEN, CNODES, and EU-ADR. As this is a new emerging field, we note in this paper the conceptual similarities and differences between the analysis of distributed networks and the now well-established field of meta-analysis of randomized clinical trials (RCTs). We recommend, wherever appropriate, to apply learnings from meta-analysis to help guide the development of distributed network analyses of longitudinal observational databases.

Evaluation of Electronic Healthcare Databases for Post-Marketing Drug Safety Surveillance and Pharmacoepidemiology in China

INTRODUCTION

Electronic healthcare databases (EHDs) are used increasingly for post-marketing drug safety surveillance and pharmacoepidemiology in Europe and North America. However, few studies have examined the potential of these data sources in China.

METHODS

Three major types of EHDs in China (i.e., a regional community-based database, a national claims database, and an electronic medical records [EMR] database) were selected for evaluation. Forty core variables were derived based on the US Mini-Sentinel (MS) Common Data Model (CDM) as well as the data features in China that would be desirable to support drug safety surveillance. An email survey of these core variables and eight general questions as well as follow-up inquiries on additional variables was conducted. These 40 core variables across the three EHDs and all variables in each EHD along with those in the US MS CDM and Observational Medical Outcomes Partnership (OMOP) CDM were compared for availability and labeled based on specific standards.

RESULTS

All of the EHDs' custodians confirmed their willingness to share their databases with academic institutions after appropriate approval was obtained. The regional community-based database contained 1.19 million people in 2015 with 85% of core variables. Resampled annually nationwide, the national claims database included 5.4 million people in 2014 with 55% of core variables, and the EMR database included 3 million inpatients from 60 hospitals in 2015 with 80% of core variables. Compared with MS CDM or OMOP CDM, the proportion of variables across the three EHDs available or able to be transformed/derived from the original sources are 24-83% or 45-73%, respectively.

CONCLUSIONS

These EHDs provide potential value to post-marketing drug safety surveillance and pharmacoepidemiology in China. Future research is warranted to assess the quality and completeness of these EHDs or additional data sources in China.

The Canadian Chronic Disease Surveillance System: A model for collaborative surveillance

Chronic diseases have a major impact on populations and healthcare systems worldwide. Administrative health data are an ideal resource for chronic disease surveillance because they are population-based and routinely collected. For multi-jurisdictional surveillance, a distributed model is advantageous because it does not require individual-level data to be shared across jurisdictional boundaries. Our objective is to describe the process, structure, benefits, and challenges of a distributed model for chronic disease surveillance across all Canadian provinces and territories (P/Ts) using linked administrative data. The Public Health Agency of Canada (PHAC) established the Canadian Chronic Disease Surveillance System (CCDSS) in 2009 to facilitate standardized, national estimates of chronic disease prevalence, incidence, and outcomes. The CCDSS primarily relies on linked health insurance registration files, physician billing claims, and hospital discharge abstracts. Standardized case definitions and common analytic protocols are applied to the data for each P/T; aggregate data are shared with PHAC and summarized for reports and open access data initiatives. Advantages of this distributed model include: it uses the rich data resources available in all P/Ts; it supports chronic disease surveillance capacity building in all P/Ts; and changes in surveillance methodology can be easily developed by PHAC and implemented by the P/Ts. However, there are challenges: heterogeneity in administrative databases across jurisdictions and changes in data quality over time threaten the production of standardized disease estimates; a limited set of databases are common to all P/Ts, which hinders potential CCDSS expansion; and there is a need to balance comprehensive reporting with P/T disclosure requirements to protect privacy. The CCDSS distributed model for chronic disease surveillance has been successfully implemented and sustained by PHAC and its P/T partners. Many lessons have been learned about national surveillance involving jurisdictions that are heterogeneous with respect to healthcare databases, expertise and analytical capacity, population characteristics, and priorities.

Applying a common data model to Asian databases for multinational pharmacoepidemiologic studies: opportunities and challenges

Objective

The goal of the Asian Pharmacoepidemiology Network is to study the effectiveness and safety of medications commonly used in Asia using databases from individual Asian countries. An efficient infrastructure to support multinational pharmacoepidemiologic studies is critical to this effort.

Study design and setting

We converted data from the Japan Medical Data Center database, Taiwan’s National Health Insurance Research Database, Hong Kong’s Clinical Data Analysis and Reporting System, South Korea’s Ajou University School of Medicine database, and the US Medicare 5% sample to the Observational Medical Outcome Partnership common data model (CDM).

Results

We completed and documented the process for the CDM conversion. The coordinating center and participating sites reviewed the documents and refined the conversions based on the comments. The time required to convert data to the CDM varied widely across sites and included conversion to standard terminology codes and refinements of the conversion based on reviews. We mapped 97.2%, 86.7%, 92.6%, and 80.1% of domestic drug codes from the USA, Taiwan, Hong Kong, and Korea to RxNorm, respectively. The mapping rate from Japanese domestic drug codes to RxNorm (70.7%) was lower than from other countries, and we mapped remaining unmapped drugs to Anatomical Therapeutic Chemical Classification System codes. Because the native databases used international procedure coding systems for which mapping tables have been established, we were able to map >90% of diagnosis and procedure codes to standard terminology codes.

Conclusion

The CDM established the foundation and reinforced collaboration for multinational pharmacoepidemiologic studies in Asia. Mapping of terminology codes was the greatest challenge, because of differences in health systems, cultures, and coding systems.

Real-World Data Analytics Fit for Regulatory Decision-Making

Healthcare database analyses (claims, electronic health records) have been identified by various regulatory initiatives, including the 21^st Century Cures Act and Prescription Drug User Fee Act ("PDUFA"), as useful supplements to randomized clinical trials to generate evidence on the effectiveness, harm, and value of medical products in routine care. Specific applications include accelerated drug approval pathways and secondary indications for approved medical products. Such real-world data ("RWD") analyses reflect how medical products impact health outside a highly controlled research environment. A constant stream of data from the routine operation of modern healthcare systems that can be analyzed in rapid cycles enables incremental evidence development for regulatory decision-making. Key evidentiary needs by regulators include 1) monitoring of medication performance in routine care, including the effectiveness, safety and value; 2) identifying new patient strata in which a drug may have added value or unacceptable harms; and 3) monitoring targeted utilization. Four broad requirements have been proposed to enable successful regulatory decision-making based on healthcare database analyses (collectively, "MVET"): Meaningful evidence that provides relevant and context-informed evidence sufficient for interpretation, drawing conclusions, and making decisions; valid evidence that meets scientific and technical quality standards to allow causal interpretations; expedited evidence that provides incremental evidence that is synchronized with the decision-making process; and transparent evidence that is audible, reproducible, robust, and ultimately trusted by decision-makers. Evidence generation systems that satisfy MVET requirements to a high degree will contribute to effective regulatory decision-making. Rapid-cycle analytics of healthcare databases is maturing at a time when regulatory overhaul increasingly demands such evidence. Governance, regulations, and data quality are catching up as the utility of this resource is demonstrated in multiple contexts.

Towards Implementation of OMOP in a German University Hospital Consortium

Background  In 2015, the German Federal Ministry of Education and Research initiated a large data integration and data sharing research initiative to improve the reuse of data from patient care and translational research. The Observational Medical Outcomes Partnership (OMOP) common data model and the Observational Health Data Sciences and Informatics (OHDSI) tools could be used as a core element in this initiative for harmonizing the terminologies used as well as facilitating the federation of research analyses across institutions.

Objective  To realize an OMOP/OHDSI-based pilot implementation within a consortium of eight German university hospitals, evaluate the applicability to support data harmonization and sharing among them, and identify potential enhancement requirements.

Methods  The vocabularies and terminological mapping required for importing the fact data were prepared, and the process for importing the data from the source files was designed. For eight German university hospitals, a virtual machine preconfigured with the OMOP database and the OHDSI tools as well as the jobs to import the data and conduct the analysis was provided. Last, a federated/distributed query to test the approach was executed.

Results  While the mapping of ICD-10 German Modification succeeded with a rate of 98.8% of all terms for diagnoses, the procedures could not be mapped and hence an extension to the OMOP standard terminologies had to be made.

Overall, the data of 3 million inpatients with approximately 26 million conditions, 21 million procedures, and 23 million observations have been imported.

A federated query to identify a cohort of colorectal cancer patients was successfully executed and yielded 16,701 patient cases visualized in a Sunburst plot.

Conclusion  OMOP/OHDSI is a viable open source solution for data integration in a German research consortium. Once the terminology problems can be solved, researchers can build on an active community for further development.

Assessing performance of sequential analysis methods for active drug safety surveillance using observational data

The routine use of sequential methods is well established in clinical studies. Recently, there has been increasing interest in applying these methods to prospectively monitor the safety of newly approved drugs through accrual of real-world data. However, the application to marketed drugs using real-world data has been limited and work is needed to determine which sequential approaches are most suited to such data. In this study, the conditional sequential sampling procedure (CSSP), a group sequential method, was compared with a log-linear model with Poisson distribution (LLMP) through a SAS procedure (PROC GENMOD) combined with an alpha-spending function on two large longitudinal US administrative health claims databases. Relative performance in identifying known drug-outcome associations was examined using a set of 50 well-studied drug-outcome pairs. The study finds that neither method correctly identified all pairs but that LLMP often provides better ability and shorter time for identifying the known drug-outcome associations with superior computational performance when compared with CSSP, albeit with more false positives. With the features of flexible confounding control and ease of implementation, LLMP may be a good alternative or complement to CSSP.

Data Sharing and Data Registries in Physical Medicine and Rehabilitation

The field of physical medicine & rehabilitation (PM&R), along with all the disciplines it encompasses, has evolved rapidly in the past 50 years. The number of controlled trials, systematic reviews, and meta-analyses in PM&R increased 5-fold from 1998 to 2013. In recent years, professional, private, and governmental institutions have identified the need to track function and functional status across providers and settings of care and on a larger scale. Because function and functional status are key aspects of PM&R, access to and sharing of reliable data will have an important impact on clinical practice. We reviewed the current landscape of PM&R databases and data repositories, the clinical applicability and practice implications of data sharing, and challenges and future directions. We included articles that (1) addressed any aspect of function, disability, or participation; (2) focused on recovery or maintenance of any function; and (3) used data repositories or research databases. We identified 398 articles that cited 244 data sources. The data sources included 66 data repositories and 179 research databases. We categorized the data sources based on their purposes and uses, geographic distribution, and other characteristics. This study collates the range of databases, data repositories, and data-sharing mechanisms that have been used in PM&R internationally. In recent years, these data sources have provided significant information for the field, especially at the population-health level. Implications and future directions for data sources also are discussed.

Evaluating common data models for use with a longitudinal community registry

OBJECTIVE

To evaluate common data models (CDMs) to determine which is best suited for sharing data from a large, longitudinal, electronic health record (EHR)-based community registry.

MATERIALS AND METHODS

Four CDMs were chosen from models in use for clinical research data: Sentinel v5.0 (referred to as the Mini-Sentinel CDM in previous versions), PCORnet v3.0 (an extension of the Mini-Sentinel CDM), OMOP v5.0, and CDISC SDTM v1.4. Each model was evaluated against 11 criteria adapted from previous research. The criteria fell into six categories: content coverage, integrity, flexibility, ease of querying, standards compatibility, and ease and extent of implementation.

RESULTS

The OMOP CDM accommodated the highest percentage of our data elements (76%), fared well on other requirements, and had broader terminology coverage than the other models. Sentinel and PCORnet fell short in content coverage with 37% and 48% matches respectively. Although SDTM accommodated a significant percentage of data elements (55% true matches), 45% of the data elements mapped to SDTM's extension mechanism, known as Supplemental Qualifiers, increasing the number of joins required to query the data.

CONCLUSION

The OMOP CDM best met the criteria for supporting data sharing from longitudinal EHR-based studies. Conclusions may differ for other uses and associated data element sets, but the methodology reported here is easily adaptable to common data model evaluation for other uses.

Multisite Evaluation of a Data Quality Tool for Patient-Level Clinical Data Sets

Introduction:

Data quality and fitness for analysis are crucial if outputs of analyses of electronic health record data or administrative claims data should be trusted by the public and the research community.

Methods:

We describe a data quality analysis tool (called Achilles Heel) developed by the Observational Health Data Sciences and Informatics Collaborative (OHDSI) and compare outputs from this tool as it was applied to 24 large healthcare datasets across seven different organizations.

Results:

We highlight 12 data quality rules that identified issues in at least 10 of the 24 datasets and provide a full set of 71 rules identified in at least one dataset. Achilles Heel is a freely available software that provides a useful starter set of data quality rules with the ability to add additional rules. We also present results of a structured email-based interview of all participating sites that collected qualitative comments about the value of Achilles Heel for data quality evaluation.

Discussion:

Our analysis represents the first comparison of outputs from a data quality tool that implements a fixed (but extensible) set of data quality rules. Thanks to a common data model, we were able to compare quickly multiple datasets originating from several countries in America, Europe and Asia.

Successful Comparison of US Food and Drug Administration Sentinel Analysis Tools to Traditional Approaches in Quantifying a Known Drug-Adverse Event Association

The US Food and Drug Administration's Sentinel system has developed the capability to conduct active safety surveillance of marketed medical products in a large network of electronic healthcare databases. We assessed the extent to which the newly developed, semiautomated Sentinel Propensity Score Matching (PSM) tool could produce the same results as a customized protocol-driven assessment, which found an adjusted hazard ratio (HR) of 3.04 (95% confidence interval [CI], 2.81-3.27) comparing angioedema in patients initiating angiotensin-converting enzyme (ACE) inhibitors vs. beta-blockers. Using data from 13 Data Partners between 1 January 2008, and 30 September 2013, the PSM tool identified 2,211,215 eligible ACE inhibitor and 1,673,682 eligible beta-blocker initiators. The tool produced an HR of 3.14 (95% CI, 2.86-3.44). This comparison provides initial evidence that Sentinel analytic tools can produce findings similar to those produced by a highly customized protocol-driven assessment.

Big Data: transforming drug development and health policy decision making

The explosion of data sources, accompanied by the evolution of technology and analytical techniques, has created considerable challenges and opportunities for drug development and healthcare resource utilization. We present a systematic overview these phenomena, and suggest measures to be taken for effective integration of the new developments in the traditional medical research paradigm and health policy decision making. Special attention is paid to pertinent issues in emerging areas, including rare disease drug development, personalized medicine, Comparative Effectiveness Research, and privacy and confidentiality concerns.