An evaluation of the THIN database in the OMOP Common Data Model for active drug safety surveillance

Data harmonization and federated learning for multi-cohort dementia research using the OMOP common data model: A Netherlands consortium of dementia cohorts case study

BACKGROUND

Establishing collaborations between cohort studies has been fundamental for progress in health research. However, such collaborations are hampered by heterogeneous data representations across cohorts and legal constraints to data sharing. The first arises from a lack of consensus in standards of data collection and representation across cohort studies and is usually tackled by applying data harmonization processes. The second is increasingly important due to raised awareness for privacy protection and stricter regulations, such as the GDPR. Federated learning has emerged as a privacy-preserving alternative to transferring data between institutions through analyzing data in a decentralized manner.

METHODS

In this study, we set up a federated learning infrastructure for a consortium of nine Dutch cohorts with appropriate data available to the etiology of dementia, including an extract, transform, and load (ETL) pipeline for data harmonization. Additionally, we assessed the challenges of transforming and standardizing cohort data using the Observational Medical Outcomes Partnership (OMOP) common data model (CDM) and evaluated our tool in one of the cohorts employing federated algorithms.

RESULTS

We successfully applied our ETL tool and observed a complete coverage of the cohorts' data by the OMOP CDM. The OMOP CDM facilitated the data representation and standardization, but we identified limitations for cohort-specific data fields and in the scope of the vocabularies available. Specific challenges arise in a multi-cohort federated collaboration due to technical constraints in local environments, data heterogeneity, and lack of direct access to the data.

CONCLUSION

In this article, we describe the solutions to these challenges and limitations encountered in our study. Our study shows the potential of federated learning as a privacy-preserving solution for multi-cohort studies that enhance reproducibility and reuse of both data and analyses.

MetaLAB-HOI: Template standardization of health outcomes enable massive and accurate detection of adverse drug reactions from electronic health records

PURPOSE

This study aimed to advance the MetaLAB algorithm and verify its performance with multicenter data to effectively detect major adverse drug reactions (ADRs), including drug-induced liver injury.

METHODS

Based on MetaLAB, we created an optimal scenario for detecting ADRs by considering demographic and clinical records. MetaLAB-HOI was developed to identify ADR signals using common model-based multicenter electronic health record (EHR) data from the clinical health outcomes of interest (HOI) template and design for drug-exposed and nonexposed groups. In this study, we calculated the odds ratio of 101 drugs for HOI in Konyang University Hospital, Seoul National University Hospital, Chungbuk National University Hospital, and Seoul National University Bundang Hospital.

RESULTS

The overlapping drugs in four medical centers are amlodipine, aspirin, bisoprolol, carvedilol, clopidogrel, clozapine, digoxin, diltiazem, methotrexate, and rosuvastatin. We developed MetaLAB-HOI, an algorithm that can detect ADRs more efficiently using EHR. We compared the detection results of four medical centers, with drug-induced liver injuries as representative ADRs.

CONCLUSIONS

MetaLAB-HOI's strength lies in fully utilizing the patient's clinical information, such as prescription, procedure, and laboratory results, to detect ADR signals. Considering changes in the patient's condition over time, we created an algorithm based on a scenario that accounted for each drug exposure and onset period supervised by specialists for HOI. We determined that when a template capable of detecting ADR based on clinical evidence is developed and manualized, it can be applied in medical centers for new drugs with insufficient data.

European Health Data & Evidence Network-learnings from building out a standardized international health data network

OBJECTIVE

Health data standardized to a common data model (CDM) simplifies and facilitates research. This study examines the factors that make standardizing observational health data to the Observational Medical Outcomes Partnership (OMOP) CDM successful.

MATERIALS AND METHODS

Twenty-five data partners (DPs) from 11 countries received funding from the European Health Data Evidence Network (EHDEN) to standardize their data. Three surveys, DataQualityDashboard results, and statistics from the conversion process were analyzed qualitatively and quantitatively. Our measures of success were the total number of days to transform source data into the OMOP CDM and participation in network research.

RESULTS

The health data converted to CDM represented more than 133 million patients. 100%, 88%, and 84% of DPs took Surveys 1, 2, and 3. The median duration of the 6 key extract, transform, and load (ETL) processes ranged from 4 to 115 days. Of the 25 DPs, 21 DPs were considered applicable for analysis of which 52% standardized their data on time, and 48% participated in an international collaborative study.

DISCUSSION

This study shows that the consistent workflow used by EHDEN proves appropriate to support the successful standardization of observational data across Europe. Over the 25 successful transformations, we confirmed that getting the right people for the ETL is critical and vocabulary mapping requires specific expertise and support of tools. Additionally, we learned that teams that proactively prepared for data governance issues were able to avoid considerable delays improving their ability to finish on time.

CONCLUSION

This study provides guidance for future DPs to standardize to the OMOP CDM and participate in distributed networks. We demonstrate that the Observational Health Data Sciences and Informatics community must continue to evaluate and provide guidance and support for what ultimately develops the backbone of how community members generate evidence.

Advancing Toward a Common Data Model in Ophthalmology: Gap Analysis of General Eye Examination Concepts to Standard Observational Medical Outcomes Partnership (OMOP) Concepts

Purpose

Evaluate the degree of concept coverage of the general eye examination in one widely used electronic health record (EHR) system using the Observational Health Data Sciences and Informatics Observational Medical Outcomes Partnership (OMOP) common data model (CDM).

Design

Study of data elements.

Participants

Not applicable.

Methods

Data elements (field names and predefined entry values) from the general eye examination in the Epic foundation system were mapped to OMOP concepts and analyzed. Each mapping was given a Health Level 7 equivalence designation-equal when the OMOP concept had the same meaning as the source EHR concept, wider when it was missing information, narrower when it was overly specific, and unmatched when there was no match. Initial mappings were reviewed by 2 graders. Intergrader agreement for equivalence designation was calculated using Cohen's kappa. Agreement on the mapped OMOP concept was calculated as a percentage of total mappable concepts. Discrepancies were discussed and a final consensus created. Quantitative analysis was performed on wider and unmatched concepts.

Main Outcome Measures

Gaps in OMOP concept coverage of EHR elements and intergrader agreement of mapped OMOP concepts.

Results

A total of 698 data elements (210 fields, 488 values) from the EHR were analyzed. The intergrader kappa on the equivalence designation was 0.88 (standard error 0.03, P < 0.001). There was a 96% agreement on the mapped OMOP concept. In the final consensus mapping, 25% (1% fields, 31% values) of the EHR to OMOP concept mappings were considered equal, 50% (27% fields, 60% values) wider, 4% (8% fields, 2% values) narrower, and 21% (52% fields, 8% values) unmatched. Of the wider mapped elements, 46% were missing the laterality specification, 24% had other missing attributes, and 30% had both issues. Wider and unmatched EHR elements could be found in all areas of the general eye examination.

Conclusions

Most data elements in the general eye examination could not be represented precisely using the OMOP CDM. Our work suggests multiple ways to improve the incorporation of important ophthalmology concepts in OMOP, including adding laterality to existing concepts. There exists a strong need to improve the coverage of ophthalmic concepts in source vocabularies so that the OMOP CDM can better accommodate vision research.

Financial Disclosures

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

LeafAI: query generator for clinical cohort discovery rivaling a human programmer

OBJECTIVE

Identifying study-eligible patients within clinical databases is a critical step in clinical research. However, accurate query design typically requires extensive technical and biomedical expertise. We sought to create a system capable of generating data model-agnostic queries while also providing novel logical reasoning capabilities for complex clinical trial eligibility criteria.

MATERIALS AND METHODS

The task of query creation from eligibility criteria requires solving several text-processing problems, including named entity recognition and relation extraction, sequence-to-sequence transformation, normalization, and reasoning. We incorporated hybrid deep learning and rule-based modules for these, as well as a knowledge base of the Unified Medical Language System (UMLS) and linked ontologies. To enable data-model agnostic query creation, we introduce a novel method for tagging database schema elements using UMLS concepts. To evaluate our system, called LeafAI, we compared the capability of LeafAI to a human database programmer to identify patients who had been enrolled in 8 clinical trials conducted at our institution. We measured performance by the number of actual enrolled patients matched by generated queries.

RESULTS

LeafAI matched a mean 43% of enrolled patients with 27 225 eligible across 8 clinical trials, compared to 27% matched and 14 587 eligible in queries by a human database programmer. The human programmer spent 26 total hours crafting queries compared to several minutes by LeafAI.

CONCLUSIONS

Our work contributes a state-of-the-art data model-agnostic query generation system capable of conditional reasoning using a knowledge base. We demonstrate that LeafAI can rival an experienced human programmer in finding patients eligible for clinical trials.

Development and Application of an Active Pharmacovigilance Framework Based on Electronic Healthcare Records from Multiple Centers in Korea

INTRODUCTION

With the availability of retrospective pharmacovigilance data, the common data model (CDM) has been identified as an efficient approach towards anonymized multicenter analysis; however, the establishment of a suitable model for individual medical systems and applications supporting their analysis is a challenge.

OBJECTIVE

The aim of this study was to construct a specialized Korean CDM (K-CDM) for pharmacovigilance systems based on a clinical scenario to detect adverse drug reactions (ADRs).

METHODS

De-identified patient records (n = 5,402,129) from 13 institutions were converted to the K-CDM. From 2005 to 2017, 37,698,535 visits, 39,910,849 conditions, 259,594,727 drug exposures, and 30,176,929 procedures were recorded. The K-CDM, which comprises three layers, is compatible with existing models and is potentially adaptable to extended clinical research. Local codes for electronic medical records (EMRs), including diagnosis, drug prescriptions, and procedures, were mapped using standard vocabulary. Distributed queries based on clinical scenarios were developed and applied to K-CDM through decentralized or distributed networks.

RESULTS

Meta-analysis of drug relative risk ratios from ten institutions revealed that non-steroidal anti-inflammatory drugs (NSAIDs) increased the risk of gastrointestinal hemorrhage by twofold compared with aspirin, and non-vitamin K anticoagulants decreased cerebrovascular bleeding risk by 0.18-fold compared with warfarin.

CONCLUSION

These results are similar to those from previous studies and are conducive for new research, thereby demonstrating the feasibility of K-CDM for pharmacovigilance. However, the low quality of original EMR data, incomplete mapping, and heterogeneity between institutions reduced the validity of the analysis, thus necessitating continuous calibration among researchers, clinicians, and the government.

Extract, transform, load framework for the conversion of health databases to OMOP

Common data models standardize the structures and semantics of health datasets, enabling reproducibility and large-scale studies that leverage the data from multiple locations and settings. The Observational Medical Outcomes Partnership Common Data Model (OMOP CDM) is one of the leading common data models. While there is a strong incentive to convert datasets to OMOP, the conversion is time and resource-intensive, leaving the research community in need of tools for mapping data to OMOP. We propose an extract, transform, load (ETL) framework that is metadata-driven and generic across source datasets. The ETL framework uses a new data manipulation language (DML) that organizes SQL snippets in YAML. Our framework includes a compiler that converts YAML files with mapping logic into an ETL script. Access to the ETL framework is available via a web application, allowing users to upload and edit YAML files via web editor and obtain an ETL SQL script for use in development environments. The structure of the DML maximizes readability, refactoring, and maintainability, while minimizing technical debt and standardizing the writing of ETL operations for mapping to OMOP. Our framework also supports transparency of the mapping process and reuse by different institutions.

Preliminary feasibility assessment of CDM-based active surveillance using current status of medical device data in medical records and OMOP-CDM

In recent years, there has been an emerging interest in the use of claims and electronic health record (EHR) data for evaluation of medical device safety and effectiveness. In Korea, national insurance electronic data interchange (EDI) code has been used as a medical device data source for common data model (CDM). This study performed a preliminary feasibility assessment of CDM-based vigilance. A cross-sectional study of target medical device data in EHR and CDM was conducted. A total of 155 medical devices were finally enrolled, with 58.7% of them having EDI codes. Femoral head prosthesis was selected as a focus group. It was registered in our institute with 11 EDI codes. However, only three EDI codes were converted to systematized nomenclature of medicine clinical terms concept. EDI code was matched in one-to-many (up to 104) with unique device identifier (UDI), including devices classified as different global medical device nomenclature. The use of UDI rather than EDI code as a medical device data source is recommended. We hope that this study will share the current state of medical device data recorded in the EHR and contribute to the introduction of CDM-based medical device vigilance by selecting appropriate medical device data sources.

Standardizing registry data to the OMOP Common Data Model: experience from three pulmonary hypertension databases

Background

The Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM) can be used to transform observational health data to a common format. CDM transformation allows for analysis across disparate databases for the generation of new, real-word evidence, which is especially important in rare disease where data are limited. Pulmonary hypertension (PH) is a progressive, life-threatening disease, with rare subgroups such as pulmonary arterial hypertension (PAH), for which generating real-world evidence is challenging. Our objective is to document the process and outcomes of transforming registry data in PH to the OMOP CDM, and highlight challenges and our potential solutions.

Methods

Three observational studies were transformed from the Clinical Data Interchange Standards Consortium study data tabulation model (SDTM) to OMOP CDM format. OPUS was a prospective, multi-centre registry (2014–2020) and OrPHeUS was a retrospective, multi-centre chart review (2013–2017); both enrolled patients newly treated with macitentan in the US. EXPOSURE is a prospective, multi-centre cohort study (2017–ongoing) of patients newly treated with selexipag or any PAH-specific therapy in Europe and Canada. OMOP CDM version 5.3.1 with recent OMOP CDM vocabulary was used. Imputation rules were defined and applied for missing dates to avoid exclusion of data. Custom target concepts were introduced when existing concepts did not provide sufficient granularity.

Results

Of the 6622 patients in the three registry studies, records were mapped for 6457. Custom target concepts were introduced for PAH subgroups (by combining SNOMED concepts or creating custom concepts) and World Health Organization functional class. Per the OMOP CDM convention, records about the absence of an event, or the lack of information, were not mapped. Excluding these non-event records, 4% (OPUS), 2% (OrPHeUS) and 1% (EXPOSURE) of records were not mapped.

Conclusions

SDTM data from three registries were transformed to the OMOP CDM with limited exclusion of data and deviation from the SDTM database content. Future researchers can apply our strategy and methods in different disease areas, with tailoring as necessary. Mapping registry data to the OMOP CDM facilitates more efficient collaborations between researchers and establishment of federated data networks, which is an unmet need in rare diseases.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-021-01434-3.

Association between the use of allopurinol and risk of increased thyroid-stimulating hormone level

Allopurinol is the first-line agent for patients with gout, including those with moderate-to-severe chronic kidney disease. However, increased thyroid-stimulating hormone (TSH) levels are observed in patients with long-term allopurinol treatment. This large-scale, nested case-control, retrospective observational study analysed the association between allopurinol use and increased TSH levels. A common data model based on an electronic medical record database of 19,200,973 patients from seven hospitals between January 1997 and September 2020 was used. Individuals aged > 19 years in South Korea with at least one record of a blood TSH test were included. Data of 59,307 cases with TSH levels > 4.5 mIU/L and 236,508 controls matched for sex, age (± 5), and cohort registration date (± 30 days) were analysed. An association between the risk of increased TSH and allopurinol use in participants from five hospitals was observed. A meta-analysis (I² = 0) showed that the OR was 1.51 (95% confidence interval: 1.32-1.72) in both the fixed and random effects models. The allopurinol intake group demonstrated that increased TSH did not significantly affect free thyroxine and thyroxine levels. After the index date, some diseases were likely to occur in patients with subclinical hypothyroidism and hypothyroidism. Allopurinol administration may induce subclinical hypothyroidism.

Transformation of Electronic Health Records and Questionnaire Data to OMOP CDM: A Feasibility Study Using SG_T2DM Dataset

Background  Diabetes mellitus (DM) is an important public health concern in Singapore and places a massive burden on health care spending. Tackling chronic diseases such as DM requires innovative strategies to integrate patients' data from diverse sources and use scientific discovery to inform clinical practice that can help better manage the disease. The Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM) was chosen as the framework for integrating data with disparate formats.

Objective  The study aimed to evaluate the feasibility of converting Singapore based data source, comprising of electronic health records (EHR), cognitive and depression assessment questionnaire data to OMOP CDM standard. Additionally, we also validate whether our OMOP CDM instance is fit for the purpose of research by executing a simple treatment pathways study using Atlas, a graphical user interface tool to conduct analysis on OMOP CDM data as a proof of concept.

Methods  We used de-identified EHR, cognitive, and depression assessment questionnaires data from a tertiary care hospital in Singapore to convert it to version 5.3.1 of OMOP CDM standard. We evaluate the OMOP CDM conversion by (1) assessing the mapping coverage (that is the percentage of source terms mapped to OMOP CDM standard); (2) local raw dataset versus CDM dataset analysis; and (3) Implementing Harmonized Intrinsic Data Quality Framework using an open-source R package called Data Quality Dashboard.

Results  The content coverage of OMOP CDM vocabularies is more than 90% for clinical data, but only around 11% for questionnaire data. The comparison of characteristics between source and target data returned consistent results and our transformed data did not pass 38 (1.4%) out of 2,622 quality checks.

Conclusion  Adoption of OMOP CDM at our site demonstrated that EHR data are feasible for standardization with minimal information loss, whereas challenges remain for standardizing cognitive and depression assessment questionnaire data that requires further work.

Do Nonsteroidal Anti-Inflammatory or COX-2 Inhibitor Drugs Increase the Nonunion or Delayed Union Rates After Fracture Surgery?: A Propensity-Score-Matched Study

BACKGROUND

The effects of nonsteroidal anti-inflammatory drugs (NSAIDs)/cyclooxygenase (COX)-2 inhibitors on postoperative fracture-healing are controversial. Thus, we investigated the association between NSAID/COX-2 inhibitor administration and postoperative nonunion or delayed union of fractures. We aimed to determine the effects of NSAID/COX-2 inhibitor administration on postoperative fracture-healing with use of a common data model.

METHODS

Patients who underwent operative treatment of a fracture between 1998 and 2018 were included. To determine the effects of NSAID/COX-2 inhibitor administration on fracture-healing, postoperative NSAID/COX-2 inhibitor users were compared and 1:1 matched to nonusers, with 3,264 patients matched. The effect of each agent on bone-healing was determined on the basis of the primary outcome (nonunion/delayed union), defined as having a diagnosis code for nonunion or delayed union ≥6 months after surgery. The secondary outcome was reoperation for nonunion/delayed union. To examine the effect of NSAIDs/COX-2 inhibitors on bone union according to medication duration, a Kaplan-Meier survival analysis was performed.

RESULTS

Of the 8,693 patients who were included in the analysis, 208 had nonunion (178 patients; 2.05%) or delayed union (30 patients; 0.35%). Sixty-four (30.8%) of those 208 patients had a reoperation for nonunion or delayed union. NSAID users showed a significantly lower hazard of nonunion compared with the matched cohort of nonusers (hazard ratio, 0.69 [95% confidence interval, 0.48 to 0.98]; p = 0.040) but did not show a significant difference in the other matched comparison for any other outcomes. Kaplan-Meier survival analysis revealed significantly lower and higher nonunion/delayed union rates when the medication durations were ≤3 and >3 weeks, respectively (p = 0.001). For COX-2 inhibitors, the survival curve according to the medication duration showed no significant difference among the groups (p = 0.9).

CONCLUSIONS

Our study demonstrated no short-term impact of NSAIDs/COX-2 inhibitors on long-bone fracture-healing. However, continued use of these medications for a period of >3 weeks may be associated with higher rates of nonunion or delayed union.

LEVEL OF EVIDENCE

Therapeutic Level III. See Instructions for Authors for a complete description of levels of evidence.

Evaluation of the Privacy Risks of Personal Health Identifiers and Quasi-Identifiers in a Distributed Research Network: Development and Validation Study

Background

Privacy should be protected in medical data that include patient information. A distributed research network (DRN) is one of the challenges in privacy protection and in the encouragement of multi-institutional clinical research. A DRN standardizes multi-institutional data into a common structure and terminology called a common data model (CDM), and it only shares analysis results. It is necessary to measure how a DRN protects patient information privacy even without sharing data in practice.

Objective

This study aimed to quantify the privacy risk of a DRN by comparing different deidentification levels focusing on personal health identifiers (PHIs) and quasi-identifiers (QIs).

Methods

We detected PHIs and QIs in an Observational Medical Outcomes Partnership (OMOP) CDM as threatening privacy, based on 18 Health Insurance Portability and Accountability Act of 1996 (HIPPA) identifiers and previous studies. To compare the privacy risk according to the different privacy policies, we generated limited and safe harbor data sets based on 16 PHIs and 12 QIs as threatening privacy from the Synthetic Public Use File 5 Percent (SynPUF5PCT) data set, which is a public data set of the OMOP CDM. With minimum cell size and equivalence class methods, we measured the privacy risk reduction with a trust differential gap obtained by comparing the two data sets. We also measured the gap in randomly sampled records from the two data sets to adjust the number of PHI or QI records.

Results

The gaps averaged 31.448% and 73.798% for PHIs and QIs, respectively, with a minimum cell size of one, which represents a unique record in a data set. Among PHIs, the national provider identifier had the highest gap of 71.236% (71.244% and 0.007% in the limited and safe harbor data sets, respectively). The maximum size of the equivalence class, which has the largest size of an indistinguishable set of records, averaged 771. In 1000 random samples of PHIs, Device_exposure_start_date had the highest gap of 33.730% (87.705% and 53.975% in the data sets). Among QIs, Death had the highest gap of 99.212% (99.997% and 0.784% in the data sets). In 1000, 10,000, and 100,000 random samples of QIs, Device_treatment had the highest gaps of 12.980% (99.980% and 87.000% in the data sets), 60.118% (99.831% and 39.713%), and 93.597% (98.805% and 5.207%), respectively, and in 1 million random samples, Death had the highest gap of 99.063% (99.998% and 0.934% in the data sets).

Conclusions

In this study, we verified and quantified the privacy risk of PHIs and QIs in the DRN. Although this study used limited PHIs and QIs for verification, the privacy limitations found in this study could be used as a quality measurement index for deidentification of multi-institutional collaboration research, thereby increasing DRN safety.

An OMOP-CDM based pharmacovigilance data-processing pipeline (PDP) providing active surveillance for ADR signal detection from real-world data sources

Background

Adverse drug reactions (ADRs) are regarded as a major cause of death and a major contributor to public health costs. For the active surveillance of drug safety, the use of real-world data and real-world evidence as part of the overall pharmacovigilance process is important. In this regard, many studies apply the data-driven approaches to support pharmacovigilance. We developed a pharmacovigilance data-processing pipeline (PDP) that utilized electronic health records (EHR) and spontaneous reporting system (SRS) data to explore pharmacovigilance signals.

Methods

To this end, we integrated two medical data sources: Konyang University Hospital (KYUH) EHR and the United States Food and Drug Administration (FDA) Adverse Event Reporting System (FAERS). As part of the presented PDP, we converted EHR data on the Observation Medical Outcomes Partnership (OMOP) data model. To evaluate the ability of using the proposed PDP for pharmacovigilance purposes, we performed a statistical validation using drugs that induce ear disorders.

Results

To validate the presented PDP, we extracted six drugs from the EHR that were significantly involved in ADRs causing ear disorders: nortriptyline, (hazard ratio [HR] 8.06, 95% CI 2.41–26.91); metoclopramide (HR 3.35, 95% CI 3.01–3.74); doxycycline (HR 1.73, 95% CI 1.14–2.62); digoxin (HR 1.60, 95% CI 1.08–2.38); acetaminophen (HR 1.59, 95% CI 1.47–1.72); and sucralfate (HR 1.21, 95% CI 1.06–1.38). In FAERS, the strongest associations were found for nortriptyline (reporting odds ratio [ROR] 1.94, 95% CI 1.73–2.16), sucralfate (ROR 1.22, 95% CI 1.01–1.45), doxycycline (ROR 1.30, 95% CI 1.20–1.40), and hydroxyzine (ROR 1.17, 95% CI 1.06–1.29). We confirmed the results in a meta-analysis using random and fixed models for doxycycline, hydroxyzine, metoclopramide, nortriptyline, and sucralfate.

Conclusions

The proposed PDP could support active surveillance and the strengthening of potential ADR signals via real-world data sources. In addition, the PDP was able to generate real-world evidence for drug safety.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-021-01520-y.

Using and improving distributed data networks to generate actionable evidence: the case of real-world outcomes in the Food and Drug Administration's Sentinel system

The US Food and Drug Administration (FDA) Sentinel System uses a distributed data network, a common data model, curated real-world data, and distributed analytic tools to generate evidence for FDA decision-making. Sentinel system needs include analytic flexibility, transparency, and reproducibility while protecting patient privacy. Based on over a decade of experience, a critical system limitation is the inability to identify enough medical conditions of interest in observational data to a satisfactory level of accuracy. Improving the system's ability to use computable phenotypes will require an "all of the above" approach that improves use of electronic health data while incorporating the growing array of complementary electronic health record data sources. FDA recently funded a Sentinel System Innovation Center and a Community Building and Outreach Center that will provide a platform for collaboration across disciplines to promote better use of real-world data for decision-making.

Common Problems, Common Data Model Solutions: Evidence Generation for Health Technology Assessment

There is growing interest in using observational data to assess the safety, effectiveness, and cost effectiveness of medical technologies, but operational, technical, and methodological challenges limit its more widespread use. Common data models and federated data networks offer a potential solution to many of these problems. The open-source Observational and Medical Outcomes Partnerships (OMOP) common data model standardises the structure, format, and terminologies of otherwise disparate datasets, enabling the execution of common analytical code across a federated data network in which only code and aggregate results are shared. While common data models are increasingly used in regulatory decision making, relatively little attention has been given to their use in health technology assessment (HTA). We show that the common data model has the potential to facilitate access to relevant data, enable multidatabase studies to enhance statistical power and transfer results across populations and settings to meet the needs of local HTA decision makers, and validate findings. The use of open-source and standardised analytics improves transparency and reduces coding errors, thereby increasing confidence in the results. Further engagement from the HTA community is required to inform the appropriate standards for mapping data to the common data model and to design tools that can support evidence generation and decision making.

Estimation of sodium-glucose cotransporter 2 inhibitor-related genital and urinary tract infections via electronic medical record-based common data model

WHAT IS KNOWN AND OBJECTIVES

In Korea, the side effects of sodium-glucose cotransporter 2 inhibitors (SGLT2i) have not been clearly reported, aside from voluntary reporting. We aimed to develop detection algorithms for SGLT2i-related genital tract infections (GTIs) and urinary tract infections (UTIs) via a common data model (CDM), an electronic medical record-based database for supporting multi-hospital clinical research. We estimated the occurrence of GTIs and UTIs and-by assessing the status of each step of the algorithm-we also aimed to determine how clinicians responded to the SGLT2i-related GTIs and UTIs.

METHODS

We targeted all patients who were prescribed SGLT2i at Catholic University Seoul St. Mary's Hospital and Hallym University Dongtan Sacred Heart Hospital from January 2014 to August 2018. We developed algorithms for detection of SGLT2i-related GTIs or UTIs that divided patients into "most likely," "possibly" or "less likely" categories of GTIs or UTIs. The numbers of patients at each step were extracted.

RESULTS AND DISCUSSION

A total of 4253 patients received their first prescription of SGLT2i. According to the algorithm used in this study, the proportions of "most likely GTI" and "possibly GTI" were 0.9% (37 out of 4253) and 19.4% (826 out of 4253 patients), respectively. Similarly, the proportions of "most likely UTI" and "possibly UTI" were 0.9% (38 out of 4253) and 20.2% (858 out of 4253 patients), respectively. Compared to the various existing prospective studies, both GTIs and UTIs showed lower occurrence among patients who met "most likely" criteria and higher occurrence among those who met "possibly" criteria. When a GTI or UTI occurred or was suspected, the overall rate of discontinuing SGLT2i was 51.8% (1721 out of 3323). Despite a confirmed or suspected GTI and an UTI, 62.8% (1460 out of 2323) and 14.2% (142 out of 1000) of patients continued to take SGLT2i, respectively. The discontinuation rate for suspected GTIs was significantly lower than that for suspected UTIs (37.2% vs. 85.8%, p < 0.001).

WHAT IS NEW AND CONCLUSION

In this study, although the GTIs appeared to have a similar occurrence as UTIs, however, the discontinuation rate of SGLT2i for suspected GTIs was relatively lower. Our study is novel in that we identified how the physicians approached SGLT2i-related GTIs or UTIs at each step in a real-world clinical practice setting. Although we could estimate SGLT2i-related GTIs and UTIs via CDM, we were limited in our ability to accurately detect mild drug side effects via CDM, which lacked data for operational definition.

Transforming and evaluating electronic health record disease phenotyping algorithms using the OMOP common data model: a case study in heart failure

Objective

The aim of the study was to transform a resource of linked electronic health records (EHR) to the OMOP common data model (CDM) and evaluate the process in terms of syntactic and semantic consistency and quality when implementing disease and risk factor phenotyping algorithms.

Materials and Methods

Using heart failure (HF) as an exemplar, we represented three national EHR sources (Clinical Practice Research Datalink, Hospital Episode Statistics Admitted Patient Care, Office for National Statistics) into the OMOP CDM 5.2. We compared the original and CDM HF patient population by calculating and presenting descriptive statistics of demographics, related comorbidities, and relevant clinical biomarkers.

Results

We identified a cohort of 502 536 patients with the incident and prevalent HF and converted 1 099 195 384 rows of data from 216 581 914 encounters across three EHR sources to the OMOP CDM. The largest percentage (65%) of unmapped events was related to medication prescriptions in primary care. The average coverage of source vocabularies was >98% with the exception of laboratory tests recorded in primary care. The raw and transformed data were similar in terms of demographics and comorbidities with the largest difference observed being 3.78% in the prevalence of chronic obstructive pulmonary disease (COPD).

Conclusion

Our study demonstrated that the OMOP CDM can successfully be applied to convert EHR linked across multiple healthcare settings and represent phenotyping algorithms spanning multiple sources. Similar to previous research, challenges mapping primary care prescriptions and laboratory measurements still persist and require further work. The use of OMOP CDM in national UK EHR is a valuable research tool that can enable large-scale reproducible observational research.

Real-world reproducibility study characterizing patients newly diagnosed with multiple myeloma using Clinical Practice Research Datalink, a UK-based electronic health records database

Purpose

We evaluated the reproducibility of a study characterizing newly‐diagnosed multiple myeloma (MM) patients within an electronic health records (EHR) database using different analytic tools.

Methods

We reproduced the findings of a descriptive cohort study using an iterative two‐phase approach. In Phase I, a common protocol and statistical analysis plan (SAP) were implemented by independent investigators using the Aetion Evidence Platform® (AEP), a rapid‐cycle analytics tool, and SAS statistical software as a gold standard for statistical analyses. Using the UK Clinical Practice Research Datalink (CPRD) dataset, the study included patients newly diagnosed with MM within primary care setting and assessed baseline demographics, conditions, drug exposure, and laboratory procedures. Phase II incorporated analysis revisions based on our initial comparison of the Phase I findings. Reproducibility of findings was evaluate by calculating the match rate and absolute difference in prevalence between the SAS and AEP study results.

Results

Phase I yielded slightly discrepant results, prompting amendments to SAP to add more clarity to operational decisions. After detailed specification of data and operational choices, exact concordance was achieved for the number of eligible patients (N = 2646), demographics, comorbidities (i.e., osteopenia, osteoporosis, cardiovascular disease [CVD], and hypertension), bone pain, skeletal‐related events, drug exposure, and laboratory investigations in the Phase II analyses.

Conclusions

In this reproducibility study, a rapid‐cycle analytics tool and traditional statistical software achieved near‐exact findings after detailed specification of data and operational choices. Transparency and communication of the study design, operational and analytical choices between independent investigators were critical to achieve this reproducibility.

Content Coverage Evaluation of the OMOP Vocabulary on the Transplant Domain Focusing on Concepts Relevant for Kidney Transplant Outcomes Analysis

BACKGROUND

Improving outcomes of transplant recipients within and across transplant centers is important with the increasing number of organ transplantations being performed. The current practice is to analyze the outcomes based on patient level data submitted to the United Network for Organ Sharing (UNOS). Augmenting the UNOS data with other sources such as the electronic health record will enrich the outcomes analysis, for which a common data model (CDM) can be a helpful tool for transforming heterogeneous source data into a uniform format.

OBJECTIVES

In this study, we evaluated the feasibility of representing concepts from the UNOS transplant registry forms with the Observational Medical Outcomes Partnership (OMOP) CDM vocabulary to understand the content coverage of OMOP vocabulary on transplant-specific concepts.

METHODS

Two annotators manually mapped a total of 3,571 unique concepts extracted from the UNOS registry forms to concepts in the OMOP vocabulary. Concept mappings were evaluated by (1) examining the agreement among the initial two annotators and (2) investigating the number of UNOS concepts not mapped to a concept in the OMOP vocabulary and then classifying them. A subset of mappings was validated by clinicians.

RESULTS

There was a substantial agreement between annotators with a kappa score of 0.71. We found that 55.5% of UNOS concepts could not be represented with OMOP standard concepts. The majority of unmapped UNOS concepts were categorized into transplant, measurement, condition, and procedure concepts.

CONCLUSION

We identified categories of unmapped concepts and found that some transplant-specific concepts do not exist in the OMOP vocabulary. We suggest that adding these missing concepts to OMOP would facilitate further research in the transplant domain.

Data extraction for epidemiological research (DExtER): a novel tool for automated clinical epidemiology studies

The use of primary care electronic health records for research is abundant. The benefits gained from utilising such records lies in their size, longitudinal data collection and data quality. However, the use of such data to undertake high quality epidemiological studies, can lead to significant challenges particularly in dealing with misclassification, variation in coding and the significant effort required to pre-process the data in a meaningful format for statistical analysis. In this paper, we describe a methodology to aid with the extraction and processing of such databases, delivered by a novel software programme; the "Data extraction for epidemiological research" (DExtER). The basis of DExtER relies on principles of extract, transform and load processes. The tool initially provides the ability for the healthcare dataset to be extracted, then transformed in a format whereby data is normalised, converted and reformatted. DExtER has a user interface designed to obtain data extracts specific to each research question and observational study design. There are facilities to input the requirements for; eligible study period, definition of exposed and unexposed groups, outcome measures and important baseline covariates. To date the tool has been utilised and validated in a multitude of settings. There have been over 35 peer-reviewed publications using the tool, and DExtER has been implemented as a validated public health surveillance tool for obtaining accurate statistics on epidemiology of key morbidities. Future direction of this work will be the application of the framework to linked as well as international datasets and the development of standardised methods for conducting electronic pre-processing and extraction from datasets for research purposes.

EHR-Independent Predictive Decision Support Architecture Based on OMOP

BACKGROUND

The increasing availability of molecular and clinical data of cancer patients combined with novel machine learning techniques has the potential to enhance clinical decision support, example, for assessing a patient's relapse risk. While these prediction models often produce promising results, a deployment in clinical settings is rarely pursued.

OBJECTIVES

In this study, we demonstrate how prediction tools can be integrated generically into a clinical setting and provide an exemplary use case for predicting relapse risk in melanoma patients.

METHODS

To make the decision support architecture independent of the electronic health record (EHR) and transferable to different hospital environments, it was based on the widely used Observational Medical Outcomes Partnership (OMOP) common data model (CDM) rather than on a proprietary EHR data structure. The usability of our exemplary implementation was evaluated by means of conducting user interviews including the thinking-aloud protocol and the system usability scale (SUS) questionnaire.

RESULTS

An extract-transform-load process was developed to extract relevant clinical and molecular data from their original sources and map them to OMOP. Further, the OMOP WebAPI was adapted to retrieve all data for a single patient and transfer them into the decision support Web application for enabling physicians to easily consult the prediction service including monitoring of transferred data. The evaluation of the application resulted in a SUS score of 86.7.

CONCLUSION

This work proposes an EHR-independent means of integrating prediction models for deployment in clinical settings, utilizing the OMOP CDM. The usability evaluation revealed that the application is generally suitable for routine use while also illustrating small aspects for improvement.

Effect of Age on the Initiation of Biologic Agent Therapy in Patients With Inflammatory Bowel Disease: Korean Common Data Model Cohort Study

BACKGROUND

The Observational Health Data Sciences and Informatics (OHDSI) network is an international collaboration established to apply open-source data analytics to a large network of health databases, including the Korean common data model (K-CDM) network.

OBJECTIVE

The aim of this study is to analyze the effect that age at diagnosis has on the prognosis of inflammatory bowel disease (IBD) in Korea using a CDM network database.

METHODS

We retrospectively analyzed the K-CDM network database from 2005 to 2015. We transformed the electronic medical record into the CDM version 5.0 used in OHDSI. A worsened IBD prognosis was defined as the initiation of therapy with biologic agents, including infliximab and adalimumab. To evaluate the effect that age at diagnosis had on the prognosis of IBD, we divided the patients into an early-onset (EO) IBD group (age at diagnosis <40 years) and a late-onset (LO) IBD group (age at diagnosis ≥40 years) with the cutoff value of age at diagnosis as 40 years, which was calculated using the Youden index method. We then used the logrank test and Cox proportional hazards model to analyze the effect that age at diagnosis (EO group vs LO group) had on the prognosis in patients with IBD.

RESULTS

A total of 3480 patients were enrolled. There was 2017 patients with ulcerative colitis (UC) and 1463 with Crohn's disease (CD). The median follow up period was 109.5 weeks. The EO UC group was statistically significant and showed less event-free survival (ie, experiences of biologic agents) than the LO UC group (P<.001). In CD, the EO CD group showed less event-free survival (ie, experiences of biologic agents) than the LO CD group. In the Cox proportional hazard analysis, the odds ratio (OR) of the EO UC group on experiences of biologic agents compared with the LO UC group was 2.3 (95% CI 1.3-3.8, P=.002). The OR of the EO CD group on experiences of biologic agents compared with the LO CD group was 5.4 (95% CI 1.9-14.9, P=.001).

CONCLUSIONS

The EO IBD group showed a worse prognosis than the LO IBD group in Korean patients with IBD. In addition, this study successfully verified the CDM model in gastrointestinal research.

Impact of risk minimisation measures on the use of strontium ranelate in Europe: a multi-national cohort study in 5 EU countries by the EU-ADR Alliance

INTRODUCTION

In May 2013 and March 2014, the European Medicines Agency (EMA) issued two decisions restricting the use of strontium ranelate (SR). These risk minimisation measures (RMM) introduced new contraindications and limited the indications of SR therapy. The EMA required an assessment of the impact of RMMs on the use of SR in Europe. Methods design: multi-national, multi-database cohort Setting: electronic medical record databases based on hospital (Denmark) and primary care provenance (Italy, Spain, the Netherlands, UK).

PARTICIPANTS

the database source populations were included for population-based analyses, and SR users for patient-level analyses.

INTERVENTION

New RMMs included contraindications (ischaemic heart disease, peripheral arterial disease, cerebrovascular disease, uncontrolled hypertension) and restricted SR indication to severe osteoporosis with initiation by experienced physician and not as first line anti-osteoporosis therapy.

METHODS

Prevalence and incidence rates of SR use in the population; prevalence of contraindications and restricted indications in SR users, plus 1-year therapy persistence. Drug use measures were calculated in three periods for comparison: reference (2004 to May 2013), transition (June 2013 to March 2014) and assessment (from April 2014 to end 2016).

RESULTS

The study population included 143 million person-years(PY) of follow-up and 76,141 incident episodes of SR treatment. Average monthly prevalence rates of SR use dropped by 86.4% from 62.6/10,000 PY (95 CI 62.4-62.9) in the reference to 8.5 (8.5-8.6) in the assessment period. Similarly, the incidence rate of SR use fell by 97.3% from 7.4/10,000 PY (7.4-7.4) to 0.2 (0.2-0.2) between the reference and assessment period. The prevalence of any contraindication decreased, whilst the prevalence of restricted indications increased in these periods. One-year persistence decreased in the assessment compared with reference period.

CONCLUSIONS

Our study demonstrates a substantial impact of the regulatory action to restrict use of SR in Europe: SR utilisation overall decreased strongly. The proportion of patients fulfilling the restricted indications, without contraindications, increased after the proposed RMMs.

Analysis of antiseizure drug-related adverse reactions from the electronic health record using the common data model

OBJECTIVE

Antiseizure drugs (ASDs) are known to cause a wide range of adverse drug reactions (ADRs). Recently, electronic health care data using the common data model (CDM) have been introduced and commonly adopted in pharmacovigilance research. We aimed to analyze ASD-related ADRs using CDM and to assess the feasibility of CDM analysis in monitoring ADR in a single tertiary hospital.

METHODS

We selected five ASDs: oxcarbazepine (OXC), lamotrigine (LTG), levetiracetam (LEV), valproic acid (VPA), and topiramate (TPM). Patients diagnosed with epilepsy and exposed to monotherapy with one of the ASDs before age 18 years were included. We measured four ADR outcomes: (1) hematologic abnormality, (2) hyponatremia, (3) elevation of liver enzymes, and (4) subclinical hypothyroidism. We performed a subgroup analysis to exclude the effects of concomitant medications.

RESULTS

From the database, 1344 patients were included for the study. Of the 1344 patients, 436 were receiving OXC, 293 were receiving LTG, 275 were receiving LEV, 180 were receiving VPA, and 160 were receiving TPM. Thrombocytopenia developed in 14.1% of patients taking VPA. Hyponatremia occurred in 10.5% of patients taking OXC. Variable ranges of liver enzyme elevation were detected in 19.3% of patients taking VPA. Subclinical hypothyroidism occurred in approximately 21.5% to 28% of patients with ASD monotherapy, which did not significantly differ according to the type of ASD. In a subgroup analysis, we observed similar ADR tendencies, but with less thrombocytopenia in the TPM group.

SIGNIFICANCE

The incidence and trends of ADRs that were evaluated by CDM were similar to the previous literature. CDM can be a useful tool for analyzing ASD-related ADRs in a multicenter study. The strengths and limitations of CDM should be carefully addressed.

Can We Rely on Results From IQVIA Medical Research Data UK Converted to the Observational Medical Outcome Partnership Common Data Model?: A Validation Study Based on Prescribing Codeine in Children

Exploring and combining results from more than one real‐world data (RWD) source might be necessary in order to explore variability and demonstrate generalizability of the results or for regulatory requirements. However, the heterogeneous nature of RWD poses challenges when working with more than one source, some of which can be solved by analyzing databases converted into a common data model (CDM). The main objective of the study was to evaluate the implementation of the Observational Medical Outcome Partnership (OMOP) CDM on IQVIA Medical Research Data (IMRD)‐UK data. A drug utilization study describing the prescribing of codeine for pain in children was used as a case study to be replicated in IMRD‐UK and its corresponding OMOP CDM transformation. Differences between IMRD‐UK source and OMOP CDM were identified and investigated. In IMRD‐UK updated to May 2017, results were similar between source and transformed data with few discrepancies. These were the result of different conventions applied during the transformation regarding the date of birth for children younger than 15 years and the start of the observation period, and of a misclassification of two drug treatments. After the initial analysis and feedback provided, a rerun of the analysis in IMRD‐UK updated to September 2018 showed almost identical results for all the measures analyzed. For this study, the conversion to OMOP CDM was adequate. Although some decisions and mapping could be improved, these impacted on the absolute results but not on the study inferences. This validation study supports six recommendations for good practice in transforming to CDMs.

Using clinical registries, administrative data and electronic medical records to improve medication safety and effectiveness in dementia

PURPOSE OF REVIEW

Clinical registries, routinely collected administrative data and electronic medical records (EMRs) provide new opportunities to investigate medication safety and effectiveness. This review outlines the strengths and limitations of these data, and highlights recent research related to safe and effective medication use in dementia.

RECENT FINDINGS

Clinical registries, administrative data and EMRs facilitate observational research among people often excluded from randomized controlled trials (RCTs). Larger sample sizes and longer follow-up times permit research into less common adverse events not apparent in RCTs. The validity of diagnoses recorded in administrative data and EMRs remains variable, although positive predictive values are typically high and sensitivity is low. Dispensing records are a rich source of data for estimating medication exposure. Recent research has investigated medications and prescribing patterns as risk factors for incident dementia, strategies to alleviate behavioural symptoms and the management of comorbidity. Common study protocols and common data models are examples of distributed network approaches increasingly used to conduct large and generalizable multi-database studies across different countries.

SUMMARY

Greater availability of electronic health data provides important opportunities to address evidence-practice gaps in relation to medication use and safety in people with dementia.

Transforming French Electronic Health Records into the Observational Medical Outcome Partnership's Common Data Model: A Feasibility Study

BACKGROUND

Common data models (CDMs) enable data to be standardized, and facilitate data exchange, sharing, and storage, particularly when the data have been collected via distinct, heterogeneous systems. Moreover, CDMs provide tools for data quality assessment, integration into models, visualization, and analysis. The observational medical outcome partnership (OMOP) provides a CDM for organizing and standardizing databases. Common data models not only facilitate data integration but also (and especially for the OMOP model) extends the range of available statistical analyses.

OBJECTIVE

This study aimed to evaluate the feasibility of implementing French national electronic health records in the OMOP CDM.

METHODS

The OMOP's specifications were used to audit the source data, specify the transformation into the OMOP CDM, implement an extract-transform-load process to feed data from the French health care system into the OMOP CDM, and evaluate the final database.

RESULTS

Seventeen vocabularies corresponding to the French context were added to the OMOP CDM's concepts. Three French terminologies were automatically mapped to standardized vocabularies. We loaded nine tables from the OMOP CDM's "standardized clinical data" section, and three tables from the "standardized health system data" section. Outpatient and inpatient data from 38,730 individuals were integrated. The median (interquartile range) number of outpatient and inpatient stays per patient was 160 (19-364).

CONCLUSION

Our results demonstrated that data from the French national health care system can be integrated into the OMOP CDM. One of the main challenges was the use of international OMOP concepts to annotate data recorded in a French context. The use of local terminologies was an obstacle to conceptual mapping; with the exception of an adaptation of the International Classification of Diseases 10th Revision, the French health care system does not use international terminologies. It would be interesting to extend our present findings to the 65 million people registered in the French health care system.

Choosing Among Common Data Models for Real-World Data Analyses Fit for Making Decisions About the Effectiveness of Medical Products

Many real-world data analyses use common data models (CDMs) to standardize terminologies for medication use, medical events and procedures, data structures, and interpretations of data to facilitate analyses across data sources. For decision makers, key aspects that influence the choice of a CDM may include (i) adaptability to a specific question; (ii) transparency to reproduce findings, assess validity, and instill confidence in findings; and (iii) ease and speed of use. Organizing CDMs preserve the original information from a data source and have maximum adaptability. Full mapping data models, or preconfigured rules systems, are easy to use, since all raw codes are mapped to medical constructs. Adaptive rule systems grow libraries of reusable measures that can easily adjust to preserve adaptability, expedite analyses, and ensure study-specific transparency.

The Generalized Data Model for clinical research

BACKGROUND

Most healthcare data sources store information within their own unique schemas, making reliable and reproducible research challenging. Consequently, researchers have adopted various data models to improve the efficiency of research. Transforming and loading data into these models is a labor-intensive process that can alter the semantics of the original data. Therefore, we created a data model with a hierarchical structure that simplifies the transformation process and minimizes data alteration.

METHODS

There were two design goals in constructing the tables and table relationships for the Generalized Data Model (GDM). The first was to focus on clinical codes in their original vocabularies to retain the original semantic representation of the data. The second was to retain hierarchical information present in the original data while retaining provenance. The model was tested by transforming synthetic Medicare data; Surveillance, Epidemiology, and End Results data linked to Medicare claims; and electronic health records from the Clinical Practice Research Datalink. We also tested a subsequent transformation from the GDM into the Sentinel data model.

RESULTS

The resulting data model contains 19 tables, with the Clinical Codes, Contexts, and Collections tables serving as the core of the model, and containing most of the clinical, provenance, and hierarchical information. In addition, a Mapping table allows users to apply an arbitrarily complex set of relationships among vocabulary elements to facilitate automated analyses.

CONCLUSIONS

The GDM offers researchers a simpler process for transforming data, clear data provenance, and a path for users to transform their data into other data models. The GDM is designed to retain hierarchical relationships among data elements as well as the original semantic representation of the data, ensuring consistency in protocol implementation as part of a complete data pipeline for researchers.

Creation of a Multicenter Pediatric Inpatient Data Repository Derived from Electronic Health Records

BACKGROUND

Integration of electronic health records (EHRs) data across sites and access to that data remain limited.

OBJECTIVE

We developed an EHR-based pediatric inpatient repository using nine U.S. centers from the National Institute of Child Health and Human Development Pediatric Trials Network.

METHODS

A data model encompassing 147 mandatory and 99 optional elements was developed to provide an EHR data extract of all inpatient encounters from patients <17 years of age discharged between January 6, 2013 and June 30, 2017. Sites received instructions on extractions, transformation, testing, and transmission to the coordinating center.

RESULTS

We generated 177 staging reports to process all nine sites' 147 mandatory and 99 optional data elements to the repository. Based on 520 prespecified criteria, all sites achieved 0% errors and <2% warnings. The repository includes 386,159 inpatient encounters from 264,709 children to support study design and conduct of future trials in children.

CONCLUSION

Our EHR-based data repository of pediatric inpatient encounters utilized a customized data model heavily influenced by the PCORnet format, site-based data mapping, a comprehensive set of data testing rules, and an iterative process of data submission. The common data model, site-based extraction, and technical expertise were key to our success. Data from this repository will be used in support of Pediatric Trials Network studies and the labeling of drugs and devices for children.

ADEpedia-on-OHDSI: A next generation pharmacovigilance signal detection platform using the OHDSI common data model

OBJECTIVE

Supplementing the Spontaneous Reporting System (SRS) with Electronic Health Record (EHR) data for adverse drug reaction detection could augment sample size, increase population heterogeneity and cross-validate results for pharmacovigilance research. The difference in the underlying data structures and terminologies between SRS and EHR data presents challenges when attempting to integrate the two into a single database. The Observational Health Data Sciences and Informatics (OHDSI) collaboration provides a Common Data Model (CDM) for organizing and standardizing EHR data to support large-scale observational studies. The objective of the study is to develop and evaluate an informatics platform known as ADEpedia-on-OHDSI, where spontaneous reporting data from FDA's Adverse Event Reporting System (FAERS) is converted into the OHDSI CDM format towards building a next generation pharmacovigilance signal detection platform.

METHODS

An extraction, transformation and loading (ETL) tool was designed, developed, and implemented to convert FAERS data into the OHDSI CDM format. A comprehensive evaluation, including overall ETL evaluation, mapping quality evaluation of drug names to RxNorm, and an evaluation of transformation and imputation quality, was then performed to assess the mapping accuracy and information loss using the FAERS data collected between 2012 and 2017. Previously published findings related to vascular safety profile of triptans were validated using ADEpedia-on-OHDSI in pharmacovigilance research. For the triptan-related vascular event detection, signals were detected by Reporting Odds Ratio (ROR) in high-level group terms (HLGT) level, high-level terms (HLT) level and preferred term (PT) level using the original FAERS data and CDM-based FAERS respectively. In addition, six standardized MedDRA queries (SMQs) related to vascular events were applied.

RESULTS

A total of 4,619,362 adverse event cases were loaded into 8 tables in the OHDSI CDM. For drug name mapping, 93.9% records and 47.0% unique names were matched with RxNorm codes. Mapping accuracy of drug names was 96% based on a manual verification of randomly sampled 500 unique mappings. Information loss evaluation showed that more than 93% of the data is loaded into the OHDSI CDM for most fields, with the exception of drug route data (66%). The replication study detected 5, 18, 47 and 6, 18, 50 triptan-related vascular event signals in MedDRA HLGT level, HLT level, and PT level for the original FAERS data and CDM-based FAERS respectively. The signal detection scores of six standardized MedDRA queries (SMQs) of vascular events in the raw data study were found to be lower than those scores in the CDM study.

CONCLUSION

The outcome of this work would facilitate seamless integration and combined analyses of both SRS and EHR data for pharmacovigilance in ADEpedia-on-OHDSI, our platform for next generation pharmacovigilance.

A 16-Year Cohort Analysis of Autism Spectrum Disorder-Associated Morbidity in a Pediatric Population

Introduction: This chapter presents the analysis of physician-diagnosed International Classification of Diseases (ICD version 9) disorders and diseases associated with autism spectrum disorders (ASD) in a 16-year pediatric cohort. Materials and Methods: The sample (n = 47,180; 62% male) consisted of children in the Alberta Health Services Calgary Health Region catchment under the age of 3 years, who received any physician-assigned ICD 9 diagnosis before the age of three between April 1993 and December 31, 1994. There were 111 females and 609 males with ASD diagnosed at any time between 1993 and 2010. The results detail the 16-year odds ratio (OR) associations of ASD diagnosis within the major classes of international classification of diseases (ICD 9) stratified by age and sex in the cohort. Further, for those suffering from ASD and any other disorder or disease, the analysis presents by sex, age, and duration, the proportions of all index physician-assigned ICD diagnoses, arising significantly before and after the index ASD diagnosis. Results: The rate of treated ASD in the cohort was 1 in 65 and the 16-year population rate of ASD was 62 per 10,000. For males with an ASD over the 16 year period, the ORs were significantly greater than the value one for 15 of the 17 main ICD classes and for 10 of the main ICD classes for females. Different age strata presented a more specific account of the main ICD class OR profiles. More specifically, 28 ICD disorders significantly preceded and 95 ICD disorders significantly followed ASD for females. Thirty-eight ICD disorders significantly preceded and 234 ICD disorders significantly followed ASD for males. Conclusions: The results largely confirm past studies focusing on more constrained sets of ASD morbidity. The age-stratified ORs gauge the order of risk in time for the cohort. The proportions of specific ICD disorders arising before and after ASD may be useful in respect to informing basic ASD research and ASD clinical management. Limitations are discussed.

Lessons from meta-analyses of randomized clinical trials for analysis of distributed networks of observational databases

Networks of constellations of longitudinal observational databases, often electronic medical records or transactional insurance claims or both, are increasingly being used for studying the effects of medicinal products in real-world use. Such databases are frequently configured as distributed networks. That is, patient-level data are kept behind firewalls and not communicated outside of the data vendor other than in aggregate form. Instead, data are standardized across the network, and queries of the network are executed locally by data partners, and summary results provided to a central research partner(s) for amalgamation, aggregation, and summarization. Such networks can be huge covering years of data on upwards of 100 million patients. Examples of such networks include the FDA Sentinel Network, ASPEN, CNODES, and EU-ADR. As this is a new emerging field, we note in this paper the conceptual similarities and differences between the analysis of distributed networks and the now well-established field of meta-analysis of randomized clinical trials (RCTs). We recommend, wherever appropriate, to apply learnings from meta-analysis to help guide the development of distributed network analyses of longitudinal observational databases.

Evaluation of Electronic Healthcare Databases for Post-Marketing Drug Safety Surveillance and Pharmacoepidemiology in China

INTRODUCTION

Electronic healthcare databases (EHDs) are used increasingly for post-marketing drug safety surveillance and pharmacoepidemiology in Europe and North America. However, few studies have examined the potential of these data sources in China.

METHODS

Three major types of EHDs in China (i.e., a regional community-based database, a national claims database, and an electronic medical records [EMR] database) were selected for evaluation. Forty core variables were derived based on the US Mini-Sentinel (MS) Common Data Model (CDM) as well as the data features in China that would be desirable to support drug safety surveillance. An email survey of these core variables and eight general questions as well as follow-up inquiries on additional variables was conducted. These 40 core variables across the three EHDs and all variables in each EHD along with those in the US MS CDM and Observational Medical Outcomes Partnership (OMOP) CDM were compared for availability and labeled based on specific standards.

RESULTS

All of the EHDs' custodians confirmed their willingness to share their databases with academic institutions after appropriate approval was obtained. The regional community-based database contained 1.19 million people in 2015 with 85% of core variables. Resampled annually nationwide, the national claims database included 5.4 million people in 2014 with 55% of core variables, and the EMR database included 3 million inpatients from 60 hospitals in 2015 with 80% of core variables. Compared with MS CDM or OMOP CDM, the proportion of variables across the three EHDs available or able to be transformed/derived from the original sources are 24-83% or 45-73%, respectively.

CONCLUSIONS

These EHDs provide potential value to post-marketing drug safety surveillance and pharmacoepidemiology in China. Future research is warranted to assess the quality and completeness of these EHDs or additional data sources in China.

Signal Detection for Recently Approved Products: Adapting and Evaluating Self-Controlled Case Series Method Using a US Claims and UK Electronic Medical Records Database

INTRODUCTION

The Self-Controlled Case Series (SCCS) method has been widely used for hypothesis testing, but there is limited evidence of its performance for safety signal detection.

OBJECTIVE

The objective of this study was to evaluate SCCS for signal detection on recently approved products.

METHODS

A retrospective study covered the period after three recently marketed drugs were launched through to 31 December 2010 using The Health Improvement Network, a UK primary care database, and Optum, a US claims database. The SCCS method was applied to examine five heterogenous outcomes with desvenlafaxine and escitalopram and six outcomes with adalimumab for Signals of Disproportional Recording (SDRs); a positive finding was determined to be when the lower bound of 95% Confidence Interval of the incidence rate ratio (IRR) estimate was >  1. Multiple design choices were tested and the trend in IRR estimates over calendar time for one drug event pair was examined.

RESULTS

All six outcomes with adalimumab, three of five outcomes with desvenlafaxine, and four of five outcomes with escitalopram had SDRs. SCCS highlighted all acute events in the primary analysis but was less successful with slower-onset outcomes. Performance varied by risk period definition. Changes in IRR estimates over quarterly intervals for adalimumab with herpes zoster showed marked higher SDR within 9 months of drug launch.

CONCLUSION

SCCS shows promise for signal detection: it may highlight known associations for recent marketed products and has potential for early signal identification. SCCS performance varied by design choice and the nature of both exposure and event pair. Future work is needed to determine how effective the approach is in prospective testing and determining the performance characteristics of the approach.

Applying a common data model to Asian databases for multinational pharmacoepidemiologic studies: opportunities and challenges

Objective

The goal of the Asian Pharmacoepidemiology Network is to study the effectiveness and safety of medications commonly used in Asia using databases from individual Asian countries. An efficient infrastructure to support multinational pharmacoepidemiologic studies is critical to this effort.

Study design and setting

We converted data from the Japan Medical Data Center database, Taiwan’s National Health Insurance Research Database, Hong Kong’s Clinical Data Analysis and Reporting System, South Korea’s Ajou University School of Medicine database, and the US Medicare 5% sample to the Observational Medical Outcome Partnership common data model (CDM).

Results

We completed and documented the process for the CDM conversion. The coordinating center and participating sites reviewed the documents and refined the conversions based on the comments. The time required to convert data to the CDM varied widely across sites and included conversion to standard terminology codes and refinements of the conversion based on reviews. We mapped 97.2%, 86.7%, 92.6%, and 80.1% of domestic drug codes from the USA, Taiwan, Hong Kong, and Korea to RxNorm, respectively. The mapping rate from Japanese domestic drug codes to RxNorm (70.7%) was lower than from other countries, and we mapped remaining unmapped drugs to Anatomical Therapeutic Chemical Classification System codes. Because the native databases used international procedure coding systems for which mapping tables have been established, we were able to map >90% of diagnosis and procedure codes to standard terminology codes.

Conclusion

The CDM established the foundation and reinforced collaboration for multinational pharmacoepidemiologic studies in Asia. Mapping of terminology codes was the greatest challenge, because of differences in health systems, cultures, and coding systems.

Reporting to Improve Reproducibility and Facilitate Validity Assessment for Healthcare Database Studies V1.0

Purpose

Defining a study population and creating an analytic dataset from longitudinal healthcare databases involves many decisions. Our objective was to catalogue scientific decisions underpinning study execution that should be reported to facilitate replication and enable assessment of validity of studies conducted in large healthcare databases.

Methods

We reviewed key investigator decisions required to operate a sample of macros and software tools designed to create and analyze analytic cohorts from longitudinal streams of healthcare data. A panel of academic, regulatory, and industry experts in healthcare database analytics discussed and added to this list.

Conclusion

Evidence generated from large healthcare encounter and reimbursement databases is increasingly being sought by decision‐makers. Varied terminology is used around the world for the same concepts. Agreeing on terminology and which parameters from a large catalogue are the most essential to report for replicable research would improve transparency and facilitate assessment of validity. At a minimum, reporting for a database study should provide clarity regarding operational definitions for key temporal anchors and their relation to each other when creating the analytic dataset, accompanied by an attrition table and a design diagram.

A substantial improvement in reproducibility, rigor and confidence in real world evidence generated from healthcare databases could be achieved with greater transparency about operational study parameters used to create analytic datasets from longitudinal healthcare databases.

A Semantic Transformation Methodology for the Secondary Use of Observational Healthcare Data in Postmarketing Safety Studies

Background: Utilization of the available observational healthcare datasets is key to complement and strengthen the postmarketing safety studies. Use of common data models (CDM) is the predominant approach in order to enable large scale systematic analyses on disparate data models and vocabularies. Current CDM transformation practices depend on proprietarily developed Extract-Transform-Load (ETL) procedures, which require knowledge both on the semantics and technical characteristics of the source datasets and target CDM. Purpose: In this study, our aim is to develop a modular but coordinated transformation approach in order to separate semantic and technical steps of transformation processes, which do not have a strict separation in traditional ETL approaches. Such an approach would discretize the operations to extract data from source electronic health record systems, alignment of the source, and target models on the semantic level and the operations to populate target common data repositories. Approach: In order to separate the activities that are required to transform heterogeneous data sources to a target CDM, we introduce a semantic transformation approach composed of three steps: (1) transformation of source datasets to Resource Description Framework (RDF) format, (2) application of semantic conversion rules to get the data as instances of ontological model of the target CDM, and (3) population of repositories, which comply with the specifications of the CDM, by processing the RDF instances from step 2. The proposed approach has been implemented on real healthcare settings where Observational Medical Outcomes Partnership (OMOP) CDM has been chosen as the common data model and a comprehensive comparative analysis between the native and transformed data has been conducted. Results: Health records of ~1 million patients have been successfully transformed to an OMOP CDM based database from the source database. Descriptive statistics obtained from the source and target databases present analogous and consistent results. Discussion and Conclusion: Our method goes beyond the traditional ETL approaches by being more declarative and rigorous. Declarative because the use of RDF based mapping rules makes each mapping more transparent and understandable to humans while retaining logic-based computability. Rigorous because the mappings would be based on computer readable semantics which are amenable to validation through logic-based inference methods.

Assessing performance of sequential analysis methods for active drug safety surveillance using observational data

The routine use of sequential methods is well established in clinical studies. Recently, there has been increasing interest in applying these methods to prospectively monitor the safety of newly approved drugs through accrual of real-world data. However, the application to marketed drugs using real-world data has been limited and work is needed to determine which sequential approaches are most suited to such data. In this study, the conditional sequential sampling procedure (CSSP), a group sequential method, was compared with a log-linear model with Poisson distribution (LLMP) through a SAS procedure (PROC GENMOD) combined with an alpha-spending function on two large longitudinal US administrative health claims databases. Relative performance in identifying known drug-outcome associations was examined using a set of 50 well-studied drug-outcome pairs. The study finds that neither method correctly identified all pairs but that LLMP often provides better ability and shorter time for identifying the known drug-outcome associations with superior computational performance when compared with CSSP, albeit with more false positives. With the features of flexible confounding control and ease of implementation, LLMP may be a good alternative or complement to CSSP.

Reporting to Improve Reproducibility and Facilitate Validity Assessment for Healthcare Database Studies V1.0

PURPOSE

Defining a study population and creating an analytic dataset from longitudinal healthcare databases involves many decisions. Our objective was to catalogue scientific decisions underpinning study execution that should be reported to facilitate replication and enable assessment of validity of studies conducted in large healthcare databases.

METHODS

We reviewed key investigator decisions required to operate a sample of macros and software tools designed to create and analyze analytic cohorts from longitudinal streams of healthcare data. A panel of academic, regulatory, and industry experts in healthcare database analytics discussed and added to this list.

CONCLUSION

Evidence generated from large healthcare encounter and reimbursement databases is increasingly being sought by decision-makers. Varied terminology is used around the world for the same concepts. Agreeing on terminology and which parameters from a large catalogue are the most essential to report for replicable research would improve transparency and facilitate assessment of validity. At a minimum, reporting for a database study should provide clarity regarding operational definitions for key temporal anchors and their relation to each other when creating the analytic dataset, accompanied by an attrition table and a design diagram. A substantial improvement in reproducibility, rigor and confidence in real world evidence generated from healthcare databases could be achieved with greater transparency about operational study parameters used to create analytic datasets from longitudinal healthcare databases.

Multisite Evaluation of a Data Quality Tool for Patient-Level Clinical Data Sets

Introduction:

Data quality and fitness for analysis are crucial if outputs of analyses of electronic health record data or administrative claims data should be trusted by the public and the research community.

Methods:

We describe a data quality analysis tool (called Achilles Heel) developed by the Observational Health Data Sciences and Informatics Collaborative (OHDSI) and compare outputs from this tool as it was applied to 24 large healthcare datasets across seven different organizations.

Results:

We highlight 12 data quality rules that identified issues in at least 10 of the 24 datasets and provide a full set of 71 rules identified in at least one dataset. Achilles Heel is a freely available software that provides a useful starter set of data quality rules with the ability to add additional rules. We also present results of a structured email-based interview of all participating sites that collected qualitative comments about the value of Achilles Heel for data quality evaluation.

Discussion:

Our analysis represents the first comparison of outputs from a data quality tool that implements a fixed (but extensible) set of data quality rules. Thanks to a common data model, we were able to compare quickly multiple datasets originating from several countries in America, Europe and Asia.

Fidelity assessment of a clinical practice research datalink conversion to the OMOP common data model

Background

The unique structure and coding of the Clinical Practice Research Datalink (CPRD) presents challenges for epidemiologic analysis and for comparisons with other databases. To address this limitation we sought to transform CPRD into the Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM).

Methods

An extraction, transformation and loading process was developed, which detailed source code mappings, Read code domain classification, an imputation algorithm for drug duration and special handling of lifestyle/clinical data. Completeness and accuracy of the above elements were assessed. A final validation exercise involved replication of a published case–control study that examined use of nonsteroidal anti-inflammatory drugs (NSAIDs) and the risk of first-time acute myocardial infarction (AMI) in raw CPRD data and the CPRD CDM.

Findings

All elements of the CPRD CDM transformation were assessed to be of high quality. 99.9 % of database condition records and 89.7 % of database drug records were mapped (majority unmapped drugs were devices and over-the-counter products); 3.1 % of duration imputations were deemed possibly erroneous and prevalences for selected conditions and drugs across CPRD raw and CDM data were equivalent. Results between the replication raw data and CDM study agreed for conditions, demographics and lifestyle data with slight NSAID exposure data loss owing to unmapped drugs.

Conclusion

CPRD can be accurately transformed into the OMOP CDM with acceptable information loss across drugs, conditions and observations. We determined that for a particular use, case CDM structure was adequate and mappings could be improved but did not substantially change the results of our analysis.

Electronic supplementary material

The online version of this article (doi:10.1007/s40264-014-0214-3) contains supplementary material, which is available to authorized users.

Creating a Common Data Model for Comparative Effectiveness with the Observational Medical Outcomes Partnership

BACKGROUND

Adoption of a common data model across health systems is a key infrastructure requirement to allow large scale distributed comparative effectiveness analyses. There are a growing number of common data models (CDM), such as Mini-Sentinel, and the Observational Medical Outcomes Partnership (OMOP) CDMs.

OBJECTIVES

In this case study, we describe the challenges and opportunities of a study specific use of the OMOP CDM by two health systems and describe three comparative effectiveness use cases developed from the CDM.

METHODS

The project transformed two health system databases (using crosswalks provided) into the OMOP CDM. Cohorts were developed from the transformed CDMs for three comparative effectiveness use case examples. Administrative/billing, demographic, order history, medication, and laboratory were included in the CDM transformation and cohort development rules.

RESULTS

Record counts per person month are presented for the eligible cohorts, highlighting differences between the civilian and federal datasets, e.g. the federal data set had more outpatient visits per person month (6.44 vs. 2.05 per person month). The count of medications per person month reflected the fact that one system's medications were extracted from orders while the other system had pharmacy fills and medication administration records. The federal system also had a higher prevalence of the conditions in all three use cases. Both systems required manual coding of some types of data to convert to the CDM.

CONCLUSIONS

The data transformation to the CDM was time consuming and resources required were substantial, beyond requirements for collecting native source data. The need to manually code subsets of data limited the conversion. However, once the native data was converted to the CDM, both systems were then able to use the same queries to identify cohorts. Thus, the CDM minimized the effort to develop cohorts and analyze the results across the sites.

A comparison of active adverse event surveillance systems worldwide

Post-marketing drug surveillance for adverse drug events (ADEs) has typically relied on spontaneous reporting. Recently, regulatory agencies have turned their attention to more preemptive approaches that use existing data for surveillance. We conducted an environmental scan to identify active surveillance systems worldwide that use existing data for the detection of ADEs. We extracted data about the systems' structures, data, and functions. We synthesized the information across systems to identify common features of these systems. We identified nine active surveillance systems. Two systems are US based-the FDA Sentinel Initiative (including both the Mini-Sentinel Initiative and the Federal Partner Collaboration) and the Vaccine Safety Datalink (VSD); two are Canadian-the Canadian Network for Observational Drug Effect Studies (CNODES) and the Vaccine and Immunization Surveillance in Ontario (VISION); and two are European-the Exploring and Understanding Adverse Drug Reactions by Integrative Mining of Clinical Records and Biomedical Knowledge (EU-ADR) Alliance and the Vaccine Adverse Event Surveillance and Communication (VAESCO). Additionally, there is the Asian Pharmacoepidemiology Network (AsPEN) and the Shanghai Drug Monitoring and Evaluative System (SDMES). We identified two systems in the UK-the Vigilance and Risk Management of Medicines (VRMM) Division and the Drug Safety Research Unit (DSRU), an independent academic unit. These surveillance systems mostly use administrative claims or electronic medical records; most conduct pharmacovigilance on behalf of a regulatory agency. Either a common data model or a centralized model is used to access existing data. The systems have been built using national data alone or via partnership with other countries. However, active surveillance systems using existing data remain rare. North America and Europe have the most population coverage; with Asian countries making good advances.