Bridging islands of information to establish an integrated knowledge base of drugs and health outcomes of interest

Exploring the impact of design criteria for reference sets on performance evaluation of signal detection algorithms: The case of drug-drug interactions

PURPOSE

To evaluate the impact of multiple design criteria for reference sets that are used to quantitatively assess the performance of pharmacovigilance signal detection algorithms (SDAs) for drug-drug interactions (DDIs).

METHODS

Starting from a large and diversified reference set for two-way DDIs, we generated custom-made reference sets of various sizes considering multiple design criteria (e.g., adverse event background prevalence). We assessed differences observed in the performance metrics of three SDAs when applied to FDA Adverse Event Reporting System (FAERS) data.

RESULTS

For some design criteria, the impact on the performance metrics was neglectable for the different SDAs (e.g., theoretical evidence associated with positive controls), while others (e.g., restriction to designated medical events, event background prevalence) seemed to have opposing and effects of different sizes on the Area Under the Curve (AUC) and positive predictive value (PPV) estimates.

CONCLUSIONS

The relative composition of reference sets can significantly impact the evaluation metrics, potentially altering the conclusions regarding which methodologies are perceived to perform best. We therefore need to carefully consider the selection of controls to avoid misinterpretation of signals triggered by confounding factors rather than true associations as well as adding biases to our evaluation by "favoring" some algorithms while penalizing others.

Development and Application of an Active Pharmacovigilance Framework Based on Electronic Healthcare Records from Multiple Centers in Korea

INTRODUCTION

With the availability of retrospective pharmacovigilance data, the common data model (CDM) has been identified as an efficient approach towards anonymized multicenter analysis; however, the establishment of a suitable model for individual medical systems and applications supporting their analysis is a challenge.

OBJECTIVE

The aim of this study was to construct a specialized Korean CDM (K-CDM) for pharmacovigilance systems based on a clinical scenario to detect adverse drug reactions (ADRs).

METHODS

De-identified patient records (n = 5,402,129) from 13 institutions were converted to the K-CDM. From 2005 to 2017, 37,698,535 visits, 39,910,849 conditions, 259,594,727 drug exposures, and 30,176,929 procedures were recorded. The K-CDM, which comprises three layers, is compatible with existing models and is potentially adaptable to extended clinical research. Local codes for electronic medical records (EMRs), including diagnosis, drug prescriptions, and procedures, were mapped using standard vocabulary. Distributed queries based on clinical scenarios were developed and applied to K-CDM through decentralized or distributed networks.

RESULTS

Meta-analysis of drug relative risk ratios from ten institutions revealed that non-steroidal anti-inflammatory drugs (NSAIDs) increased the risk of gastrointestinal hemorrhage by twofold compared with aspirin, and non-vitamin K anticoagulants decreased cerebrovascular bleeding risk by 0.18-fold compared with warfarin.

CONCLUSION

These results are similar to those from previous studies and are conducive for new research, thereby demonstrating the feasibility of K-CDM for pharmacovigilance. However, the low quality of original EMR data, incomplete mapping, and heterogeneity between institutions reduced the validity of the analysis, thus necessitating continuous calibration among researchers, clinicians, and the government.

Hip Fracture Risk After Treatment with Tramadol or Codeine: An Observational Study

INTRODUCTION

Hip fractures among older people are a major public health issue, which can impact quality of life and increase mortality within the year after they occur. A recent observational study found an increased risk of hip fracture in subjects who were new users of tramadol compared with codeine. These drugs have somewhat different indications. Tramadol is indicated for moderate to severe pain and can be used for an extended period; codeine is indicated for mild to moderate pain and cough suppression.

OBJECTIVE

In this observational study, we compared the risk of hip fracture in new users of tramadol or codeine, using multiple databases and analytical methods.

METHODS

Using data from the Clinical Practice Research Datalink and three US claims databases, we compared the risk of hip fracture after exposure to tramadol or codeine in subjects aged 50-89 years. To ensure comparability, large-scale propensity scores were used to adjust for confounding.

RESULTS

We observed a calibrated hazard ratio of 1.10 (95% calibrated confidence interval 0.99-1.21) in the Clinical Practice Research Datalink database, and a pooled estimate across the US databases yielded a calibrated hazard ratio of 1.06 (95% calibrated confidence interval 0.97-1.16).

CONCLUSIONS

Our results did not demonstrate a statistically significant difference between subjects treated for pain with tramadol compared with codeine for the outcome of hip fracture risk.

Development of a Data-Driven Reference Standard for Adverse Drug Reaction (RS-ADR) Signal Assessment (Preprint)

Background

Pharmacovigilance using real-world data (RWD), such as multicenter electronic health records (EHRs), yields massively parallel adverse drug reaction (ADR) signals. However, proper validation of computationally detected ADR signals is not possible due to the lack of a reference standard for positive and negative associations.

Objective

This study aimed to develop a reference standard for ADR (RS-ADR) to streamline the systematic detection, assessment, and understanding of almost all drug-ADR associations suggested by RWD analyses.

Methods

We integrated well-known reference sets for drug-ADR pairs, including Side Effect Resource, Observational Medical Outcomes Partnership, and EU-ADR. We created a pharmacovigilance dictionary using controlled vocabularies and systematically annotated EHR data. Drug-ADR associations computed from MetaLAB and MetaNurse analyses of multicenter EHRs and extracted from the Food and Drug Administration Adverse Event Reporting System were integrated as “empirically determined” positive and negative reference sets by means of cross-validation between institutions.

Results

The RS-ADR consisted of 1344 drugs, 4485 ADRs, and 6,027,840 drug-ADR pairs with positive and negative consensus votes as pharmacovigilance reference sets. After the curation of the initial version of RS-ADR, novel ADR signals such as “famotidine–hepatic function abnormal” were detected and reasonably validated by RS-ADR. Although the validation of the entire reference standard is challenging, especially with this initial version, the reference standard will improve as more RWD participate in the consensus voting with advanced pharmacovigilance dictionaries and analytic algorithms. One can check if a drug-ADR pair has been reported by our web-based search interface for RS-ADRs.

Conclusions

RS-ADRs enriched with the pharmacovigilance dictionary, ADR knowledge, and real-world evidence from EHRs may streamline the systematic detection, evaluation, and causality assessment of computationally detected ADR signals.

The role of machine learning in clinical research: transforming the future of evidence generation

Background

Interest in the application of machine learning (ML) to the design, conduct, and analysis of clinical trials has grown, but the evidence base for such applications has not been surveyed. This manuscript reviews the proceedings of a multi-stakeholder conference to discuss the current and future state of ML for clinical research. Key areas of clinical trial methodology in which ML holds particular promise and priority areas for further investigation are presented alongside a narrative review of evidence supporting the use of ML across the clinical trial spectrum.

Results

Conference attendees included stakeholders, such as biomedical and ML researchers, representatives from the US Food and Drug Administration (FDA), artificial intelligence technology and data analytics companies, non-profit organizations, patient advocacy groups, and pharmaceutical companies. ML contributions to clinical research were highlighted in the pre-trial phase, cohort selection and participant management, and data collection and analysis. A particular focus was paid to the operational and philosophical barriers to ML in clinical research. Peer-reviewed evidence was noted to be lacking in several areas.

Conclusions

ML holds great promise for improving the efficiency and quality of clinical research, but substantial barriers remain, the surmounting of which will require addressing significant gaps in evidence.

iDISK: the integrated DIetary Supplements Knowledge base

OBJECTIVE

To build a knowledge base of dietary supplement (DS) information, called the integrated DIetary Supplement Knowledge base (iDISK), which integrates and standardizes DS-related information from 4 existing resources.

MATERIALS AND METHODS

iDISK was built through an iterative process comprising 3 phases: 1) establishment of the content scope, 2) development of the data model, and 3) integration of existing resources. Four well-regarded DS resources were integrated into iDISK: The Natural Medicines Comprehensive Database, the "About Herbs" page on the Memorial Sloan Kettering Cancer Center website, the Dietary Supplement Label Database, and the Natural Health Products Database. We evaluated the iDISK build process by manually checking that the data elements associated with 50 randomly selected ingredients were correctly extracted and integrated from their respective sources.

RESULTS

iDISK encompasses a terminology of 4208 DS ingredient concepts, which are linked via 6 relationship types to 495 drugs, 776 diseases, 985 symptoms, 605 therapeutic classes, 17 system organ classes, and 137 568 DS products. iDISK also contains 7 concept attribute types and 3 relationship attribute types. Evaluation of the data extraction and integration process showed average errors of 0.3%, 2.6%, and 0.4% for concepts, relationships and attributes, respectively.

CONCLUSION

We developed iDISK, a publicly available standardized DS knowledge base that can facilitate more efficient and meaningful dissemination of DS knowledge.

Exploring Novel Computable Knowledge in Structured Drug Product Labels

This paper introduces a database derived from Structured Product Labels (SPLs). SPLs are legally mandated snapshots containing information on all drugs released to market in the United States. Since publication is not required for pre-trial findings, we hypothesize that SPLs may contain knowledge absent in the literature, and hence "novel." SemMedDB is an existing database of computable knowledge derived from the literature. If SPL content could be similarly transformed, novel clinically relevant assertions in the SPLs could be identified through comparison with SemMedDB. After we derive a database (containing 4,297,481 assertions), we compare the extracted content with SemMedDB for recent FDA drug approvals. We find that novelty between the SPLs and the literature is nuanced, due to the redundancy of SPLs. Highlighting areas for improvement and future work, we conclude that SPLs contain a wealth of novel knowledge relevant to research and complementary to the literature.

Computational Advances in Drug Safety: Systematic and Mapping Review of Knowledge Engineering Based Approaches

Drug Safety (DS) is a domain with significant public health and social impact. Knowledge Engineering (KE) is the Computer Science discipline elaborating on methods and tools for developing “knowledge-intensive” systems, depending on a conceptual “knowledge” schema and some kind of “reasoning” process. The present systematic and mapping review aims to investigate KE-based approaches employed for DS and highlight the introduced added value as well as trends and possible gaps in the domain. Journal articles published between 2006 and 2017 were retrieved from PubMed/MEDLINE and Web of Science® (873 in total) and filtered based on a comprehensive set of inclusion/exclusion criteria. The 80 finally selected articles were reviewed on full-text, while the mapping process relied on a set of concrete criteria (concerning specific KE and DS core activities, special DS topics, employed data sources, reference ontologies/terminologies, and computational methods, etc.). The analysis results are publicly available as online interactive analytics graphs. The review clearly depicted increased use of KE approaches for DS. The collected data illustrate the use of KE for various DS aspects, such as Adverse Drug Event (ADE) information collection, detection, and assessment. Moreover, the quantified analysis of using KE for the respective DS core activities highlighted room for intensifying research on KE for ADE monitoring, prevention and reporting. Finally, the assessed use of the various data sources for DS special topics demonstrated extensive use of dominant data sources for DS surveillance, i.e., Spontaneous Reporting Systems, but also increasing interest in the use of emerging data sources, e.g., observational healthcare databases, biochemical/genetic databases, and social media. Various exemplar applications were identified with promising results, e.g., improvement in Adverse Drug Reaction (ADR) prediction, detection of drug interactions, and novel ADE profiles related with specific mechanisms of action, etc. Nevertheless, since the reviewed studies mostly concerned proof-of-concept implementations, more intense research is required to increase the maturity level that is necessary for KE approaches to reach routine DS practice. In conclusion, we argue that efficiently addressing DS data analytics and management challenges requires the introduction of high-throughput KE-based methods for effective knowledge discovery and management, resulting ultimately, in the establishment of a continuous learning DS system.

Methodological variations in lagged regression for detecting physiologic drug effects in EHR data

We studied how lagged linear regression can be used to detect the physiologic effects of drugs from data in the electronic health record (EHR). We systematically examined the effect of methodological variations ((i) time series construction, (ii) temporal parameterization, (iii) intra-subject normalization, (iv) differencing (lagged rates of change achieved by taking differences between consecutive measurements), (v) explanatory variables, and (vi) regression models) on performance of lagged linear methods in this context. We generated two gold standards (one knowledge-base derived, one expert-curated) for expected pairwise relationships between 7 drugs and 4 labs, and evaluated how the 64 unique combinations of methodological perturbations reproduce the gold standards. Our 28 cohorts included patients in the Columbia University Medical Center/NewYork-Presbyterian Hospital clinical database, and ranged from 2820 to 79,514 patients with between 8 and 209 average time points per patient. The most accurate methods achieved AUROC of 0.794 for knowledge-base derived gold standard (95%CI [0.741, 0.847]) and 0.705 for expert-curated gold standard (95% CI [0.629, 0.781]). We observed a mean AUROC of 0.633 (95%CI [0.610, 0.657], expert-curated gold standard) across all methods that re-parameterize time according to sequence and use either a joint autoregressive model with time-series differencing or an independent lag model without differencing. The complement of this set of methods achieved a mean AUROC close to 0.5, indicating the importance of these choices. We conclude that time-series analysis of EHR data will likely rely on some of the beneficial pre-processing and modeling methodologies identified, and will certainly benefit from continued careful analysis of methodological perturbations. This study found that methodological variations, such as pre-processing and representations, have a large effect on results, exposing the importance of thoroughly evaluating these components when comparing machine-learning methods.

A Census of Disease Ontologies

For centuries, humans have sought to classify diseases based on phenotypic presentation and available treatments. Today, a wide landscape of strategies, resources, and tools exist to classify patients and diseases. Ontologies can provide a robust foundation of logic for precise stratification and classification along diverse axes such as etiology, development, treatment, and genetics. Disease and phenotype ontologies are used in four primary ways: ( a) search, retrieval, and annotation of knowledge; ( b) data integration and analysis; ( c) clinical decision support; and ( d) knowledge discovery. Computational inference can connect existing knowledge and generate new insights and hypotheses about drug targets, prognosis prediction, or diagnosis. In this review, we examine the rise of disease and phenotype ontologies and the diverse ways they are represented and applied in biomedicine.

OpenPVSignal: Advancing Information Search, Sharing and Reuse on Pharmacovigilance Signals via FAIR Principles and Semantic Web Technologies

Signal detection and management is a key activity in pharmacovigilance (PV). When a new PV signal is identified, the respective information is publicly communicated in the form of periodic newsletters or reports by organizations that monitor and investigate PV-related information (such as the World Health Organization and national PV centers). However, this type of communication does not allow for systematic access, discovery and explicit data interlinking and, therefore, does not facilitate automated data sharing and reuse. In this paper, we present OpenPVSignal, a novel ontology aiming to support the semantic enrichment and rigorous communication of PV signal information in a systematic way, focusing on two key aspects: (a) publishing signal information according to the FAIR (Findable, Accessible, Interoperable, and Re-usable) data principles, and (b) exploiting automatic reasoning capabilities upon the interlinked PV signal report data. OpenPVSignal is developed as a reusable, extendable and machine-understandable model based on Semantic Web standards/recommendations. In particular, it can be used to model PV signal report data focusing on: (a) heterogeneous data interlinking, (b) semantic and syntactic interoperability, (c) provenance tracking and (d) knowledge expressiveness. OpenPVSignal is built upon widely-accepted semantic models, namely, the provenance ontology (PROV-O), the Micropublications semantic model, the Web Annotation Data Model (WADM), the Ontology of Adverse Events (OAE) and the Time ontology. To this end, we describe the design of OpenPVSignal and demonstrate its applicability as well as the reasoning capabilities enabled by its use. We also provide an evaluation of the model against the FAIR data principles. The applicability of OpenPVSignal is demonstrated by using PV signal information published in: (a) the World Health Organization's Pharmaceuticals Newsletter, (b) the Netherlands Pharmacovigilance Centre Lareb Web site and (c) the U.S. Food and Drug Administration (FDA) Drug Safety Communications, also available on the FDA Web site.

Ontology-based literature mining and class effect analysis of adverse drug reactions associated with neuropathy-inducing drugs

BACKGROUND

Adverse drug reactions (ADRs), also called as drug adverse events (AEs), are reported in the FDA drug labels; however, it is a big challenge to properly retrieve and analyze the ADRs and their potential relationships from textual data. Previously, we identified and ontologically modeled over 240 drugs that can induce peripheral neuropathy through mining public drug-related databases and drug labels. However, the ADR mechanisms of these drugs are still unclear. In this study, we aimed to develop an ontology-based literature mining system to identify ADRs from drug labels and to elucidate potential mechanisms of the neuropathy-inducing drugs (NIDs).

RESULTS

We developed and applied an ontology-based SciMiner literature mining strategy to mine ADRs from the drug labels provided in the Text Analysis Conference (TAC) 2017, which included drug labels for 53 neuropathy-inducing drugs (NIDs). We identified an average of 243 ADRs per NID and constructed an ADR-ADR network, which consists of 29 ADR nodes and 149 edges, including only those ADR-ADR pairs found in at least 50% of NIDs. Comparison to the ADR-ADR network of non-NIDs revealed that the ADRs such as pruritus, pyrexia, thrombocytopenia, nervousness, asthenia, acute lymphocytic leukaemia were highly enriched in the NID network. Our ChEBI-based ontology analysis identified three benzimidazole NIDs (i.e., lansoprazole, omeprazole, and pantoprazole), which were associated with 43 ADRs. Based on ontology-based drug class effect definition, the benzimidazole drug group has a drug class effect on all of these 43 ADRs. Many of these 43 ADRs also exist in the enriched NID ADR network. Our Ontology of Adverse Events (OAE) classification further found that these 43 benzimidazole-related ADRs were distributed in many systems, primarily in behavioral and neurological, digestive, skin, and immune systems.

CONCLUSIONS

Our study demonstrates that ontology-based literature mining and network analysis can efficiently identify and study specific group of drugs and their associated ADRs. Furthermore, our analysis of drug class effects identified 3 benzimidazole drugs sharing 43 ADRs, leading to new hypothesis generation and possible mechanism understanding of drug-induced peripheral neuropathy.

Risk of lower extremity amputations in people with type 2 diabetes mellitus treated with sodium-glucose co-transporter-2 inhibitors in the USA: A retrospective cohort study

AIMS

To examine the incidence of amputation in patients with type 2 diabetes mellitus (T2DM) treated with sodium glucose co-transporter 2 (SGLT2) inhibitors overall, and canagliflozin specifically, compared with non-SGLT2 inhibitor antihyperglycaemic agents (AHAs).

MATERIALS AND METHODS

Patients with T2DM newly exposed to SGLT2 inhibitors or non-SGLT2 inhibitor AHAs were identified using the Truven MarketScan database. The incidence of below-knee lower extremity (BKLE) amputation was calculated for patients treated with SGLT2 inhibitors, canagliflozin, or non-SGLT2 inhibitor AHAs. Patients newly exposed to canagliflozin and non-SGLT2 inhibitor AHAs were matched 1:1 on propensity scores, and a Cox proportional hazards model was used for comparative analysis. Negative controls (outcomes not believed to be associated with any AHA) were used to calibrate P values.

RESULTS

Between April 1, 2013 and October 31, 2016, 118 018 new users of SGLT2 inhibitors, including 73 024 of canagliflozin, and 226 623 new users of non-SGLT2 inhibitor AHAs were identified. The crude incidence rates of BKLE amputation were 1.22, 1.26 and 1.87 events per 1000 person-years with SGLT2 inhibitors, canagliflozin and non-SGLT2 inhibitor AHAs, respectively. For the comparative analysis, 63 845 new users of canagliflozin were matched with 63 845 new users of non-SGLT2 inhibitor AHAs, resulting in well-balanced baseline covariates. The incidence rates of BKLE amputation were 1.18 and 1.12 events per 1000 person-years with canagliflozin and non-SGLT2 inhibitor AHAs, respectively; the hazard ratio was 0.98 (95% confidence interval 0.68-1.41; P = .92, calibrated P = .95).

CONCLUSIONS

This real-world study observed no evidence of increased risk of BKLE amputation for new users of canagliflozin compared with non-SGLT2 inhibitor AHAs in a broad population of patients with T2DM.

Standard-based comprehensive detection of adverse drug reaction signals from nursing statements and laboratory results in electronic health records

Objective. We propose 2 Medical Dictionary for Regulatory Activities–enabled pharmacovigilance algorithms, MetaLAB and MetaNurse, powered by a per-year meta-analysis technique and improved subject sampling strategy.

Matrials and methods. This study developed 2 novel algorithms, MetaLAB for laboratory abnormalities and MetaNurse for standard nursing statements, as significantly improved versions of our previous electronic health record (EHR)–based pharmacovigilance method, called CLEAR. Adverse drug reaction (ADR) signals from 117 laboratory abnormalities and 1357 standard nursing statements for all precautionary drugs (n  = 101) were comprehensively detected and validated against SIDER (Side Effect Resource) by MetaLAB and MetaNurse against 11 817 and 76 457 drug-ADR pairs, respectively.

Results. We demonstrate that MetaLAB (area under the curve, AUC = 0.61 ± 0.18) outperformed CLEAR (AUC = 0.55 ± 0.06) when we applied the same 470 drug-event pairs as the gold standard, as in our previous research. Receiver operating characteristic curves for 101 precautionary terms in the Medical Dictionary for Regulatory Activities Preferred Terms were obtained for MetaLAB and MetaNurse (0.69 ± 0.11; 0.62 ± 0.07), which complemented each other in terms of ADR signal coverage. Novel ADR signals discovered by MetaLAB and MetaNurse were successfully validated against spontaneous reports in the US Food and Drug Administration Adverse Event Reporting System database.

Discussion. The present study demonstrates the symbiosis of laboratory test results and nursing statements for ADR signal detection in terms of their system organ class coverage and performance profiles.

Conclusion. Systematic discovery and evaluation of the wide spectrum of ADR signals using standard-based observational electronic health record data across many institutions will affect drug development and use, as well as postmarketing surveillance and regulation.

Drug knowledge bases and their applications in biomedical informatics research

Recent advances in biomedical research have generated a large volume of drug-related data. To effectively handle this flood of data, many initiatives have been taken to help researchers make good use of them. As the results of these initiatives, many drug knowledge bases have been constructed. They range from simple ones with specific focuses to comprehensive ones that contain information on almost every aspect of a drug. These curated drug knowledge bases have made significant contributions to the development of efficient and effective health information technologies for better health-care service delivery. Understanding and comparing existing drug knowledge bases and how they are applied in various biomedical studies will help us recognize the state of the art and design better knowledge bases in the future. In addition, researchers can get insights on novel applications of the drug knowledge bases through a review of successful use cases. In this study, we provide a review of existing popular drug knowledge bases and their applications in drug-related studies. We discuss challenges in constructing and using drug knowledge bases as well as future research directions toward a better ecosystem of drug knowledge bases.

Identifying natural health product and dietary supplement information within adverse event reporting systems

Data on safety and efficacy issues associated with natural health products and dietary supplements (NHP&S) remains largely cloistered within domain specific databases or embedded within general biomedical data sources. A major challenge in leveraging analytic approaches on such data is due to the inefficient ability to retrieve relevant data, which includes a general lack of interoperability among related sources. This study developed a thesaurus of NHP&S ingredient terms that can be used by existing biomedical natural language processing (NLP) tools for extracting information of interest. This process was evaluated relative to intervention name strings sampled from the United States Food and Drug Administration Adverse Event Reporting System (FAERS). A use case was used to demonstrate the potential to utilize FAERS for monitoring NHP&S adverse events. The results from this study provide insights on approaches for identifying additional knowledge from extant repositories of knowledge, and potentially as information that can be included into larger curation efforts.

Representing Knowledge Consistently Across Health Systems

Objectives: Electronic health records (EHRs) have increasingly emerged as a powerful source of clinical data that can be leveraged for reuse in research and in modular health apps that integrate into diverse health information technologies. A key challenge to these use cases is representing the knowledge contained within data from different EHR systems in a uniform fashion. Method: We reviewed several recent studies covering the knowledge representation in the common data models for the Observational Medical Outcomes Partnership (OMOP) and its Observational Health Data Sciences and Informatics program, and the United States Patient Centered Outcomes Research Network (PCORNet). We also reviewed the Health Level 7 Fast Healthcare Interoperability Resource standard supporting app-like programs that can be used across multiple EHR and research systems. Results: There has been a recent growth in high-impact efforts to support quality-assured and standardized clinical data sharing across different institutions and EHR systems. We focused on three major efforts as part of a larger landscape moving towards shareable, transportable, and computable clinical data. Conclusion: The growth in approaches to developing common data models to support interoperable knowledge representation portends an increasing availability of high-quality clinical data in support of research. Building on these efforts will allow a future whereby significant portions of the populations in the world may be able to share their data for research.

Incidence of diabetic ketoacidosis among patients with type 2 diabetes mellitus treated with SGLT2 inhibitors and other antihyperglycemic agents

AIMS

To estimate and compare incidence of diabetes ketoacidosis (DKA) among patients with type 2 diabetes who are newly treated with SGLT2 inhibitors (SGLT2i) versus non-SGLT2i antihyperglycemic agents (AHAs) in actual clinical practice.

METHODS

A new-user cohort study design using a large insurance claims database in the US. DKA incidence was compared between new users of SGLT2i and new users of non-SGLT2i AHAs pair-matched on exposure propensity scores (EPS) using Cox regression models.

RESULTS

Overall, crude incidence rates (95% CI) per 1000 patient-years for DKA were 1.69 (1.22-2.30) and 1.83 (1.58-2.10) among new users of SGLT2i (n=34,442) and non-SGLT2i AHAs (n=126,703). These rates more than doubled among patients with prior insulin prescriptions but decreased by more than half in analyses that excluded potential autoimmune diabetes (PAD). The hazard ratio (95% CI) for DKA comparing new users of SGLT2i to new users of non-SGLT2i AHAs was 1.91 (0.94-4.11) (p=0.09) among the 30,196 EPS-matched pairs overall, and 1.13 (0.43-3.00) (p=0.81) among the 27,515 EPS-matched pairs that excluded PAD.

CONCLUSIONS

This was the first observational study that compared DKA risk between new users of SGLT2i and non-SGLT2i AHAs among patients with type 2 diabetes, and overall no statistically significant difference was detected.

Boosting Quality Registries with Clinical Decision Support Functionality*. User Acceptance of a Prototype Applied to HIV/TB Drug Therapy

BACKGROUND

The care of HIV-related tuberculosis (HIV/TB) is complex and challenging. Clinical decision support (CDS) systems can contribute to improve quality of care, but more knowledge is needed on factors determining user acceptance of CDS.

OBJECTIVES

To analyze physicians' and nurses' acceptance of a CDS prototype for evidence-based drug therapy recommendations for HIV/TB treatment.

METHODS

Physicians and nurses were involved in designing a CDS prototype intended for future integration with the Swedish national HIV quality registry. Focus group evaluation was performed with ten nurses and four physicians, respectively. The Unified Theory of Acceptance and Use of Technology (UTAUT) was used to analyze acceptance.

RESULTS

We identified several potential benefits with the CDS prototype as well as some concerns that could be addressed by redesign. There was also concern about dependence on physician attitudes, as well as technical, organizational, and legal issues.

CONCLUSIONS

Acceptance evaluation at a prototype stage provided rich data to improve the future design of a CDS prototype. Apart from design and development efforts, substantial organizational efforts are needed to enable the implementation and maintenance of a future CDS system.

Large-scale adverse effects related to treatment evidence standardization (LAERTES): an open scalable system for linking pharmacovigilance evidence sources with clinical data

BACKGROUND

Integrating multiple sources of pharmacovigilance evidence has the potential to advance the science of safety signal detection and evaluation. In this regard, there is a need for more research on how to integrate multiple disparate evidence sources while making the evidence computable from a knowledge representation perspective (i.e., semantic enrichment). Existing frameworks suggest well-promising outcomes for such integration but employ a rather limited number of sources. In particular, none have been specifically designed to support both regulatory and clinical use cases, nor have any been designed to add new resources and use cases through an open architecture. This paper discusses the architecture and functionality of a system called Large-scale Adverse Effects Related to Treatment Evidence Standardization (LAERTES) that aims to address these shortcomings.

RESULTS

LAERTES provides a standardized, open, and scalable architecture for linking evidence sources relevant to the association of drugs with health outcomes of interest (HOIs). Standard terminologies are used to represent different entities. For example, drugs and HOIs are represented in RxNorm and Systematized Nomenclature of Medicine -- Clinical Terms respectively. At the time of this writing, six evidence sources have been loaded into the LAERTES evidence base and are accessible through prototype evidence exploration user interface and a set of Web application programming interface services. This system operates within a larger software stack provided by the Observational Health Data Sciences and Informatics clinical research framework, including the relational Common Data Model for observational patient data created by the Observational Medical Outcomes Partnership. Elements of the Linked Data paradigm facilitate the systematic and scalable integration of relevant evidence sources.

CONCLUSIONS

The prototype LAERTES system provides useful functionality while creating opportunities for further research. Future work will involve improving the method for normalizing drug and HOI concepts across the integrated sources, aggregated evidence at different levels of a hierarchy of HOI concepts, and developing more advanced user interface for drug-HOI investigations.

Accuracy of an automated knowledge base for identifying drug adverse reactions

INTRODUCTION

Drug safety researchers seek to know the degree of certainty with which a particular drug is associated with an adverse drug reaction. There are different sources of information used in pharmacovigilance to identify, evaluate, and disseminate medical product safety evidence including spontaneous reports, published peer-reviewed literature, and product labels. Automated data processing and classification using these evidence sources can greatly reduce the manual curation currently required to develop reference sets of positive and negative controls (i.e. drugs that cause adverse drug events and those that do not) to be used in drug safety research.

METHODS

In this paper we explore a method for automatically aggregating disparate sources of information together into a single repository, developing a predictive model to classify drug-adverse event relationships, and applying those predictions to a real world problem of identifying negative controls for statistical method calibration.

RESULTS

Our results showed high predictive accuracy for the models combining all available evidence, with an area under the receiver-operator curve of ⩾0.92 when tested on three manually generated lists of drugs and conditions that are known to either have or not have an association with an adverse drug event.

CONCLUSIONS

Results from a pilot implementation of the method suggests that it is feasible to develop a scalable alternative to the time-and-resource-intensive, manual curation exercise previously applied to develop reference sets of positive and negative controls to be used in drug safety research.

Feasibility of Prioritizing Drug-Drug-Event Associations Found in Electronic Health Records

Background and Objective

Several studies have demonstrated the ability to detect adverse events potentially related to multiple drug exposure via data mining. However, the number of putative associations produced by such computational approaches is typically large, making experimental validation difficult. We theorized that those potential associations for which there is evidence from multiple complementary sources are more likely to be true, and explored this idea using a published database of drug–drug-adverse event associations derived from electronic health records (EHRs).

Methods

We prioritized drug–drug-event associations derived from EHRs using four sources of information: (1) public databases, (2) sources of spontaneous reports, (3) literature, and (4) non-EHR drug–drug interaction (DDI) prediction methods. After pre-filtering the associations by removing those found in public databases, we devised a ranking for associations based on the support from the remaining sources, and evaluated the results of this rank-based prioritization.

Results

We collected information for 5983 putative EHR-derived drug–drug-event associations involving 345 drugs and ten adverse events from four data sources and four prediction methods. Only seven drug–drug-event associations (<0.5 %) had support from the majority of evidence sources, and about one third (1777) had support from at least one of the evidence sources.

Conclusions

Our proof-of-concept method for scoring putative drug–drug-event associations from EHRs offers a systematic and reproducible way of prioritizing associations for further study. Our findings also quantify the agreement (or lack thereof) among complementary sources of evidence for drug–drug-event associations and highlight the challenges of developing a robust approach for prioritizing signals of these associations.

Electronic supplementary material

The online version of this article (doi:10.1007/s40264-015-0352-2) contains supplementary material, which is available to authorized users.

A time-indexed reference standard of adverse drug reactions

Undetected adverse drug reactions (ADRs) pose a major burden on the health system. Data mining methodologies designed to identify signals of novel ADRs are of deep importance for drug safety surveillance. The development and evaluation of these methodologies requires proper reference benchmarks. While progress has recently been made in developing such benchmarks, our understanding of the performance characteristics of the data mining methodologies is limited because existing benchmarks do not support prospective performance evaluations. We address this shortcoming by providing a reference standard to support prospective performance evaluations. The reference standard was systematically curated from drug labeling revisions, such as new warnings, which were issued and communicated by the US Food and Drug Administration in 2013. The reference standard includes 62 positive test cases and 75 negative controls, and covers 44 drugs and 38 events. We provide usage guidance and empirical support for the reference standard by applying it to analyze two data sources commonly mined for drug safety surveillance.

Good Signal Detection Practices: Evidence from IMI PROTECT

Over a period of 5 years, the Innovative Medicines Initiative PROTECT (Pharmacoepidemiological Research on Outcomes of Therapeutics by a European ConsorTium) project has addressed key research questions relevant to the science of safety signal detection. The results of studies conducted into quantitative signal detection in spontaneous reporting, clinical trial and electronic health records databases are summarised and 39 recommendations have been formulated, many based on comparative analyses across a range of databases (e.g. regulatory, pharmaceutical company). The recommendations point to pragmatic steps that those working in the pharmacovigilance community can take to improve signal detection practices, whether in a national or international agency or in a pharmaceutical company setting. PROTECT has also pointed to areas of potentially fruitful future research and some areas where further effort is likely to yield less.

A curated and standardized adverse drug event resource to accelerate drug safety research

Identification of adverse drug reactions (ADRs) during the post-marketing phase is one of the most important goals of drug safety surveillance. Spontaneous reporting systems (SRS) data, which are the mainstay of traditional drug safety surveillance, are used for hypothesis generation and to validate the newer approaches. The publicly available US Food and Drug Administration (FDA) Adverse Event Reporting System (FAERS) data requires substantial curation before they can be used appropriately, and applying different strategies for data cleaning and normalization can have material impact on analysis results. We provide a curated and standardized version of FAERS removing duplicate case records, applying standardized vocabularies with drug names mapped to RxNorm concepts and outcomes mapped to SNOMED-CT concepts, and pre-computed summary statistics about drug-outcome relationships for general consumption. This publicly available resource, along with the source code, will accelerate drug safety research by reducing the amount of time spent performing data management on the source FAERS reports, improving the quality of the underlying data, and enabling standardized analyses using common vocabularies.

Integrating Heterogeneous Biomedical Data for Cancer Research: the CARPEM infrastructure

Cancer research involves numerous disciplines. The multiplicity of data sources and their heterogeneous nature render the integration and the exploration of the data more and more complex. Translational research platforms are a promising way to assist scientists in these tasks. In this article, we identify a set of scientific and technical principles needed to build a translational research platform compatible with ethical requirements, data protection and data-integration problems. We describe the solution adopted by the CARPEM cancer research program to design and deploy a platform able to integrate retrospective, prospective, and day-to-day care data. We designed a three-layer architecture composed of a data collection layer, a data integration layer and a data access layer. We leverage a set of open-source resources including i2b2 and tranSMART.

Variation in adverse drug reactions listed in product information for antidepressants and anticonvulsants, between the USA and Europe: a comparison review of paired regulatory documents

OBJECTIVE

To compare consistency of adverse drug reaction (ADR) data in publicly available product information documents for brand drugs, between the USA and Europe. To assess the usefulness of information for prescribers and patients.

DESIGN

A comparison review of product information documents for antidepressants and anticonvulsants concurrently marketed by the same pharmaceutical company in the USA and Europe.

SETTING

For each drug, data were extracted from the US Product Inserts and the European Summary of Product Characteristics documents between 09/2013 and 01/2015.

PARTICIPANTS

Individuals contributing ADR information to product information documents.

MAIN OUTCOMES MEASURES

All ADRs reported in product information sections 5 and 6 (USA), and 4·4 and 4·8 (Europe).

RESULTS

Twelve brand drugs--24 paired documents--were included. On average, there were 77 more ADRs reported in the USA compared with in the European product information document, with a median number of 201 ADRs (range: 65-425) and 114 (range: 56-265), respectively. More product information documents in the USA reported information on the source of evidence (10 vs 5) and risk (9 vs 5) for greater than 80% of ADRs included in the document. There was negligible information included regarding duration, severity, reversibility or recurrence of ADRs. On average, only 29% of ADR terms were reported in both paired documents.

CONCLUSIONS

Product information documents contained a large number of ADRs, but lacked contextual data and information important to patients and prescribers, such as duration, severity and reversibility. The ADR profile was found to be inconsistently reported between the USA and Europe, for the same drug. Identifying, selecting, summarising and presenting multidimensional harm data should be underpinned by practical evidence-based guidelines. In order for prescribers to provide considered risk-benefit advice across competing drug therapies to patients, they need access to comprehensible and reliable ADR information.

Conversion and Data Quality Assessment of Electronic Health Record Data at a Korean Tertiary Teaching Hospital to a Common Data Model for Distributed Network Research

OBJECTIVES

A distributed research network (DRN) has the advantages of improved statistical power, and it can reveal more significant relationships by increasing sample size. However, differences in data structure constitute a major barrier to integrating data among DRN partners. We describe our experience converting Electronic Health Records (EHR) to the Observational Health Data Sciences and Informatics (OHDSI) Common Data Model (CDM).

METHODS

We transformed the EHR of a hospital into Observational Medical Outcomes Partnership (OMOP) CDM ver. 4.0 used in OHDSI. All EHR codes were mapped and converted into the standard vocabulary of the CDM. All data required by the CDM were extracted, transformed, and loaded (ETL) into the CDM structure. To validate and improve the quality of the transformed dataset, the open-source data characterization program ACHILLES was run on the converted data.

RESULTS

Patient, drug, condition, procedure, and visit data from 2.07 million patients who visited the subject hospital from July 1994 to November 2014 were transformed into the CDM. The transformed dataset was named the AUSOM. ACHILLES revealed 36 errors and 13 warnings in the AUSOM. We reviewed and corrected 28 errors. The summarized results of the AUSOM processed with ACHILLES are available at http://ami.ajou.ac.kr:8080/.

CONCLUSIONS

We successfully converted our EHRs to a CDM and were able to participate as a data partner in an international DRN. Converting local records in this manner will provide various opportunities for researchers and data holders.

SPIRIT: Systematic Planning of Intelligent Reuse of Integrated Clinical Routine Data. A Conceptual Best-practice Framework and Procedure Model

BACKGROUND

Secondary use of clinical routine data is receiving an increasing amount of attention in biomedicine and healthcare. However, building and analysing integrated clinical routine data repositories are nontrivial, challenging tasks. As in most evolving fields, recognized standards, well-proven methodological frameworks, or accurately described best-practice approaches for the systematic planning of solutions for secondary use of routine medical record data are missing.

OBJECTIVE

We propose a conceptual best-practice framework and procedure model for the systematic planning of intelligent reuse of integrated clinical routine data (SPIRIT).

METHODS

SPIRIT was developed based on a broad literature overview and further refined in two case studies with different kinds of clinical routine data, including process-oriented nursing data from a large hospital group and high-volume multimodal clinical data from a neurologic intensive care unit.

RESULTS

SPIRIT aims at tailoring secondary use solutions to specific needs of single departments without losing sight of the institution as a whole. It provides a general conceptual best-practice framework consisting of three parts: First, a secondary use strategy for the whole organization is determined. Second, comprehensive analyses are conducted from two different viewpoints to define the requirements regarding a clinical routine data reuse solution at the system level from the data perspective (BOTTOM UP) and at the strategic level from the future users perspective (TOP DOWN). An obligatory clinical context analysis (IN BETWEEN) facilitates refinement, combination, and integration of the different requirements. The third part of SPIRIT is dedicated to implementation, which comprises design and realization of clinical data integration and management as well as data analysis solutions.

CONCLUSIONS

The SPIRIT framework is intended to be used to systematically plan the intelligent reuse of clinical routine data for multiple purposes, which often was not intended when the primary clinical documentation systems were implemented. SPIRIT helps to overcome this gap. It can be applied in healthcare institutions of any size or specialization and allows a stepwise setup and evolution of holistic clinical routine data reuse solutions.

Desiderata for computable representations of electronic health records-driven phenotype algorithms

BACKGROUND

Electronic health records (EHRs) are increasingly used for clinical and translational research through the creation of phenotype algorithms. Currently, phenotype algorithms are most commonly represented as noncomputable descriptive documents and knowledge artifacts that detail the protocols for querying diagnoses, symptoms, procedures, medications, and/or text-driven medical concepts, and are primarily meant for human comprehension. We present desiderata for developing a computable phenotype representation model (PheRM).

METHODS

A team of clinicians and informaticians reviewed common features for multisite phenotype algorithms published in PheKB.org and existing phenotype representation platforms. We also evaluated well-known diagnostic criteria and clinical decision-making guidelines to encompass a broader category of algorithms.

RESULTS

We propose 10 desired characteristics for a flexible, computable PheRM: (1) structure clinical data into queryable forms; (2) recommend use of a common data model, but also support customization for the variability and availability of EHR data among sites; (3) support both human-readable and computable representations of phenotype algorithms; (4) implement set operations and relational algebra for modeling phenotype algorithms; (5) represent phenotype criteria with structured rules; (6) support defining temporal relations between events; (7) use standardized terminologies and ontologies, and facilitate reuse of value sets; (8) define representations for text searching and natural language processing; (9) provide interfaces for external software algorithms; and (10) maintain backward compatibility.

CONCLUSION

A computable PheRM is needed for true phenotype portability and reliability across different EHR products and healthcare systems. These desiderata are a guide to inform the establishment and evolution of EHR phenotype algorithm authoring platforms and languages.

Text mining for adverse drug events: the promise, challenges, and state of the art

Text mining is the computational process of extracting meaningful information from large amounts of unstructured text. It is emerging as a tool to leverage underutilized data sources that can improve pharmacovigilance, including the objective of adverse drug event (ADE) detection and assessment. This article provides an overview of recent advances in pharmacovigilance driven by the application of text mining, and discusses several data sources-such as biomedical literature, clinical narratives, product labeling, social media, and Web search logs-that are amenable to text mining for pharmacovigilance. Given the state of the art, it appears text mining can be applied to extract useful ADE-related information from multiple textual sources. Nonetheless, further research is required to address remaining technical challenges associated with the text mining methodologies, and to conclusively determine the relative contribution of each textual source to improving pharmacovigilance.

Zoo or savannah? Choice of training ground for evidence-based pharmacovigilance

Pharmacovigilance seeks to detect and describe adverse drug reactions early. Ideally, we would like to see objective evidence that a chosen signal detection approach can be expected to be effective. The development and evaluation of evidence-based methods require benchmarks for signal detection performance, and recent years have seen unprecedented efforts to build such reference sets. Here, we argue that evaluation should be made against emerging and not established adverse drug reactions, and we present real-world examples that illustrate the relevance of this to pharmacovigilance methods development for both individual case reports and longitudinal health records. The establishment of broader reference sets of emerging safety signals must be made a top priority to achieve more effective pharmacovigilance methods development and evaluation.

Computational approaches for pharmacovigilance signal detection: toward integrated and semantically-enriched frameworks

Computational signal detection constitutes a key element of postmarketing drug monitoring and surveillance. Diverse data sources are considered within the 'search space' of pharmacovigilance scientists, and respective data analysis methods are employed, all with their qualities and shortcomings, towards more timely and accurate signal detection. Recent systematic comparative studies highlighted not only event-based and data-source-based differential performance across methods but also their complementarity. These findings reinforce the arguments for exploiting all possible information sources for drug safety and the parallel use of multiple signal detection methods. Combinatorial signal detection has been pursued in few studies up to now, employing a rather limited number of methods and data sources but illustrating well-promising outcomes. However, the large-scale realization of this approach requires systematic frameworks to address the challenges of the concurrent analysis setting. In this paper, we argue that semantic technologies provide the means to address some of these challenges, and we particularly highlight their contribution in (a) annotating data sources and analysis methods with quality attributes to facilitate their selection given the analysis scope; (b) consistently defining study parameters such as health outcomes and drugs of interest, and providing guidance for study setup; (c) expressing analysis outcomes in a common format enabling data sharing and systematic comparisons; and (d) assessing/supporting the novelty of the aggregated outcomes through access to reference knowledge sources related to drug safety. A semantically-enriched framework can facilitate seamless access and use of different data sources and computational methods in an integrated fashion, bringing a new perspective for large-scale, knowledge-intensive signal detection.

Using electronic health records for clinical research: the case of the EHR4CR project

OBJECTIVES

To describe the IMI EHR4CR project which is designing and developing, and aims to demonstrate, a scalable, widely acceptable and efficient approach to interoperability between EHR systems and clinical research systems.

METHODS

The IMI EHR4CR project is combining and extending several previously isolated state-of-the-art technical components through a new approach to develop a platform for reusing EHR data to support medical research. This will be achieved through multiple but unified initiatives across different major disease areas (e.g. cardiovascular, cancer) and clinical research use cases (protocol feasibility, patient identification and recruitment, clinical trial execution and serious adverse event reporting), with various local and national stakeholders across several countries and therefore under various legal frameworks.

RESULTS

An initial instance of the platform has been built, providing communication, security and terminology services to the eleven participating hospitals and ten pharmaceutical companies located in seven European countries. Proof-of-concept demonstrators have been built and evaluated for the protocol feasibility and patient recruitment scenarios. The specifications of the clinical trial execution and the adverse event reporting scenarios have been documented and reviewed.

CONCLUSIONS

Through a combination of a consortium that brings collectively many years of experience from previous relevant EU projects and of the global conduct of clinical trials, of an approach to ethics that engages many important stakeholders across Europe to ensure acceptability, of a robust iterative design methodology for the platform services that is anchored on requirements of an underlying Service Oriented Architecture that has been designed to be scalable and adaptable, EHR4CR could be well placed to deliver a sound, useful and well accepted pan-European solution for the reuse of hospital EHR data to support clinical research studies.