Large-scale adverse effects related to treatment evidence standardization (LAERTES): an open scalable system for linking pharmacovigilance evidence sources with clinical data

A scoping review of semantic integration of health data and information

OBJECTIVE

We summarized a decade of new research focusing on semantic data integration (SDI) since 2009, and we aim to: (1) summarize the state-of-art approaches on integrating health data and information; and (2) identify the main gaps and challenges of integrating health data and information from multiple levels and domains.

MATERIALS AND METHODS

We used PubMed as our focus is applications of SDI in biomedical domains and followed the Preferred Reporting Items for Systematic Review and Meta-Analyses (PRISMA) to search and report for relevant studies published between January 1, 2009 and December 31, 2021. We used Covidence-a systematic review management system-to carry out this scoping review.

RESULTS

The initial search from PubMed resulted in 5,326 articles using the two sets of keywords. We then removed 44 duplicates and 5,282 articles were retained for abstract screening. After abstract screening, we included 246 articles for full-text screening, among which 87 articles were deemed eligible for full-text extraction. We summarized the 87 articles from four aspects: (1) methods for the global schema; (2) data integration strategies (i.e., federated system vs. data warehousing); (3) the sources of the data; and (4) downstream applications.

CONCLUSION

SDI approach can effectively resolve the semantic heterogeneities across different data sources. We identified two key gaps and challenges in existing SDI studies that (1) many of the existing SDI studies used data from only single-level data sources (e.g., integrating individual-level patient records from different hospital systems), and (2) documentation of the data integration processes is sparse, threatening the reproducibility of SDI studies.

COVID-19 vaccination effectiveness rates by week and sources of bias: a retrospective cohort study

OBJECTIVE

To examine COVID-19 vaccine effectiveness over six 7-day intervals after the first dose and assess underlying bias in observational data.

DESIGN AND SETTING

Retrospective cohort study using Columbia University Irving Medical Center data linked to state and city immunisation registries.

OUTCOMES AND MEASURES

We used large-scale propensity score matching with up to 54 987 covariates, fitted Cox proportional hazards models and constructed Kaplan-Meier plots for two main outcomes (COVID-19 infection and COVID-19-associated hospitalisation). We conducted manual chart review of cases in week 1 in both groups along with a set of secondary analyses for other index date, outcome and population choices.

RESULTS

The study included 179 666 patients. We observed increasing effectiveness after the first dose of mRNA vaccines with week 6 effectiveness approximating 84% (95% CI 72% to 91%) for COVID-19 infection and 86% (95% CI 69% to 95%) for COVID-19-associated hospitalisation. When analysing unexpectedly high effectiveness in week 1, chart review revealed that vaccinated patients are less likely to seek care after vaccination and are more likely to be diagnosed with COVID-19 during the encounters for other conditions. Secondary analyses highlighted potential outcome misclassification for International Classification of Diseases, Tenth Revision, Clinical Modification diagnosis, the influence of excluding patients with prior COVID-19 infection and anchoring in the unexposed group. Long-term vaccine effectiveness in fully vaccinated patients matched the results of the randomised trials.

CONCLUSIONS

For vaccine effectiveness studies, observational data need to be scrutinised to ensure compared groups exhibit similar health-seeking behaviour and are equally likely to be captured in the data. While we found that studies may be capable of accurately estimating long-term effectiveness despite bias in early weeks, the early week results should be reported in every study so that we may gain a better understanding of the biases. Given the difference in temporal trends of vaccine exposure and patients' baseline characteristics, indirect comparison of vaccines may produce biased results.

COVID-19 vaccination effectiveness rates by week and sources of bias.. [PMID: 34981073 PMCID: PMC8722616 DOI: 10.1101/2021.12.22.21268253] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

Importance

Randomized clinical trials and observational studies have demonstrated high overall effectiveness for the three US-authorized COVID-19 vaccines against symptomatic COVID-19 infection. Nevertheless, the challenges associated with the use of observational data can undermine the results of the studies.

Objective

To assess the feasibility of using observational data for vaccine effectiveness studies by examining granular weekly effectiveness.

Design, Settings and Participants

In this retrospective cohort study, we used Columbia University Medical Center data linked to State and City Immunization Registries to assess the weekly effectiveness of mRNA COVID-19 vaccines. We conducted manual chart review of cases in week one in both groups along with a set of sensitivity analyses for Pfizer- BioNTech, Moderna and Janssen vaccines.

Main Outcomes and Measures

We used propensity score matching with up to 54,987 covariates and fitted Cox proportional hazards models to estimate hazard ratios and constructed Kaplan-Meier plots for two main outcomes (COVID-19 infection and COVID-19-associated hospitalization).

Results

The study included 179,666 patients. We observed increasing effectiveness after the first dose of mRNA vaccines with week 6 effectiveness approximating 84% (95% CI 72–91%) for COVID-19 infection and 86% (95% CI 69–95) for COVID-19-associated hospitalization. When analyzing unexpectedly high effectiveness in week one, chart review revealed that vaccinated patients are less likely to seek care after vaccination and are more likely to be diagnosed with COVID-19 during the encounters for other conditions. Sensitivity analyses showed potential outcome misclassification for COVID-19 ICD10-CM diagnosis and the influence of excluding patients with prior COVID-19 infection and anchoring in the unexposed group. Overall vaccine effectiveness analysis in fully vaccinated patients matched the results of the randomized trials.

Conclusions and Relevance

Observational data can be used to ascertain vaccine effectiveness if potential biases are accounted for. The data need to be scrutinized to ensure that compared groups exhibit similar health seeking behavior and are equally likely to be captured in the data. Given the difference in temporal trends of vaccine exposure and baseline characteristics, indirect comparison of vaccines may produce biased results.

An updated, computable MEDication-Indication resource for biomedical research

The MEDication-Indication (MEDI) knowledgebase has been utilized in research with electronic health records (EHRs) since its publication in 2013. To account for new drugs and terminology updates, we rebuilt MEDI to overhaul the knowledgebase for modern EHRs. Indications for prescribable medications were extracted using natural language processing and ontology relationships from six publicly available resources: RxNorm, Side Effect Resource 4.1, Mayo Clinic, WebMD, MedlinePlus, and Wikipedia. We compared the estimated precision and recall between the previous MEDI (MEDI-1) and the updated version (MEDI-2) with manual review. MEDI-2 contains 3031 medications and 186,064 indications. The MEDI-2 high precision subset (HPS) includes indications found within RxNorm or at least three other resources. MEDI-2 and MEDI-2 HPS contain 13% more medications and over triple the indications compared to MEDI-1 and MEDI-1 HPS, respectively. Manual review showed MEDI-2 achieves the same precision (0.60) with better recall (0.89 vs. 0.79) compared to MEDI-1. Likewise, MEDI-2 HPS had the same precision (0.92) and improved recall (0.65 vs. 0.55) than MEDI-1 HPS. The combination of MEDI-1 and MEDI-2 achieved a recall of 0.95. In updating MEDI, we present a more comprehensive medication-indication knowledgebase that can continue to facilitate applications and research with EHRs.

The 2011–2020 Trends of Data-Driven Approaches in Medical Informatics for Active Pharmacovigilance

Pharmacovigilance, the scientific discipline pertaining to drug safety, has been studied extensively and is progressing continuously. In this field, medical informatics techniques and interpretation play important roles, and appropriate approaches are required. In this study, we investigated and analyzed the trends of pharmacovigilance systems, especially the data collection, detection, assessment, and monitoring processes. We used PubMed to collect papers on pharmacovigilance published over the past 10 years, and analyzed a total of 40 significant papers to determine the characteristics of the databases and data analysis methods used to identify drug safety indicators. Through systematic reviews, we identified the difficulty of standardizing data and terminology and establishing an adverse drug reactions (ADR) evaluation system in pharmacovigilance, and their corresponding implications. We found that appropriate methods and guidelines for active pharmacovigilance using medical big data are still required and should continue to be developed.

OSIRIS: A Minimum Data Set for Data Sharing and Interoperability in Oncology

PURPOSE

Many institutions throughout the world have launched precision medicine initiatives in oncology, and a large amount of clinical and genomic data is being produced. Although there have been attempts at data sharing with the community, initiatives are still limited. In this context, a French task force composed of Integrated Cancer Research Sites (SIRICs), comprehensive cancer centers from the Unicancer network (one of Europe's largest cancer research organization), and university hospitals launched an initiative to improve and accelerate retrospective and prospective clinical and genomic data sharing in oncology.

MATERIALS AND METHODS

For 5 years, the OSIRIS group has worked on structuring data and identifying technical solutions for collecting and sharing them. The group used a multidisciplinary approach that included weekly scientific and technical meetings over several months to foster a national consensus on a minimal data set.

RESULTS

The resulting OSIRIS set and event-based data model, which is able to capture the disease course, was built with 67 clinical and 65 omics items. The group made it compatible with the HL7 Fast Healthcare Interoperability Resources (FHIR) format to maximize interoperability. The OSIRIS set was reviewed, approved by a National Plan Strategic Committee, and freely released to the community. A proof-of-concept study was carried out to put the OSIRIS set and Common Data Model into practice using a cohort of 300 patients.

CONCLUSION

Using a national and bottom-up approach, the OSIRIS group has defined a model including a minimal set of clinical and genomic data that can be used to accelerate data sharing produced in oncology. The model relies on clear and formally defined terminologies and, as such, may also benefit the larger international community.

Combined unsupervised-supervised machine learning for phenotyping complex diseases with its application to obstructive sleep apnea

Unsupervised clustering models have been widely used for multimetric phenotyping of complex and heterogeneous diseases such as diabetes and obstructive sleep apnea (OSA) to more precisely characterize the disease beyond simplistic conventional diagnosis standards. However, the number of clusters and key phenotypic features have been subjectively selected, reducing the reliability of the phenotyping results. Here, to minimize such subjective decisions for highly confident phenotyping, we develop a multimetric phenotyping framework by combining supervised and unsupervised machine learning. This clusters 2277 OSA patients to six phenotypes based on their multidimensional polysomnography (PSG) data. Importantly, these new phenotypes show statistically different comorbidity development for OSA-related cardio-neuro-metabolic diseases, unlike the conventional single-metric apnea-hypopnea index-based phenotypes. Furthermore, the key features of highly comorbid phenotypes were identified through supervised learning rather than subjective choice. These results can also be used to automatically phenotype new patients and predict their comorbidity risks solely based on their PSG data. The phenotyping framework based on the combination of unsupervised and supervised machine learning methods can also be applied to other complex, heterogeneous diseases for phenotyping patients and identifying important features for high-risk phenotypes.

Deep Learning Approach to Parse Eligibility Criteria in Dietary Supplements Clinical Trials Following OMOP Common Data Model

Dietary supplements (DSs) have been widely used in the U.S. and evaluated in clinical trials as potential interventions for various diseases. However, many clinical trials face challenges in recruiting enough eligible patients in a timely fashion, causing delays or even early termination. Using electronic health records to find eligible patients who meet clinical trial eligibility criteria has been shown as a promising way to assess recruitment feasibility and accelerate the recruitment process. In this study, we analyzed the eligibility criteria of 100 randomly selected DS clinical trials and identified both computable and non-computable criteria. We mapped annotated entities to OMOP Common Data Model (CDM) with novel entities (e.g., DS). We also evaluated a deep learning model (Bi-LSTM-CRF) for extracting these entities on CLAMP platform, with an average F1 measure of 0.601. This study shows the feasibility of automatic parsing of the eligibility criteria following OMOP CDM for future cohort identification.

A Federated Online Search Tool for Biospecimens (Sample Locator): Usability Study

BACKGROUND

The German Biobank Alliance (GBA) aims to establish a cross-site biobank network. For this endeavor, the so-called Sample Locator, a federated search tool for biospecimens and related data, has been developed, forming the heart of its information technology (IT) infrastructure.

OBJECTIVE

To ensure the sustainable use of such a tool, we included researchers as participants in an end user-based usability evaluation.

METHODS

To develop a prototype ready for evaluation, we needed input from GBA IT experts. Thus, we conducted a 2-day workshop with 8 GBA IT team members. The focus was on the respective steps of a user-centered design process. With the acquired knowledge, the participants designed low-fidelity mock-ups. The main ideas of these mock-ups were discussed, extracted, and summarized into a comprehensive prototype using Microsoft PowerPoint. Furthermore, we created a questionnaire concerning the usability of the prototype, including the System Usability Scale (SUS), questions on negative and positive aspects, and typical tasks to be fulfilled with the tool. Subsequently, the prototype was pretested on the basis of this questionnaire with researchers who have a biobank background. Based on this preliminary work, the usability analysis was ultimately carried out with researchers and the results were evaluated.

RESULTS

Altogether, 27 researchers familiar with sample requests evaluated the prototype. The analysis of the feedback certified a good usability, given that the Sample Locator prototype was seen as intuitive and user-friendly by 74% (20/27) of the participants. The total SUS score by the 25 persons that completed the questionnaire was 80.4, indicating good system usability. Still, the evaluation provided useful advice on optimization potential (eg, offering a help function).

CONCLUSIONS

The findings of this usability analysis indicate that the considerations regarding a user-friendly application that have been made in the development process so far strongly coincide with the perception of the study participants. Nevertheless, it was important to engage prospective end users to ensure that the previous development is going in the desired direction and that the Sample Locator will be used in the future. The user comments and suggestions for improvement will be considered in upcoming iterations for refinement.

The Use of Inter-terminology Maps for the Creation and Maintenance of Value Sets

Value sets are essential in activities such as electronic clinical quality measures (eCQM) and patient cohort definition. Creation and maintenance of value sets is labor intensive and error prone. Our method aims to use existing inter-terminology maps to improve the quality of value sets that are defined in more than one terminology. For 197 eCQM value sets defined in SNOMED CT plus ICD-9-CM and/or ICD-10-CM, the map-generated codes showed good overlap with the value set codes. Manual review showed that some new codes identified by mapping should probably be included in the value sets. This could potentially augment the ICD-9-CM codes by 45% (1.5 codes), ICD-10-CM codes by 25% (1.8 codes) and SNOMED CT codes by up to 42% (4.8 codes) per value set on average. The mapping between SNOMED CT and ICD-10-PCS did not perform as well because of the granularity discrepancy in the map.

Utilizing Advanced Technologies to Augment Pharmacovigilance Systems: Challenges and Opportunities

There are significant challenges and opportunities in deploying and utilizing advanced information technology (IT) within pharmacovigilance (PV) systems and across the pharmaceutical industry. Various aspects of PV will benefit from automation (e.g., by improving standardization or increasing data quality). Several themes are developed, highlighting the challenges faced, exploring solutions, and assessing the potential for further research. Automation of the workflow for processing of individual case safety reports (ICSRs) is adopted as a use case. This involves a logical progression through a series of steps that when linked together comprise the complete work process required for the effective management of ICSRs. We recognize that the rapid development of new technologies will invariably outpace the regulations applicable to PV systems. Nevertheless, we believe that such systems may be improved by intelligent automation. It is incumbent on the owners of these systems to explore opportunities presented by new technologies with regulators in order to evaluate the applicability, design, deployment, performance, validation and maintenance of advanced technologies to ensure that the PV system continues to be fit for purpose. Proposed approaches to the validation of automated PV systems are presented. A series of definitions and a critical appraisal of important considerations are provided in the form of use cases. We summarize progress made and opportunities for the development of automation of future systems. The overall goal of automation is to provide high quality safety data in the correct format, in context, more quickly, and with less manual effort. This will improve the evidence available for scientific assessment and helps to inform and expedite decisions about the minimization of risks associated with medicines.

A 2018 workshop: vaccine and drug ontology studies (VDOS 2018)

This Editorial first introduces the background of the vaccine and drug relations and how biomedical terminologies and ontologies have been used to support their studies. The history of the seven workshops, initially named VDOSME, and then named VDOS, is also summarized and introduced. Then the 7th International Workshop on Vaccine and Drug Ontology Studies (VDOS 2018), held on August 10th, 2018, Corvallis, Oregon, USA, is introduced in detail. These VDOS workshops have greatly supported the development, applications, and discussion of vaccine- and drug-related terminology and drug studies.

E-Synthesis: A Bayesian Framework for Causal Assessment in Pharmacosurveillance

Background: Evidence suggesting adverse drug reactions often emerges unsystematically and unpredictably in form of anecdotal reports, case series and survey data. Safety trials and observational studies also provide crucial information regarding the (un-)safety of drugs. Hence, integrating multiple types of pharmacovigilance evidence is key to minimising the risks of harm.

Methods: In previous work, we began the development of a Bayesian framework for aggregating multiple types of evidence to assess the probability of a putative causal link between drugs and side effects. This framework arose out of a philosophical analysis of the Bradford Hill Guidelines. In this article, we expand the Bayesian framework and add “evidential modulators,” which bear on the assessment of the reliability of incoming study results. The overall framework for evidence synthesis, “E-Synthesis”, is then applied to a case study.

Results: Theoretically and computationally, E-Synthesis exploits coherence of partly or fully independent evidence converging towards the hypothesis of interest (or of conflicting evidence with respect to it), in order to update its posterior probability. With respect to other frameworks for evidence synthesis, our Bayesian model has the unique feature of grounding its inferential machinery on a consolidated theory of hypothesis confirmation (Bayesian epistemology), and in allowing any data from heterogeneous sources (cell-data, clinical trials, epidemiological studies), and methods (e.g., frequentist hypothesis testing, Bayesian adaptive trials, etc.) to be quantitatively integrated into the same inferential framework.

Conclusions: E-Synthesis is highly flexible concerning the allowed input, while at the same time relying on a consistent computational system, that is philosophically and statistically grounded. Furthermore, by introducing evidential modulators, and thereby breaking up the different dimensions of evidence (strength, relevance, reliability), E-Synthesis allows them to be explicitly tracked in updating causal hypotheses.

Complementing Observational Signals with Literature-Derived Distributed Representations for Post-Marketing Drug Surveillance

INTRODUCTION

As a result of the well documented limitations of data collected by spontaneous reporting systems (SRS), such as bias and under-reporting, a number of authors have evaluated the utility of other data sources for the purpose of pharmacovigilance, including the biomedical literature. Previous work has demonstrated the utility of literature-derived distributed representations (concept embeddings) with machine learning for the purpose of drug side-effect prediction. In terms of data sources, these methods are complementary, observing drug safety from two different perspectives (knowledge extracted from the literature and statistics from SRS data). However, the combined utility of these pharmacovigilance methods has yet to be evaluated.

OBJECTIVE

This research investigates the utility of directly or indirectly combining an observational signal from SRS with literature-derived distributed representations into a single feature vector or in an ensemble approach for downstream machine learning (logistic regression).

METHODS

Leveraging a recently developed representation scheme, concept embeddings were generated from relational connections extracted from the literature and composed to represent drug and associated adverse reactions, as defined by two reference standards of positive (likely causal) and negative (no causal evidence) pairs. Embeddings were presented with and without common measures of observational signal from SRS sources to logistic regressors, and performance was evaluated with the receiver operating characteristic (ROC) area under the curve (AUC) metric.

RESULTS

ROC AUC performance with these composite models improves up to ≈ 20% over SRS-based disproportionality metrics alone and exceeds the best prior results reported in the literature when models leverage both sources of information.

CONCLUSIONS

Results from this study support the hypothesis that knowledge extracted from the literature can enhance the performance of SRS-based methods (and vice versa). Across reference sets, using literature and SRS information together performed better than using either source alone, providing strong support for the complementary nature of these approaches to post-marketing drug surveillance.

An Interactive Visualization Tool for HL7 FHIR Specification Browsing and Profiling

The rich semantic representation and sophisticated structure definition of the HL7 Fast Healthcare Interoperability Resources (FHIR) specification requires relatively great efforts to understand and utilize. The objective of our study is to design, develop and evaluate an open-source and user-friendly visualization interface for exploring the FHIR specification. We prototyped an interactive visualization tool for navigating and manipulating the FHIR core resources, profiles and extensions. The utility of the tool was evaluated using evaluation metrics mainly focusing on its interactive mechanism and content expressiveness. We demonstrated that the visualization techniques are helpful for navigating the HL7 FHIR specification and aiding its profiling.

Computational Advances in Drug Safety: Systematic and Mapping Review of Knowledge Engineering Based Approaches

Drug Safety (DS) is a domain with significant public health and social impact. Knowledge Engineering (KE) is the Computer Science discipline elaborating on methods and tools for developing “knowledge-intensive” systems, depending on a conceptual “knowledge” schema and some kind of “reasoning” process. The present systematic and mapping review aims to investigate KE-based approaches employed for DS and highlight the introduced added value as well as trends and possible gaps in the domain. Journal articles published between 2006 and 2017 were retrieved from PubMed/MEDLINE and Web of Science® (873 in total) and filtered based on a comprehensive set of inclusion/exclusion criteria. The 80 finally selected articles were reviewed on full-text, while the mapping process relied on a set of concrete criteria (concerning specific KE and DS core activities, special DS topics, employed data sources, reference ontologies/terminologies, and computational methods, etc.). The analysis results are publicly available as online interactive analytics graphs. The review clearly depicted increased use of KE approaches for DS. The collected data illustrate the use of KE for various DS aspects, such as Adverse Drug Event (ADE) information collection, detection, and assessment. Moreover, the quantified analysis of using KE for the respective DS core activities highlighted room for intensifying research on KE for ADE monitoring, prevention and reporting. Finally, the assessed use of the various data sources for DS special topics demonstrated extensive use of dominant data sources for DS surveillance, i.e., Spontaneous Reporting Systems, but also increasing interest in the use of emerging data sources, e.g., observational healthcare databases, biochemical/genetic databases, and social media. Various exemplar applications were identified with promising results, e.g., improvement in Adverse Drug Reaction (ADR) prediction, detection of drug interactions, and novel ADE profiles related with specific mechanisms of action, etc. Nevertheless, since the reviewed studies mostly concerned proof-of-concept implementations, more intense research is required to increase the maturity level that is necessary for KE approaches to reach routine DS practice. In conclusion, we argue that efficiently addressing DS data analytics and management challenges requires the introduction of high-throughput KE-based methods for effective knowledge discovery and management, resulting ultimately, in the establishment of a continuous learning DS system.

ODAL: A one-shot distributed algorithm to perform logistic regressions on electronic health records data from multiple clinical sites

Electronic Health Records (EHR) contain extensive information on various health outcomes and risk factors, and therefore have been broadly used in healthcare research. Integrating EHR data from multiple clinical sites can accelerate knowledge discovery and risk prediction by providing a larger sample size in a more general population which potentially reduces clinical bias and improves estimation and prediction accuracy. To overcome the barrier of patient-level data sharing, distributed algorithms are developed to conduct statistical analyses across multiple sites through sharing only aggregated information. The current distributed algorithm often requires iterative information evaluation and transferring across sites, which can potentially lead to a high communication cost in practical settings. In this study, we propose a privacy-preserving and communication-efficient distributed algorithm for logistic regression without requiring iterative communications across sites. Our simulation study showed our algorithm reached comparative accuracy comparing to the oracle estimator where data are pooled together. We applied our algorithm to an EHR data from the University of Pennsylvania health system to evaluate the risks of fetal loss due to various medication exposures.

Machine learning model combining features from algorithms with different analytical methodologies to detect laboratory-event-related adverse drug reaction signals

Background

The importance of identifying and evaluating adverse drug reactions (ADRs) has been widely recognized. Many studies have developed algorithms for ADR signal detection using electronic health record (EHR) data. In this study, we propose a machine learning (ML) model that enables accurate ADR signal detection by integrating features from existing algorithms based on inpatient EHR laboratory results.

Materials and methods

To construct an ADR reference dataset, we extracted known drug–laboratory event pairs represented by a laboratory test from the EU-SPC and SIDER databases. All possible drug–laboratory event pairs, except known ones, are considered unknown. To detect a known drug–laboratory event pair, three existing algorithms—CERT, CLEAR, and PACE—were applied to 21-year inpatient EHR data. We also constructed ML models (based on random forest, L1 regularized logistic regression, support vector machine, and a neural network) that use the intermediate products of the CERT, CLEAR, and PACE algorithms as inputs and determine whether a drug–laboratory event pair is associated. For performance comparison, we evaluated the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), F1-measure, and area under receiver operating characteristic (AUROC).

Results

All measures of ML models outperformed those of existing algorithms with sensitivity of 0.593–0.793, specificity of 0.619–0.796, NPV of 0.645–0.727, PPV of 0.680–0.777, F1-measure of 0.629–0.709, and AUROC of 0.737–0.816. Features related to change or distribution of shape were considered important for detecting ADR signals.

Conclusions

Improved performance of ML models indicated that applying our model to EHR data is feasible and promising for detecting more accurate and comprehensive ADR signals.

Clinical Research Informatics: Contributions from 2017

Objectives:  To summarize key contributions to current research in the field of Clinical Research Informatics (CRI) and to select best papers published in 2017.

Method:  A bibliographic search using a combination of MeSH descriptors and free terms on CRI was performed using PubMed, followed by a double-blind review in order to select a list of candidate best papers to be then peer-reviewed by external reviewers. A consensus meeting between the two section editors and the editorial team was organized to finally conclude on the selection of best papers.

Results:  Among the 741 returned papers published in 2017 in the various areas of CRI, the full review process selected five best papers. The first best paper reports on the implementation of consent management considering patient preferences for the use of de-identified data of electronic health records for research. The second best paper describes an approach using natural language processing to extract symptoms of severe mental illness from clinical text. The authors of the third best paper describe the challenges and lessons learned when leveraging the EHR4CR platform to support patient inclusion in academic studies in the context of an important collaboration between private industry and public health institutions. The fourth best paper describes a method and an interactive tool for case-crossover analyses of electronic medical records for patient safety. The last best paper proposes a new method for bias reduction in association studies using electronic health records data.

Conclusions:  Research in the CRI field continues to accelerate and to mature, leading to tools and platforms deployed at national or international scales with encouraging results. Beyond securing these new platforms for exploiting large-scale health data, another major challenge is the limitation of biases related to the use of “real-world” data. Controlling these biases is a prerequisite for the development of learning health systems.

Recent Developments in Clinical Terminologies - SNOMED CT, LOINC, and RxNorm

OBJECTIVE

 To discuss recent developments in clinical terminologies. SNOMED CT (Systematized Nomenclature of Medicine Clinical Terms) is the world's largest clinical terminology, developed by an international consortium. LOINC (Logical Observation Identifiers, Names, and Codes) is an international terminology widely used for clinical and laboratory observations. RxNorm is the standard drug terminology in the U.S.

METHODS AND RESULTS

 We present a brief review of the history, current state, and future development of SNOMED CT, LOINC and RxNorm. We also analyze their similarities and differences, and outline areas for greater interoperability among them.

CONCLUSIONS

 With different starting points, representation formalisms, funding sources, and evolutionary paths, SNOMED CT, LOINC, and RxNorm have evolved over the past few decades into three major clinical terminologies supporting key use cases in clinical practice. Despite their differences, partnerships have been created among their development teams to facilitate interoperability and minimize duplication of effort.

Natural Language Processing and Its Implications for the Future of Medication Safety: A Narrative Review of Recent Advances and Challenges

The safety of medication use has been a priority in the United States since the late 1930s. Recently, it has gained prominence due to the increasing amount of data suggesting that a large amount of patient harm is preventable and can be mitigated with effective risk strategies that have not been sufficiently adopted. Adverse events from medications are part of clinical practice, but the ability to identify a patient's risk and to minimize that risk must be a priority. The ability to identify adverse events has been a challenge due to limitations of available data sources, which are often free text. The use of natural language processing (NLP) may help to address these limitations. NLP is the artificial intelligence domain of computer science that uses computers to manipulate unstructured data (i.e., narrative text or speech data) in the context of a specific task. In this narrative review, we illustrate the fundamentals of NLP and discuss NLP's application to medication safety in four data sources: electronic health records, Internet-based data, published literature, and reporting systems. Given the magnitude of available data from these sources, a growing area is the use of computer algorithms to help automatically detect associations between medications and adverse effects. The main benefit of NLP is in the time savings associated with automation of various medication safety tasks such as the medication reconciliation process facilitated by computers, as well as the potential for near-real-time identification of adverse events for postmarketing surveillance such as those posted on social media that would otherwise go unanalyzed. NLP is limited by a lack of data sharing between health care organizations due to insufficient interoperability capabilities, inhibiting large-scale adverse event monitoring across populations. We anticipate that future work in this area will focus on the integration of data sources from different domains to improve the ability to identify potential adverse events more quickly and to improve clinical decision support with regard to a patient's estimated risk for specific adverse events at the time of medication prescription or review.

OpenPVSignal: Advancing Information Search, Sharing and Reuse on Pharmacovigilance Signals via FAIR Principles and Semantic Web Technologies

Signal detection and management is a key activity in pharmacovigilance (PV). When a new PV signal is identified, the respective information is publicly communicated in the form of periodic newsletters or reports by organizations that monitor and investigate PV-related information (such as the World Health Organization and national PV centers). However, this type of communication does not allow for systematic access, discovery and explicit data interlinking and, therefore, does not facilitate automated data sharing and reuse. In this paper, we present OpenPVSignal, a novel ontology aiming to support the semantic enrichment and rigorous communication of PV signal information in a systematic way, focusing on two key aspects: (a) publishing signal information according to the FAIR (Findable, Accessible, Interoperable, and Re-usable) data principles, and (b) exploiting automatic reasoning capabilities upon the interlinked PV signal report data. OpenPVSignal is developed as a reusable, extendable and machine-understandable model based on Semantic Web standards/recommendations. In particular, it can be used to model PV signal report data focusing on: (a) heterogeneous data interlinking, (b) semantic and syntactic interoperability, (c) provenance tracking and (d) knowledge expressiveness. OpenPVSignal is built upon widely-accepted semantic models, namely, the provenance ontology (PROV-O), the Micropublications semantic model, the Web Annotation Data Model (WADM), the Ontology of Adverse Events (OAE) and the Time ontology. To this end, we describe the design of OpenPVSignal and demonstrate its applicability as well as the reasoning capabilities enabled by its use. We also provide an evaluation of the model against the FAIR data principles. The applicability of OpenPVSignal is demonstrated by using PV signal information published in: (a) the World Health Organization's Pharmaceuticals Newsletter, (b) the Netherlands Pharmacovigilance Centre Lareb Web site and (c) the U.S. Food and Drug Administration (FDA) Drug Safety Communications, also available on the FDA Web site.

High-fidelity phenotyping: richness and freedom from bias

Electronic health record phenotyping is the use of raw electronic health record data to assert characterizations about patients. Researchers have been doing it since the beginning of biomedical informatics, under different names. Phenotyping will benefit from an increasing focus on fidelity, both in the sense of increasing richness, such as measured levels, degree or severity, timing, probability, or conceptual relationships, and in the sense of reducing bias. Research agendas should shift from merely improving binary assignment to studying and improving richer representations. The field is actively researching new temporal directions and abstract representations, including deep learning. The field would benefit from research in nonlinear dynamics, in combining mechanistic models with empirical data, including data assimilation, and in topology. The health care process produces substantial bias, and studying that bias explicitly rather than treating it as merely another source of noise would facilitate addressing it.

Accuracy of an automated knowledge base for identifying drug adverse reactions

INTRODUCTION

Drug safety researchers seek to know the degree of certainty with which a particular drug is associated with an adverse drug reaction. There are different sources of information used in pharmacovigilance to identify, evaluate, and disseminate medical product safety evidence including spontaneous reports, published peer-reviewed literature, and product labels. Automated data processing and classification using these evidence sources can greatly reduce the manual curation currently required to develop reference sets of positive and negative controls (i.e. drugs that cause adverse drug events and those that do not) to be used in drug safety research.

METHODS

In this paper we explore a method for automatically aggregating disparate sources of information together into a single repository, developing a predictive model to classify drug-adverse event relationships, and applying those predictions to a real world problem of identifying negative controls for statistical method calibration.

RESULTS

Our results showed high predictive accuracy for the models combining all available evidence, with an area under the receiver-operator curve of ⩾0.92 when tested on three manually generated lists of drugs and conditions that are known to either have or not have an association with an adverse drug event.

CONCLUSIONS

Results from a pilot implementation of the method suggests that it is feasible to develop a scalable alternative to the time-and-resource-intensive, manual curation exercise previously applied to develop reference sets of positive and negative controls to be used in drug safety research.

Learning statistical models of phenotypes using noisy labeled training data

OBJECTIVE

Traditionally, patient groups with a phenotype are selected through rule-based definitions whose creation and validation are time-consuming. Machine learning approaches to electronic phenotyping are limited by the paucity of labeled training datasets. We demonstrate the feasibility of utilizing semi-automatically labeled training sets to create phenotype models via machine learning, using a comprehensive representation of the patient medical record.

METHODS

We use a list of keywords specific to the phenotype of interest to generate noisy labeled training data. We train L1 penalized logistic regression models for a chronic and an acute disease and evaluate the performance of the models against a gold standard.

RESULTS

Our models for Type 2 diabetes mellitus and myocardial infarction achieve precision and accuracy of 0.90, 0.89, and 0.86, 0.89, respectively. Local implementations of the previously validated rule-based definitions for Type 2 diabetes mellitus and myocardial infarction achieve precision and accuracy of 0.96, 0.92 and 0.84, 0.87, respectively.We have demonstrated feasibility of learning phenotype models using imperfectly labeled data for a chronic and acute phenotype. Further research in feature engineering and in specification of the keyword list can improve the performance of the models and the scalability of the approach.

CONCLUSIONS

Our method provides an alternative to manual labeling for creating training sets for statistical models of phenotypes. Such an approach can accelerate research with large observational healthcare datasets and may also be used to create local phenotype models.

Preparing Nursing Home Data from Multiple Sites for Clinical Research - A Case Study Using Observational Health Data Sciences and Informatics

INTRODUCTION

A potential barrier to nursing home research is the limited availability of research quality data in electronic form. We describe a case study of converting electronic health data from five skilled nursing facilities to a research quality longitudinal dataset by means of open-source tools produced by the Observational Health Data Sciences and Informatics (OHDSI) collaborative.

METHODS

The Long-Term Care Minimum Data Set (MDS), drug dispensing, and fall incident data from five SNFs were extracted, translated, and loaded into version 4 of the OHDSI common data model. Quality assurance involved identifying errors using the Achilles data characterization tool and comparing both quality measures and drug exposures in the new database for concordance with externally available sources.

FINDINGS

Records for a total 4,519 patients (95.1%) made it into the final database. Achilles identified 10 different types of errors that were addressed in the final dataset. Drug exposures based on dispensing were generally accurate when compared with medication administration data from the pharmacy services provider. Quality measures were generally concordant between the new database and Nursing Home Compare for measures with a prevalence ≥ 10%. Fall data recorded in MDS was found to be more complete than data from fall incident reports.

CONCLUSIONS

The new dataset is ready to support observational research on topics of clinical importance in the nursing home including patient-level prediction of falls. The extraction, translation, and loading process enabled the use of OHDSI data characterization tools that improved the quality of the final dataset.

Clinical phenotyping in selected national networks: demonstrating the need for high-throughput, portable, and computational methods

OBJECTIVE

The combination of phenomic data from electronic health records (EHR) and clinical data repositories with dense biological data has enabled genomic and pharmacogenomic discovery, a first step toward precision medicine. Computational methods for the identification of clinical phenotypes from EHR data will advance our understanding of disease risk and drug response, and support the practice of precision medicine on a national scale.

METHODS

Based on our experience within three national research networks, we summarize the broad approaches to clinical phenotyping and highlight the important role of these networks in the progression of high-throughput phenotyping and precision medicine. We provide supporting literature in the form of a non-systematic review.

RESULTS

The practice of clinical phenotyping is evolving to meet the growing demand for scalable, portable, and data driven methods and tools. The resources required for traditional phenotyping algorithms from expert defined rules are significant. In contrast, machine learning approaches that rely on data patterns will require fewer clinical domain experts and resources.

CONCLUSIONS

Machine learning approaches that generate phenotype definitions from patient features and clinical profiles will result in truly computational phenotypes, derived from data rather than experts. Research networks and phenotype developers should cooperate to develop methods, collaboration platforms, and data standards that will enable computational phenotyping and truly modernize biomedical research and precision medicine.

Feasibility and utility of applications of the common data model to multiple, disparate observational health databases

OBJECTIVES

To evaluate the utility of applying the Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM) across multiple observational databases within an organization and to apply standardized analytics tools for conducting observational research.

MATERIALS AND METHODS

Six deidentified patient-level datasets were transformed to the OMOP CDM. We evaluated the extent of information loss that occurred through the standardization process. We developed a standardized analytic tool to replicate the cohort construction process from a published epidemiology protocol and applied the analysis to all 6 databases to assess time-to-execution and comparability of results.

RESULTS

Transformation to the CDM resulted in minimal information loss across all 6 databases. Patients and observations excluded were due to identified data quality issues in the source system, 96% to 99% of condition records and 90% to 99% of drug records were successfully mapped into the CDM using the standard vocabulary. The full cohort replication and descriptive baseline summary was executed for 2 cohorts in 6 databases in less than 1 hour.

DISCUSSION

The standardization process improved data quality, increased efficiency, and facilitated cross-database comparisons to support a more systematic approach to observational research. Comparisons across data sources showed consistency in the impact of inclusion criteria, using the protocol and identified differences in patient characteristics and coding practices across databases.

CONCLUSION

Standardizing data structure (through a CDM), content (through a standard vocabulary with source code mappings), and analytics can enable an institution to apply a network-based approach to observational research across multiple, disparate observational health databases.