Efficiency and effectiveness evaluation of an automated multi-country patient count cohort system

Leveraging the EHR4CR platform to support patient inclusion in academic studies: challenges and lessons learned

Background

The development of Electronic Health Records (EHRs) in hospitals offers the ability to reuse data from patient care activities for clinical research. EHR4CR is a European public-private partnership aiming to develop a computerized platform that enables the re-use of data collected from EHRs over its network. However, the reproducibility of queries may depend on attributes of the local data. Our objective was 1/ to describe the different steps that were achieved in order to use the EHR4CR platform and 2/ to identify the specific issues that could impact the final performance of the platform.

Methods

We selected three institutional studies covering various medical domains. The studies included a total of 67 inclusion and exclusion criteria and ran in two University Hospitals. We described the steps required to use the EHR4CR platform for a feasibility study. We also defined metrics to assess each of the steps (including criteria complexity, normalization quality, and data completeness of EHRs).

Results

We identified 114 distinct medical concepts from a total of 67 eligibility criteria Among the 114 concepts: 23 (20.2%) corresponded to non-structured data (i.e. for which transformation is needed before analysis), 92 (81%) could be mapped to terminologies used in EHR4CR, and 86 (75%) could be mapped to local terminologies. We identified 51 computable criteria following the normalization process. The normalization was considered by experts to be satisfactory or higher for 64.2% (43/67) of the computable criteria. All of the computable criteria could be expressed using the EHR4CR platform.

Conclusions

We identified a set of issues that could affect the future results of the platform: (a) the normalization of free-text criteria, (b) the translation into computer-friendly criteria and (c) issues related to the execution of the query to clinical data warehouses. We developed and evaluated metrics to better describe the platforms and their result. These metrics could be used for future reports of Clinical Trial Recruitment Support Systems assessment studies, and provide experts and readers with tools to insure the quality of constructed dataset.

Electronic supplementary material

The online version of this article (doi:10.1186/s12874-017-0299-3) contains supplementary material, which is available to authorized users.

Cross border semantic interoperability for learning health systems: The EHR4CR semantic resources and services

With the development of platforms enabling the integration and use of phenome, genome, and exposome data in the context of international research, data management challenges are increasing, and scalable solutions for cross border and cross domain semantic interoperability need to be developed. Reusing routinely collected clinical data, especially, requires computable portable phenotype algorithms running across different electronic health record (EHR) products and healthcare systems. We propose a framework for describing and comparing mediation platforms enabling cross border phenotype identification within federated EHRs. This framework was used to describe the experience gained during the EHR4CR project and the evaluation of the platform developed for accessing semantically equivalent data elements across 11 European participating EHR systems from 5 countries. Developers of semantic interoperability platforms are beginning to address a core set of requirements in order to reach the goal of developing cross border semantic integration of data.

Clinical Research Informatics Contributions from 2015

OBJECTIVES

To summarize key contributions to current research in the field of Clinical Research Informatics (CRI) and to select best papers published in 2015.

METHOD

A bibliographic search using a combination of MeSH and free terms search over PubMed on Clinical Research Informatics (CRI) was performed followed by a double-blind review in order to select a list of candidate best papers to be then peer-reviewed by external reviewers. A consensus meeting between the two section editors and the editorial team was finally organized to conclude on the selection of best papers.

RESULTS

Among the 579 returned papers published in the past year in the various areas of Clinical Research Informatics (CRI) - i) methods supporting clinical research, ii) data sharing and interoperability, iii) re-use of healthcare data for research, iv) patient recruitment and engagement, v) data privacy, security and regulatory issues and vi) policy and perspectives - the full review process selected four best papers. The first selected paper evaluates the capability of the Clinical Data Interchange Standards Consortium (CDISC) Operational Data Model (ODM) to support the representation of case report forms (in both the design stage and with patient level data) during a complete clinical study lifecycle. The second selected paper describes a prototype for secondary use of electronic health records data captured in non-standardized text. The third selected paper presents a privacy preserving electronic health record linkage tool and the last selected paper describes how big data use in US relies on access to health information governed by varying and often misunderstood legal requirements and ethical considerations.

CONCLUSIONS

A major trend in the 2015 publications is the analysis of observational, "nonexperimental" information and the potential biases and confounding factors hidden in the data that will have to be carefully taken into account to validate new predictive models. In addiction, researchers have to understand complicated and sometimes contradictory legal requirements and to consider ethical obligations in order to balance privacy and promoting discovery.

User Satisfaction Evaluation of the EHR4CR Query Builder: A Multisite Patient Count Cohort System

The Electronic Health Records for Clinical Research (EHR4CR) project aims to develop services and technology for the leverage reuse of Electronic Health Records with the purpose of improving the efficiency of clinical research processes. A pilot program was implemented to generate evidence of the value of using the EHR4CR platform. The user acceptance of the platform is a key success factor in driving the adoption of the EHR4CR platform; thus, it was decided to evaluate the user satisfaction. In this paper, we present the results of a user satisfaction evaluation for the EHR4CR multisite patient count cohort system. This study examined the ability of testers (n = 22 and n = 16 from 5 countries) to perform three main tasks (around 20 minutes per task), after a 30-minute period of self-training. The System Usability Scale score obtained was 55.83 (SD: 15.37), indicating a moderate user satisfaction. The responses to an additional satisfaction questionnaire were positive about the design of the interface and the required procedure to design a query. Nevertheless, the most complex of the three tasks proposed in this test was rated as difficult, indicating a need to improve the system regarding complicated queries.