Developing electronic data methods infrastructure to participate in collaborative research networks

Learning health systems in primary care: a systematic scoping review

BACKGROUND

Learning health systems have been gaining traction over the past decade. The purpose of this study was to understand the spread of learning health systems in primary care, including where they have been implemented, how they are operating, and potential challenges and solutions.

METHODS

We completed a scoping review by systematically searching OVID Medline®, Embase®, IEEE Xplore®, and reviewing specific journals from 2007 to 2020. We also completed a Google search to identify gray literature.

RESULTS

We reviewed 1924 articles through our database search and 51 articles from other sources, from which we identified 21 unique learning health systems based on 62 data sources. Only one of these learning health systems was implemented exclusively in a primary care setting, where all others were integrated health systems or networks that also included other care settings. Eighteen of the 21 were in the United States. Examples of how these learning health systems were being used included real-time clinical surveillance, quality improvement initiatives, pragmatic trials at the point of care, and decision support. Many challenges and potential solutions were identified regarding data, sustainability, promoting a learning culture, prioritization processes, involvement of community, and balancing quality improvement versus research.

CONCLUSIONS

We identified 21 learning health systems, which all appear at an early stage of development, and only one was primary care only. We summarized and provided examples of integrated health systems and data networks that can be considered early models in the growing global movement to advance learning health systems in primary care.

A method for interoperable knowledge-based data quality assessment

BACKGROUND

Assessing the quality of healthcare data is a complex task including the selection of suitable measurement methods (MM) and adequately assessing their results.

OBJECTIVES

To present an interoperable data quality (DQ) assessment method that formalizes MMs based on standardized data definitions and intends to support collaborative governance of DQ-assessment knowledge, e.g. which MMs to apply and how to assess their results in different situations.

METHODS

We describe and explain central concepts of our method using the example of its first real world application in a study on predictive biomarkers for rejection and other injuries of kidney transplants. We applied our open source tool-openCQA-that implements our method utilizing the openEHR specifications. Means to support collaborative governance of DQ-assessment knowledge are the version-control system git and openEHR clinical information models.

RESULTS

Applying the method on the study's dataset showed satisfactory practicability of the described concepts and produced useful results for DQ-assessment.

CONCLUSIONS

The main contribution of our work is to provide applicable concepts and a tested exemplary open source implementation for interoperable and knowledge-based DQ-assessment in healthcare that considers the need for flexible task and domain specific requirements.

A Data Driven Approach to Achieving High Value Healthcare

The purpose of this special issue is to disseminate learning from the High Value Healthcare Collaborative (HVHC). The HVHC is a voluntary, member-led organization based on trusted, working relationships among delivery system leaders. HVHC's mission is to be a provider-based learning health system committed to improving healthcare value through data, evidence, and collaboration. We begin by describing the organization and structure of HVHC in order to lay the context for a series of papers that feature work from this learning health system. HVHC was awarded a grant from the John and Laura Arnold Foundation to develop a generalizable model for dissemination and implementation. Implementation of the 3-hour sepsis bundle was used as a prototypic, complex intervention with an in-depth mixed methods evaluation across 16 member sites. The first four articles in this issue describe, in detail, various data and methodological challenges encountered together with strategies for overcoming these (see Knowlton et al., von Recklinghausen et al., Welch et al., and Taenzer et al.). Next, we illustrate how the Data Trust can support emerging questions relevant to member organizations. The paper by Albritton et al., explores the impact of observation stays on readmission rates. Knighton et al., explore the use of an area-based measure for health literacy to assess risk in disadvantaged populations. Two final papers illustrate the importance of fundamental data sources needed to support advanced data science.

Data Cleaning in the Evaluation of a Multi-Site Intervention Project

Context:

The High Value Healthcare Collaborative (HVHC) sepsis project was a two-year multi-site project where Member health care delivery systems worked on improving sepsis care using a dissemination & implementation framework designed by HVHC. As part of the project evaluation, participating Members provided 5 data submissions over the project period. Members created data files using a uniform specification, but the data sources and methods used to create the data sets differed. Extensive data cleaning was necessary to get a data set usable for the evaluation analysis.

Case Description:

HVHC was the coordinating center for the project and received and cleaned all data submissions. Submissions received 3 sequentially more detailed levels of checking by HVHC. The most detailed level evaluated validity by comparing values within-Member over time and between Member. For a subset of episodes Member-submitted data were compared to matched Medicare claims data.

Findings:

Inconsistencies in data submissions, particularly for length-of-stay variables were common in early submissions and decreased with subsequent submissions. Multiple resubmissions were sometimes required to get clean data. Data checking also uncovered a systematic difference in the way Medicare and some members defined intensive care unit stay.

Conclusions:

Data checking is a critical for ensuring valid analytic results for projects using electronic health record data. It is important to budget sufficient resources for data checking. Interim data submissions and checks help find anomalies early. Data resubmissions should be checked as fixes can introduce new errors. Communicating with those responsible for creating the data set provides critical information.

A Framework for Classification of Electronic Health Data Extraction-Transformation-Loading Challenges in Data Network Participation

BACKGROUND

Contributing health data to national, regional, and local networks or registries requires data stored in local systems with local structures and codes to be extracted, transformed, and loaded into a standard format called a Common Data Model (CDM). These processes called Extract, Transform, Load (ETL) require data partners or contributors to invest in costly technical resources with specialized skills in data models, terminologies, and programming. Given the wide range of tasks, skills, and technologies required to transform data into a CDM, a classification of ETL challenges can help identify needed resources, which in turn may encourage data partners with less-technical capabilities to participate in data-sharing networks.

METHODS

We conducted key-informant interviews with data partner representatives to survey the ETL challenges faced in clinical data research networks (CDRNs) and registries. A list of ETL challenges, organized into six themes was vetted during a one-day workshop with a wide range of network stakeholders including data partners, researchers, and policy experts.

RESULTS

We identified 24 technical ETL challenges related to the data sharing process. All of these ETL challenges were rated as "important" or "very important" by workshop participants using a five point Likert scale. Based on these findings, a framework for categorizing ETL challenges according to ETL phases, themes, and levels of data network participation was developed.

CONCLUSIONS

Overcoming ETL technical challenges require significant investments in a broad array of information technologies and human resources. Identifying these technical obstacles can inform optimal resource allocation to minimize the barriers and cost of entry for new data partners into extant networks, which in turn can expand data networks' inclusiveness and diversity. This paper offers pertinent information and guiding framework that are relevant for data partners in ascertaining challenges associated with contributing data in data networks.