Assessing the suitability of general practice electronic health records for clinical prediction model development: a data quality assessment

Generalizability of a Musculoskeletal Therapist Electronic Health Record for Modelling Outcomes to Work-Related Musculoskeletal Disorders

PURPOSE

Electronic Health Records (EHRs) can contain vast amounts of clinical information that could be reused in modelling outcomes of work-related musculoskeletal disorders (WMSDs). Determining the generalizability of an EHR dataset is an important step in determining the appropriateness of its reuse. The study aims to describe the EHR dataset used by occupational musculoskeletal therapists and determine whether the EHR dataset is generalizable to the Australian workers' population and injury characteristics seen in workers' compensation claims.

METHODS

Variables were considered if they were associated with outcomes of WMSDs and variables data were available. Completeness and external validity assessment analysed frequency distributions, percentage of records and confidence intervals.

RESULTS

There were 48,434 patient care plans across 10 industries from 2014 to 2021. The EHR collects information related to clinical interventions, health and psychosocial factors, job demands, work accommodations as well as workplace culture, which have all been shown to be valuable variables in determining outcomes to WMSDs. Distributions of age, duration of employment, gender and region of birth were mostly similar to the Australian workforce. Upper limb WMSDs were higher in the EHR compared to workers' compensation claims and diagnoses were similar.

CONCLUSION

The study shows the EHR has strong potential to be used for further research into WMSDs as it has a similar population to the Australian workforce, manufacturing industry and workers' compensation claims. It contains many variables that may be relevant in modelling outcomes to WMSDs that are not typically available in existing datasets.

Preconception health risk factors documented in general practice electronic medical records

BACKGROUND

Documenting medical and lifestyle preconception health risk factors in electronic medical records (EMRs) could assist general practitioners (GPs) to identify those reproductive-aged women who could most benefit from preconception care (PCC). However, it is unclear to what extent PCC risk factors are identifiable in general practice records. This study aimed to determine the extent to which medical and lifestyle preconception health risk factors are documented in general practice EMRs.

METHODS

We conducted an audit of the documentation of medical and lifestyle preconception risk factors in 10 general practice EMRs in Melbourne, Australia. We retrospectively analysed the EMRs of 100 consecutive women aged 18-44 years who visited each practice between January and September 2022. Using a template informed by PCC guidelines, we extracted data from structured fields in the EMR and conducted a descriptive analysis.

RESULTS

Among the data extracted, the more commonly documented medical and lifestyle preconception health risk factors in the EMRs included smoking (79%), blood pressure (74%), alcohol consumption (63%) and body mass index (57%). Among the women audited, 14% were smokers, 24% were obese, 7% had high blood pressure, 5% had diabetes, 28% had a mental health condition, 13% had asthma, 6% had thyroid disease and 17% had been prescribed and could be using a potentially teratogenic medication.

CONCLUSIONS

Better documentation of medical and lifestyle preconception health risk factors in structured fields in EMRs may potentially assist primary care providers including GPs in identifying and providing PCC to women who could most benefit from it.

Electronic health record data quality assessment and tools: a systematic review

OBJECTIVE

We extended a 2013 literature review on electronic health record (EHR) data quality assessment approaches and tools to determine recent improvements or changes in EHR data quality assessment methodologies.

MATERIALS AND METHODS

We completed a systematic review of PubMed articles from 2013 to April 2023 that discussed the quality assessment of EHR data. We screened and reviewed papers for the dimensions and methods defined in the original 2013 manuscript. We categorized papers as data quality outcomes of interest, tools, or opinion pieces. We abstracted and defined additional themes and methods though an iterative review process.

RESULTS

We included 103 papers in the review, of which 73 were data quality outcomes of interest papers, 22 were tools, and 8 were opinion pieces. The most common dimension of data quality assessed was completeness, followed by correctness, concordance, plausibility, and currency. We abstracted conformance and bias as 2 additional dimensions of data quality and structural agreement as an additional methodology.

DISCUSSION

There has been an increase in EHR data quality assessment publications since the original 2013 review. Consistent dimensions of EHR data quality continue to be assessed across applications. Despite consistent patterns of assessment, there still does not exist a standard approach for assessing EHR data quality.

CONCLUSION

Guidelines are needed for EHR data quality assessment to improve the efficiency, transparency, comparability, and interoperability of data quality assessment. These guidelines must be both scalable and flexible. Automation could be helpful in generalizing this process.

Data Quality in Health Care: Main Concepts and Assessment Methodologies

INTRODUCTION

 In the health care environment, a huge volume of data is produced on a daily basis. However, the processes of collecting, storing, sharing, analyzing, and reporting health data usually face with numerous challenges that lead to producing incomplete, inaccurate, and untimely data. As a result, data quality issues have received more attention than before.

OBJECTIVE

 The purpose of this article is to provide an insight into the data quality definitions, dimensions, and assessment methodologies.

METHODS

 In this article, a scoping literature review approach was used to describe and summarize the main concepts related to data quality and data quality assessment methodologies. Search terms were selected to find the relevant articles published between January 1, 2012 and September 31, 2022. The retrieved articles were then reviewed and the results were reported narratively.

RESULTS

 In total, 23 papers were included in the study. According to the results, data quality dimensions were various and different methodologies were used to assess them. Most studies used quantitative methods to measure data quality dimensions either in paper-based or computer-based medical records. Only two studies investigated respondents' opinions about data quality.

CONCLUSION

 In health care, high-quality data not only are important for patient care, but also are vital for improving quality of health care services and better decision making. Therefore, using technical and nontechnical solutions as well as constant assessment and supervision is suggested to improve data quality.

The Alliance for Healthier Communities as a Learning Health System for primary care: A qualitative analysis in Ontario, Canada

RATIONALE, AIMS AND OBJECTIVES

A learning health system model can be used to efficiently evaluate and incorporate evidence-based care into practice. However, there is a paucity of evidence describing key organizational attributes needed to ensure a successful learning health system within primary care. We interviewed stakeholders for a primary care learning health system in Ontario, Canada (the Alliance for Healthier Communities) to identify strengths and areas for improvement.

METHOD

We conducted a qualitative descriptive study using individual semistructured interviews with Alliance stakeholders between December 2019 and March 2020. The Alliance delivers community-governed primary healthcare through 109 organizations including Community Health Centres (CHCs). All CHC staff within the Alliance were invited to participate. Interviews were audio-recorded and transcribed verbatim. We performed a thematic analysis using a team approach.

RESULTS

We interviewed 29 participants across six CHCs, including Executive Directors, managers, healthcare providers and data support staff. We observed three foundational elements necessary for a successful learning health system within primary care: shared organizational goals and culture, data quality and resources. Building on this foundation, people are needed to drive the learning health system, and this is conditional on their level of engagement. The main factors motivating staff member's engagement with the learning health system included their drive to help improve patient care, focusing on initiatives of personal interest and understanding the purpose of different initiatives. Areas for improvement were identified such as the ability to extract and use data to inform changes in real-time, better engagement and protected time for providers to do improvement work, and more staff dedicated to data extraction and analysis.

CONCLUSIONS

We identified key components needed to establish a learning health system in primary care. Similar primary care organizations in Canada and elsewhere can use these insights to guide their development as learning health systems.

Describing a complex primary health care population to support future decision support initiatives

Introduction

Developing decision support tools using data from a health care organization, to support care within that organization, is a promising paradigm to improve care delivery and population health. Descriptive epidemiology may be a valuable supplement to stakeholder input towards selection of potential initiatives and to inform methodological decisions throughout tool development. We additionally propose that to properly characterize complex populations in large-scale descriptive studies, both simple statistical and machine learning techniques can be useful.

Objective

To describe sociodemographic, clinical, and health care use characteristics of primary care clients served by the Alliance for Healthier Communities, which provides team-based primary health care through Community Health Centres (CHCs) across Ontario, Canada.

Methods

We used electronic health record data from adult ongoing primary care clients served by CHCs in 2009-2019. We performed traditional table-based summaries for each characteristic; and applied three unsupervised learning techniques to explore patterns of common condition co-occurrence, care provider teams, and care frequency.

Results

There were 221,047 eligible clients. Sociodemographics: We described 13 characteristics, stratified by CHC type and client multimorbidity status. Clinical characteristics: Eleven-year prevalence of 24 investigated conditions ranged from 1% (Hepatitis C) to 63% (chronic musculoskeletal problem) with non-uniform risk across the care history; multimorbidity was common (81%) with variable co-occurrence patterns. Health care use characteristics: Most care was provided by physician and nursing providers, with heterogeneous combinations of other provider types. A subset of clients had many issues addressed within single-visits and there was within- and between-client variability in care frequency. In addition to substantive findings, we discuss methodological considerations for future decision support initiatives.

Conclusions

We demonstrated the use of methods from statistics and machine learning, applied with an epidemiological lens, to provide an overview of a complex primary care population and lay a foundation for stakeholder engagement and decision support tool development.

Is primary health care ready for artificial intelligence? What do primary health care stakeholders say?

Background

Effective deployment of AI tools in primary health care requires the engagement of practitioners in the development and testing of these tools, and a match between the resulting AI tools and clinical/system needs in primary health care. To set the stage for these developments, we must gain a more in-depth understanding of the views of practitioners and decision-makers about the use of AI in primary health care. The objective of this study was to identify key issues regarding the use of AI tools in primary health care by exploring the views of primary health care and digital health stakeholders.

Methods

This study utilized a descriptive qualitative approach, including thematic data analysis. Fourteen in-depth interviews were conducted with primary health care and digital health stakeholders in Ontario. NVivo software was utilized in the coding of the interviews.

Results

Five main interconnected themes emerged: (1) Mismatch Between Envisioned Uses and Current Reality—denoting the importance of potential applications of AI in primary health care practice, with a recognition of the current reality characterized by a lack of available tools; (2) Mechanics of AI Don’t Matter: Just Another Tool in the Toolbox– reflecting an interest in what value AI tools could bring to practice, rather than concern with the mechanics of the AI tools themselves; (3) AI in Practice: A Double-Edged Sword—the possible benefits of AI use in primary health care contrasted with fundamental concern about the possible threats posed by AI in terms of clinical skills and capacity, mistakes, and loss of control; (4) The Non-Starters: A Guarded Stance Regarding AI Adoption in Primary Health Care—broader concerns centred on the ethical, legal, and social implications of AI use in primary health care; and (5) Necessary Elements: Facilitators of AI in Primary Health Care—elements required to support the uptake of AI tools, including co-creation, availability and use of high quality data, and the need for evaluation.

Conclusion

The use of AI in primary health care may have a positive impact, but many factors need to be considered regarding its implementation. This study may help to inform the development and deployment of AI tools in primary health care.

DQAgui: a graphical user interface for the MIRACUM data quality assessment tool

Background

With the growing impact of observational research studies, there is also a growing focus on data quality (DQ). As opposed to experimental study designs, observational research studies are performed using data mostly collected in a non-research context (secondary use). Depending on the number of data elements to be analyzed, DQ reports of data stored within research networks can grow very large. They might be cumbersome to read and important information could be overseen quickly. To address this issue, a DQ assessment (DQA) tool with a graphical user interface (GUI) was developed and provided as a web application.

Methods

The aim was to provide an easy-to-use interface for users without prior programming knowledge to carry out DQ checks and to present the results in a clearly structured way. This interface serves as a starting point for a more detailed investigation of possible DQ irregularities. A user-centered development process ensured the practical feasibility of the interactive GUI. The interface was implemented in the R programming language and aligned to Kahn et al.’s DQ categories conformance, completeness and plausibility.

Results

With DQAgui, an R package with a web-app frontend for DQ assessment was developed. The GUI allows users to perform DQ analyses of tabular data sets and to systematically evaluate the results. During the development of the GUI, additional features were implemented, such as analyzing a subset of the data by defining time periods and restricting the analyses to certain data elements.

Conclusions

As part of the MIRACUM project, DQAgui is now being used at ten German university hospitals for DQ assessment and to provide a central overview of the availability of important data elements in a datamap over 2 years. Future development efforts should focus on design optimization and include a usability evaluation.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-022-01961-z.