Assessing function of electronic health records for real-world data generation

Data Quality-Driven Improvement in Health Care: Systematic Literature Review

BACKGROUND

The promise of real-world evidence and the learning health care system primarily depends on access to high-quality data. Despite widespread awareness of the prevalence and potential impacts of poor data quality (DQ), best practices for its assessment and improvement are unknown.

OBJECTIVE

This review aims to investigate how existing research studies define, assess, and improve the quality of structured real-world health care data.

METHODS

A systematic literature search of studies in the English language was implemented in the Embase and PubMed databases to select studies that specifically aimed to measure and improve the quality of structured real-world data within any clinical setting. The time frame for the analysis was from January 1945 to June 2023. We standardized DQ concepts according to the Data Management Association (DAMA) DQ framework to enable comparison between studies. After screening and filtering by 2 independent authors, we identified 39 relevant articles reporting DQ improvement initiatives.

RESULTS

The studies were characterized by considerable heterogeneity in settings and approaches to DQ assessment and improvement. Affiliated institutions were from 18 different countries and 18 different health domains. DQ assessment methods were largely manual and targeted completeness and 1 other DQ dimension. Use of DQ frameworks was limited to the Weiskopf and Weng (3/6, 50%) or Kahn harmonized model (3/6, 50%). Use of standardized methodologies to design and implement quality improvement was lacking, but mainly included plan-do-study-act (PDSA) or define-measure-analyze-improve-control (DMAIC) cycles. Most studies reported DQ improvements using multiple interventions, which included either DQ reporting and personalized feedback (24/39, 61%), IT-related solutions (21/39, 54%), training (17/39, 44%), improvements in workflows (5/39, 13%), or data cleaning (3/39, 8%). Most studies reported improvements in DQ through a combination of these interventions. Statistical methods were used to determine significance of treatment effect (22/39, 56% times), but only 1 study implemented a randomized controlled study design. Variability in study designs, approaches to delivering interventions, and reporting DQ changes hindered a robust meta-analysis of treatment effects.

CONCLUSIONS

There is an urgent need for standardized guidelines in DQ improvement research to enable comparison and effective synthesis of lessons learned. Frameworks such as PDSA learning cycles and the DAMA DQ framework can facilitate this unmet need. In addition, DQ improvement studies can also benefit from prioritizing root cause analysis of DQ issues to ensure the most appropriate intervention is implemented, thereby ensuring long-term, sustainable improvement. Despite the rise in DQ improvement studies in the last decade, significant heterogeneity in methodologies and reporting remains a challenge. Adopting standardized frameworks for DQ assessment, analysis, and improvement can enhance the effectiveness, comparability, and generalizability of DQ improvement initiatives.

Can requests for real-world evidence by the French HTA body be planned? An exhaustive retrospective case-control study of medicinal products appraisals from 2016 to 2021

OBJECTIVES

In France, decisions for pricing and reimbursement for medicinal products are based on appraisals performed by the National authority for health (Haute Autorité de Santé (HAS)). During the appraisal process, additional real-world evidence can be requested as "Post-Registration Studies" (PRS) when there are uncertainties in evidence that could be resolved by additional data collection. To facilitate PRS planning, a retrospective exploratory analysis was conducted to identify the characteristics of medicinal products associated with a PRS request.

METHODS

This analysis encompassed all appraisals finalized between January 1, 2016 and December 31, 2021 and compared products for which the appraisal led to a PRS request with those that did not.

RESULTS

Six hundred positive opinions for reimbursement were identified, with a PRS request present in 17 percent (n = 103) of cases. The independent characteristics associated with a PRS request were a mild or moderate clinical benefit score, a major to moderate or minor clinical added value score, previous availability under an early access program, and certain therapeutic areas (neurology, pulmonology, and endocrinology). These findings suggest two different profiles of PRS requests: (i) products for which there is uncertainty in the size of the clinical benefit and (ii) innovative products for which a substantial benefit is expected but uncertainties persist.

CONCLUSIONS

These results will assist health technology developers to better anticipate data generation to promptly address uncertainties identified by HAS. It may also help HAS and other assessment agencies to work together to improve postlaunch evidence generation according to the characteristics of the medicinal products.

Replication of Real-World Evidence in Oncology Using Electronic Health Record Data Extracted by Machine Learning

Meaningful real-world evidence (RWE) generation requires unstructured data found in electronic health records (EHRs) which are often missing from administrative claims; however, obtaining relevant data from unstructured EHR sources is resource-intensive. In response, researchers are using natural language processing (NLP) with machine learning (ML) techniques (i.e., ML extraction) to extract real-world data (RWD) at scale. This study assessed the quality and fitness-for-use of EHR-derived oncology data curated using NLP with ML as compared to the reference standard of expert abstraction. Using a sample of 186,313 patients with lung cancer from a nationwide EHR-derived de-identified database, we performed a series of replication analyses demonstrating some common analyses conducted in retrospective observational research with complex EHR-derived data to generate evidence. Eligible patients were selected into biomarker- and treatment-defined cohorts, first with expert-abstracted then with ML-extracted data. We utilized the biomarker- and treatment-defined cohorts to perform analyses related to biomarker-associated survival and treatment comparative effectiveness, respectively. Across all analyses, the results differed by less than 8% between the data curation methods, and similar conclusions were reached. These results highlight that high-performance ML-extracted variables trained on expert-abstracted data can achieve similar results as when using abstracted data, unlocking the ability to perform oncology research at scale.

Perspectives on emerging technologies, personalised medicine, and clinical research for cancer control in Latin America and the Caribbean

Challenges of health systems in Latin America and the Caribbean include accessibility, inequity, segmentation, and poverty. These challenges are similar in different countries of the region and transcend national borders. The increasing digital transformation of health care holds promise of more precise interventions, improved health outcomes, increased efficiency, and ultimately reduced health-care costs. In Latin America and the Caribbean, the adoption of digital health tools is in early stages and the quality of cancer registries, electronic health records, and structured databases are problematic. Cancer research and innovation in the region are limited due to inadequate academic resources and translational research is almost fully dependent on public funding. Regulatory complexity and extended timelines jeopardise the potential improvement in participation in international studies. Emerging technologies, artificial intelligence, big data, and cancer research represent an opportunity to address the health-care challenges in Latin America and the Caribbean collectively, by optimising national capacities, sharing and comparing best practices, and transferring scientific and technical capabilities.

Telogen effluvium in daily practice: Patient characteristics, laboratory parameters, and treatment modalities of 3028 patients with telogen effluvium

BACKGROUND

Telogen effluvium (TE) is a common form of non-scarring alopecia, characterized by excessive shedding of telogen club hairs.

OBJECTIVES

The aim of the present study was to investigate patient characteristics, laboratory parameters, and treatment strategies in TE.

METHODS

Electronic records of 3028 patients were retrospectively analyzed. Demographic and clinical data, as well as serum parameters screening for iron, vitamin B12, vitamin D, folate and zinc deficiencies, thyroid function, and ANA titers, were evaluated.

RESULTS

In the study group, the most frequently performed test type was serum ferritin level (82.3%), followed by complete blood count (81%), both of which revealed that 6.2% of the patients had iron deficiency anemia. 4.6% of the patients had thyroid dysfunction. In screened patients, vitamin and mineral deficiencies were as follows: vitamin D (72.2%), vitamin B12 (30.7%), folate (4.4%), and zinc (2.1%). Women were more likely to be prescribed vitamin D replacement therapy. Iron replacement was the most frequently ordered treatment, comprising 37.5% of total prescriptions.

CONCLUSION

To the best of our knowledge, this is the most comprehensive retrospective study having the largest number of patients with TE. Our results will not only help to augment knowledge about TE, but also provide a diagnostic algorithm for the laboratory and clinical workup of patients with TE.

Clinical decision support in a hospital electronic prescribing system informed by local data: experience at a tertiary New Zealand centre

BACKGROUND

An electronic prescribing and administration (ePA) system has been progressively rolled out to Canterbury District Health Board (CDHB, Christchurch, New Zealand) public hospitals since 2014, and is currently used for around 1300 tertiary beds. ePA data can be used to monitor user behaviour, and to evaluate and inform the local customisation of clinical decision support (CDS) tools within the ePA system.

AIMS

To describe retrospectively illustrative vignettes of CDHB ePA analyses that have been used for CDS.

METHODS

Alerts were developed according to a set of common principles agreed upon by the CDHB CDS Working Group. Alerts were informed and evaluated by extracting and parsing data for various time periods during 2016 to 2018 from the CDHB ePA database.

RESULTS

There was a median of 74 000 prescriptions a month. After examining 525 spironolactone prescriptions, the high dose alert threshold was set at 100 mg with an expected alert burden of 3%. The presence of a ceftriaxone shortage prescribing alert for 1 week was associated with a prescribing rate that was lower than 95% of the preceding 52 weeks. Following review of 367 fentanyl patch alerts, revision of the alert led to false positives falling from 43% to 3% (P < 0.0001). At the point of firing, 6% of antithrombotic drug interactions alerts led to immediate changes in prescriptions (94% overridden), and a further 22% were changed within 30 min after the alert.

CONCLUSIONS

Local data extracts from ePA systems can inform iterative configuration of the software and monitor user behaviour.

Improving Collection of Real-World Data: The Experience of the Joven & Fuerte Prospective Cohort for Mexican Young Women With Breast Cancer

The prospective collection of clinical data can generate detailed information on heterogeneous populations. This article reviews the strengths and limitations of the collection of real-world data and provides insight into the feasibility of routine collection of high-quality evidence even in a resource-constrained setting. The acquisition of high-quality data to assess the clinical and psychosocial needs of young Mexican patients with breast cancer has been enhanced through the use of preplanned, standardized data definitions and instrumentation to provide internally and externally comparable results, optimization of data collection with web-based surveys, engagement of participants to minimize missing data, and routine review for data consistency. A similar approach by other research groups could improve the quality of real-world data and accomplish enhanced inference of information.

Mobile apps for real-world evidence in health care

The use of real-world evidence for health care research and evaluation is growing. Mobile health apps have often-overlooked potential to contribute valuable real-world data that are not captured by other sources and could provide data that are more cost-effective and generalizable than can randomized controlled trials. However, there are several challenges that must be overcome to realize the potential value of patient-used mobile health app real-world data, including data quality, motivation for long-term use, privacy and security, methods of analysis, and standardization and integration. Addressing these challenges will increase the value of data from mobile health apps to inform real-world evidence and improve patient empowerment, clinical management, disease research, and treatment development.