Methods for enhancing the reproducibility of biomedical research findings using electronic health records

Determining prescriptions in electronic healthcare record data: methods for development of standardized, reproducible drug codelists

Objective

To develop a standardizable, reproducible method for creating drug codelists that incorporates clinical expertise and is adaptable to other studies and databases.

Materials and Methods

We developed methods to generate drug codelists and tested this using the Clinical Practice Research Datalink (CPRD) Aurum database, accounting for missing data in the database. We generated codelists for: (1) cardiovascular disease and (2) inhaled Chronic Obstructive Pulmonary Disease (COPD) therapies, applying them to a sample cohort of 335 931 COPD patients. We compared searching all drug dictionary variables (A) against searching only (B) chemical or (C) ontological variables.

Results

In Search A, we identified 165 150 patients prescribed cardiovascular drugs (49.2% of cohort), and 317 963 prescribed COPD inhalers (94.7% of cohort). Evaluating output per search strategy, Search C missed numerous prescriptions, including vasodilator anti-hypertensives (A and B:19 696 prescriptions; C:1145) and SAMA inhalers (A and B:35 310; C:564).

Discussion

We recommend the full search (A) for comprehensiveness. There are special considerations when generating adaptable and generalizable drug codelists, including fluctuating status, cohort-specific drug indications, underlying hierarchical ontology, and statistical analyses.

Conclusions

Methods must have end-to-end clinical input, and be standardizable, reproducible, and understandable to all researchers across data contexts.

Development and Validation of a Tool to Identify Patients Diagnosed With Castration-Resistant Prostate Cancer

PURPOSE

Several novel therapies for castration-resistant prostate cancer (CRPC) have been approved with randomized phase III studies with continuing observational research either planned or ongoing. Accurately identifying patients with CRPC in electronic health care data is critical for quality observational research, resource allocation, and quality improvement. Previous work in this area has relied on either structured laboratory results and medication data or natural language processing (NLP) methods. However, a computable phenotype using both structured data and NLP identifies these patients with more accuracy.

METHODS

The Corporate Data Warehouse (CDW) of the Veterans Health Administration (VHA) was used to collect PCa diagnoses, prostate-specific antigen test results, and information regarding patient characteristics and medication use. The final system used for validation and subsequent analysis combined the NLP system and an algorithm of structured laboratory and medication data to identify patients as being diagnosed with CRPC. Patients with both a documented diagnosis of CRPC and a documented diagnosis of metastatic PCa were classified as having mCRPC by this system.

RESULTS

Among 1.2 million veterans with PCa, the International Classification of Diseases (ICD)-10 diagnosis code for CRPC (Z19.2) identifies 3,791 patients from 2016 when the code was created until 2022, compared with the combined algorithm which identifies 14,103, 10,312 more than ICD-10 codes alone, from 2016 to 2022. The combined algorithm showed a sensitivity of 97.9% and a specificity of 99.2%.

CONCLUSION

ICD-10 codes proved to be insufficient for capturing CRPC in the VHA CDW data. Using both structured and unstructured data identified more than double the number of patients compared with ICD-10 codes alone. Application of this combined approach drastically improved identification of real-world patients and enables high-quality observational research in mCRPC.

Transforming and evaluating electronic health record disease phenotyping algorithms using the OMOP common data model: a case study in heart failure

Objective

The aim of the study was to transform a resource of linked electronic health records (EHR) to the OMOP common data model (CDM) and evaluate the process in terms of syntactic and semantic consistency and quality when implementing disease and risk factor phenotyping algorithms.

Materials and Methods

Using heart failure (HF) as an exemplar, we represented three national EHR sources (Clinical Practice Research Datalink, Hospital Episode Statistics Admitted Patient Care, Office for National Statistics) into the OMOP CDM 5.2. We compared the original and CDM HF patient population by calculating and presenting descriptive statistics of demographics, related comorbidities, and relevant clinical biomarkers.

Results

We identified a cohort of 502 536 patients with the incident and prevalent HF and converted 1 099 195 384 rows of data from 216 581 914 encounters across three EHR sources to the OMOP CDM. The largest percentage (65%) of unmapped events was related to medication prescriptions in primary care. The average coverage of source vocabularies was >98% with the exception of laboratory tests recorded in primary care. The raw and transformed data were similar in terms of demographics and comorbidities with the largest difference observed being 3.78% in the prevalence of chronic obstructive pulmonary disease (COPD).

Conclusion

Our study demonstrated that the OMOP CDM can successfully be applied to convert EHR linked across multiple healthcare settings and represent phenotyping algorithms spanning multiple sources. Similar to previous research, challenges mapping primary care prescriptions and laboratory measurements still persist and require further work. The use of OMOP CDM in national UK EHR is a valuable research tool that can enable large-scale reproducible observational research.

Big data in epilepsy: Clinical and research considerations. Report from the Epilepsy Big Data Task Force of the International League Against Epilepsy

Epilepsy is a heterogeneous condition with disparate etiologies and phenotypic and genotypic characteristics. Clinical and research aspects are accordingly varied, ranging from epidemiological to molecular, spanning clinical trials and outcomes, gene and drug discovery, imaging, electroencephalography, pathology, epilepsy surgery, digital technologies, and numerous others. Epilepsy data are collected in the terabytes and petabytes, pushing the limits of current capabilities. Modern computing firepower and advances in machine and deep learning, pioneered in other diseases, open up exciting possibilities for epilepsy too. However, without carefully designed approaches to acquiring, standardizing, curating, and making available such data, there is a risk of failure. Thus, careful construction of relevant ontologies, with intimate stakeholder inputs, provides the requisite scaffolding for more ambitious big data undertakings, such as an epilepsy data commons. In this review, we assess the clinical and research epilepsy landscapes in the big data arena, current challenges, and future directions, and make the case for a systematic approach to epilepsy big data.

Assessing the impact of introductory programming workshops on the computational reproducibility of biomedical workflows

INTRODUCTION

As biomedical research becomes more data-intensive, computational reproducibility is a growing area of importance. Unfortunately, many biomedical researchers have not received formal computational training and often struggle to produce results that can be reproduced using the same data, code, and methods. Programming workshops can be a tool to teach new computational methods, but it is not always clear whether researchers are able to use their new skills to make their work more computationally reproducible.

METHODS

This mixed methods study consisted of in-depth interviews with 14 biomedical researchers before and after participation in an introductory programming workshop. During the interviews, participants described their research workflows and responded to a quantitative checklist measuring reproducible behaviors. The interview data was analyzed using a thematic analysis approach, and the pre and post workshop checklist scores were compared to assess the impact of the workshop on the computational reproducibility of the researchers' workflows.

RESULTS

Pre and post scores on a checklist of reproducible behaviors did not change in a statistically significant manner. The qualitative interviews revealed that several participants had made small changes to their workflows including switching to open source programming languages for their data cleaning, analysis, and visualization. Overall many of the participants indicated higher levels of programming literacy, and an interest in further training. Factors that enabled change included supportive environments and an immediate research need, while barriers included collaborators that were resistant to new tools, and a lack of time.

CONCLUSION

While none of the workshop participants completely changed their workflows, many of them did incorporate new practices, tools, or methods that helped make their work more reproducible and transparent to other researchers. This indicates that programming workshops now offered by libraries and other organizations contribute to computational reproducibility training for researchers.

Electronic health records and polygenic risk scores for predicting disease risk

Accurate prediction of disease risk based on the genetic make-up of an individual is essential for effective prevention and personalized treatment. Nevertheless, to date, individual genetic variants from genome-wide association studies have achieved only moderate prediction of disease risk. The aggregation of genetic variants under a polygenic model shows promising improvements in prediction accuracies. Increasingly, electronic health records (EHRs) are being linked to patient genetic data in biobanks, which provides new opportunities for developing and applying polygenic risk scores in the clinic, to systematically examine and evaluate patient susceptibilities to disease. However, the heterogeneous nature of EHR data brings forth many practical challenges along every step of designing and implementing risk prediction strategies. In this Review, we present the unique considerations for using genotype and phenotype data from biobank-linked EHRs for polygenic risk prediction.

UK phenomics platform for developing and validating electronic health record phenotypes: CALIBER

OBJECTIVE

Electronic health records (EHRs) are a rich source of information on human diseases, but the information is variably structured, fragmented, curated using different coding systems, and collected for purposes other than medical research. We describe an approach for developing, validating, and sharing reproducible phenotypes from national structured EHR in the United Kingdom with applications for translational research.

MATERIALS AND METHODS

We implemented a rule-based phenotyping framework, with up to 6 approaches of validation. We applied our framework to a sample of 15 million individuals in a national EHR data source (population-based primary care, all ages) linked to hospitalization and death records in England. Data comprised continuous measurements (for example, blood pressure; medication information; coded diagnoses, symptoms, procedures, and referrals), recorded using 5 controlled clinical terminologies: (1) read (primary care, subset of SNOMED-CT [Systematized Nomenclature of Medicine Clinical Terms]), (2) International Classification of Diseases-Ninth Revision and Tenth Revision (secondary care diagnoses and cause of mortality), (3) Office of Population Censuses and Surveys Classification of Surgical Operations and Procedures, Fourth Revision (hospital surgical procedures), and (4) DM+D prescription codes.

RESULTS

Using the CALIBER phenotyping framework, we created algorithms for 51 diseases, syndromes, biomarkers, and lifestyle risk factors and provide up to 6 validation approaches. The EHR phenotypes are curated in the open-access CALIBER Portal (https://www.caliberresearch.org/portal) and have been used by 40 national and international research groups in 60 peer-reviewed publications.

CONCLUSIONS

We describe a UK EHR phenomics approach within the CALIBER EHR data platform with initial evidence of validity and use, as an important step toward international use of UK EHR data for health research.

One- and two-year visual outcomes from the Moorfields age-related macular degeneration database: a retrospective cohort study and an open science resource

OBJECTIVES

To analyse treatment outcomes and share clinical data from a large, single-centre, well-curated database (8174 eyes/6664 patients with 120 756 single entries) of patients with neovascular age-related macular degeneration (AMD) treated with anti-vascular endothelial growth factor (VEGF). By making our depersonalised raw data openly available, we aim to stimulate further research in AMD, as well as set a precedent for future work in this area.

SETTING

Retrospective, comparative, non-randomised electronic medical record (EMR) database cohort study of the UK Moorfields AMD database with data extracted between 2008 and 2018.

PARTICIPANTS

Including one eye per patient, 3357 eyes/patients (61% female). Extraction criteria were ≥1 ranibizumab or aflibercept injection, entry of 'AMD' in the diagnosis field of the EMR and a minimum of 1 year of follow-up. Exclusion criteria were unknown date of first injection and treatment outside of routine clinical care at Moorfields before the first recorded injection in the database.

MAIN OUTCOME MEASURES

Primary outcome measure was change in VA at 1 and 2 years from baseline as measured in Early Treatment Diabetic Retinopathy Study letters. Secondary outcomes were the number of injections and predictive factors for VA gain.

RESULTS

Mean VA gain at 1 year and 2 years were +5.5 (95% CI 5.0 to 6.0) and +4.9 (95% CI 4.2 to 5.6) letters, respectively. Fifty-four per cent of eyes gained ≥5 letters at 2 years, 63% had stable VA (±≤14 letters), 44% of eyes maintained good VA (≥70 letters). Patients received a mean of 7.7 (95% CI 7.6 to 7.8) injections during year 1 and 13.0 (95% CI 12.8 to 13.2) injections over 2 years. Younger age, lower baseline VA and more injections were associated with higher VA gain at 2 years.

CONCLUSION

This study benchmarks high quality EMR study results of real life AMD treatment and promotes open science in clinical AMD research by making the underlying data publicly available.

Code sets for respiratory symptoms in electronic health records research: a systematic review protocol

INTRODUCTION

Asthma and chronic obstructive pulmonary disease (COPD) are common respiratory conditions, which result in significant morbidity worldwide. These conditions are associated with a range of non-specific symptoms, which in themselves are a target for health research. Such research is increasingly being conducted using electronic health records (EHRs), but computable phenotype definitions, in the form of code sets or code lists, are required to extract structured data from these large routine databases in a systematic and reproducible way. The aim of this protocol is to specify a systematic review to identify code sets for respiratory symptoms in EHRs research.

METHODS AND ANALYSIS

MEDLINE and Embase databases will be searched using terms relating to EHRs, respiratory symptoms and use of code sets. The search will cover all English-language studies in these databases between January 1990 and December 2017. Two reviewers will independently screen identified studies for inclusion, and key data will be extracted into a uniform table, facilitating cross-comparison of codes used. Disagreements between the reviewers will be adjudicated by a third reviewer. This protocol has been produced in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocol guidelines.

ETHICS AND DISSEMINATION

As a review of previously published studies, no ethical approval is required. The results of this review will be submitted to a peer-reviewed journal for publication and can be used in future research into respiratory symptoms that uses electronic healthcare databases.

PROSPERO REGISTRATION NUMBER

CRD42018100830.