Transparent Reporting on Research Using Unstructured Electronic Health Record Data to Generate ‘Real World’ Evidence of Comparative Effectiveness and Safety

Transparency in the secondary use of health data: assessing the status quo of guidance and best practices

We evaluated what guidance exists in the literature to improve the transparency of studies that make secondary use of health data. To find peer-reviewed papers, we searched PubMed and Google Scholar. To find institutional documents, we used our personal expertise to draft a list of health organizations and searched their websites. We quantitatively and qualitatively coded different types of research transparency: registration, methods reporting, results reporting, data sharing and code sharing. We found 56 documents that provide recommendations to improve the transparency of studies making secondary use of health data, mainly in relation to study registration (n = 27) and/or methods reporting (n = 39). Only three documents made recommendations on data sharing or code sharing. Recommendations for study registration and methods reporting mainly came in the form of structured documents like registration templates and reporting guidelines. Aside from the recommendations aimed directly at researchers, we also found recommendations aimed at the wider research community, typically on how to improve research infrastructure. Limitations or challenges of improving transparency were rarely mentioned, highlighting the need for more nuance in providing transparency guidance for studies that make secondary use of health data.

Real-World Evidence BRIDGE: A Tool to Connect Protocol With Code Programming

OBJECTIVE

To enhance documentation on programming decisions in Real World Evidence (RWE) studies.

MATERIALS AND METHODS

We analyzed several statistical analysis plans (SAP) within the Vaccine Monitoring Collaboration for Europe (VAC4EU) to identify study design sections and specifications for programming RWE studies. We designed a machine-readable metadata schema containing study sections, codelists, and time anchoring definitions specified in the SAPs with adaptability and user-friendliness.

RESULTS

We developed the RWE-BRIDGE, a metadata schema in form of relational database divided into four study design sections with 12 tables: Study Variable Definition (two tables), Cohort Definition (two tables), Post-Exposure Outcome Analysis (one table), and Data Retrieval (seven tables). We provide a guide to populate this metadata schema and a Shiny app that checks the tables. RWE-BRIDGE is available on GitHub (github.com/UMC-Utrecht-RWE/RWE-BRIDGE).

DISCUSSION

The RWE-BRIDGE has been designed to support the translation of study design sections from statistical analysis plans into analytical pipelines and to adhere to the FAIR principles, facilitating collaboration and transparency between researcher and programmers. This metadata schema strategy is flexible as it can support different common data models and programming languages, and it is adaptable to the specific needs of each SAP by adding further tables or fields, if necessary. Modified versions of the RWE-BRIGE have been applied in several RWE studies within VAC4EU.

CONCLUSION

RWE-BRIDGE offers a systematic approach to detailing variables, time anchoring, and algorithms for RWE studies. This metadata schema facilitates communication between researcher and programmers.

Natural Language Processing for Smart Healthcare

Smart healthcare has achieved significant progress in recent years. Emerging artificial intelligence (AI) technologies enable various smart applications across various healthcare scenarios. As an essential technology powered by AI, natural language processing (NLP) plays a key role in smart healthcare due to its capability of analysing and understanding human language. In this work, we review existing studies that concern NLP for smart healthcare from the perspectives of technique and application. We first elaborate on different NLP approaches and the NLP pipeline for smart healthcare from the technical point of view. Then, in the context of smart healthcare applications employing NLP techniques, we introduce representative smart healthcare scenarios, including clinical practice, hospital management, personal care, public health, and drug development. We further discuss two specific medical issues, i.e., the coronavirus disease 2019 (COVID-19) pandemic and mental health, in which NLP-driven smart healthcare plays an important role. Finally, we discuss the limitations of current works and identify the directions for future works.

Translating clinical notes into quantitative measures-a real-world observation on the response to cholinesterase inhibitors or selective serotonin reuptake inhibitors prescribed to outpatients with dementia using electronic medical records

Objective: Cholinesterase inhibitors (CEIs) are prescribed for dementia to maintain or improve memory. Selective serotonin reuptake inhibitors (SSRIs) are also prescribed to manage psychiatric symptoms seen in dementia. What proportion of outpatients actually responds to these drugs is still unclear. Our objective was to investigate the responder rates of these medications in an outpatient setting using the electronic medical record (EMR). Methods: We used the Johns Hopkins EMR system to identify patients with dementia who were prescribed a CEI or SSRI for the first time between 2010 and 2021. Treatment effects were assessed through routinely documented clinical notes and free-text entries in which healthcare providers record clinical findings and impressions of patients. Responses were scored using a three-point Likert scale named the NOte-based evaluation method for Treatment Efficacy (NOTE) in addition to the Clinician's Interview-Based Impression of Change Plus caregiver input (CIBIC-plus), a seven-point Likert scale used in clinical trials. To validate NOTE, the relationships between NOTE and CIBIC-plus and between NOTE and change in MMSE (Mini-Mental State Examination) before and after medication were examined. Inter-rater reliability was evaluated using Krippendorff's alpha. The responder rates were calculated. Results: NOTE showed excellent inter-rater reliability and correlated well with CIBIC-plus and changes in MMSEs. Out of 115 CEI cases, 27.0% reported improvement and 34.8% reported stable symptoms in cognition; out of 225 SSRI cases, 69.3% reported an improvement in neuropsychiatric symptoms. Conclusion: NOTE showed high validity in measuring the pharmacotherapy effects based on unstructured clinical entries. Although our real-world observation included various types of dementia, the results were remarkably similar to what was reported in controlled clinical trials of Alzheimer's disease and its related neuropsychiatric symptoms.

Real-world treatment response in Japanese patients with cancer using unstructured data from electronic health records

Purpose

We generated methods for evaluating clinical outcomes including treatment response in oncology using the unstructured data from electronic health records (EHR) in Japanese language.

Methods

This retrospective analysis used medical record database and administrative data of University of Miyazaki Hospital in Japan of patients with lung/breast cancer. Treatment response (objective response [OR], stable disease [SD] or progressive disease [PD]) was adjudicated by two evaluators using clinicians’ progress notes, radiology reports and pathological reports of 15 patients with lung cancer (training data set). For assessing key terms to describe treatment response, natural language processing (NLP) rules were created from the texts identified by the evaluators and broken down by morphological analysis. The NLP rules were applied for assessing data of other 70 lung cancer and 30 breast cancer patients, who were not adjudicated, to examine if any difference in using key terms exist between these patients.

Results

A total of 2,039 records in progress notes, 131 in radiology reports and 60 in pathological reports of 15 patients, were adjudicated. Progress notes were the most common primary source data for treatment assessment (60.7%), wherein, the most common key terms with high sensitivity and specificity to describe OR were “reduction/shrink”, for SD were “(no) remarkable change/(no) aggravation)” and for PD were “(limited) effect” and “enlargement/grow”. These key terms were also found in other larger cohorts of 70 patients with lung cancer and 30 patients with breast cancer.

Conclusion

This study demonstrated that assessing response to anticancer therapy using Japanese EHRs is feasible by interpreting progress notes, radiology reports and Japanese key terms using NLP.

An overview of real-world data sources for oncology and considerations for research

Generating evidence on the use, effectiveness, and safety of new cancer therapies is a priority for researchers, health care providers, payers, and regulators given the rapid pace of change in cancer diagnosis and treatments. The use of real-world data (RWD) is integral to understanding the utilization patterns and outcomes of these new treatments among patients with cancer who are treated in clinical practice and community settings. An initial step in the use of RWD is careful study design to assess the suitability of an RWD source. This pivotal process can be guided by using a conceptual model that encourages predesign conceptualization. The primary types of RWD included are electronic health records, administrative claims data, cancer registries, and specialty data providers and networks. Careful consideration of each data type is necessary because they are collected for a specific purpose, capturing a set of data elements within a certain population for that purpose, and they vary by population coverage and longitudinality. In this review, the authors provide a high-level assessment of the strengths and limitations of each data category to inform data source selection appropriate to the study question. Overall, the development and accessibility of RWD sources for cancer research are rapidly increasing, and the use of these data requires careful consideration of composition and utility to assess important questions in understanding the use and effectiveness of new therapies.

Safety of medicines and vaccines - building next generation capability

The systematic safety surveillance of real-world use of medicinal products and related activities (pharmacovigilance) started in earnest as a scientific field only in the 1960s. While developments have occurred over the past 50 years, adding to its complexity and sophistication, the extent to which some of these advances have positively impacted the capability for ensuring patient safety is questionable. We review how the conduct of safety surveillance has changed, highlight recent scientific advances, and argue how they need to be harnessed to enhance pharmacovigilance in the future. Specifically, we describe five changes that we believe should and will need to happen globally in the coming years: (i) better, more diverse data used for safety; (ii) the switch from manual activities to automation; (iii) removal of limited value, extraneous transactional activities and replacement with sharpened focus on scientific efforts to improve patient safety; (iv) patient-involved and focussed safety; and (v) personalised safety.

Invited Commentary: Code Review-An Important Step Toward Reproducible Research

In this issue of the Journal, Vable et al. (Am J Epidemiol. 2021;190(10):2172-2177) discuss a systematic approach to code review as a way to improve reproducibility in epidemiologic research. Reproducibility needs to become a cornerstone of our work. In the present commentary, I discuss some of the implications of their proposal, other methods to reduce coding mistakes, and other methods to improve reproducibility in research in general. Finally, I discuss the fact that no one of these approaches is sufficient on its own; rather, these different steps need to become part of a culture that prioritizes reproducibility in research.

Measuring the Effectiveness of Real-World Evidence to Ensure Appropriate Impact

The value of real-world evidence (RWE) in medicines regulation and health technology assessment has been increasingly emphasized. Nevertheless, although RWE is increasingly used, there has been limited systematic evidence of its value. A recent study that examined the role and impact of RWE in regulatory assessments conducted through the European Medicines Agency provided such evidence. Results of the study demonstrated RWE was important to decision making, particularly for certain questions such as the quantification of adverse events, the evaluation of risk minimization measures, and the assessment of product usage. The study suggested, however, that in many of the assessments further RWE would have been valuable and concluded that RWE has, as yet, played a limited role in hypothesis generation and in the assessment of medication effectiveness. This study had been possible only because of the transparency of the European Medicines Agency decision making. Ensuring transparency of RWE evidence collection, study design and conduct, and of decision making based on this evidence will facilitate further development of the uses and value of RWE. Keywords: benefit-risk assessment; medicines regulation; real-world evidence; regulatory decision making.

Study designs and statistical approaches to suicide and prevention research in real-world data

OBJECTIVE

To provide researchers, clinicians and policy makers with a primer to study designs, statistical approaches and graphical reporting methods for suicide research in real world data (RWD).

METHODS

Study designs, statistical method and graphical reporting standards are detailed with examples from the recently published literature.

RESULTS

Data sources and codes for identifying suicidal behavior are described. Study designs are described in detail for post-market surveillance, retrospective cohort studies, case control and nested case-control studies, and self-controlled (within-individual) studies including applications of marginal structural models. Graphical reporting of designs is described using an original research study.

CONCLUSIONS

Compared to RCTs, RWE studies offer larger sample sizes, greater generalizability, and real-world validity. However, these non-experimental data risk uncontrolled confounding and potential introduction of bias unless data, design and statistical approaches are rigorously aligned.

Artificial Intelligence, Real-World Automation and the Safety of Medicines

Despite huge technological advances in the capabilities to capture, store, link and analyse data electronically, there has been some but limited impact on routine pharmacovigilance. We discuss emerging research in the use of artificial intelligence, machine learning and automation across the pharmacovigilance lifecycle including pre-licensure. Reasons are provided on why adoption is challenging and we also provide a perspective on changes needed to accelerate adoption, and thereby improve patient safety. Last, we make clear that while technologies could be superimposed on existing pharmacovigilance processes for incremental improvements, these great societal advances in data and technology also provide us with a timely opportunity to reconsider everything we do in pharmacovigilance operations to maximise the benefit of these advances.

User Preferences for the Design of a Mobile Health System to Support Transition-Age Youth With Epilepsy

BACKGROUND

Transition-age youth with epilepsy (TAYWE) experience poor self management and adverse health outcomes. The purpose of this study was to gain the perspectives of TAYWE, their caregivers, and clinicians to inform the design of a mobile health (mHealth) system to support the self-management needs of TAYWE.

METHODS

Individual semi-structured interviews and focus groups were conducted with TAYWE, their caregivers, and clinicians who manage their care.

RESULTS

Sixteen TAYWE and seven caregivers participated in focus group sessions, and four clinicians were interviewed. Participants expressed the need for an mHealth system that addressed privacy, supervision of caregiver involvement, a user-friendly system design, and motivation to sustain ongoing use. Three themes evolved: current mobile app use, mHealth systems features and functions, and implementation concerns.

DISCUSSION

Data from this study informs the design of an mHealth system to support self-management in TAYWE and identifies important areas for practitioners to address when providing health care to TAYWE.