Representing and utilizing clinical textual data for real world studies: An OHDSI approach

Identifying patients with neurofibromatosis type 1 related optic pathway glioma using the OMOP CDM

Neurofibromatosis type 1 (NF1) is a rare tumour predisposition syndrome. Optic pathway gliomas (NF1-related OPG) are a well-characterised tumour type. There is great need for tools that can efficiently identify patients with NF1-related OPG at hospitals. Computable phenotypes algorithms can be used to find patients with certain clinical features in an electronic database. We developed computable phenotype algorithms using the Observational Medical Outcome Partnership (OMOP) Common Data Model. We subsequently assessed if these algorithms could identify patients with NF1-related OPG in an electronic health records (EHR) derived database. We created phenotype algorithms based on diagnosis codes, visits, and radiologic procedures. These phenotypes were applied to the EHR-derived database of an academic hospital. To assess the performance of the phenotypes, we calculated the precision, recall, and F2 score against a list of known cases (n = 61), provided by a clinician. To evaluate the ability of the phenotypes to identify additional cases, we manually reviewed the predicted positives of each phenotype algorithm. The phenotype algorithm based on the diagnosis codes 'Neurofibromatosis syndrome' and 'Neoplasm of optic nerve' performed best (precision = 1.000, recall = 0.614, F2-score = 0.665). The phenotype 'Neurofibromatosis syndrome and three or more Ophthalmology visits and one or more MRI of brain' performed best of the phenotypes based on visits and radiologic procedures (precision = 0.489, recall = 0.511, F2-score = 0.507). Generally, increased precision came at the cost of a decrease in recall. Following review of the predicted positives of each phenotype, 27 additional cases were identified. OMOP computable phenotype algorithms successfully identified NF1-related OPG patients in an EHR-derived database. They provided swift insight into the number of NF1-related OPG cases and were able to identify additional cases, which were not included in the original list of known cases. Phenotype algorithms created with OMOP could be an invaluable tool to facilitate patient screening, especially in multi-centric trials for rare diseases.

Generative Artificial Intelligence for Health Technology Assessment: Opportunities, Challenges, and Policy Considerations: An ISPOR Working Group Report

OBJECTIVES

To provide an introduction to the uses of generative artificial intelligence (AI) and foundation models, including large language models, in the field of health technology assessment (HTA).

METHODS

We reviewed applications of generative AI in 3 areas: systematic literature reviews, real-world evidence, and health economic modeling.

RESULTS

(1) Literature reviews: generative AI has the potential to assist in automating aspects of systematic literature reviews by proposing search terms, screening abstracts, extracting data, and generating code for meta-analyses; (2) real-world evidence: generative AI can facilitate automating processes and analyze large collections of real-world data, including unstructured clinical notes and imaging; (3) health economic modeling: generative AI can aid in the development of health economic models, from conceptualization to validation. Limitations in the use of foundation models and large language models include challenges surrounding their scientific rigor and reliability, the potential for bias, implications for equity, as well as nontrivial concerns regarding adherence to regulatory and ethical standards, particularly in terms of data privacy and security. Additionally, we survey the current policy landscape and provide suggestions for HTA agencies on responsibly integrating generative AI into their workflows, emphasizing the importance of human oversight and the fast-evolving nature of these tools.

CONCLUSIONS

Although generative AI technology holds promise with respect to HTA applications, it is still undergoing rapid developments and improvements. Continued careful evaluation of their applications to HTA is required. Both developers and users of research incorporating these tools, should familiarize themselves with their current capabilities and limitations.

Automated Integration of AI Results into Radiology Reports Using Common Data Elements

Integration of artificial intelligence (AI) into radiology practice can create opportunities to improve diagnostic accuracy, workflow efficiency, and patient outcomes. Integration demands the ability to seamlessly incorporate AI-derived measurements into radiology reports. Common data elements (CDEs) define standardized, interoperable units of information. This article describes the application of CDEs as a standardized framework to embed AI-derived results into radiology reports. The authors defined a set of CDEs for measurements of the volume and attenuation of the liver and spleen. An AI system segmented the liver and spleen on non-contrast CT images of the abdomen and pelvis, and it recorded their measurements as CDEs using the Digital Imaging and Communications in Medicine Structured Reporting (DICOM-SR) framework to express the corresponding labels and values. The AI system successfully segmented the liver and spleen in non-contrast CT images and generated measurements of organ volume and attenuation. Automated systems extracted corresponding CDE labels and values from the AI-generated data, incorporated CDE values into the radiology report, and transmitted the generated image series to the Picture Archiving and Communication System (PACS) for storage and display. This study demonstrates the use of radiology CDEs in clinical practice to record and transfer AI-generated data. This approach can improve communication among radiologists and referring providers, harmonize data to enable large-scale research efforts, and enhance the performance of decision support systems. CDEs ensure consistency, interoperability, and clarity in reporting AI findings across diverse healthcare systems.

Advancing Medical Imaging Research Through Standardization: The Path to Rapid Development, Rigorous Validation, and Robust Reproducibility

ABSTRACT

Artificial intelligence (AI) has made significant advances in radiology. Nonetheless, challenges in AI development, validation, and reproducibility persist, primarily due to the lack of high-quality, large-scale, standardized data across the world. Addressing these challenges requires comprehensive standardization of medical imaging data and seamless integration with structured medical data.Developed by the Observational Health Data Sciences and Informatics community, the OMOP Common Data Model enables large-scale international collaborations with structured medical data. It ensures syntactic and semantic interoperability, while supporting the privacy-protected distribution of research across borders. The recently proposed Medical Imaging Common Data Model is designed to encompass all DICOM-formatted medical imaging data and integrate imaging-derived features with clinical data, ensuring their provenance.The harmonization of medical imaging data and its seamless integration with structured clinical data at a global scale will pave the way for advanced AI research in radiology. This standardization will enable federated learning, ensuring privacy-preserving collaboration across institutions and promoting equitable AI through the inclusion of diverse patient populations. Moreover, it will facilitate the development of foundation models trained on large-scale, multimodal datasets, serving as powerful starting points for specialized AI applications. Objective and transparent algorithm validation on a standardized data infrastructure will enhance reproducibility and interoperability of AI systems, driving innovation and reliability in clinical applications.

Clinical Research Informatics: a Decade-in-Review

BACKGROUND

Clinical Research Informatics (CRI) is a subspeciality of biomedical informatics that has substantially matured during the last decade. Advances in CRI have transformed the way clinical research is conducted. In recent years, there has been growing interest in CRI, as reflected by a vast and expanding scientific literature focused on the topic. The main objectives of this review are: 1) to provide an overview of the evolving definition and scope of this biomedical informatics subspecialty over the past 10 years; 2) to highlight major contributions to the field during the past decade; and 3) to provide insights about more recent CRI research trends and perspectives.

METHODS

We adopted a modified thematic review approach focused on understanding the evolution and current status of the CRI field based on literature sources identified through two complementary review processes (AMIA CRI year-in-review/IMIA Yearbook of Medical Informatics) conducted annually during the last decade.

RESULTS

More than 1,500 potentially relevant publications were considered, and 205 sources were included in the final review. The review identified key publications defining the scope of CRI and/or capturing its evolution over time as illustrated by impactful tools and methods in different categories of CRI focus. The review also revealed current topics of interest in CRI and prevailing research trends.

CONCLUSION

This scoping review provides an overview of a decade of research in CRI, highlighting major changes in the core CRI discoveries as well as increasingly impactful methods and tools that have bridged the principles-to-practice gap. Practical CRI solutions as well as examples of CRI-enabled large-scale, multi-organizational and/or multi-national research projects demonstrate the maturity of the field. Despite the progress demonstrated, some topics remain challenging, highlighting the need for ongoing CRI development and research, including the need of more rigorous evaluations of CRI solutions and further formalization and maturation of CRI services and capabilities across the research enterprise.

Clinical Research Informatics: Contributions from 2023

OBJECTIVES

To summarize key contributions to current research in the field of Clinical Research Informatics (CRI) and to select the best papers published in 2023.

METHODS

A bibliographic search using a combination of MeSH descriptors and free-text terms on CRI was performed using PubMed, followed by a double-blind review in order to select a list of candidate best papers to be then peer-reviewed by external reviewers. After peer-review ranking, a consensus meeting between the two section editors and the editorial team was organized to finally conclude on the selected three best papers.

RESULTS

Among the 1,119 papers returned by the search, published in 2023, that were in the scope of the various areas of CRI, the full review process selected three best papers. The first best paper describes the process undertaken in Germany, under the national Medical Informatics Initiative, to define and validate a provenance metadata framework to enable the interpretation including quality assessment of health data reused for research. The authors of the second-best paper present a methodology for the generation of computable phenotypes and the covariates associated with success rates in e-phenotype validation. The third-best presents a review of published and accessible tools that enable the assessment of health data quality through an automated process. This year's survey paper marks the tenth anniversary of the CRI section of the Yearbook by reviewing the dominant themes within CRI over the past decade and the major milestone innovations within this field.

CONCLUSIONS

The literature relevant to CRI in 2023 has largely been populated by publications that assess and enhance the reusability of health data for clinical research, in particular data quality assessment and metadata management.

Large Language Models for Social Determinants of Health Information Extraction from Clinical Notes - A Generalizable Approach across Institutions

The consistent and persuasive evidence illustrating the influence of social determinants on health has prompted a growing realization throughout the health care sector that enhancing health and health equity will likely depend, at least to some extent, on addressing detrimental social determinants. However, detailed social determinants of health (SDoH) information is often buried within clinical narrative text in electronic health records (EHRs), necessitating natural language processing (NLP) methods to automatically extract these details. Most current NLP efforts for SDoH extraction have been limited, investigating on limited types of SDoH elements, deriving data from a single institution, focusing on specific patient cohorts or note types, with reduced focus on generalizability. This study aims to address these issues by creating cross-institutional corpora spanning different note types and healthcare systems, and developing and evaluating the generalizability of classification models, including novel large language models (LLMs), for detecting SDoH factors from diverse types of notes from four institutions: Harris County Psychiatric Center, University of Texas Physician Practice, Beth Israel Deaconess Medical Center, and Mayo Clinic. Four corpora of deidentified clinical notes were annotated with 21 SDoH factors at two levels: level 1 with SDoH factor types only and level 2 with SDoH factors along with associated values. Three traditional classification algorithms (XGBoost, TextCNN, Sentence BERT) and an instruction tuned LLM-based approach (LLaMA) were developed to identify multiple SDoH factors. Substantial variation was noted in SDoH documentation practices and label distributions based on patient cohorts, note types, and hospitals. The LLM achieved top performance with micro-averaged F1 scores over 0.9 on level 1 annotated corpora and an F1 over 0.84 on level 2 annotated corpora. While models performed well when trained and tested on individual datasets, cross-dataset generalization highlighted remaining obstacles. To foster collaboration, access to partial annotated corpora and models trained by merging all annotated datasets will be made available on the PhysioNet repository.

Development of Medical Imaging Data Standardization for Imaging-Based Observational Research: OMOP Common Data Model Extension

The rapid growth of artificial intelligence (AI) and deep learning techniques require access to large inter-institutional cohorts of data to enable the development of robust models, e.g., targeting the identification of disease biomarkers and quantifying disease progression and treatment efficacy. The Observational Medical Outcomes Partnership Common Data Model (OMOP CDM) has been designed to accommodate a harmonized representation of observational healthcare data. This study proposes the Medical Imaging CDM (MI-CDM) extension, adding two new tables and two vocabularies to the OMOP CDM to address the structural and semantic requirements to support imaging research. The tables provide the capabilities of linking DICOM data sources as well as tracking the provenance of imaging features derived from those images. The implementation of the extension enables phenotype definitions using imaging features and expanding standardized computable imaging biomarkers. This proposal offers a comprehensive and unified approach for conducting imaging research and outcome studies utilizing imaging features.

Making causal inferences from transactional data: A narrative review of opportunities and challenges when implementing the target trial framework

The target trial framework has emerged as a powerful tool for addressing causal questions in clinical practice and in public health. In the healthcare sector, where decision-making is increasingly data-driven, transactional databases, such as electronic health records (EHR) and insurance claims, present an untapped potential for answering complex causal questions. This narrative review explores the potential of the integration of the target trial framework with real-world data to enhance healthcare decision-making processes. We outline essential elements of the target trial framework, and identify pertinent challenges in data quality, privacy concerns, and methodological limitations, proposing solutions to overcome these obstacles and optimize the framework's application.

The Growing Impact of Natural Language Processing in Healthcare and Public Health

Natural Language Processing (NLP) is a subset of Artificial Intelligence, specifically focused on understanding and generating human language. NLP technologies are becoming more prevalent in healthcare and hold potential solutions to current problems. Some examples of existing and future uses include: public sentiment analysis in relation to health policies, electronic health record (EHR) screening, use of speech to text technology for extracting EHR data from point of care, patient communications, accelerated identification of eligible clinical trial candidates through automated searches and access of health data to assist in informed treatment decisions. This narrative review aims to summarize the current uses of NLP in healthcare, highlight successful implementation of computational linguistics-based approaches, and identify gaps, limitations, and emerging trends within the subfield of NLP in public health. The online databases Google Scholar and PubMed were scanned for papers published between 2018 and 2023. Keywords "Natural Language Processing, Health Policy, Large Language Models" were utilized in the initial search. Then, papers were limited to those written in English. Each of the 27 selected papers was subject to careful analysis, and their relevance in relation to NLP and healthcare respectively is utilized in this review. NLP and deep learning technologies scan large datasets, extracting valuable insights in various realms. This is especially significant in healthcare where huge amounts of data exist in the form of unstructured text. Automating labor intensive and tedious tasks with language processing algorithms, using text analytics systems and machine learning to analyze social media data and extracting insights from unstructured data allows for better public sentiment analysis, enhancement of risk prediction models, improved patient communication, and informed treatment decisions. In the recent past, some studies have applied NLP tools to social media posts to evaluate public sentiment regarding COVID-19 vaccine use. Social media data also has the capacity to be harnessed to develop pandemic prediction models based on reported symptoms. Furthermore, NLP has the potential to enhance healthcare delivery across the globe. Advanced language processing techniques such as Speech Recognition (SR) and Natural Language Understanding (NLU) tools can help overcome linguistic barriers and facilitate efficient communication between patients and healthcare providers.