The Future of the Patient-Centered Outcomes Research Institute (PCORI)

Patient-Centered Research Through Artificial Intelligence to Identify Priorities in Cancer Care

Importance

Patient-centered research is essential for bridging the gap between research and patient care, yet patient perspectives are often inadequately represented in health research.

Objective

To leverage artificial intelligence (AI) and natural language processing (NLP) to analyze a large dataset of patient messages, defining patient concerns and generating relevant research topics, and to quantify the quality of these AI-generated topics.

Design, Setting, and Participants

This case series was conducted using an automated framework involving a 2-staged unsupervised NLP topic model and AI-generated research topic suggestions. The study was based on deidentified patient portal message data from individuals with breast or skin cancer at Stanford Health Care and 22 affiliated centers over July 2013 to April 2024.

Exposures

A widely used large language model (ChatGPT-4o [OpenAI]; April 2024) was used and guided through multiple prompt-engineering strategies to perform multilevel tasks, including knowledge interpretation and summarization (eg, interpreting and summarizing the NLP-defined topics), knowledge generation (eg, generating research ideas corresponding to patients' issues), self-reflection and correction (eg, ensuring and revising the research ideas after searching for scientific articles), and self-reassurance (eg, confirming and finalizing the research ideas).

Main Outcomes and Measures

Three breast oncologists (J.L.C., A.W.K., F.R) and 3 dermatologists (K.Y.S, J.Y.T., E.L.) evaluated the meaningfulness and novelty of the AI-generated research topics using a 5-point Likert scale (1 representing exceptional to 5 representing poor). Mean (SD) scores for meaningfulness and novelty were computed for each topic.

Results

A total of 614 464 patient messages were analyzed from 25 549 individuals, 10 665 with breast cancer (98.6% female) and 14 884 had skin cancer (49.0% female). The overall mean (SD) scores for meaningfulness and novelty were 3.00 (0.50) and 3.29 (0.74), respectively, for breast cancer topics and 2.67 (0.45) and 3.09 (0.68), respectively, for skin cancer topics. One-third of the AI-suggested research topics were highly meaningful and novel when both scores were lower than the average (5 of 15 for breast cancer and 6 of 15 for skin cancer). Notably, two-thirds of the AI-suggested topics were novel (10 of 15 for breast cancer and 11 of 15 for skin cancer).

Conclusions and Relevance

This case series demonstrates that AI/NLP-driven analysis of large volumes of patient messages can generate quality research topics in cancer care that reflect patient perspectives, providing valuable guidance for future patient-centered health research endeavors.

Twenty-Five Years of Evolution and Hurdles in Electronic Health Records and Interoperability in Medical Research: Comprehensive Review

BACKGROUND

Electronic health records (EHRs) facilitate the accessibility and sharing of patient data among various health care providers, contributing to more coordinated and efficient care.

OBJECTIVE

This study aimed to summarize the evolution of secondary use of EHRs and their interoperability in medical research over the past 25 years.

METHODS

We conducted an extensive literature search in the PubMed, Scopus, and Web of Science databases using the keywords Electronic health record and Electronic medical record in the title or abstract and Medical research in all fields from 2000 to 2024. Specific terms were applied to different time periods.

RESULTS

The review yielded 2212 studies, all of which were then screened and processed in a structured manner. Of these 2212 studies, 2102 (93.03%) were included in the review analysis, of which 1079 (51.33%) studies were from 2000 to 2009, 582 (27.69%) were from 2010 to 2019, 251 (11.94%) were from 2020 to 2023, and 190 (9.04%) were from 2024.

CONCLUSIONS

The evolution of EHRs marks an important milestone in health care's journey toward integrating technology and medicine. From early documentation practices to the sophisticated use of artificial intelligence and big data analytics today, EHRs have become central to improving patient care, enhancing public health surveillance, and advancing medical research.

Intention to Use Different Formulations of Longer Acting HIV Pre-Exposure Prophylaxis Among Transgender and Gender Expansive Individuals: The Roles of Social Vulnerability and Medical Mistrust

In 2012, the Federal Drug Administration approved daily oral pre-exposure prophylaxis (PrEP) for HIV prevention in adults. Longer acting injectable PrEP (LA PrEP) has been approved and other formulations are in development. A successful LA PrEP rollout requires examining potential facilitators and barriers to PrEP uptake. Given that transgender and gender expansive (TGE) individuals experience more social vulnerability and higher levels of medical mistrust compared to other populations, examining the role of these two factors in LA PrEP uptake is important. This study, PrEP for ALL, is a community-based participatory research project in Texas that engaged TGE community members and organizational partners through a community advisory board. In total, 482 TGE individuals were recruited and responded to all relevant questions in an online survey, including their intentions to use three formulations: a monthly oral pill, a bimonthly intramuscular injection, and an annual subdermal implant. Multiple regression analysis was used to examine the influence of social vulnerability and medical mistrust on intention to use each LA PrEP formulation adjusting for other relevant factors. Findings suggest that individuals with higher levels of social vulnerability had greater intentions to use the monthly oral pill (β = 0.12, p = 0.009), the bimonthly intramuscular injection (β = 0.18, p < 0.001), and annual subdermal implant (β = 0.17, p < 0.001), whereas medical mistrust reduced intentions to use the bimonthly intramuscular injection (β = -0.18, p < 0.001) and annual subdermal implant (β = -0.11, p = 0.021). Improvements in gender-affirming clinical care are needed along with LA PrEP formulations that allow for greater autonomy and reduced clinical contact. Clinical Trial Registration number: NCT05044286.

Design and comparison of a hybrid to a traditional in-person point-of-care ultrasound course

Background

Traditional introductory point-of-care ultrasound (POCUS) courses are resource intensive, typically requiring 2–3 days at a remote site, consisting of lectures and hands-on components. Social distancing requirements resulting from the COVID-19 pandemic led us to create a novel hybrid course curriculum consisting of virtual and in-person components.

Methods

Faculty, chief residents, fellows and advanced practice providers (APPs) in the Department of Medicine were invited to participate in the hybrid curriculum. The course structure included 4 modules of recorded lectures, quizzes, online image interpretation sessions, online case discussions, and hands-on sessions at the bedside of course participant’s patients. The components of the course were delivered over approximately 8 months. Those participants who completed a minimum of 3 modules over the year were invited for final assessments. Results from the hybrid curriculum cohort were compared to the year-end data from a prior traditional in-person cohort.

Results

Participant knowledge scores were not different between traditional (n = 19) and hybrid (n = 24) groups (81% and 84%, respectively, P = 0.9). There was no change in POCUS skills as measured by the hands-on test from both groups at end-of-course (76% and 76%, respectively, P = 0.93). Confidence ratings were similar across groups from 2.73 traditional to 3.0 hybrid (out of possible 4, P = 0.46). Participants rated the course highly, with an average overall rating of 4.6 out 5.

Conclusions

A hybrid virtual and in-person POCUS course was highly rated and as successful as a traditional course in improving learner knowledge, hands-on skill and confidence at 8 months after course initiation. These results support expanding virtual elements of POCUS educational curricula.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13089-022-00261-x.

Development and Application of a Patient Group Engagement Prioritization Tool for Use in Medical Product Development

Introduction

Patient group engagement is increasingly used to inform the design, conduct, and dissemination of clinical trials and other medical research activities. However, the priorities of industry sponsors and patient groups differ, and there is currently no framework to help these groups identify mutually beneficial engagement activities.

Methods

We conducted 28 qualitative, semi-structured interviews with representatives from research sponsor organizations (n = 14) and patient groups (n = 14) to determine: (1) how representatives define benefits and investments of patient group engagement in medical product development, and (2) to refine a list of 31 predefined patient group engagement activities.

Results

Patient group and sponsor representatives described similar benefits: engagement activities can enhance the quality and efficiency of clinical trials by improving patient recruitment and retention, reduce costs, and help trials meet expectations of regulators and payers. All representatives indicated that investments include both dedicated staff time and expertise, and financial resources. Factors to consider when evaluating benefits and investments were also identified as were suggestions for clarifying the list of engagement activities.

Discussion

Using these findings, we refined the 31 engagement activities to 24 unique activities across the medical product development lifecycle. We also developed a web-based prioritization tool (https://prioritizationtool.ctti-clinicaltrials.org/) to help clinical research sponsors and patient groups identify high-priority engagement activities. Use of this tool can help sponsors and patient groups identify the engagement activities that they believe will provide the most benefit for the least investment and may lead to more meaningful and mutually beneficial partnerships in medical product development.

Electronic supplementary material

The online version of this article (10.1007/s43441-020-00217-0) contains supplementary material, which is available to authorized users.