1
|
Craig KJ, Ji YJ, Zhang YC, Berk A, Zaleski A, Abdelsamad O, Coetzer H, Verbrugge DJ, Hua G. Real-world Application of Racial and Ethnic Imputation and Cohort Balancing Techniques to Deliver Equitable Clinical Trial Recruitment. AMIA Annu Symp Proc 2024; 2023:319-328. [PMID: 38222354 PMCID: PMC10785904] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Subscribe] [Scholar Register] [Indexed: 01/16/2024]
Abstract
Enhancing diversity and inclusion in clinical trial recruitment, especially for historically marginalized populations including Black, Indigenous, and People of Color individuals, is essential. This practice ensures that generalizable trial results are achieved to deliver safe, effective, and equitable health and healthcare. However, recruitment is limited by two inextricably linked barriers - the inability to recruit and retain enough trial participants, and the lack of diversity amongst trial populations whereby racial and ethnic groups are underrepresented when compared to national composition. To overcome these barriers, this study describes and evaluates a framework that combines 1) probabilistic and machine learning models to accurately impute missing race and ethnicity fields in real-world data including medical and pharmacy claims for the identification of eligible trial participants, 2) randomized controlled trial experimentation to deliver an optimal patient outreach strategy, and 3) stratified sampling techniques to effectively balance cohorts to continuously improve engagement and recruitment metrics.
Collapse
Affiliation(s)
- Kelly J Craig
- Clinical Evidence Development, Aetna Medical Affairs, CVS Health, Wellesley, MA, US
| | | | | | | | - Amanda Zaleski
- Clinical Evidence Development, Aetna Medical Affairs, CVS Health, Wellesley, MA, US
| | | | | | - Dorothea J Verbrugge
- Clinical Evidence Development, Aetna Medical Affairs, CVS Health, Wellesley, MA, US
| | - Guangying Hua
- Clinical Trial Services, CVS Health, Woonsocket, RI, US
| |
Collapse
|
2
|
Simon GE, Shortreed SM, Johnson E, Yaseen ZS, Stone M, Mosholder AD, Ahmedani BK, Coleman KJ, Coley RY, Penfold RB, Toh S. Predicting risk of suicidal behavior from insurance claims data vs. linked data from insurance claims and electronic health records. Pharmacoepidemiol Drug Saf 2024; 33:e5734. [PMID: 38112287 PMCID: PMC10843611 DOI: 10.1002/pds.5734] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/19/2023] [Revised: 10/16/2023] [Accepted: 11/10/2023] [Indexed: 12/21/2023]
Abstract
PURPOSE Observational studies assessing effects of medical products on suicidal behavior often rely on health record data to account for pre-existing risk. We assess whether high-dimensional models predicting suicide risk using data derived from insurance claims and electronic health records (EHRs) are superior to models using data from insurance claims alone. METHODS Data were from seven large health systems identified outpatient mental health visits by patients aged 11 or older between 1/1/2009 and 9/30/2017. Data for the 5 years prior to each visit identified potential predictors of suicidal behavior typically available from insurance claims (e.g., mental health diagnoses, procedure codes, medication dispensings) and additional potential predictors available from EHRs (self-reported race and ethnicity, responses to Patient Health Questionnaire or PHQ-9 depression questionnaires). Nonfatal self-harm events following each visit were identified from insurance claims data and fatal self-harm events were identified by linkage to state mortality records. Random forest models predicting nonfatal or fatal self-harm over 90 days following each visit were developed in a 70% random sample of visits and validated in a held-out sample of 30%. Performance of models using linked claims and EHR data was compared to models using claims data only. RESULTS Among 15 845 047 encounters by 1 574 612 patients, 99 098 (0.6%) were followed by a self-harm event within 90 days. Overall classification performance did not differ between the best-fitting model using all data (area under the receiver operating curve or AUC = 0.846, 95% CI 0.839-0.854) and the best-fitting model limited to data available from insurance claims (AUC = 0.846, 95% CI 0.838-0.853). Competing models showed similar classification performance across a range of cut-points and similar calibration performance across a range of risk strata. Results were similar when the sample was limited to health systems and time periods where PHQ-9 depression questionnaires were recorded more frequently. CONCLUSION Investigators using health record data to account for pre-existing risk in observational studies of suicidal behavior need not limit that research to databases including linked EHR data.
Collapse
Affiliation(s)
- Gregory E Simon
- Kaiser Permanente Washington Health Research Institute, Seattle, Washington, USA
- Department of Health Systems Science, Bernard J. Tyson Kaiser Permanente School of Medicine, Pasadena, California, USA
| | - Susan M Shortreed
- Kaiser Permanente Washington Health Research Institute, Seattle, Washington, USA
- Department of Biostatistics, University of Washington, Seattle, Washington, USA
| | - Eric Johnson
- Kaiser Permanente Washington Health Research Institute, Seattle, Washington, USA
| | - Zimri S Yaseen
- U.S. Food and Drug Administration, Silver Spring, Maryland, USA
| | - Marc Stone
- U.S. Food and Drug Administration, Silver Spring, Maryland, USA
| | | | - Brian K Ahmedani
- Center for Health Policy and Health Services Research, Henry Ford Health, Detroit, Michigan, USA
| | - Karen J Coleman
- Department of Health Systems Science, Bernard J. Tyson Kaiser Permanente School of Medicine, Pasadena, California, USA
- Department of Research and Evaluation, Kaiser Permanente Southern California, Pasadena, California, USA
| | - R Yates Coley
- Kaiser Permanente Washington Health Research Institute, Seattle, Washington, USA
- Department of Biostatistics, University of Washington, Seattle, Washington, USA
| | - Robert B Penfold
- Kaiser Permanente Washington Health Research Institute, Seattle, Washington, USA
| | - Sengwee Toh
- Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, Boston, Massachusetts, USA
| |
Collapse
|
3
|
Leviton A, Loddenkemper T. Design, implementation, and inferential issues associated with clinical trials that rely on data in electronic medical records: a narrative review. BMC Med Res Methodol 2023; 23:271. [PMID: 37974111 PMCID: PMC10652539 DOI: 10.1186/s12874-023-02102-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/17/2022] [Accepted: 11/08/2023] [Indexed: 11/19/2023] Open
Abstract
Real world evidence is now accepted by authorities charged with assessing the benefits and harms of new therapies. Clinical trials based on real world evidence are much less expensive than randomized clinical trials that do not rely on "real world evidence" such as contained in electronic health records (EHR). Consequently, we can expect an increase in the number of reports of these types of trials, which we identify here as 'EHR-sourced trials.' 'In this selected literature review, we discuss the various designs and the ethical issues they raise. EHR-sourced trials have the potential to improve/increase common data elements and other aspects of the EHR and related systems. Caution is advised, however, in drawing causal inferences about the relationships among EHR variables. Nevertheless, we anticipate that EHR-CTs will play a central role in answering research and regulatory questions.
Collapse
Affiliation(s)
- Alan Leviton
- Department of Neurology, Boston Children's Hospital, Harvard Medical School, Boston, MA, USA.
| | - Tobias Loddenkemper
- Department of Neurology, Boston Children's Hospital, Harvard Medical School, Boston, MA, USA
| |
Collapse
|
4
|
Jenssen BP, Schnoll R, Beidas RS, Bekelman J, Bauer AM, Evers-Casey S, Fisher T, Scott C, Nicoloso J, Gabriel P, Asch DA, Buttenheim AM, Chen J, Melo J, Grant D, Horst M, Oyer R, Shulman LN, Clifton AB, Lieberman A, Salam T, Rendle KA, Chaiyachati KH, Shelton RC, Fayanju O, Wileyto EP, Ware S, Blumenthal D, Ragusano D, Leone FT. Cluster Randomized Pragmatic Clinical Trial Testing Behavioral Economic Implementation Strategies to Improve Tobacco Treatment for Patients With Cancer Who Smoke. J Clin Oncol 2023; 41:4511-4521. [PMID: 37467454 PMCID: PMC10552951 DOI: 10.1200/jco.23.00355] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2023] [Revised: 05/25/2023] [Accepted: 06/15/2023] [Indexed: 07/21/2023] Open
Abstract
PURPOSE Few cancer centers systematically engage patients with evidence-based tobacco treatment despite its positive effect on quality of life and survival. Implementation strategies directed at patients, clinicians, or both may increase tobacco use treatment (TUT) within oncology. METHODS We conducted a four-arm cluster-randomized pragmatic trial across 11 clinical sites comparing the effect of strategies informed by behavioral economics on TUT engagement during oncology encounters with cancer patients. We delivered electronic health record (EHR)-based nudges promoting TUT across four nudge conditions: patient only, clinician only, patient and clinician, or usual care. Nudges were designed to counteract cognitive biases that reduce TUT engagement. The primary outcome was TUT penetration, defined as the proportion of patients with documented TUT referral or a medication prescription in the EHR. Generalized estimating equations were used to estimate the parameters of a linear model. RESULTS From June 2021 to July 2022, we randomly assigned 246 clinicians in 95 clusters, and collected TUT penetration data from their encounters with 2,146 eligible patients who smoke receiving oncologic care. Intent-to-treat (ITT) analysis showed that the clinician nudge led to a significant increase in TUT penetration versus usual care (35.6% v 13.5%; OR = 3.64; 95% CI, 2.52 to 5.24; P < .0001). Completer-only analysis (N = 1,795) showed similar impact (37.7% clinician nudge v 13.5% usual care; OR = 3.77; 95% CI, 2.73 to 5.19; P < .0001). Clinician type affected TUT penetration, with physicians less likely to provide TUT than advanced practice providers (ITT OR = 0.67; 95% CI, 0.51 to 0.88; P = .004). CONCLUSION EHR nudges, informed by behavioral economics and aimed at oncology clinicians, appear to substantially increase TUT penetration. Adding patient nudges to the implementation strategy did not affect TUT penetration rates.
Collapse
Affiliation(s)
- Brian P. Jenssen
- Department of Pediatrics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA
| | - Robert Schnoll
- Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA
- Abramson Cancer Center, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA
| | - Rinad S. Beidas
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA
- Department of Medical Social Sciences, Feinberg School of Medicine, Northwestern University, Chicago, IL
| | - Justin Bekelman
- Department of Radiation Oncology, Hospital of the University of Pennsylvania, Philadelphia, PA
| | - Anna-Marika Bauer
- Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA
| | - Sarah Evers-Casey
- Comprehensive Smoking Treatment Program, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA
| | - Tierney Fisher
- Comprehensive Smoking Treatment Program, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA
| | - Callie Scott
- Abramson Cancer Center, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA
| | - Jody Nicoloso
- Comprehensive Smoking Treatment Program, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA
| | - Peter Gabriel
- Abramson Cancer Center, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA
| | - David A. Asch
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA
| | - Alison M. Buttenheim
- Department of Family and Community Health, School of Nursing, University of Pennsylvania, Philadelphia, PA
| | - Jessica Chen
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA
| | - Julissa Melo
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA
| | - Dwayne Grant
- Penn Medicine Lancaster General Health, Lancaster, PA
| | - Michael Horst
- Penn Medicine Lancaster General Health, Lancaster, PA
| | - Randall Oyer
- Penn Medicine Lancaster General Health, Lancaster, PA
| | - Lawrence N. Shulman
- Abramson Cancer Center, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA
| | - Alicia B.W. Clifton
- Abramson Cancer Center, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA
| | - Adina Lieberman
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA
| | - Tasnim Salam
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA
| | - Katharine A. Rendle
- Department of Family Medicine and Community Health, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA
- Center for Clinical Epidemiology and Biostatistics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA
| | - Krisda H. Chaiyachati
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA
- Verily Life Sciences, San Francisco, CA
| | - Rachel C. Shelton
- Department of Sociomedical Sciences, Mailman School of Public Health, Columbia University, New York, NY
| | - Oluwadamilola Fayanju
- Department of Surgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA
| | - E. Paul Wileyto
- Center for Clinical Epidemiology and Biostatistics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA
| | - Sue Ware
- Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA
| | - Daniel Blumenthal
- Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA
| | - Daniel Ragusano
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA
| | - Frank T. Leone
- Pulmonary, Allergy, & Critical Care, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA
| |
Collapse
|
5
|
Venkatesh SS, Ganjgahi H, Palmer DS, Coley K, Wittemans LBL, Nellaker C, Holmes C, Lindgren CM, Nicholson G. The genetic architecture of changes in adiposity during adulthood. medRxiv 2023:2023.01.09.23284364. [PMID: 36711652 PMCID: PMC9882550 DOI: 10.1101/2023.01.09.23284364] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Indexed: 01/13/2023]
Abstract
Obesity is a heritable disease, characterised by excess adiposity that is measured by body mass index (BMI). While over 1,000 genetic loci are associated with BMI, less is known about the genetic contribution to adiposity trajectories over adulthood. We derive adiposity-change phenotypes from 1.5 million primary-care health records in over 177,000 individuals in UK Biobank to study the genetic architecture of weight-change. Using multiple BMI measurements over time increases power to identify genetic factors affecting baseline BMI. In the largest reported genome-wide study of adiposity-change in adulthood, we identify novel associations with BMI-change at six independent loci, including rs429358 (a missense variant in APOE). The SNP-based heritability of BMI-change (1.98%) is 9-fold lower than that of BMI, and higher in women than in men. The modest genetic correlation between BMI-change and BMI (45.2%) indicates that genetic studies of longitudinal trajectories could uncover novel biology driving quantitative trait values in adulthood.
Collapse
Affiliation(s)
- Samvida S. Venkatesh
- Wellcome Centre for Human Genetics, Nuffield Department of Medicine, University of Oxford, UK
- Big Data Institute at the Li Ka Shing Centre for Health Information and Discovery, University of Oxford, UK
| | | | - Duncan S. Palmer
- Big Data Institute at the Li Ka Shing Centre for Health Information and Discovery, University of Oxford, UK
- Nuffield Department of Women’s and Reproductive Health, Medical Sciences Division, University of Oxford, UK
| | - Kayesha Coley
- Department of Population Health Sciences, University of Leicester, UK
| | - Laura B. L. Wittemans
- Big Data Institute at the Li Ka Shing Centre for Health Information and Discovery, University of Oxford, UK
- Nuffield Department of Women’s and Reproductive Health, Medical Sciences Division, University of Oxford, UK
| | - Christoffer Nellaker
- Big Data Institute at the Li Ka Shing Centre for Health Information and Discovery, University of Oxford, UK
- Nuffield Department of Women’s and Reproductive Health, Medical Sciences Division, University of Oxford, UK
| | - Chris Holmes
- Department of Statistics, University of Oxford, UK
- Nuffield Department of Medicine, Medical Sciences Division, University of Oxford, UK
- The Alan Turing Institute, London, UK
| | - Cecilia M. Lindgren
- Wellcome Centre for Human Genetics, Nuffield Department of Medicine, University of Oxford, UK
- Big Data Institute at the Li Ka Shing Centre for Health Information and Discovery, University of Oxford, UK
- Nuffield Department of Women’s and Reproductive Health, Medical Sciences Division, University of Oxford, UK
- Broad Institute of Harvard and MIT, Cambridge, Massachusetts, USA
| | | |
Collapse
|
6
|
Wartko PD, Qiu H, Idu AE, Yu O, McCormack J, Matthews AG, Bobb JF, Saxon AJ, Campbell CI, Liu D, Braciszewski JM, Murphy SM, Burganowski RP, Murphy MT, Horigian VE, Hamilton LK, Lee AK, Boudreau DM, Bradley KA. Baseline representativeness of patients in clinics enrolled in the PRimary care Opioid Use Disorders treatment (PROUD) trial: comparison of trial and non-trial clinics in the same health systems. BMC Health Serv Res 2022; 22:1593. [PMID: 36581845 PMCID: PMC9801668 DOI: 10.1186/s12913-022-08915-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2022] [Accepted: 11/30/2022] [Indexed: 12/31/2022] Open
Abstract
BACKGROUND Pragmatic primary care trials aim to test interventions in "real world" health care settings, but clinics willing and able to participate in trials may not be representative of typical clinics. This analysis compared patients in participating and non-participating clinics from the same health systems at baseline in the PRimary care Opioid Use Disorders treatment (PROUD) trial. METHODS This observational analysis relied on secondary electronic health record and administrative claims data in 5 of 6 health systems in the PROUD trial. The sample included patients 16-90 years at an eligible primary care visit in the 3 years before randomization. Each system contributed 2 randomized PROUD trial clinics and 4 similarly sized non-trial clinics. We summarized patient characteristics in trial and non-trial clinics in the 2 years before randomization ("baseline"). Using mixed-effect regression models, we compared trial and non-trial clinics on a baseline measure of the primary trial outcome (clinic-level patient-years of opioid use disorder (OUD) treatment, scaled per 10,000 primary care patients seen) and a baseline measure of the secondary trial outcome (patient-level days of acute care utilization among patients with OUD). RESULTS Patients were generally similar between the 10 trial clinics (n = 248,436) and 20 non-trial clinics (n = 341,130), although trial clinics' patients were slightly younger, more likely to be Hispanic/Latinx, less likely to be white, more likely to have Medicaid/subsidized insurance, and lived in less wealthy neighborhoods. Baseline outcomes did not differ between trial and non-trial clinics: trial clinics had 1.0 more patient-year of OUD treatment per 10,000 patients (95% CI: - 2.9, 5.0) and a 4% higher rate of days of acute care utilization than non-trial clinics (rate ratio: 1.04; 95% CI: 0.76, 1.42). CONCLUSIONS trial clinics and non-trial clinics were similar regarding most measured patient characteristics, and no differences were observed in baseline measures of trial primary and secondary outcomes. These findings suggest trial clinics were representative of comparably sized clinics within the same health systems. Although results do not reflect generalizability more broadly, this study illustrates an approach to assess representativeness of clinics in future pragmatic primary care trials.
Collapse
Affiliation(s)
- Paige D Wartko
- Kaiser Permanente Washington Health Research Institute, 1730 Minor Ave, Ste 1600, Seattle, WA, 98101, United States.
| | - Hongxiang Qiu
- Kaiser Permanente Washington Health Research Institute, 1730 Minor Ave, Ste 1600, Seattle, WA, 98101, United States
- Department of Biostatistics, University of Washington, 1705 NE Pacific Street, Seattle, WA, 98195, United States
- Department of Statistics and Data Science, University of Pennsylvania, 3451 Walnut St Philadelphia, Philadelphia, PA, 19104, United States
| | - Abisola E Idu
- Kaiser Permanente Washington Health Research Institute, 1730 Minor Ave, Ste 1600, Seattle, WA, 98101, United States
| | - Onchee Yu
- Kaiser Permanente Washington Health Research Institute, 1730 Minor Ave, Ste 1600, Seattle, WA, 98101, United States
| | - Jennifer McCormack
- The Emmes Company, 401 N Washington St #700, Rockville, MD, 20850, United States
| | - Abigail G Matthews
- The Emmes Company, 401 N Washington St #700, Rockville, MD, 20850, United States
| | - Jennifer F Bobb
- Kaiser Permanente Washington Health Research Institute, 1730 Minor Ave, Ste 1600, Seattle, WA, 98101, United States
| | - Andrew J Saxon
- Center of Excellence in Substance Addiction Treatment and Education, VA Puget Sound Health Care System, 1660 S Columbian Way, Seattle, WA, 98108, United States
| | - Cynthia I Campbell
- Kaiser Permanente Northern California Division of Research, 2000 Broadway, Oakland, CA, 94612, United States
| | - David Liu
- National Institute on Drug Abuse Center for Clinical Trials Network, Three White Flint North, 11601 Landsdown Street, North Bethesda, MD, 20852, United States
| | | | - Sean M Murphy
- Department of Population Health Sciences, Weill Cornell Medical College, 1300 York Ave, New York, NY, 10065, United States
| | - Rachael P Burganowski
- Kaiser Permanente Washington Health Research Institute, 1730 Minor Ave, Ste 1600, Seattle, WA, 98101, United States
| | - Mark T Murphy
- MultiCare Health System, 315 Martin Luther King Jr. Way, Tacoma, WA, 98415, United States
| | - Viviana E Horigian
- Department of Public Health Sciences, Miller School of Medicine, University of Miami, 1120 NW 14th St, CRB 919, Miami, FL, 33136, United States
| | - Leah K Hamilton
- Kaiser Permanente Washington Health Research Institute, 1730 Minor Ave, Ste 1600, Seattle, WA, 98101, United States
| | - Amy K Lee
- Kaiser Permanente Washington Health Research Institute, 1730 Minor Ave, Ste 1600, Seattle, WA, 98101, United States
| | - Denise M Boudreau
- Kaiser Permanente Washington Health Research Institute, 1730 Minor Ave, Ste 1600, Seattle, WA, 98101, United States
- Genentech, 1 DNA Way, South San Francisco, CA, 94080, United States
| | - Katharine A Bradley
- Kaiser Permanente Washington Health Research Institute, 1730 Minor Ave, Ste 1600, Seattle, WA, 98101, United States
| |
Collapse
|
7
|
Mitra N, Roy J, Small D. The Future of Causal Inference. Am J Epidemiol 2022; 191:1671-1676. [PMID: 35762132 PMCID: PMC9991894 DOI: 10.1093/aje/kwac108] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/01/2022] [Revised: 06/16/2022] [Accepted: 06/17/2022] [Indexed: 01/29/2023] Open
Abstract
The past several decades have seen exponential growth in causal inference approaches and their applications. In this commentary, we provide our top-10 list of emerging and exciting areas of research in causal inference. These include methods for high-dimensional data and precision medicine, causal machine learning, causal discovery, and others. These methods are not meant to be an exhaustive list; instead, we hope that this list will serve as a springboard for stimulating the development of new research.
Collapse
Affiliation(s)
- Nandita Mitra
- Correspondence to Dr. Nandita Mitra, Department of Biostatistics, Epidemiology and Informatics, University of Pennsylvania, 423 Guardian Drive, Philadelphia, PA (e-mail: )
| | | | | |
Collapse
|
8
|
Rogers JR, Pavisic J, Ta CN, Liu C, Soroush A, Cheung YK, Hripcsak G, Weng C. Leveraging electronic health record data for clinical trial planning by assessing eligibility criteria's impact on patient count and safety. J Biomed Inform 2022; 127:104032. [PMID: 35189334 PMCID: PMC8920749 DOI: 10.1016/j.jbi.2022.104032] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/18/2021] [Revised: 02/14/2022] [Accepted: 02/15/2022] [Indexed: 10/19/2022]
Abstract
OBJECTIVE To present an approach on using electronic health record (EHR) data that assesses how different eligibility criteria, either individually or in combination, can impact patient count and safety (exemplified by all-cause hospitalization risk) and further assist with criteria selection for prospective clinical trials. MATERIALS AND METHODS Trials in three disease domains - relapsed/refractory (r/r) lymphoma/leukemia; hepatitis C virus (HCV); stages 3 and 4 chronic kidney disease (CKD) - were analyzed as case studies for this approach. For each disease domain, criteria were identified and all criteria combinations were used to create EHR cohorts. Per combination, two values were derived: (1) number of eligible patients meeting the selected criteria; (2) hospitalization risk, measured as the hazard ratio between those that qualified and those that did not. From these values, k-means clustering was applied to derive which criteria combinations maximized patient counts but minimized hospitalization risk. RESULTS Criteria combinations that reduced hospitalization risk without substantial reductions on patient counts were as follows: for r/r lymphoma/leukemia (23 trials; 9 criteria; 623 patients), applying no infection and adequate absolute neutrophil count while forgoing no prior malignancy; for HCV (15; 7; 751), applying no human immunodeficiency virus and no hepatocellular carcinoma while forgoing no decompensated liver disease/cirrhosis; for CKD (10; 9; 23893), applying no congestive heart failure. CONCLUSIONS Within each disease domain, the more drastic effects were generally driven by a few criteria. Similar criteria across different disease domains introduce different changes. Although results are contingent on the trial sample and the EHR data used, this approach demonstrates how EHR data can inform the impact on safety and available patients when exploring different criteria combinations for designing clinical trials.
Collapse
Affiliation(s)
- James R. Rogers
- Department of Biomedical Informatics, Columbia University, New York, NY
| | - Jovana Pavisic
- Department of Pediatrics, Division of Pediatric Hematology, Oncology, and Stem Cell Transplantation, Columbia University Irving Medical Center, New York, NY
| | - Casey N. Ta
- Department of Biomedical Informatics, Columbia University, New York, NY
| | - Cong Liu
- Department of Biomedical Informatics, Columbia University, New York, NY
| | - Ali Soroush
- Department of Biomedical Informatics, Columbia University, New York, NY,Division of Gastroenterology, Department of Medicine, Columbia University Irving Medical Center, New York, NY
| | | | - George Hripcsak
- Department of Biomedical Informatics, Columbia University, New York, NY,Medical Informatics Services, New York-Presbyterian Hospital, New York, NY
| | - Chunhua Weng
- Department of Biomedical Informatics, Columbia University, New York, NY, United States.
| |
Collapse
|
9
|
Simon GE, Shortreed SM, Rossom RC, Beck A, Clarke GN, Whiteside U, Richards JE, Penfold RB, Boggs JM, Smith J. Effect of Offering Care Management or Online Dialectical Behavior Therapy Skills Training vs Usual Care on Self-harm Among Adult Outpatients With Suicidal Ideation: A Randomized Clinical Trial. JAMA 2022; 327:630-638. [PMID: 35166800 PMCID: PMC8848197 DOI: 10.1001/jama.2022.0423] [Citation(s) in RCA: 30] [Impact Index Per Article: 15.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/14/2022]
Abstract
IMPORTANCE People at risk of self-harm or suicidal behavior can be accurately identified, but effective prevention will require effective scalable interventions. OBJECTIVE To compare 2 low-intensity outreach programs with usual care for prevention of suicidal behavior among outpatients who report recent frequent suicidal thoughts. DESIGN, SETTING, AND PARTICIPANTS Pragmatic randomized clinical trial including outpatients reporting frequent suicidal thoughts identified using routine Patient Health Questionnaire depression screening at 4 US integrated health systems. A total of 18 882 patients were randomized between March 2015 and September 2018, and ascertainment of outcomes continued through March 2020. INTERVENTIONS Patients were randomized to a care management intervention (n = 6230) that included systematic outreach and care, a skills training intervention (n = 6227) that introduced 4 dialectical behavior therapy skills (mindfulness, mindfulness of current emotion, opposite action, and paced breathing), or usual care (n = 6187). Interventions, lasting up to 12 months, were delivered primarily through electronic health record online messaging and were intended to supplement ongoing mental health care. MAIN OUTCOMES AND MEASURES The primary outcome was time to first nonfatal or fatal self-harm. Nonfatal self-harm was ascertained from health system records, and fatal self-harm was ascertained from state mortality data. Secondary outcomes included more severe self-harm (leading to death or hospitalization) and a broader definition of self-harm (selected injuries and poisonings not originally coded as self-harm). RESULTS A total of 18 644 patients (9009 [48%] aged 45 years or older; 12 543 [67%] female; 9222 [50%] from mental health specialty clinics and the remainder from primary care) contributed at least 1 day of follow-up data and were included in analyses. Thirty-one percent of participants offered care management and 39% offered skills training actively engaged in intervention programs. A total of 540 participants had a self-harm event (including 45 deaths attributed to self-harm and 495 nonfatal self-harm events) over 18 months following randomization: 172 (3.27%) in care management, 206 (3.92%) in skills training, and 162 (3.27%) in usual care. Risk of fatal or nonfatal self-harm over 18 months did not differ significantly between the care management and usual care groups (hazard ratio [HR], 1.07; 97.5% CI, 0.84-1.37) but was significantly higher in the skills training group than in usual care (HR, 1.29; 97.5% CI, 1.02-1.64). For severe self-harm, care management vs usual care had an HR of 1.03 (97.5% CI, 0.71-1.51); skills training vs usual care had an HR of 1.34 (97.5% CI, 0.94-1.91). For the broader self-harm definition, care management vs usual care had an HR of 1.10 (97.5% CI, 0.92-1.33); skills training vs usual care had an HR of 1.17 (97.5% CI, 0.97-1.41). CONCLUSIONS AND RELEVANCE Among adult outpatients with frequent suicidal ideation, offering care management did not significantly reduce risk of self-harm, and offering brief dialectical behavior therapy skills training significantly increased risk of self-harm, compared with usual care. These findings do not support implementation of the programs tested in this study. TRIAL REGISTRATION ClinicalTrials.gov Identifier: NCT02326883.
Collapse
Affiliation(s)
| | | | | | - Arne Beck
- Kaiser Permanente Colorado Institute for Health Research, Denver
| | - Gregory N. Clarke
- Kaiser Permanente Northwest Center for Health Research, Portland, Oregon
| | | | | | | | | | - Julia Smith
- Kaiser Permanente Washington Health Research Institute, Seattle
| |
Collapse
|
10
|
Weinstein SM, Coates LC, Helliwell PS, Ogdie A, Stephens-Shields AJ. Simulation-based design of pragmatic trials in psoriatic arthritis using propensity scores. Clin Trials 2021; 18:541-551. [PMID: 34431409 DOI: 10.1177/17407745211023840] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
BACKGROUND/AIMS Design of clinical trials requires careful decision-making across several dimensions, including endpoints, eligibility criteria, and subgroup enrichment. Clinical trial simulation can be an informative tool in trial design, providing empirical evidence by which to evaluate and compare the results of hypothetical trials with varying designs. We introduce a novel simulation-based approach using observational data to inform the design of a future pragmatic trial. METHODS We utilize propensity score-adjusted models to simulate hypothetical trials under alternative endpoints and enrollment criteria. We apply our approach to the design of pragmatic trials in psoriatic arthritis, using observational data embedded within the Tight Control of Inflammation in Early Psoriatic Arthritis study to simulate hypothetical open-label trials comparing treatment with tumor necrosis factor-α inhibitors to methotrexate. We first validate our simulations of a trial with traditional enrollment criteria and endpoints against a recently published trial. Next, we compare simulated treatment effects in patient populations defined by traditional and broadened enrollment criteria, where the latter is consistent with a future pragmatic trial. In each trial, we also consider five candidate primary endpoints. RESULTS Our results highlight how changes in the enrolled population and primary endpoints may qualitatively alter study findings and the ability to detect heterogeneous treatment effects between clinical subgroups. For treatments of interest in the study of psoriatic arthritis, broadened enrollment criteria led to diluted estimated treatment effects. Endpoints with greater responsiveness to treatment compared with a traditionally used endpoint were identified. These considerations, among others, are important for designing a future pragmatic trial aimed at having high external validity with relevance for real-world clinical practice. CONCLUSION Observational data may be leveraged to inform design decisions in pragmatic trials. Our approach may be generalized to the study of other conditions where existing trial data are limited or do not generalize well to real-world clinical practice, but where observational data are available.
Collapse
Affiliation(s)
- Sarah M Weinstein
- Department of Biostatistics, Epidemiology, and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Laura C Coates
- Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK
| | - Philip S Helliwell
- Leeds Institute of Rheumatic and Musculoskeletal Medicine, University of Leeds, Leeds, UK
| | - Alexis Ogdie
- Department of Biostatistics, Epidemiology, and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA.,Division of Rheumatology, Center for Pharmacoepidemiology Research and Training, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Alisa J Stephens-Shields
- Department of Biostatistics, Epidemiology, and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| |
Collapse
|
11
|
Rogers JR, Lee J, Zhou Z, Cheung YK, Hripcsak G, Weng C. Contemporary use of real-world data for clinical trial conduct in the United States: a scoping review. J Am Med Inform Assoc 2021; 28:144-154. [PMID: 33164065 DOI: 10.1093/jamia/ocaa224] [Citation(s) in RCA: 23] [Impact Index Per Article: 7.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/05/2020] [Revised: 08/11/2020] [Accepted: 09/02/2020] [Indexed: 12/28/2022] Open
Abstract
OBJECTIVE Real-world data (RWD), defined as routinely collected healthcare data, can be a potential catalyst for addressing challenges faced in clinical trials. We performed a scoping review of database-specific RWD applications within clinical trial contexts, synthesizing prominent uses and themes. MATERIALS AND METHODS Querying 3 biomedical literature databases, research articles using electronic health records, administrative claims databases, or clinical registries either within a clinical trial or in tandem with methodology related to clinical trials were included. Articles were required to use at least 1 US RWD source. All abstract screening, full-text screening, and data extraction was performed by 1 reviewer. Two reviewers independently verified all decisions. RESULTS Of 2020 screened articles, 89 qualified: 59 articles used electronic health records, 29 used administrative claims, and 26 used registries. Our synthesis was driven by the general life cycle of a clinical trial, culminating into 3 major themes: trial process tasks (51 articles); dissemination strategies (6); and generalizability assessments (34). Despite a diverse set of diseases studied, <10% of trials using RWD for trial process tasks evaluated medications or procedures (5/51). All articles highlighted data-related challenges, such as missing values. DISCUSSION Database-specific RWD have been occasionally leveraged for various clinical trial tasks. We observed underuse of RWD within conducted medication or procedure trials, though it is subject to the confounder of implicit report of RWD use. CONCLUSION Enhanced incorporation of RWD should be further explored for medication or procedure trials, including better understanding of how to handle related data quality issues to facilitate RWD use.
Collapse
Affiliation(s)
- James R Rogers
- Department of Biomedical Informatics, Columbia University, New York, New York, USA
| | - Junghwan Lee
- Department of Biomedical Informatics, Columbia University, New York, New York, USA
| | - Ziheng Zhou
- Institute of Human Nutrition, Columbia University, New York, New York, USA
| | - Ying Kuen Cheung
- Department of Biostatistics, Columbia University, New York, New York, USA, and
| | - George Hripcsak
- Department of Biomedical Informatics, Columbia University, New York, New York, USA.,Medical Informatics Services, New York-Presbyterian Hospital, New York, New York, USA
| | - Chunhua Weng
- Department of Biomedical Informatics, Columbia University, New York, New York, USA
| |
Collapse
|
12
|
Abstract
Clinical trials embedded in health systems can randomize large populations using automated data sources to determine trial eligibility and assess outcomes. The suicide prevention outreach trial used real-world data for trial design and randomized 18,868 individuals in four health systems using patient-reported thoughts of death or self-harm (Patient Health Questionnaire item 9). This took 3.5 years. We consider if using predictive analytics, that is, suicide risk estimates based on prediction models, could improve trial "efficiency." We used data on mental health outpatient visits between 1 January 2009 and 30 September 2017 in seven health systems (HealthPartners; Henry Ford Health System; and Colorado, Hawaii, Northwest, Southern California, and Washington Kaiser Permanente regions). We used a suicide risk prediction model developed in these same systems. We compared five trial designs with different eligibility criteria: a response of a 2 or 3 on Patient Health Questionnaire item 9, a response of a 3, suicide risk score above 90th, 95th, or 99th percentile. We compared the sample that met each criterion, 90-day suicide attempt rate following first eligible visit, and necessary sample sizes to detect a 15%, 25%, and 35% relative reduction in the suicide attempt rate, assuming 90% power, for each eligibility criterion. Our sample included 24,355,599 outpatient visits. Despite wide-spread use of Patient Health Questionnaire, 21,026,985 (86.3%) visits did not have a recorded Patient Health Questionnaire. Of the 2,928,927 individuals in our sample, 109,861 had a recorded Patient Health Questionnaire item 9 response of a 2 or 3 over the study years with a 1.40% 90-day suicide attempt rate and 50,047 had a response of a 3 (suicide attempt rate 1.98%). More patients met criteria requiring a certain risk score or higher: 331,273 had a 90th percentile risk score or higher (suicide attempt rate: 1.36%); 182,316 a 95th percentile or higher (suicide attempt rate 2.16%), and 78,655 a 99th percentile or higher (suicide attempt rate: 3.95%). Eligibility criterion of a Patient Health Questionnaire item 9 response of a 2 or 3 would require randomizing 44,081 individuals (40.2% of eligible population in our sample); eligibility criterion of a 3 would require 31,024 individuals (62.0% of eligible population). Eligibility criterion of a suicide risk score of 90th percentile or higher would require 45,675 individuals (13.8% of eligible population), 95th percentile 28,699 individuals (15.7% of eligible population), and 99th percentile 15,509 (19.7% of eligible population). A suicide risk prediction calculator could improve trial "efficiency"; identifying more individuals at increased suicide risk than relying on patient-report. It is an open scientific question if individuals identified using predictive analytics would respond differently to interventions than those identified by more traditional means.
Collapse
Affiliation(s)
- Susan M Shortreed
- Kaiser Permanente Washington Health Research Institute, Seattle, WA, USA.,Department of Biostatistics, University of Washington, Seattle, WA, USA
| | - Gregory E Simon
- Kaiser Permanente Washington Health Research Institute, Seattle, WA, USA
| |
Collapse
|
13
|
Simon GE, Shortreed SM, Rossom RC, Penfold RB, Sperl-Hillen JAM, O'Connor P. Principles and procedures for data and safety monitoring in pragmatic clinical trials. Trials 2019; 20:690. [PMID: 31815644 PMCID: PMC6902512 DOI: 10.1186/s13063-019-3869-3] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/26/2019] [Accepted: 10/31/2019] [Indexed: 11/27/2022] Open
Abstract
Background All clinical trial investigators have ethical and regulatory obligations to monitor participant safety and trial integrity. Specific procedures for meeting these obligations, however, may differ substantially between pragmatic trials and traditional explanatory clinical trials. Methods/Results Appropriate monitoring of clinical trials typically includes assessing rate of recruitment or enrollment; monitoring safe and effective delivery of study treatments; assuring that study staff act to minimize risks; monitoring quality and timeliness of study data; and considering interim analyses for early detection of benefit, harm, or futility. Each of these responsibilities applies to pragmatic clinical trials. Just as design of pragmatic trials typically involves specific and necessary departures from methods of explanatory clinical trials, appropriate monitoring of pragmatic trials typically requires specific departures from monitoring procedures used in explanatory clinical trials. We discuss how specific aspects of pragmatic trial design and operations influence selection of monitoring procedures and illustrate those choices using examples from three ongoing pragmatic trials conducted by the Mental Health Research Network. Conclusions Pragmatic trial investigators should not routinely adopt monitoring procedures used in explanatory clinical trials. Instead, investigators should consider core principles of trial monitoring and design monitoring procedures appropriate for each pragmatic trial.
Collapse
Affiliation(s)
- Gregory E Simon
- Kaiser Permanente Washington Health Research Institute, Seattle, WA, USA.
| | - Susan M Shortreed
- Kaiser Permanente Washington Health Research Institute, Seattle, WA, USA
| | | | - Robert B Penfold
- Kaiser Permanente Washington Health Research Institute, Seattle, WA, USA
| | | | | |
Collapse
|