A strategy for validation of variables derived from large-scale electronic health record data

Development and Validation of the Veterans Affairs Eosinophilic Esophagitis Cohort

BACKGROUND & AIMS

Gaps remain in understanding the epidemiology of eosinophilic esophagitis (EoE). Our aim was to identify and validate a national cohort of individuals with EoE using Veterans Health Administration (VHA) data.

METHODS

We used 2 validation strategies to develop algorithms that identified adults with EoE between 1999 and 2020. The first validation strategy applied International Classification of Diseases (ICD) code algorithms to a base cohort of individuals who underwent esophagogastroduodenoscopy with esophageal biopsy specimens. The second applied ICD code algorithms to a base cohort of all individuals in the VHA. For each ICD algorithm applied, a random sample of candidate EoE cases and non-EoE controls were selected and the charts were reviewed manually by a blinded reviewer. Each algorithm was modified iteratively until the prespecified diagnostic accuracy end point (95% confidence lower bound for a positive predictive value [PPV], >88%) was achieved. We compiled individuals from each strategy's maximum performance algorithm to construct the Veterans Affairs Eosinophilic Esophagitis Cohort.

RESULTS

The maximum performance algorithm from the first validation strategy included 2 or more ICD code encounters for EoE separated by more than 30 days and achieved a 93.3% PPV (lower bound, 88.1%) for identifying true EoE cases. The maximum performance algorithm from the second validation strategy included 4 or more ICD code encounters for EoE in which 2 codes were separated by at least 30 days, and similarly achieved a 93.3% PPV (lower bound, 88.1%). Combining both strategies yielded 6637 individuals, which comprised the Veterans Affairs Eosinophilic Esophagitis Cohort.

CONCLUSIONS

We developed and validated 2 highly accurate coding algorithms for EoE and established a nationwide VHA cohort of adults with EoE for future studies.

Risk of Incident and Fatal Colorectal Cancer After Young-Onset Adenoma Diagnosis: A National Cohort Study

INTRODUCTION

Colorectal cancer (CRC) incidence and mortality rates are increasing in adults aged <50 years. Young-onset adenoma (YOA)-adenoma detected in adults younger than 50 years-may signify increased CRC risk, but this association has not been widely studied. Our aim was to compare the risk of incident and fatal CRC in adults aged <50 years with YOA diagnosis compared with those with a normal colonoscopy.

METHODS

We conducted a cohort study of US Veterans aged 18-49 years who received colonoscopy between 2005 and 2016. The primary exposure of interest was YOA. Primary outcomes included incident and fatal CRC. We used Kaplan-Meier curves to calculate cumulative incident and fatal CRC risk and Cox models to examine relative CRC risk.

RESULTS

The study cohort included 54,284 Veterans aged <50 years exposed to colonoscopy, among whom 13% (n = 7,233) had YOA at start of follow-up. Cumulative 10-year CRC incidence was 0.11% (95% confidence interval [CI]: 0.00%-0.27%) after any adenoma diagnosis, 0.18% (95% CI: 0.02%-0.53%) after advanced YOA diagnosis, 0.10% (95% CI: 0.00%-0.28%) after nonadvanced adenoma diagnosis, and 0.06% (95% CI: 0.02%-0.09%) after normal colonoscopy. Veterans with advanced adenoma had 8-fold greater incident CRC risk than those with normal colonoscopy (hazard ratio: 8.0; 95% CI: 1.8-35.6). Across groups, no differences in fatal CRC risk were observed.

DISCUSSION

Young-onset advanced adenoma diagnosis was associated with 8-fold increased incident CRC risk compared with normal colonoscopy. However, cumulative CRC incidence and mortality at 10 years among individuals with either young onset non-advanced or advanced adenoma diagnosis were both relatively low.

Development and Validation of Quality Measures for Testosterone Prescribing

Context

Accurate measures to assess appropriateness of testosterone prescribing are needed to improve prescribing practices.

Objective

This work aimed to develop and validate quality measures around the initiation and monitoring of testosterone prescribing.

Methods

This retrospective cohort study comprised a national cohort of male patients receiving care in the Veterans Health Administration who initiated testosterone during January or February 2020. Using laboratory data and diagnostic codes, we developed 9 initiation and 7 monitoring measures. These were based on the current Endocrine Society guidelines supplemented by expert opinion and prior work. We chose measures that could be operationalized using national VA electronic health record (EHR) data. We assessed criterion validity for these 16 measures by manual review of 142 charts. Main outcome measures included positive and negative predictive values (PPVs, NPVs), overall accuracy (OA), and Matthews Correlation Coefficients (MCCs).

Results

We found high PPVs (>78%), NPVs (>98%), OA (≥94%), and MCCs (>0.85) for the 10 measures based on laboratory data (5 initiation and 5 monitoring). For the 6 measures relying on diagnostic codes, we similarly found high NPVs (100%) and OAs (≥98%). However, PPVs for measures of acute conditions occurring before testosterone initiation (ie, acute myocardial infarction or stroke) or new conditions occurring after initiation (ie, prostate or breast cancer) PPVs were much lower (0% to 50%) due to few or no cases.

Conclusion

We developed several valid EHR-based quality measures for assessing testosterone-prescribing practices. Deployment of these measures in health care systems can facilitate identification of quality gaps in testosterone-prescribing and improve care of men with hypogonadism.

Development and Validation of a National US Achalasia Cohort: The Veterans Affairs Achalasia Cohort (VA-AC)

BACKGROUND & AIMS

Achalasia is an esophageal motility disorder associated with significant morbidity, yet achalasia-associated risk factors and outcomes are not well-characterized. Our aim was to establish a national cohort of individuals with achalasia, utilizing Veterans Health Administration (VHA) data.

METHODS

We iteratively developed combinations of International Classification of Diseases and Current Procedural Terminology code algorithms to validate an approach for identifying achalasia cases. We assessed algorithm accuracy for achalasia diagnosis through manual chart review of candidate achalasia cases and candidate non-achalasia controls. The prespecified end point chosen to establish algorithm performance success was achieving a 1-sided 95% confidence lower bound for a positive predictive value >85% for a random sample of 100 candidate achalasia cases. Once adequate performance was validated, we queried national VHA data to establish and characterize a cohort of individuals diagnosed with achalasia between 1999 and 2020.

RESULTS

Three rounds of algorithm modification and validation were conducted to achieve the prespecified performance endpoint. In the final round, a combination of 3 or more International Classification of Diseases codes for achalasia in the subject's lifetime and a Current Procedural Terminology code for esophageal manometry achieved an observed 94% positive predictive value (1-sided 95% confidence lower bound of 88.5%) for identifying achalasia. Applying the algorithm to national VHA data identified a cohort of 2100 individuals with achalasia, with a median age 65 years and who were 93% male.

CONCLUSIONS

Using a rigorous validation approach, we established a national cohort of 2100 individuals with achalasia within the VHA, one of the largest established to date. This cohort can be utilized to study risk factors for achalasia and outcomes over time.

Reducing disparities and achieving health equity in colorectal cancer screening

Increases in colorectal cancer screening are linked to the declining incidence of the disease over the past three decades. These favorable trends, however, are not observed in marginalized racial and ethnic populations with disproportionately lower rates of screening, higher disease incidence, and increased mortality despite advances in health technology and policy. This review describes the differences in screening uptake and test selection amongst racial and ethnic groups, discusses known obstacles and facilitators that impact screening, and highlights existing frameworks developed to achieve health equity in colorectal cancer screening.

Clinical Research Informatics

OBJECTIVES

To summarize key contributions to current research in the field of Clinical Research Informatics (CRI) and to select best papers published in 2021.

METHOD

Using PubMed, we did a bibliographic search using a combination of MeSH descriptors and free-text terms on CRI, followed by a double-blind review in order to select a list of candidate best papers to be peer-reviewed by external reviewers. After peer-review ranking, three section editors met for a consensus meeting and the editorial team was organized to finally conclude on the selected three best papers.

RESULTS

Among the 1,096 papers (published in 2021) returned by the search and in the scope of the various areas of CRI, the full review process selected three best papers. The first best paper describes an operational and scalable framework for generating EHR datasets based on a detailed clinical model with an application in the domain of the COVID-19 pandemics. The authors of the second best paper present a secure and scalable platform for the preprocessing of biomedical data for deep data-driven health management applied for the detection of pre-symptomatic COVID-19 cases and for biological characterization of insulin-resistance heterogeneity. The third best paper provides a contribution to the integration of care and research activities with the REDCap Clinical Data and Interoperability sServices (CDIS) module improving the accuracy and efficiency of data collection.

CONCLUSIONS

The COVID-19 pandemic is still significantly stimulating research efforts in the CRI field to improve the process deeply and widely for conducting real-world studies as well as for optimizing clinical trials, the duration and cost of which are constantly increasing. The current health crisis highlights the need for healthcare institutions to continue the development and deployment of Big Data spaces, to strengthen their expertise in data science and to implement efficient data quality evaluation and improvement programs.

Adenoma Detection Rate and Clinical Characteristics Influence Advanced Neoplasia Risk After Colorectal Polypectomy

BACKGROUND AND AIMS

Postpolypectomy risk stratification for subsequent metachronous advanced neoplasia (MAN) is imprecise and does not account for colonoscopist adenoma detection rate (ADR). Our aim was to assess association of ADR with MAN and create a prediction model for postpolypectomy risk stratification incorporating ADR and other factors.

METHODS

We conducted a retrospective cohort study of individuals with baseline polypectomy and subsequent surveillance colonoscopy from 2004 to 2016 within the U.S. Department of Veterans Affairs (VA). Clinical factors, polyp findings, and baseline colonoscopist ADR were considered for the model. Model performance (sensitivity, specificity, and area under the curve) for identifying individuals with MAN was compared with 2020 U.S. Multi-Society Task Force on Colorectal Cancer (USMSTF) surveillance recommendations.

RESULTS

A total of 30,897 individuals were randomly assigned 2:1 into independent model training and validation sets. Increasing age, male sex, diabetes, current smoking, adenoma number, polyp location, adenoma ≥10 mm or with tubulovillous/villous features, and decreasing colonoscopist ADR were independently associated with MAN. A range of 1.48- to 1.66-fold increased risk for MAN was observed for ADR in the lowest 3 quintiles (ADR <19.7%-39.3%) vs the highest quintile (ADR >47.0%). When the final model selected based on the training set was applied to the validation set, improved sensitivity and specificity over 2020 USMSTF risk stratification were achieved (P = .001), with an area under the curve of 0.62 (95% confidence interval, 0.60-0.64).

CONCLUSIONS

Colonoscopist ADR is associated with MAN. Combining clinical factors and ADR for risk stratification has potential to improve postpolypectomy risk stratification. Improving ADR is likely to improve postpolypectomy outcomes.

Natural Language Processing for Computer-Assisted Chart Review to Assess Documentation of Substance use and Psychopathology in Heart Failure Patients Awaiting Cardiac Resynchronization Therapy

CONTEXT

Advanced heart failure (HF) patients often experience distressing psychological symptoms, frequently meeting diagnostic criteria for psychological disorders, including anxiety, depression, and substance use disorder. Patients with device-based HF therapies have added risk for psychological disorders, with consequences for their physiological functioning, including adverse cardiac outcomes.

OBJECTIVES

This study used natural language processing (NLP) for computer-assisted chart review to assess documentation of mental health and substance use in HF patients awaiting cardiac resynchronization therapy (CRT), a device-based HF therapy.

METHODS

We applied NLP to clinical notes from electronic health records (EHR) of 965 consecutive patients, with 9821 total clinical notes, at two academic medical centers between 2004 and 2015. We developed and validated a keyword library capturing terms related to mental health and substance use, while balancing specificity and sensitivity.

RESULTS

Mean age was 71.6 years (SD = 11.8), 78% male, and 87% non-Hispanic White. Of the 544 patients (56.4%) with documentation of mental health history, 9.7% had their mental health assessed and 6.6% had a plan documented. Of the 773 patients (80.1%) with documentation of substance use history, 10 (1.0%) had an assessment, and 3 (0.3%) had a plan.

CONCLUSION

Despite clinical recommendations and standards of care, clinicians are under documenting assessments and plans prior to CRT. Future research should develop an algorithm to prompt clinicians to document this content. Such quality improvement efforts may ensure adherence to standards of care and clinical guidelines.

Monocyte distribution width as part of a broad pragmatic sepsis screen in the emergency department

Study Objective

Methods

Results

Conclusion