Clinical Informatics Researcher's Desiderata for the Data Content of the Next Generation Electronic Health Record

External Validation of an Electronic Phenotyping Algorithm Detecting Attention to High Body Mass Index in Pediatric Primary Care

OBJECTIVES

 The lack of feasible and meaningful measures of clinicians' behavior hinders efforts to assess and improve obesity management in pediatric primary care. In this study, we examined the external validity of a novel algorithm, previously validated in a single geographic region, using structured electronic health record (EHR) data to identify phenotypes of clinicians' attention to elevated body mass index (BMI) and weight-related comorbidities.

METHODS

 We extracted structured EHR data for 300 randomly selected 6- to 12-year-old children with elevated BMI seen for well-child visits from June 2018 to May 2019 at pediatric primary care practices affiliated with Yale. Using diagnosis codes, laboratory orders, referrals, and medications adapted from the original algorithm, we categorized encounters as having evidence of attention to BMI only, weight-related comorbidities only, or both BMI and comorbidities. We evaluated the algorithm's sensitivity and specificity for detecting any attention to BMI and/or comorbidities using chart review as the reference standard.

RESULTS

 The adapted algorithm yielded a sensitivity of 79.2% and specificity of 94.0% for identifying any attention to high BMI/comorbidities in clinical documentation. Of 86 encounters labeled as "no attention" by the algorithm, 83% had evidence of attention in free-text components of the progress note. The likelihood of classification as "any attention" by both chart review and the algorithm varied by BMI category and by clinician type (p < 0.001).

CONCLUSION

 The electronic phenotyping algorithm had high specificity for detecting attention to high BMI and/or comorbidities in structured EHR inputs. The algorithm's performance may be improved by incorporating unstructured data from clinical notes.

Refining Clinical Phenotypes to Improve Clinical Decision Support and Reduce Alert Fatigue: A Feasibility Study

BACKGROUND

Chronic kidney disease (CKD) is common and associated with adverse clinical outcomes. Most care for early CKD is provided in primary care, including hypertension (HTN) management. Computerized clinical decision support (CDS) can improve the quality of care for CKD but can also cause alert fatigue for primary care physicians (PCPs). Computable phenotypes (CPs) are algorithms to identify disease populations using, for example, specific laboratory data criteria.

OBJECTIVES

Our objective was to determine the feasibility of implementation of CDS alerts by developing CPs and estimating potential alert burden.

METHODS

We utilized clinical guidelines to develop a set of five CPs for patients with stage 3 to 4 CKD, uncontrolled HTN, and indications for initiation or titration of guideline-recommended antihypertensive agents. We then conducted an iterative data analytic process consisting of database queries, data validation, and subject matter expert discussion, to make iterative changes to the CPs. We estimated the potential alert burden to make final decisions about the scope of the CDS alerts. Specifically, the number of times that each alert could fire was limited to once per patient.

RESULTS

In our primary care network, there were 239,339 encounters for 105,992 primary care patients between April 1, 2018 and April 1, 2019. Of these patients, 9,081 (8.6%) had stage 3 and 4 CKD. Almost half of the CKD patients, 4,191 patients, also had uncontrolled HTN. The majority of CKD patients were female, elderly, white, and English-speaking. We estimated that 5,369 alerts would fire if alerts were triggered multiple times per patient, with a mean number of alerts shown to each PCP ranging from 0.07-to 0.17 alerts per week.

CONCLUSION

Development of CPs and estimation of alert burden allows researchers to iteratively fine-tune CDS prior to implementation. This method of assessment can help organizations balance the tradeoff between standardization of care and alert fatigue.

Assessing the Contribution of Scanned Outside Documents to the Completeness of Real-World Data Abstraction

PURPOSE

Electronic health record (EHR) data are widely used in precision medicine, quality improvement, disease surveillance, and population health management. However, a significant amount of EHR data are stored in unstructured formats including scanned documents external to the treatment facility presenting an informatics challenge for secondary use. Studies are needed to characterize the clinical information uniquely available in scanned outside documents (SODs) to understand to what extent the availability of such information affects the use of these real-world data for cancer research.

MATERIALS AND METHODS

Two independent EHR data abstractions capturing 30 variables commonly used in oncology research were conducted for 125 patients treated for advanced non-small-cell lung cancer at a comprehensive cancer center, with and without consideration of SODs. Completeness and concordance were compared between the two abstractions, overall, and by patient groups and variable types.

RESULTS

The overall completeness of the data with SODs was 77.6% as compared with 54.3% for the abstraction without SODs. The differences in completeness were driven by data related to biomarker tests, which were more likely to be uniquely available in SODs. Such data were prone to missingness among patients who were diagnosed externally.

CONCLUSION

There were no major differences in completeness between the two abstractions by demographics, diagnosis, disease progression, performance status, or oral therapy use. However, biomarker data were more likely to be uniquely contained in the SODs. Our findings may help cancer centers prioritize the types of SOD data being abstracted for research or other secondary purposes.

Data Quality of Chemotherapy-Induced Nausea and Vomiting Documentation

OBJECTIVE

The objective of the study was to characterize the completeness and concordance of the electronic health record (EHR) documentation of cancer symptoms among multidisciplinary health care professionals.

METHODS

We examined the EHRs of children, adolescents, and young adults who received highly emetogenic chemotherapy and characterized the completeness and concordance of chemotherapy-induced nausea and vomiting (CINV) documentation by clinician type and by the International Classification of Diseases 10th Revision (ICD-10) coding choice.

RESULTS

The EHRs of 127 patients, comprising 870 patient notes, were abstracted and reviewed. A CINV assessment was documented by prescribers in 75% of patients, and by nurses in 58% of patients. Of the 60 encounters where both prescribers and nurses documented, 72% agreed on the presence/absence of CINV.

CONCLUSION

Most patients receiving highly emetogenic chemotherapy had a documented assessment of CINV; however, many had incomplete or discordant documentation of CINV from different providers by role, implying the importance of incorporating pragmatic knowledge of EHR documentation patterns among multidisciplinary health professionals for EHR phenotyping and clinical decision support systems directed toward cancer-related symptom management.

Coronary Artery Disease Phenotype Detection in an Academic Hospital System Setting

BACKGROUND

The United States, and especially West Virginia, have a tremendous burden of coronary artery disease (CAD). Undiagnosed familial hypercholesterolemia (FH) is an important factor for CAD in the U.S. Identification of a CAD phenotype is an initial step to find families with FH.

OBJECTIVE

We hypothesized that a CAD phenotype detection algorithm that uses discrete data elements from electronic health records (EHRs) can be validated from EHR information housed in a data repository.

METHODS

We developed an algorithm to detect a CAD phenotype which searched through discrete data elements, such as diagnosis, problem lists, medical history, billing, and procedure (International Classification of Diseases [ICD]-9/10 and Current Procedural Terminology [CPT]) codes. The algorithm was applied to two cohorts of 500 patients, each with varying characteristics. The second (younger) cohort consisted of parents from a school child screening program. We then determined which patients had CAD by systematic, blinded review of EHRs. Following this, we revised the algorithm by refining the acceptable diagnoses and procedures. We ran the second algorithm on the same cohorts and determined the accuracy of the modification.

RESULTS

CAD phenotype Algorithm I was 89.6% accurate, 94.6% sensitive, and 85.6% specific for group 1. After revising the algorithm (denoted CAD Algorithm II) and applying it to the same groups 1 and 2, sensitivity 98.2%, specificity 87.8%, and accuracy 92.4; accuracy 93% for group 2. Group 1 F1 score was 92.4%. Specific ICD-10 and CPT codes such as "coronary angiography through a vein graft" were more useful than generic terms.

CONCLUSION

We have created an algorithm, CAD Algorithm II, that detects CAD on a large scale with high accuracy and sensitivity (recall). It has proven useful among varied patient populations. Use of this algorithm can extend to monitor a registry of patients in an EHR and/or to identify a group such as those with likely FH.

A new approach to integrating patient-generated data with expert knowledge for personalized goal setting: A pilot study

INTRODUCTION

Self-monitoring technologies produce patient-generated data that could be leveraged to personalize nutritional goal setting to improve population health; however, most computational approaches are limited when applied to individual-level personalization with sparse and irregular self-monitoring data. We applied informatics methods from expert suggestion systems to a challenging clinical problem: generating personalized nutrition goals from patient-generated diet and blood glucose data.

MATERIALS AND METHODS

We applied qualitative process coding and decision tree modeling to understand how registered dietitians translate patient-generated data into recommendations for dietary self-management of diabetes (i.e., knowledge model). We encoded this process in a set of functions that take diet and blood glucose data as an input and output diet recommendations (i.e., inference engine). Dietitians assessed face validity. Using four patient datasets, we compared our inference engine's output to clinical narratives and gold standards developed by expert clinicians.

RESULTS

To dietitians, the knowledge model represented how recommendations from patient data are made. Inference engine recommendations were 63 % consistent with the gold standard (range = 42 %-75 %) and 74 % consistent with narrative clinical observations (range = 63 %-83 %).

DISCUSSION

Qualitative modeling and automating how dietitians reason over patient data resulted in a knowledge model representing clinical knowledge. However, our knowledge model was less consistent with gold standard than narrative clinical recommendations, raising questions about how best to evaluate approaches that integrate patient-generated data with expert knowledge.

CONCLUSION

New informatics approaches that integrate data-driven methods with expert decision making for personalized goal setting, such as the knowledge base and inference engine presented here, demonstrate the potential to extend the reach of patient-generated data by synthesizing it with clinical knowledge. However, important questions remain about the strengths and weaknesses of computer algorithms developed to discern signal from patient-generated data compared to human experts.

Sync for Genes: Making Clinical Genomics Available for Precision Medicine at the Point-of-Care

BACKGROUND

Making genomic data available at the point-of-care and for research is critical for the success of the Precision Medicine Initiative (PMI), a research initiative which seeks to change health care by "tak(ing) into account individual differences in people's genes, environments, and lifestyles." The Office of the National Coordinator for Health Information Technology (ONC) led Sync for Genes, a program to develop standards that make genomic data available when and where it matters most. This article discusses lessons learned from recent Sync for Genes activities.

OBJECTIVES

The goals of Sync for Genes were to (1) demonstrate exchange of genomic data using health data standards, (2) provide feedback for refinement of health data standards, and (3) synthesize project experiences to support the integration of genomic data at the point-of-care and for research.

METHODS

Four organizations participated in a program to test the Health Level Seven International (HL7^®) Fast Healthcare Interoperability Resources (FHIR^®) standard, which supports sharing genomic data. ONC provided access to subject matter experts, resources, tools, and technical guidance to support testing activities. Three of the four organizations participated in HL7 FHIR Connectathons to test FHIR's ability to exchange genomic diagnostic reports.

RESULTS

The organizations successfully demonstrated exchange of genomic diagnostic reports using FHIR. The feedback and artifacts that resulted from these activities were shared with HL7 and made publicly available. Four areas were identified as important considerations for similar projects: (1) FHIR proficiency, (2) developer support, (3) project scope, and (4) bridging health information technology and genomic expertise.

CONCLUSION

Precision medicine is a rapidly evolving field, and there is opportunity to continue maturing health data standards for the exchange of necessary genomic data, increasing the likelihood that the standard supports the needs of users.