Usability of a Digital Registry to Promote Secondary Prevention for Peripheral Artery Disease Patients

Objective

To evaluate usability of a quality improvement tool that promotes guideline-based care for patients with peripheral arterial disease (PAD).

Patients and Methods

The study was conducted from July 19, 2018, to August 21, 2019. We compared the usability of a PAD cohort knowledge solution (CKS) with standard management supported by an electronic health record (EHR). Two scenarios were developed for usability evaluation; the first for the PAD-CKS while the second evaluated standard EHR workflow. Providers were asked to provide opinions about the PAD-CKS tool and to generate a System Usability Scale (SUS) score. Metrics analyzed included time required, number of mouse clicks, and number of keystrokes.

Results

Usability evaluations were completed by 11 providers. SUS for the PAD-CKS was excellent at 89.6. Time required to complete 21 tasks in the CKS was 4 minutes compared with 12 minutes for standard EHR workflow (median, P = .002). Completion of CKS tasks required 34 clicks compared with 148 clicks for the EHR (median, P = .002). Keystrokes for CKS task completion was 8 compared with 72 for EHR (median, P = .004). Providers indicated that overall they found the tool easy to use and the PAD mortality risk score useful.

Conclusions

Usability evaluation of the PAD-CKS tool demonstrated time savings, a high SUS score, and a reduction of mouse clicks and keystrokes for task completion compared to standard workflow using the EHR. Provider feedback regarding the strengths and weaknesses also created opportunities for iterative improvement of the PAD-CKS tool.

Automated extraction of sudden cardiac death risk factors in hypertrophic cardiomyopathy patients by natural language processing

BACKGROUND

The management of hypertrophic cardiomyopathy (HCM) patients requires the knowledge of risk factors associated with sudden cardiac death (SCD). SCD risk factors such as syncope and family history of SCD (FH-SCD) as well as family history of HCM (FH-HCM) are documented in electronic health records (EHRs) as clinical narratives. Automated extraction of risk factors from clinical narratives by natural language processing (NLP) may expedite management workflow of HCM patients. The aim of this study was to develop and deploy NLP algorithms for automated extraction of syncope, FH-SCD, and FH-HCM from clinical narratives.

METHODS AND RESULTS

We randomly selected 200 patients from the Mayo HCM registry for development (n = 100) and testing (n = 100) of NLP algorithms for extraction of syncope, FH-SCD as well as FH-HCM from clinical narratives of EHRs. The clinical reference standard was manually abstracted by 2 independent annotators. Performance of NLP algorithms was compared to aggregation and summarization of data entries in the HCM registry for syncope, FH-SCD, and FH-HCM. We also compared the NLP algorithms with billing codes for syncope as well as responses to patient survey questions for FH-SCD and FH-HCM. These analyses demonstrated NLP had superior sensitivity (0.96 vs 0.39, p < 0.001) and comparable specificity (0.90 vs 0.92, p = 0.74) and PPV (0.90 vs 0.83, p = 0.37) compared to billing codes for syncope. For FH-SCD, NLP outperformed survey responses for all parameters (sensitivity: 0.91 vs 0.59, p = 0.002; specificity: 0.98 vs 0.50, p < 0.001; PPV: 0.97 vs 0.38, p < 0.001). NLP also achieved superior sensitivity (0.95 vs 0.24, p < 0.001) with comparable specificity (0.95 vs 1.0, p-value not calculable) and positive predictive value (PPV) (0.92 vs 1.0, p = 0.09) compared to survey responses for FH-HCM.

CONCLUSIONS

Automated extraction of syncope, FH-SCD and FH-HCM using NLP is feasible and has promise to increase efficiency of workflow for providers managing HCM patients.

Provider Survey on Automated Clinical Decision Support System for Cardiovascular Risk Assessment

Despite progress made in establishing primary and secondary preventive strategies for cardiovascular diseases, there are significant gaps between guideline recommended strategies and implementation of recommendations in practice. A clinical decision support (CDS) system entitled CV Risk Profile was developed at Mayo Clinic Rochester as a targeted solution for this gap in preventive cardiovascular care. The system remained in use for 10 years until it became non-functional in 2018 during transition to a new electronic health record (EHR). This study investigated provider opinions regarding the cardiovascular disease CDS system while it was still in operation, to determine if there exists a provider reported need for a similar system to be developed for use within the new EHR.

Provider Survey on Automated Clinical Decision Support for Cardiovascular Risk Assessment

OBJECTIVE

To investigate provider opinions regarding a clinical decision support (CDS) system for cardiovascular risk assessment and for the creation of a replacement system.

METHODS

From March to April 2018, an invitation letter with a link to a self-administered web-based survey was sent via e-mail to 279 providers with primary appointment in the Department of Cardiovascular Medicine, Mayo Clinic, Rochester. The e-mail was sent to providers on March 8, 2018 and the survey closed on April 16, 2018.

RESULTS

One hundred providers responded to the survey yielding an overall response rate of 35.8%. Of these, 52 (52%) indicated they had used the cardiovascular (CV) risk profile CDS system and were classified as users and prompted to continue the survey. Among users, 42 (80.8%) indicated use of the CDS was either important (25; 48.1%) or very important (17; 32.7%) in their clinical practice; 45 (86.5%) responded that the system was very easy (17; 32.7%) or easy (28; 53.8%) to use. In addition, 48 (96.0%) users indicated that the CV risk profile supported their thought process at the point-of-care; 47 (97.9%) users indicated similar functionalities should be implemented into the new electronic health record system and 41 (85.4%) users reported new functionalities should also be incorporated.

CONCLUSIONS

For most users, the CDS system was easy to use and supported clinical thought process at the point-of-care. Users also felt their practice was supported and should continue to be supported by CDS systems providing individualized patient information at the point-of-care.

Electronic Knowledge Resources and Point-of-Care Learning: A Scoping Review

PURPOSE

The authors sought to summarize quantitative and qualitative research addressing electronic knowledge resources and point-of-care learning in a scoping review.

METHOD

The authors searched MEDLINE, Embase, PsycINFO, and the Cochrane Database for studies addressing electronic knowledge resources and point-of-care learning. They iteratively revised inclusion criteria and operational definitions of study features and research themes of interest. Two reviewers independently performed each phase of study selection and data extraction.

RESULTS

Of 10,811 studies identified, 305 were included and reviewed. Most studies (225; 74%) included physicians or medical students. The most frequently mentioned electronic resources were UpToDate (88; 29%), Micromedex (59; 19%), Epocrates (50; 16%), WebMD (46; 15%), MD Consult (32; 10%), and LexiComp (31; 10%). Eight studies (3%) evaluated electronic resources or point-of-care learning using outcomes of patient effects, and 36 studies (12%) reported objectively measured clinician behaviors. Twenty-five studies (8%) examined the clinical or educational impact of electronic knowledge resource use on patient care or clinician knowledge, 124 (41%) compared use rates of various knowledge resources, 69 (23%) examined the quality of knowledge resource content, and 115 (38%) explored the process of point-of-care learning. Two conceptual clarifications were identified, distinguishing the impact on clinical or educational outcomes versus the impact on test setting decision support, and the quality of information content versus the correctness of information obtained by a clinician-user.

CONCLUSIONS

Research on electronic knowledge resources is dominated by studies involving physicians and evaluating use rates. Studies involving nonphysician users, and evaluating resource impact and implementation, are needed.

Knowledge management in the era of digital medicine: A programmatic approach to optimize patient care in an academic medical center

INTRODUCTION

The pace of medical discovery is accelerating to the point where caregivers can no longer keep up with the latest diagnosis or treatment recommendations. At the same time, sophisticated and complex electronic medical records and clinical systems are generating increasing volumes of patient data, making it difficult to find the important information required for patient care. To address these challenges, Mayo Clinic established a knowledge management program to curate, store, and disseminate clinical knowledge.

METHODS

The authors describe AskMayoExpert, a point-of-care knowledge delivery system, and discuss the process by which the clinical knowledge is captured, vetted by clinicians, annotated, and stored in a knowledge content management system. The content generated for AskMayoExpert is considered to be core clinical content and serves as the basis for knowledge diffusion to clinicians through order sets and clinical decision support rules, as well as to patients and consumers through patient education materials and internet content. The authors evaluate alternative approaches for better integration of knowledge into the clinical workflow through development of computer-interpretable care process models.

RESULTS

Each of the modeling approaches evaluated has shown promise. However, because each of them addresses the problem from a different perspective, there have been challenges in coming to a common model. Given the current state of guideline modeling and the need for a near-term solution, Mayo Clinic will likely focus on breaking down care process models into components and on standardization of those components, deferring, for now, the orchestration.

CONCLUSION

A point-of-care knowledge resource developed to support an individualized approach to patient care has grown into a formal knowledge management program. Translation of the textual knowledge into machine executable knowledge will allow integration of the knowledge with specific patient data and truly serve as a colleague and mentor for the physicians taking care of the patient.

Knowledge as a Service at the Point of Care

An electronic health record (EHR) can assist the delivery of high-quality patient care, in part by providing the capability for a broad range of clinical decision support, including contextual references (e.g., Infobuttons), alerts and reminders, order sets, and dashboards. All of these decision support tools are based on clinical knowledge; unfortunately, the mechanisms for managing rules, order sets, Infobuttons, and dashboards are often unrelated, making it difficult to coordinate the application of clinical knowledge to various components of the clinical workflow. Additional complexity is encountered when updating enterprise-wide knowledge bases and delivering the content through multiple modalities to different consumers. We present the experience of Mayo Clinic as a case study to examine the requirements and implementation challenges related to knowledge management across a large, multi-site medical center. The lessons learned through the development of our knowledge management and delivery platform will help inform the future development of interoperable knowledge resources.

Effect of a Novel Clinical Decision Support Tool on the Efficiency and Accuracy of Treatment Recommendations for Cholesterol Management

BACKGROUND

The 2013 American College of Cardiology / American Heart Association Guidelines for the Treatment of Blood Cholesterol emphasize treatment based on cardiovascular risk. But finding time in a primary care visit to manually calculate cardiovascular risk and prescribe treatment based on risk is challenging. We developed an informatics-based clinical decision support tool, MayoExpertAdvisor, to deliver automated cardiovascular risk scores and guideline-based treatment recommendations based on patient-specific data in the electronic heath record.

OBJECTIVE

To assess the impact of our clinical decision support tool on the efficiency and accuracy of clinician calculation of cardiovascular risk and its effect on the delivery of guideline-consistent treatment recommendations.

METHODS

Clinicians were asked to review the EHR records of selected patients. We evaluated the amount of time and the number of clicks and keystrokes needed to calculate cardiovascular risk and provide a treatment recommendation with and without our clinical decision support tool. We also compared the treatment recommendation arrived at by clinicians with and without the use of our tool to those recommended by the guidelines.

RESULTS

Clinicians saved 3 minutes and 38 seconds in completing both tasks with MayoExpertAdvisor, used 94 fewer clicks and 23 fewer key strokes, and improved accuracy from the baseline of 60.61% to 100% for both the risk score calculation and guideline-consistent treatment recommendation.

CONCLUSION

Informatics solution can greatly improve the efficiency and accuracy of individualized treatment recommendations and have the potential to increase guideline compliance.