1
|
Sundermann M, Clendon O, McNeill R, Doogue M, Chin PKL. Optimising interruptive clinical decision support alerts for antithrombotic duplicate prescribing in hospital. Int J Med Inform 2024; 186:105418. [PMID: 38518676 DOI: 10.1016/j.ijmedinf.2024.105418] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2023] [Revised: 03/05/2024] [Accepted: 03/17/2024] [Indexed: 03/24/2024]
Abstract
INTRODUCTION Duplicate prescribing clinical decision support alerts can prevent important prescribing errors but are frequently the cause of much alert fatigue. Stat dose prescriptions are a known reason for overriding these alerts. This study aimed to evaluate the effect of excluding stat dose prescriptions from duplicate prescribing alerts for antithrombotic medicines on alert burden, prescriber adherence, and prescribing. MATERIALS AND METHODS A before (January 1st, 2017 to August 31st, 2022) and after (October 5th, 2022 to September 30th, 2023) study was undertaken of antithrombotic duplicate prescribing alerts and prescribing following a change in alert settings. Alert and prescribing data for antithrombotic medicines were joined, processed, and analysed to compare alert rates, adherence, and prescribing. Alert burden was assessed as alerts per 100 prescriptions. Adherence was measured at the point of the alert as whether the prescriber accepted the alert and following the alert as whether a relevant prescription was ceased within an hour. Co-prescribing of antithrombotic stat dose prescriptions was assessed pre- and post-alert reconfiguration. RESULTS Reconfiguration of the alerts reduced the alert rate by 29 % (p < 0.001). The proportion of alerts associated with cessation of antithrombotic duplication significantly increased (32.8 % to 44.5 %, p < 0.001). Adherence at the point of the alert increased 1.2 % (4.8 % to 6.0 %, p = 0.012) and 11.5 % (29.4 % to 40.9 %, p < 0.001) within one hour of the alert. When ceased after the alert over 80 % of duplicate prescriptions were ceased within 2 min of overriding. Antithrombotic stat dose co-prescribing was unchanged for 4 out of 5 antithrombotic duplication alert rules. CONCLUSION By reconfiguring our antithrombotic duplicate prescribing alerts, we reduced alert burden and increased alert adherence. Many prescribers ceased duplicate prescribing within 2 min of alert override highlighting the importance of incorporating post-alert measures in accurately determining prescriber alert adherence.
Collapse
Affiliation(s)
- Milan Sundermann
- Department of Medicine, University of Otago, Christchurch, New Zealand
| | - Olivia Clendon
- Department of Clinical Pharmacology, Te Whatu Ora Health New Zealand - Waitaha Canterbury, New Zealand
| | - Richard McNeill
- Department of Medicine, University of Otago, Christchurch, New Zealand
| | - Matthew Doogue
- Department of Medicine, University of Otago, Christchurch, New Zealand; Department of Clinical Pharmacology, Te Whatu Ora Health New Zealand - Waitaha Canterbury, New Zealand
| | - Paul K L Chin
- Department of Medicine, University of Otago, Christchurch, New Zealand; Department of Clinical Pharmacology, Te Whatu Ora Health New Zealand - Waitaha Canterbury, New Zealand.
| |
Collapse
|
2
|
Barton HJ, Maru A, Leaf MA, Hekman DJ, Wiegmann DA, Shah MN, Patterson BW. Academic Detailing as a Health Information Technology Implementation Method: Supporting the Design and Implementation of an Emergency Department-Based Clinical Decision Support Tool to Prevent Future Falls. JMIR Hum Factors 2024; 11:e52592. [PMID: 38635318 PMCID: PMC11066751 DOI: 10.2196/52592] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/11/2023] [Revised: 02/08/2024] [Accepted: 03/02/2024] [Indexed: 04/19/2024] Open
Abstract
BACKGROUND Clinical decision support (CDS) tools that incorporate machine learning-derived content have the potential to transform clinical care by augmenting clinicians' expertise. To realize this potential, such tools must be designed to fit the dynamic work systems of the clinicians who use them. We propose the use of academic detailing-personal visits to clinicians by an expert in a specific health IT tool-as a method for both ensuring the correct understanding of that tool and its evidence base and identifying factors influencing the tool's implementation. OBJECTIVE This study aimed to assess academic detailing as a method for simultaneously ensuring the correct understanding of an emergency department-based CDS tool to prevent future falls and identifying factors impacting clinicians' use of the tool through an analysis of the resultant qualitative data. METHODS Previously, our team designed a CDS tool to identify patients aged 65 years and older who are at the highest risk of future falls and prompt an interruptive alert to clinicians, suggesting the patient be referred to a mobility and falls clinic for an evidence-based preventative intervention. We conducted 10-minute academic detailing interviews (n=16) with resident emergency medicine physicians and advanced practice providers who had encountered our CDS tool in practice. We conducted an inductive, team-based content analysis to identify factors that influenced clinicians' use of the CDS tool. RESULTS The following categories of factors that impacted clinicians' use of the CDS were identified: (1) aspects of the CDS tool's design (2) clinicians' understanding (or misunderstanding) of the CDS or referral process, (3) the busy nature of the emergency department environment, (4) clinicians' perceptions of the patient and their associated fall risk, and (5) the opacity of the referral process. Additionally, clinician education was done to address any misconceptions about the CDS tool or referral process, for example, demonstrating how simple it is to place a referral via the CDS and clarifying which clinic the referral goes to. CONCLUSIONS Our study demonstrates the use of academic detailing for supporting the implementation of health information technologies, allowing us to identify factors that impacted clinicians' use of the CDS while concurrently educating clinicians to ensure the correct understanding of the CDS tool and intervention. Thus, academic detailing can inform both real-time adjustments of a tool's implementation, for example, refinement of the language used to introduce the tool, and larger scale redesign of the CDS tool to better fit the dynamic work environment of clinicians.
Collapse
Affiliation(s)
- Hanna J Barton
- BerbeeWalsh Department of Emergency Medicine, University of Wisconsin-Madison, Madison, WI, United States
| | - Apoorva Maru
- BerbeeWalsh Department of Emergency Medicine, University of Wisconsin-Madison, Madison, WI, United States
| | - Margaret A Leaf
- Department of Information Services, UW Health, Madison, WI, United States
| | - Daniel J Hekman
- BerbeeWalsh Department of Emergency Medicine, University of Wisconsin-Madison, Madison, WI, United States
| | - Douglas A Wiegmann
- Department of Industrial and Systems Engineering, University of Wisconsin-Madison, Madison, WI, United States
| | - Manish N Shah
- BerbeeWalsh Department of Emergency Medicine, University of Wisconsin-Madison, Madison, WI, United States
| | - Brian W Patterson
- BerbeeWalsh Department of Emergency Medicine, University of Wisconsin-Madison, Madison, WI, United States
- Department of Industrial and Systems Engineering, University of Wisconsin-Madison, Madison, WI, United States
| |
Collapse
|
3
|
Asgari E, Kaur J, Nuredini G, Balloch J, Taylor AM, Sebire N, Robinson R, Peters C, Sridharan S, Pimenta D. Impact of Electronic Health Record Use on Cognitive Load and Burnout Among Clinicians: Narrative Review. JMIR Med Inform 2024; 12:e55499. [PMID: 38607672 PMCID: PMC11053390 DOI: 10.2196/55499] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2023] [Revised: 02/15/2024] [Accepted: 03/11/2024] [Indexed: 04/13/2024] Open
Abstract
The cognitive load theory suggests that completing a task relies on the interplay between sensory input, working memory, and long-term memory. Cognitive overload occurs when the working memory's limited capacity is exceeded due to excessive information processing. In health care, clinicians face increasing cognitive load as the complexity of patient care has risen, leading to potential burnout. Electronic health records (EHRs) have become a common feature in modern health care, offering improved access to data and the ability to provide better patient care. They have been added to the electronic ecosystem alongside emails and other resources, such as guidelines and literature searches. Concerns have arisen in recent years that despite many benefits, the use of EHRs may lead to cognitive overload, which can impact the performance and well-being of clinicians. We aimed to review the impact of EHR use on cognitive load and how it correlates with physician burnout. Additionally, we wanted to identify potential strategies recommended in the literature that could be implemented to decrease the cognitive burden associated with the use of EHRs, with the goal of reducing clinician burnout. Using a comprehensive literature review on the topic, we have explored the link between EHR use, cognitive load, and burnout among health care professionals. We have also noted key factors that can help reduce EHR-related cognitive load, which may help reduce clinician burnout. The research findings suggest that inadequate efforts to present large amounts of clinical data to users in a manner that allows the user to control the cognitive burden in the EHR and the complexity of the user interfaces, thus adding more "work" to tasks, can lead to cognitive overload and burnout; this calls for strategies to mitigate these effects. Several factors, such as the presentation of information in the EHR, the specialty, the health care setting, and the time spent completing documentation and navigating systems, can contribute to this excess cognitive load and result in burnout. Potential strategies to mitigate this might include improving user interfaces, streamlining information, and reducing documentation burden requirements for clinicians. New technologies may facilitate these strategies. The review highlights the importance of addressing cognitive overload as one of the unintended consequences of EHR adoption and potential strategies for mitigation, identifying gaps in the current literature that require further exploration.
Collapse
Affiliation(s)
- Elham Asgari
- Guy's and St Thomas' NHS Trust, London, United Kingdom
- Tortus AI, London, United Kingdom
| | - Japsimar Kaur
- Manchester University NHS Foundation Trust, Manchester, United Kingdom
| | | | | | | | - Neil Sebire
- Great Ormond Street Hospital, London, United Kingdom
| | | | | | | | | |
Collapse
|
4
|
Molloy MJ, Muthu N, Orenstein EW, Shelov E, Luo BT. Clinical Decision Support Principles for Quality Improvement and Research. Hosp Pediatr 2024; 14:e219-e224. [PMID: 38545665 PMCID: PMC10965756 DOI: 10.1542/hpeds.2023-007540] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/02/2024]
Abstract
Pediatric hospitalists frequently interact with clinical decision support (CDS) tools in patient care and use these tools for quality improvement or research. In this method/ology paper, we provide an introduction and practical approach to developing and evaluating CDS tools within the electronic health record. First, we define CDS and describe the types of CDS interventions that exist. We then outline a stepwise approach to CDS development, which begins with defining the problem and understanding the system. We present a framework for metric development and then describe tools that can be used for CDS design (eg, 5 Rights of CDS, "10 commandments," usability heuristics, human-centered design) and testing (eg, validation, simulation, usability testing). We review approaches to evaluating CDS tools, which range from randomized studies to traditional quality improvement methods. Lastly, we discuss practical considerations for implementing CDS, including the assessment of a project team's skills and an organization's information technology resources.
Collapse
Affiliation(s)
- Matthew J. Molloy
- Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati, Ohio
- Divisions of Hospital Medicine
- Biomedical Informatics, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio
| | - Naveen Muthu
- Department of Pediatrics, Emory University School of Medicine, Atlanta, Georgia
- Division of Hospital Medicine, Children’s Healthcare of Atlanta, Atlanta, Georgia
| | - Evan W. Orenstein
- Department of Pediatrics, Emory University School of Medicine, Atlanta, Georgia
- Division of Hospital Medicine, Children’s Healthcare of Atlanta, Atlanta, Georgia
| | - Eric Shelov
- Department of Pediatrics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania
- Section of Pediatric Hospital Medicine
- Department of Biomedical and Health Informatics, Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania
| | - Brooke T. Luo
- Department of Pediatrics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania
- Section of Pediatric Hospital Medicine
- Department of Biomedical and Health Informatics, Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania
| |
Collapse
|
5
|
Sheehan KN, Cioci AL, Lucioni TM, Hernandez SM. Resident-Driven Clinical Decision Support Governance to Improve the Utility of Clinical Decision Support. Appl Clin Inform 2024; 15:335-341. [PMID: 38692282 PMCID: PMC11062759 DOI: 10.1055/s-0044-1786682] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/16/2023] [Accepted: 03/12/2024] [Indexed: 05/03/2024] Open
Abstract
OBJECTIVES This resident-driven quality improvement project aimed to better understand the known problem of a misaligned clinical decision support (CDS) strategy and improve CDS utilization. METHODS An internal survey was sent to all internal medicine (IM) residents to identify the most bothersome CDS alerts. Survey results were supported by electronic health record (EHR) data of CDS firing rates and response rates which were collected for each of the three most bothersome CDS tools. Changes to firing criteria were created to increase utilization and to better align with the five rights of CDS. Findings and proposed changes were presented to our institution's CDS Governance Committee. Changes were approved and implemented. Postintervention firing rates were then collected for 1 week. RESULTS Twenty nine residents participated in the CDS survey and identified sepsis alerts, lipid profile reminders, and telemetry renewals to be the most bothersome alerts. EHR data showed action rates for these CDS as low as 1%. We implemented changes to focus emergency department (ED)-based sepsis alerts to the right provider, better address the right information for lipid profile reminders, and select the right time in workflow for telemetry renewals to be most effective. With these changes we successfully eliminated ED-based sepsis CDS reminders for IM providers, saw a 97% reduction in firing rates for the lipid profile CDS, and noted a 55% reduction in firing rates for telemetry CDS. CONCLUSION This project highlighted that alert improvements spearheaded by resident teams can be completed successfully using robust CDS governance strategies and can effectively optimize interruptive alerts.
Collapse
Affiliation(s)
- Kristin N. Sheehan
- Department of Internal Medicine, Wake Forest School of Medicine, Winston-Salem, North Carolina, United States
| | - Anthony L. Cioci
- Department of Internal Medicine, Wake Forest School of Medicine, Winston-Salem, North Carolina, United States
| | - Tomas M. Lucioni
- Department of Internal Medicine, Wake Forest School of Medicine, Winston-Salem, North Carolina, United States
| | - Sean M. Hernandez
- Department of Internal Medicine, Wake Forest School of Medicine, Winston-Salem, North Carolina, United States
| |
Collapse
|
6
|
Fallon A, Haralambides K, Mazzillo J, Gleber C. Addressing Alert Fatigue by Replacing a Burdensome Interruptive Alert with Passive Clinical Decision Support. Appl Clin Inform 2024; 15:101-110. [PMID: 38086417 PMCID: PMC10830237 DOI: 10.1055/a-2226-8144] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/24/2023] [Accepted: 12/11/2023] [Indexed: 02/02/2024] Open
Abstract
BACKGROUND Recognizing that alert fatigue poses risks to patient safety and clinician wellness, there is a growing emphasis on evaluation and governance of electronic health record clinical decision support (CDS). This is particularly critical for interruptive alerts to ensure that they achieve desired clinical outcomes while minimizing the burden on clinicians. This study describes an improvement effort to address a problematic interruptive alert intended to notify clinicians about patients needing coronavirus disease 2019 (COVID) precautions and how we collaborated with operational leaders to develop an alternative passive CDS system in acute care areas. OBJECTIVES Our dual aim was to reduce the alert burden by redesigning the CDS to adhere to best practices for decision support while also improving the percent of admitted patients with symptoms of possible COVID who had appropriate and timely infection precautions orders. METHODS Iterative changes to CDS design included adjustment to alert triggers and acknowledgment reasons and development of a noninterruptive rule-based order panel for acute care areas. Data on alert burden and appropriate precautions orders on symptomatic admitted patients were followed over time on run and attribute (p) and individuals-moving range control charts. RESULTS At baseline, the COVID alert fired on average 8,206 times per week with an alert per encounter rate of 0.36. After our interventions, the alerts per week decreased to 1,449 and alerts per encounter to 0.07 equating to an 80% reduction for both metrics. Concurrently, the percentage of symptomatic admitted patients with COVID precautions ordered increased from 23 to 61% with a reduction in the mean time between COVID test and precautions orders from 19.7 to -1.3 minutes. CONCLUSION CDS governance, partnering with operational stakeholders, and iterative design led to successful replacement of a frequently firing interruptive alert with less burdensome passive CDS that improved timely ordering of COVID precautions.
Collapse
Affiliation(s)
- Anne Fallon
- Division of Pediatric Hospital Medicine, Department of Pediatrics, University of Rochester Medical Center, Rochester, New York, United States
| | - Kristina Haralambides
- Department of Otolaryngology, University of Rochester Medical Center, Rochester, New York, United States
| | - Justin Mazzillo
- Department of Emergency Medicine, University of Rochester Medical Center, Rochester, New York, United States
| | - Conrad Gleber
- Division of Hospital Medicine, Department of Medicine, University of Rochester Medical Center, Rochester, New York, United States
| |
Collapse
|
7
|
Kadura S, Schneider RB. Moving beyond alerts: Electronic health record strategies to improve inpatient Parkinson's disease care. Parkinsonism Relat Disord 2023; 116:105819. [PMID: 37635057 DOI: 10.1016/j.parkreldis.2023.105819] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/14/2023] [Accepted: 08/19/2023] [Indexed: 08/29/2023]
Affiliation(s)
- Sullafa Kadura
- Department of Medicine, University of Rochester, 601 Elmwood Ave, Rochester, NY, 14642, USA
| | - Ruth B Schneider
- Department of Neurology, University of Rochester, 601 Elmwood Ave, Rochester, NY, 14642, USA; Center for Health + Technology, University of Rochester, 265 Crittenden Blvd, Rochester, NY, 14642, USA.
| |
Collapse
|
8
|
Corrente C, Satkumaran S, Segal A, Butters C, Fernandez C, Babl FE, Orme LM, Thursky K, Haeusler GM. Evaluating the accuracy and efficacy of an electronic medical record alert to identify paediatric patients with low-risk febrile neutropenia. Int J Med Inform 2023; 178:105205. [PMID: 37703799 DOI: 10.1016/j.ijmedinf.2023.105205] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/14/2023] [Revised: 08/21/2023] [Accepted: 08/27/2023] [Indexed: 09/15/2023]
Abstract
BACKGROUND Point-of-care decision support, embedded into electronic medical record (EMR) workflows, has the potential to improve efficiency, reduce unwarranted variation and improve patient outcomes. A clinical-facing best practice advisory (BPA) in the Epic EMR system was developed to identify children admitted with low-risk febrile neutropenia (FN) who should be considered for treatment at home after a brief inpatient stay. We evaluated the accuracy and impact of this BPA and identify areas for improvement. METHODS The low-risk FN BPA was co-designed with key-stakeholders and implemented after a one-month testing phase. Mixed methodology was used to collect and analyse data. The sensitivity and positive predictive value of the BPA was calculated using FN episodes captured in a prospectively collected database. Overall effectiveness was defined as the proportion of alerts resulting in completion of a FN risk assessment flowsheet. RESULTS Over the 12-month period 176 FN episodes were admitted. Overall, the alert had poor sensitivity (58%) and positive predictive value (75%), failing to trigger in 62 (35%) episodes. In the episodes where the alert did trigger, the alert was frequently dismissed by clinicians (76%) and the overall effectiveness was extremely low (3%). Manual review of each FN episode without a BPA identified important design limitations and incorrect workflow assumptions. DISCUSSION Given the poor sensitivity and limited impact on clinician behaviour the low-risk BPA, in its current form, has not been an effective intervention at this site. While work is ongoing to enhance the accuracy of the BPA, alternative EMR workflows are likely required to improve the clinical impact.
Collapse
Affiliation(s)
| | | | - Ahuva Segal
- Centre for Health Analytics, Melbourne Children's Campus, Parkville, Australia; Department of Paediatrics, Faculty of Medicine, Dentistry and Health Sciences, University of Melbourne, Australia
| | - Coen Butters
- Murdoch Children's Research Institute, Parkville, Australia; Department of Paediatrics, Faculty of Medicine, Dentistry and Health Sciences, University of Melbourne, Australia
| | - Corinne Fernandez
- Children's Cancer Centre, Royal Children's Hospital, Parkville, Australia
| | - Franz E Babl
- Murdoch Children's Research Institute, Parkville, Australia; Centre for Health Analytics, Melbourne Children's Campus, Parkville, Australia; Department of Emergency Medicine, Royal Children's Hospital, Parkville, Australia; Department of Critical Care, Faculty of Medicine, Dentistry and Health Sciences, University of Melbourne, Australia
| | - Lisa M Orme
- Department of Paediatrics, Faculty of Medicine, Dentistry and Health Sciences, University of Melbourne, Australia; Children's Cancer Centre, Royal Children's Hospital, Parkville, Australia
| | - Karin Thursky
- Department of Infectious Diseases, Peter MacCallum Cancer Centre, Melbourne, Australia; NHMRC National Centre for Infections in Cancer, Sir Peter MacCallum Department of Oncology, University of Melbourne, Parkville, Australia; Sir Peter MacCallum Department of Oncology, University of Melbourne, Parkville, Australia; The Paediatric Integrated Cancer Service, Victoria, Australia
| | - Gabrielle M Haeusler
- Murdoch Children's Research Institute, Parkville, Australia; Department of Paediatrics, Faculty of Medicine, Dentistry and Health Sciences, University of Melbourne, Australia; Department of Infectious Diseases, Peter MacCallum Cancer Centre, Melbourne, Australia; NHMRC National Centre for Infections in Cancer, Sir Peter MacCallum Department of Oncology, University of Melbourne, Parkville, Australia; Sir Peter MacCallum Department of Oncology, University of Melbourne, Parkville, Australia; The Paediatric Integrated Cancer Service, Victoria, Australia; Infection Diseases Unit, Department of General Medicine, Royal Children's Hospital, Parkville, Australia.
| |
Collapse
|
9
|
Co Z, Classen DC, Cole JM, Seger DL, Madsen R, Davis T, McGaffigan P, Bates DW. How Safe are Outpatient Electronic Health Records? An Evaluation of Medication-Related Decision Support using the Ambulatory Electronic Health Record Evaluation Tool. Appl Clin Inform 2023; 14:981-991. [PMID: 38092360 PMCID: PMC10719043 DOI: 10.1055/s-0043-1777107] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2023] [Accepted: 10/24/2023] [Indexed: 12/17/2023] Open
Abstract
BACKGROUND The purpose of the Ambulatory Electronic Health Record (EHR) Evaluation Tool is to provide outpatient clinics with an assessment that they can use to measure the ability of the EHR system to detect and prevent common prescriber errors. The tool consists of a medication safety test and a medication reconciliation module. OBJECTIVES The goal of this study was to perform a broad evaluation of outpatient medication-related decision support using the Ambulatory EHR Evaluation Tool. METHODS We performed a cross-sectional study with 10 outpatient clinics using the Ambulatory EHR Evaluation Tool. For the medication safety test, clinics were provided test patients and associated medication test orders to enter in their EHR, where they recorded any advice or information they received. Once finished, clinics received an overall percentage score of unsafe orders detected and individual order category scores. For the medication reconciliation module, clinics were asked to electronically reconcile two medication lists, where modifications were made by adding and removing medications and changing the dosage of select medications. RESULTS For the medication safety test, the mean overall score was 57%, with the highest score being 70%, and the lowest score being 40%. Clinics performed well in the drug allergy (100%), drug dose daily (85%), and inappropriate medication combinations (74%) order categories. Order categories with the lowest performance were drug laboratory (10%) and drug monitoring (3%). Most clinics (90%) scored a 0% in at least one order category. For the medication reconciliation module, only one clinic (10%) could reconcile medication lists electronically; however, there was no clinical decision support available that checked for drug interactions. CONCLUSION We evaluated a sample of ambulatory practices around their medication-related decision support and found that advanced capabilities within these systems have yet to be widely implemented. The tool was practical to use and identified substantial opportunities for improvement in outpatient medication safety.
Collapse
Affiliation(s)
- Zoe Co
- Department of General Internal Medicine, Brigham and Women's Hospital, Boston, Massachusetts, United States
- Department of Learning Health Sciences, University of Michigan, Ann Arbor, Michigan, United States
| | - David C. Classen
- Division of Epidemiology, University of Utah, Salt Lake City, Utah, United States
| | - Jessica M. Cole
- Division of Epidemiology, University of Utah, Salt Lake City, Utah, United States
| | - Diane L. Seger
- Clinical and Quality Analysis, Mass General Brigham, Somerville, Massachusetts, United States
| | - Randy Madsen
- Biomedical Informatics Core, Clinical and Translational Science Institute, University of Utah, Salt Lake City, Utah, United States
| | - Terrance Davis
- Biomedical Informatics Core, Clinical and Translational Science Institute, University of Utah, Salt Lake City, Utah, United States
| | | | - David W. Bates
- Department of General Internal Medicine, Brigham and Women's Hospital, Boston, Massachusetts, United States
- Biomedical Informatics Core, Clinical and Translational Science Institute, University of Utah, Salt Lake City, Utah, United States
- Harvard Medical School, Boston, Massachusetts, United States
| |
Collapse
|
10
|
Ravi A, Arvisais-Anhalt S, Weia B, Khanna R, Adler-Milstein J, Auerbach A. Governance of Electronic Health Record Modification at U.S. Academic Medical Centers. Appl Clin Inform 2023; 14:843-854. [PMID: 37553071 PMCID: PMC10599807 DOI: 10.1055/a-2150-8523] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2023] [Accepted: 08/07/2023] [Indexed: 08/10/2023] Open
Abstract
OBJECTIVES A key aspect of electronic health record (EHR) governance involves the approach to EHR modification. We report a descriptive study to characterize EHR governance at academic medical centers (AMCs) across the United States. METHODS We conducted interviews with the Chief Medical Information Officers of 18 AMCs about the process of EHR modification for standard requests. Recordings of the interviews were analyzed to identify categories within prespecified domains. Responses were then assigned to categories for each domain. RESULTS At our AMCs, EHR requests were governed variably, with a similar number of sites using quantitative scoring systems (7, 38.9%), qualitative systems (5, 27.8%), or no scoring system (6, 33.3%). Two (11%) organizations formally review all requests for their impact on health equity. Although 14 (78%) organizations have trained physician builders/architects, their primary role was not for EHR build. Most commonly reported governance challenges included request volume (11, 61%), integrating diverse clinician input (3, 17%), and stakeholder buy-in (3, 17%). The slowest step in the process was clarifying end user requests (14, 78%). Few leaders had identified metrics for the success of EHR governance. CONCLUSION Governance approaches for managing EHR modification at AMCs are highly variable, which suggests ongoing efforts to balance EHR standardization and maintenance burden, while dealing with a high volume of requests. Developing metrics to capture the performance of governance and quantify problems may be a key step in identifying best practices.
Collapse
Affiliation(s)
- Akshay Ravi
- Department of Medicine, University of California, San Francisco, San Francisco, California, United States
| | - Simone Arvisais-Anhalt
- Department of Laboratory Medicine, University of California, San Francisco, San Francisco, California, United States
| | - Benjamin Weia
- Department of Medicine, University of California, San Francisco, San Francisco, California, United States
| | - Raman Khanna
- Department of Medicine, University of California, San Francisco, San Francisco, California, United States
| | - Julia Adler-Milstein
- Department of Medicine, University of California, San Francisco, San Francisco, California, United States
| | - Andrew Auerbach
- Department of Medicine, University of California, San Francisco, San Francisco, California, United States
| |
Collapse
|
11
|
Mills SC, Massmann A. Congruence rates for pharmacogenomic noninterruptive alerts. Pharmacogenomics 2023; 24:493-500. [PMID: 37435734 DOI: 10.2217/pgs-2023-0016] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/13/2023] Open
Abstract
Meaningful clinical decision support (CDS) recommendations are vital for implementation of pharmacogenomics (PGx) into routine clinical care. PGx CDS alerts include interruptive and noninterruptive alerts. The objective of this study was to evaluate provider ordering behavior after noninterruptive alerts are displayed. A retrospective manual chart review was conducted from the time of noninterruptive alert implementation to the time of data analysis to determine congruence with CDS recommendations. The congruence rate for noninterruptive alerts was 89.8% across all drug-gene interactions. The drug-gene interaction with the most alerts for analysis included metoclopramide (n = 138). The high rate of medication order congruence after noninterruptive alerts were deployed suggests this modality may be appropriate for PGx CDS as a method for best practice adherence.
Collapse
Affiliation(s)
- Sarah C Mills
- Sanford Imagenetics, Sanford Health, Sioux Falls, SD 57105, USA
| | - Amanda Massmann
- Sanford Imagenetics, Sanford Health, Sioux Falls, SD 57105, USA
- Department of Internal Medicine, University of South Dakota School of Medicine, Vermillion, SD 57069, USA
| |
Collapse
|
12
|
Cánovas-Segura B, Morales A, Juarez JM, Campos M. Meaningful time-related aspects of alerts in Clinical Decision Support Systems. A unified framework. J Biomed Inform 2023:104397. [PMID: 37245656 DOI: 10.1016/j.jbi.2023.104397] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/01/2022] [Revised: 03/11/2023] [Accepted: 05/15/2023] [Indexed: 05/30/2023]
Abstract
Alerts are a common functionality of clinical decision support systems (CDSSs). Although they have proven to be useful in clinical practice, the alert burden can lead to alert fatigue and significantly reduce their usability and acceptance. Based on a literature review, we propose a unified framework consisting of a set of meaningful timestamps that allows the use of state-of-the-art measures for alert burden, such as alert dwell time, alert think time, and response time. In addition, it can be used to investigate other measures that could be relevant as regards dealing with this problem. Furthermore, we provide a case study concerning three different types of alerts to which the framework was successfully applied. We consider that our framework can easily be adapted to other CDSSs and that it could be useful for dealing with alert burden measurement thus contributing to its appropriate management.
Collapse
Affiliation(s)
| | - Antonio Morales
- AIKE Research Group (INTICO), University of Murcia, Murcia, Spain.
| | - Jose M Juarez
- AIKE Research Group (INTICO), University of Murcia, Murcia, Spain.
| | - Manuel Campos
- AIKE Research Group (INTICO), University of Murcia, Murcia, Spain; Murcian Bio-Health Institute (IMIB-Arrixaca), Murcia, Spain.
| |
Collapse
|
13
|
Afshar M, Adelaine S, Resnik F, Mundt MP, Long J, Leaf M, Ampian T, Wills GJ, Schnapp B, Chao M, Brown R, Joyce C, Sharma B, Dligach D, Burnside ES, Mahoney J, Churpek MM, Patterson BW, Liao F. Deployment of Real-time Natural Language Processing and Deep Learning Clinical Decision Support in the Electronic Health Record: Pipeline Implementation for an Opioid Misuse Screener in Hospitalized Adults. JMIR Med Inform 2023; 11:e44977. [PMID: 37079367 PMCID: PMC10160938 DOI: 10.2196/44977] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/11/2022] [Revised: 02/01/2023] [Accepted: 03/26/2023] [Indexed: 04/21/2023] Open
Abstract
BACKGROUND The clinical narrative in electronic health records (EHRs) carries valuable information for predictive analytics; however, its free-text form is difficult to mine and analyze for clinical decision support (CDS). Large-scale clinical natural language processing (NLP) pipelines have focused on data warehouse applications for retrospective research efforts. There remains a paucity of evidence for implementing NLP pipelines at the bedside for health care delivery. OBJECTIVE We aimed to detail a hospital-wide, operational pipeline to implement a real-time NLP-driven CDS tool and describe a protocol for an implementation framework with a user-centered design of the CDS tool. METHODS The pipeline integrated a previously trained open-source convolutional neural network model for screening opioid misuse that leveraged EHR notes mapped to standardized medical vocabularies in the Unified Medical Language System. A sample of 100 adult encounters were reviewed by a physician informaticist for silent testing of the deep learning algorithm before deployment. An end user interview survey was developed to examine the user acceptability of a best practice alert (BPA) to provide the screening results with recommendations. The planned implementation also included a human-centered design with user feedback on the BPA, an implementation framework with cost-effectiveness, and a noninferiority patient outcome analysis plan. RESULTS The pipeline was a reproducible workflow with a shared pseudocode for a cloud service to ingest, process, and store clinical notes as Health Level 7 messages from a major EHR vendor in an elastic cloud computing environment. Feature engineering of the notes used an open-source NLP engine, and the features were fed into the deep learning algorithm, with the results returned as a BPA in the EHR. On-site silent testing of the deep learning algorithm demonstrated a sensitivity of 93% (95% CI 66%-99%) and specificity of 92% (95% CI 84%-96%), similar to published validation studies. Before deployment, approvals were received across hospital committees for inpatient operations. Five interviews were conducted; they informed the development of an educational flyer and further modified the BPA to exclude certain patients and allow the refusal of recommendations. The longest delay in pipeline development was because of cybersecurity approvals, especially because of the exchange of protected health information between the Microsoft (Microsoft Corp) and Epic (Epic Systems Corp) cloud vendors. In silent testing, the resultant pipeline provided a BPA to the bedside within minutes of a provider entering a note in the EHR. CONCLUSIONS The components of the real-time NLP pipeline were detailed with open-source tools and pseudocode for other health systems to benchmark. The deployment of medical artificial intelligence systems in routine clinical care presents an important yet unfulfilled opportunity, and our protocol aimed to close the gap in the implementation of artificial intelligence-driven CDS. TRIAL REGISTRATION ClinicalTrials.gov NCT05745480; https://www.clinicaltrials.gov/ct2/show/NCT05745480.
Collapse
Affiliation(s)
- Majid Afshar
- University of Wisconsin - Madison, Madison, WI, United States
| | | | - Felice Resnik
- University of Wisconsin - Madison, Madison, WI, United States
| | - Marlon P Mundt
- University of Wisconsin - Madison, Madison, WI, United States
| | - John Long
- University of Wisconsin - Madison, Madison, WI, United States
| | - Margaret Leaf
- University of Wisconsin - Madison, Madison, WI, United States
| | - Theodore Ampian
- University of Wisconsin - Madison, Madison, WI, United States
| | - Graham J Wills
- University of Wisconsin - Madison, Madison, WI, United States
| | | | - Michael Chao
- University of Wisconsin - Madison, Madison, WI, United States
| | - Randy Brown
- University of Wisconsin - Madison, Madison, WI, United States
| | - Cara Joyce
- Loyola University Chicago, Chicago, IL, United States
| | - Brihat Sharma
- University of Wisconsin - Madison, Madison, WI, United States
| | | | | | - Jane Mahoney
- University of Wisconsin - Madison, Madison, WI, United States
| | | | | | - Frank Liao
- University of Wisconsin - Madison, Madison, WI, United States
| |
Collapse
|
14
|
Miller SD, Murphy Z, Gray JH, Marsteller J, Oliva-Hemker M, Maslen A, Lehmann HP, Nagy P, Hutfless S, Gurses AP. Human-Centered Design of a Clinical Decision Support for Anemia Screening in Children with Inflammatory Bowel Disease. Appl Clin Inform 2023; 14:345-353. [PMID: 36809791 PMCID: PMC10171996 DOI: 10.1055/a-2040-0578] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/15/2022] [Accepted: 02/17/2023] [Indexed: 02/23/2023] Open
Abstract
BACKGROUND Inflammatory bowel disease (IBD) commonly leads to iron deficiency anemia (IDA). Rates of screening and treatment of IDA are often low. A clinical decision support system (CDSS) embedded in an electronic health record could improve adherence to evidence-based care. Rates of CDSS adoption are often low due to poor usability and fit with work processes. One solution is to use human-centered design (HCD), which designs CDSS based on identified user needs and context of use and evaluates prototypes for usefulness and usability. OBJECTIVES this study aimed to use HCD to design a CDSS tool called the IBD Anemia Diagnosis Tool, IADx. METHODS Interviews with IBD practitioners informed creation of a process map of anemia care that was used by an interdisciplinary team that used HCD principles to create a prototype CDSS. The prototype was iteratively tested with "Think Aloud" usability evaluation with clinicians as well as semi-structured interviews, a survey, and observations. Feedback was coded and informed redesign. RESULTS Process mapping showed that IADx should function at in-person encounters and asynchronous laboratory review. Clinicians desired full automation of clinical information acquisition such as laboratory trends and analysis such as calculation of iron deficit, less automation of clinical decision selection such as laboratory ordering, and no automation of action implementation such as signing medication orders. Providers preferred an interruptive alert over a noninterruptive reminder. CONCLUSION Providers preferred an interruptive alert, perhaps due to the low likelihood of noticing a noninterruptive advisory. High levels of desire for automation of information acquisition and analysis with less automation of decision selection and action may be generalizable to other CDSSs designed for chronic disease management. This underlines the ways in which CDSSs have the potential to augment rather than replace provider cognitive work.
Collapse
Affiliation(s)
- Steven D. Miller
- Division of Pediatric Gastroenterology, Hepatology, and Nutrition, Johns Hopkins University School of Medicine, Baltimore, Maryland, United States
| | - Zachary Murphy
- Division of Pediatric Gastroenterology, Hepatology, and Nutrition, Johns Hopkins University School of Medicine, Baltimore, Maryland, United States
| | - Joshua H. Gray
- Division of Pediatric Gastroenterology, Hepatology, and Nutrition, Johns Hopkins University School of Medicine, Baltimore, Maryland, United States
| | - Jill Marsteller
- Department of Health Policy and Management, Johns Hopkins University School of Medicine Armstrong Institute for Patient Safety and Quality, Baltimore, Maryland, United States
| | - Maria Oliva-Hemker
- Division of Pediatric Gastroenterology, Hepatology, and Nutrition, Johns Hopkins University School of Medicine, Baltimore, Maryland, United States
| | - Andrew Maslen
- Information Technology at Johns Hopkins Health System, Epic Project Leadership, Johns Hopkins Health System, Baltimore, Maryland, United States
| | - Harold P. Lehmann
- Division of Health Science Informatics, Johns Hopkins University, Baltimore, Maryland, United States
| | - Paul Nagy
- Department of Radiology, Johns Hopkins University School of Medicine, Johns Hopkins Technology Ventures, Baltimore, Maryland, United States
| | - Susan Hutfless
- Division of Gastroenterology and Hepatology, Johns Hopkins University, Baltimore, Maryland, United States
| | - Ayse P. Gurses
- Information Technology at Johns Hopkins Health System, Epic Project Leadership, Johns Hopkins Health System, Baltimore, Maryland, United States
| |
Collapse
|
15
|
Nguyen JQ, Crews KR, Moore BT, Kornegay NM, Baker DK, Hasan M, Campbell PK, Dean SM, Relling MV, Hoffman JM, Haidar CE. Clinician adherence to pharmacogenomics prescribing recommendations in clinical decision support alerts. J Am Med Inform Assoc 2022; 30:132-138. [PMID: 36228116 PMCID: PMC9748527 DOI: 10.1093/jamia/ocac187] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/07/2022] [Revised: 09/26/2022] [Accepted: 10/02/2022] [Indexed: 12/15/2022] Open
Abstract
Thoughtful integration of interruptive clinical decision support (CDS) alerts within the electronic health record is essential to guide clinicians on the application of pharmacogenomic results at point of care. St. Jude Children's Research Hospital implemented a preemptive pharmacogenomic testing program in 2011 in a multidisciplinary effort involving extensive education to clinicians about pharmacogenomic implications. We conducted a retrospective analysis of clinicians' adherence to 4783 pharmacogenomically guided CDS alerts that triggered for 12 genes and 60 drugs. Clinicians adhered to the therapeutic recommendations provided in 4392 alerts (92%). In our population of pediatric patients with catastrophic illnesses, the most frequently presented gene/drug CDS alerts were TPMT/NUDT15 and thiopurines (n = 3850), CYP2D6 and ondansetron (n = 667), CYP2D6 and oxycodone (n = 99), G6PD and G6PD high-risk medications (n = 51), and CYP2C19 and proton pump inhibitors (omeprazole and pantoprazole; n = 50). The high adherence rate was facilitated by our team approach to prescribing and our collaborative CDS design and delivery.
Collapse
Affiliation(s)
- Jenny Q Nguyen
- Department of Pharmacy and Pharmaceutical Sciences, St. Jude Children’s Research Hospital, Memphis, Tennessee, USA
| | - Kristine R Crews
- Department of Pharmacy and Pharmaceutical Sciences, St. Jude Children’s Research Hospital, Memphis, Tennessee, USA
| | - Ben T Moore
- Department of Pharmacy and Pharmaceutical Sciences, St. Jude Children’s Research Hospital, Memphis, Tennessee, USA
| | - Nancy M Kornegay
- Department of Pharmacy and Pharmaceutical Sciences, St. Jude Children’s Research Hospital, Memphis, Tennessee, USA
| | - Donald K Baker
- Department of Information Services, St. Jude Children’s Research Hospital, Memphis, Tennessee, USA
| | - Murad Hasan
- Department of Pharmacy and Pharmaceutical Sciences, St. Jude Children’s Research Hospital, Memphis, Tennessee, USA
| | - Patrick K Campbell
- Department of Information Services, St. Jude Children’s Research Hospital, Memphis, Tennessee, USA
- Department of Oncology, St. Jude Children’s Research Hospital, Memphis, Tennessee, USA
| | - Shannon M Dean
- Department of Information Services, St. Jude Children’s Research Hospital, Memphis, Tennessee, USA
- Department of Pediatrics, St. Jude Children’s Research Hospital, Memphis, Tennesse, USA
| | - Mary V Relling
- Department of Pharmacy and Pharmaceutical Sciences, St. Jude Children’s Research Hospital, Memphis, Tennessee, USA
| | - James M Hoffman
- Department of Pharmacy and Pharmaceutical Sciences, St. Jude Children’s Research Hospital, Memphis, Tennessee, USA
- Department of the Office of Quality and Patient Safety, St. Jude Children’s Research Hospital, Memphis, Tennesse, USA
| | - Cyrine E Haidar
- Department of Pharmacy and Pharmaceutical Sciences, St. Jude Children’s Research Hospital, Memphis, Tennessee, USA
| |
Collapse
|
16
|
Ebbers T, Kool RB, Smeele LE, Takes RP, van den Broek GB, Dirven R. Quantifying the Electronic Health Record Burden in Head and Neck Cancer Care. Appl Clin Inform 2022; 13:857-864. [PMID: 36104154 PMCID: PMC9474268 DOI: 10.1055/s-0042-1756422] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022] Open
Abstract
Background
Although the main task of health care providers is to provide patient care, studies show that increasing amounts of time are spent on documentation.
Objective
To quantify the time and effort spent on the electronic health record (EHR) in head and neck cancer care.
Methods
Cross-sectional time–motion study. Primary outcomes were the percentages of time spent on the EHR and the three main tasks (chart review, input, placing orders), number of mouse events, and keystrokes per consultation. Secondary outcome measures were perceptions of health care providers regarding EHR documentation and satisfaction.
Results
In total, 44.0% of initial oncological consultation (IOC) duration and 30.7% of follow-up consultation (FUC) duration are spent on EHR tasks. During 80.0% of an IOC and 67.9% of a FUC, the patient and provider were actively communicating. Providers required 593 mouse events and 1,664 keystrokes per IOC and 140 mouse events and 597 keystrokes per FUC, indicating almost 13 mouse clicks and close to 40 keystrokes for every minute of consultation time. Less than a quarter of providers indicated that there is enough time for documentation.
Conclusion
This study quantifies the widespread concern of high documentation burden for health care providers in oncology, which has been related to burnout and a decrease of patient–clinician interaction. Despite excessive time and effort spent on the EHR, health care providers still felt this was insufficient for proper documentation. However, the need for accurate and complete documentation is high, as reuse of information becomes increasingly important. The challenge is to decrease the documentation burden while increasing the quality of EHR data.
Collapse
Affiliation(s)
- Tom Ebbers
- Department of Otorhinolaryngology and Head and Neck Surgery, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Rudolf B Kool
- IQ Healthcare, Radboud Institute for Health Sciences, Radboud University Medical Centre, Nijmegen, The Netherlands
| | - Ludi E Smeele
- Department of Head and Neck Oncology and Surgery, Antoni van Leeuwenhoek, Amsterdam, The Netherlands
| | - Robert P Takes
- Department of Otorhinolaryngology and Head and Neck Surgery, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Guido B van den Broek
- Department of Otorhinolaryngology and Head and Neck Surgery, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Richard Dirven
- Department of Head and Neck Oncology and Surgery, Antoni van Leeuwenhoek, Amsterdam, The Netherlands
| |
Collapse
|