Mixed methods study of medication-related decision support alerts experienced during electronic prescribing for inpatients at an English hospital

Understanding "Alert Fatigue" in Primary Care: Qualitative Systematic Review of General Practitioners Attitudes and Experiences of Clinical Alerts, Prompts, and Reminders

BACKGROUND

The consistency and quality of care in modern primary care are supported by various clinical reminders (CRs), which include "alerts" describing the consequences of certain decisions and "prompts" that remind users to perform tasks promoting desirable clinical behaviors. However, not all CRs are acted upon, and many are disregarded by general practitioners (GPs), a chronic issue commonly referred to as "alert fatigue." This phenomenon has significant implications for the safety and quality of care, GP burnout, and broader medicolegal consequences. Research on mitigating alert fatigue and optimizing the use of CRs remains limited. This review offers much-needed insight into GP attitudes toward the deployment, design, and overall effectiveness of CRs.

OBJECTIVE

This systematic review aims to synthesize current qualitative research on GPs' attitudes toward CRs, enabling an exploration of the interacting influences on the occurrence of alert fatigue in GPs, including the deployment, design, and perceived efficacy of CRs.

METHODS

A systematic literature search was conducted across the Health Technology Assessment database, MEDLINE, MEDLINE In-Process, Embase, CINAHL, Conference Proceedings Citation Index, PsycINFO, and OpenGrey. The search focused on primary qualitative and mixed methods research conducted in general or family practice, specifically exploring GPs' experiences with CRs. All databases were searched from inception to December 31, 2023. To ensure structured and practicable findings, we used a directed content analysis of the data, guided by the 7 domains of the Non-adoption, Abandonment, Scale-up, Spread, and Sustainability (NASSS) framework, including domains related to Technology, Adopter attitudes, and Organization.

RESULTS

A total of 9 studies were included, and the findings were organized within the 7 domains. Regarding Condition and Value Proposition, GPs viewed CRs as an effective way to maintain or improve the safety and quality of care they provide. When considering the attributes of the Technology, the efficacy of CRs was linked to their frequency, presentation, and the accuracy of their content. Within Adopters, concerns were raised about the accuracy of CRs and the risk that their use could diminish the value of GP experience and contextual understanding. From an Organization perspective, the need for training on the use and benefits of CRs was highlighted. Finally, in the context of the Wider system and their Embedding Over Time, suggestions included sharing best practices for CR use and involving GPs in their design.

CONCLUSIONS

While GPs acknowledged that CRs, when used optimally, can enhance patient safety and quality of care, several concerns emerged regarding their design, content accuracy, and lack of contextual nuance. Suggestions to improve CR adherence included providing coherent training, enhancing their design, and incorporating more personalized content.

TRIAL REGISTRATION

PROSPERO CRD42016029418; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=29418.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.1186/s13643-017-0627-z.

A mixed methods analysis of the medication review intervention centered around the use of the 'Systematic Tool to Reduce Inappropriate Prescribing' Assistant (STRIPA) in Swiss primary care practices

BACKGROUND

Electronic clinical decision support systems (eCDSS), such as the 'Systematic Tool to Reduce Inappropriate Prescribing' Assistant (STRIPA), have become promising tools for assisting general practitioners (GPs) with conducting medication reviews in older adults. Little is known about how GPs perceive eCDSS-assisted recommendations for pharmacotherapy optimization. The aim of this study was to explore the implementation of a medication review intervention centered around STRIPA in the 'Optimising PharmacoTherapy In the multimorbid elderly in primary CAre' (OPTICA) trial.

METHODS

We used an explanatory mixed methods design combining quantitative and qualitative data. First, quantitative data about the acceptance and implementation of eCDSS-generated recommendations from GPs (n = 21) and their patients (n = 160) in the OPTICA intervention group were collected. Then, semi-structured qualitative interviews were conducted with GPs from the OPTICA intervention group (n = 8), and interview data were analyzed through thematic analysis.

RESULTS

In quantitative findings, GPs reported averages of 13 min spent per patient preparing the eCDSS, 10 min performing medication reviews, and 5 min discussing prescribing recommendations with patients. On average, out of the mean generated 3.7 recommendations (SD=1.8). One recommendation to stop or start a medication was reported to be implemented per patient in the intervention group (SD=1.2). Overall, GPs found the STRIPA useful and acceptable. They particularly appreciated its ability to generate recommendations based on large amounts of patient information. During qualitative interviews, GPs reported the main reasons for limited implementation of STRIPA were related to problems with data sourcing (e.g., incomplete data imports), preparation of the eCDSS (e.g., time expenditure for updating and adapting information), its functionality (e.g., technical problems downloading PDF recommendation reports), and appropriateness of recommendations.

CONCLUSIONS

Qualitative findings help explain the relatively low implementation of recommendations demonstrated by quantitative findings, but also show GPs' overall acceptance of STRIPA. Our results provide crucial insights for adapting STRIPA to make it more suitable for regular use in future primary care settings (e.g., necessity to improve data imports).

TRIAL REGISTRATION

Clinicaltrials.gov NCT03724539, date of first registration: 29/10/2018.

Antimicrobial stewardship using electronic prescribing systems in hospital settings: a scoping review of interventions and outcome measures

Objectives

To identify interventions implemented in hospital electronic prescribing systems and the outcome measures used to monitor their impact.

Methods

We systematically searched CINAHL, EMBASE, Google Scholar and Medline using keywords in three strands: (i) population: hospital inpatient or emergency department; (ii) intervention: electronic prescribing functionality; and (iii) outcome: antimicrobial stewardship. The interventions were grouped into six themes: alerts, order sets, restriction of access, mandated documentation, embedded guidelines and automatic prescription stop. The outcome measures were organized into those that measure the quality or quantity of prescribing or clinical decision support (CDS) activity. The impact of each intervention reported was grouped into a positive, negative or no change.

Results

A total of 28 studies were eligible for inclusion. There were 28 different interventions grouped into the six themes. Alerts visible to the practitioner in the electronic health record (EHR) were most frequently implemented (n = 11/28). Twenty different outcome measures were identified, divided into quality (n = 13/20) and quantity outcomes (n = 4/20) and CDS activity (n = 3/20). One-third of outcomes reported across the 28 studies showed positive change (34.4%, n = 42/122) and 61.4% (n = 75/122) showed no change.

Conclusions

The most frequently implemented interventions were alerts, the majority of which were to influence behaviour or decision-making of the practitioner within the EHR. Quality outcomes were most frequently selected by researchers. The review supports previous research that larger well-designed randomized studies are needed to investigate the impact of interventions on AMS and outcome measures to be standardized.

Prescribers’ reported acceptance and use of drug-drug interaction alerts: An Australian survey

Drug-drug interaction (DDI) alerts are frequently included in electronic medical record (eMR) systems to provide users with relevant information and guidance at the point of care. In this study, we aimed to examine views of DDI alerts among prescribers, including junior doctors, registrars and senior doctors, across Australia. A validated survey for assessing prescribers’ reported acceptance and use of DDI alerts was distributed among researcher networks and in newsletters. Fifty useable responses were received, more than half ( n = 28) from senior doctors. Prescribers at all levels expected DDI alerts to improve performance but junior doctors reported that this was at a high cost, with respect to time and effort. Senior doctors and registrars reported rarely reading alerts and rarely changing prescribing decisions based on alerts. Respondents identified a number of problems with current alerts including limited relevance, repetition, and poor design, highlighting some clear areas for alert improvement.

Retrospective descriptive assessment of clinical decision support medication-related alerts in two Saudi Arabian hospitals

Objectives

To determine the frequency of clinical decision support system (CDSS) medication-related alerts generated, accepted, or overridden, to assess appropriateness of alert display and overrides, and to characterise the documentation of clinician justification for these overrides in an academic medical centre in Saudi Arabia.

Materials and methods

System-generated CDSS reports for the period June 2015 to December 2017 were retrospectively reviewed and analysed. Alerts were classified into different types, and rates of alert overrides calculated as percentages of all generated alerts. A subset of 307 overridden alerts was assessed for appropriateness of display and override by two clinical pharmacists. Physician documentation of reasons for overriding alerts were categorised.

Results

A total of 4,446,730 medication-related alerts were generated from both inpatient and outpatient settings, and 4,231,743 (95.2%) were overridden. The most common alert type was ‘duplicate drug’, accounting for 3,549,736 (79.8%) of alerts. Of 307 alerts assessed for appropriateness, 246 (80%) were judged to be appropriately displayed and 244 (79%) were overridden appropriately. New drug allergy and drug allergy alerts had the highest percentage of being judged as inappropriately overridden. For 1,594,313 alerts (37.7%), ‘no overridden reason selected’ was chosen from the drop-down menu.

Conclusions

The alert generation and override rate were higher than reported previously in the literature. The small sample size of 307 alerts assessed for appropriateness of alert display and override is a potential limitation. Revision of the CDSS rules for alerts (focusing on specificity and relevance for the local context) is now recommended. Future research should prospectively assess providers’ perspectives, and determine patient harm associated with overridden alerts.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-022-01838-1.

Use of Simulation to Measure the Effects of Just-in-Time Information to Prevent Nursing Medication Errors: A Randomized Controlled Study

INTRODUCTION

Medication administration error (MAE) is the improper dispensing of medication. It is a significant contributor to the occurrence of medical errors. A novel systems thinking approach using a pediatric simulation and student nurses were used to evaluate the benefit of applying just-in-time information (JITI) to reduce medication errors. Just-in-time information applies highly focused information delivered when needed.

METHODS

A smart device app was developed to provide JITI medication administration information. The effect JITI had on MAE occurrence was assessed via a controlled study. The study population included 38 teams having 2 to 3 senior nursing students on each team. The teams were separated into a control and 2 intervention groups to complete a medication administration simulation.

RESULTS

The intervention groups (100%, N = 10) that made significant use of the JITI app demonstrated improved performance for medication administration over the control group. Familiarity with the app was pivotal to how frequently it was used and to the success of the groups in administering medications. Although those with access to the app having limited training successfully executed the simulation 27.3% (n = 11) of the time, those with extended training had a success rate of 77.8% (n = 9).

CONCLUSIONS

Providing JITI significantly reduced the occurrence of MAEs for these student nurses. Familiarity with the app, including extended training opportunities, contributed significantly to student success.

A cross-sectional study on prescribing and dispensing errors at a corporate hospital in South India

BACKGROUND

The substantial and increasing use of medications escalating the risk of harm globally. The serious medication errors in hospital and community settings resulting from patient injury and death. Hence, a cross-sectional study was aimed to analyse the prescribing and dispensing errors in the outpatient departments of a south Indian hospital.

MATERIALS AND METHODS

A prospective cross-sectional study was carried out to evaluate the prescribing, and dispensing errors in outpatients who seek patient counseling at the tertiary care multispecialty hospital. The data were collected from various sources such as patient's prescriptions and dispensing records from the pharmacy.

RESULTS

A total of 500 prescriptions were screened and identified 65.60% of prescriptions with at least any one type of medication errors. Out of 328 prescriptions, 96.04% were handwritten and 3.96% were computerised prescriptions. Among the 328 prescriptions with medication errors, 32.62% noticed prescribing errors, 37.80% with dispensing errors, and 29.58% with both prescribing and dispensing errors. Out of these 328 prescriptions, 74.09% prescriptions were found to have polypharmacy.

DISCUSSION

Medication errors are serious problems in healthcare and can be a source of significant morbidity and mortality in healthcare settings. The present study showed that dispensing errors were the most common among the types of medication errors, in these particularly wrong directions were the most common types of errors.

CONCLUSION

This study concludes that the overall prevalence of medication errors was around 80%, but there were no life-threatening events observed. A clinical pharmacist can play a major role in this situation appears to be a strong intervention and early detection and prevention of medication errors and thus can improve the quality of care to the patients.

The appropriateness of clinical decision support systems alerts in supporting clinical workflows: A systematic review

A CDSS generates a high number of inappropriate alerts that interrupt the clinical workflow. As a result, clinicians silence, disable, or ignore alerts, thereby undermining patient safety. Therefore, the effectiveness and appropriateness of CDSS alerts need to be evaluated. A systematic review was carried out to identify the factors that affect CDSS alert appropriateness in supporting clinical workflow. Seven electronic databases (PubMed, Scopus, ACM, Science Direct, IEEE, Ovid Medline, and Ebscohost) were searched for English language articles published between 1997 and 2018. Seventy six papers met the inclusion criteria, of which 26, 24, 15, and 11 papers are retrospective cohort, qualitative, quantitative, and mixed-method studies, respectively. The review highlights various factors influencing the appropriateness and efficiencies of CDSS alerts. These factors are categorized into technology, human, organization, and process aspects using a combination of approaches, including socio-technical framework, five rights of CDSS, and Lean. Most CDSS alerts were not properly designed based on human factor methods and principles, explaining high alert overrides in clinical practices. The identified factors and recommendations from the review may offer valuable insights into how CDSS alerts can be designed appropriately to support clinical workflow.

Improving smoking cessation referrals among elective surgery clinics through electronic clinical decision support

INTRODUCTION

Preoperative visits are an exceptional opportunity to encourage smoking cessation, as studies demonstrate the experience of scheduling elective surgery produces an actionable incentive to quit. However, studies suggest surgeons do not regularly assess smoking behavior or offer cessation therapies. Clinical decision support (CDS) is a system in which providers are presented with clinically integrated tools to enhance decision-making.

METHODS

A CDS tool was designed to facilitate treatment referrals for smoking cessation services among patients seeking elective surgery. Two clinics were selected: the plastic and vascular surgeries. The study objectives were to assess the utilization rate and effectiveness of this system.

RESULTS

No smoking cessation referrals had been submitted by the plastic surgery or vascular surgery clinics in the year before CDS tool implementation. Providers at the plastic surgery clinic utilized the CDS tool in 95.0% (191 of 201) eligible patient encounters. Of these patients, 16.3% were identified as active smokers, and 16.1% of these smokers accepted treatment referrals. Providers at the vascular surgery clinic utilized the CDS tool in 50.3% (98 of 195) eligible patient encounters. Of these patients, 10.2% were identified as active smokers, and 30.0% of these smokers accepted treatment referrals.

CONCLUSIONS

The CDS tool improved the incidence of smoking cessation referrals in two surgical clinics from pretest baselines and achieved satisfactory utilization rates. This report demonstrates the feasibility of CDS tools to actualize the preoperative visit as an opportunity to promote smoking cessation.

Protocol for a qualitative study to identify strategies to optimise hospital ePrescribing systems

INTRODUCTION

Electronic prescribing (ePrescribing) is a key area of development and investment in the UK and across the developed world. ePrescribing is widely understood as a vehicle for tackling medication-related safety concerns, improving care quality and making more efficient use of health resources. Nevertheless, implementation of an electronic health record does not itself ensure benefits for prescribing are maximised. We examine the process of optimisation of ePrescribing systems using case studies to provide policy recommendations based on the experiences of digitally mature hospital sites.

METHODS AND ANALYSIS

Qualitative interviews within six digitally mature sites will be carried out. The aim is to capture successful optimisation of electronic prescribing (ePrescribing) in particular health systems and hospitals. We have identified hospital sites in the UK and in three other developed countries. We used a combination of literature reviews and advice from experts at Optimising ePrescribing in Hospitals (eP Opt) Project round-table events. Sites were purposively selected based on geographical area, innovative work in ePrescribing/electronic health (eHealth) and potential transferability of practices to the UK setting. Interviews will be recorded and transcribed and transcripts coded thematically using NVivo software. Relevant policy and governance documents will be analysed, where available. Planned site visits were suspended due to the COVID-19 pandemic.

ETHICS AND DISSEMINATION

The Usher Research Ethics Group granted approval for this study. Results will be disseminated via peer-reviewed journals in medical informatics and expert round-table events, lay member meetings and the ePrescribing Toolkit (http://www.eprescribingtoolkit.com/)-an online resource supporting National Health Service (NHS) hospitals through the ePrescribing process.

Machine Learning Approach to Reduce Alert Fatigue Using a Disease Medication-Related Clinical Decision Support System: Model Development and Validation

Background

Computerized physician order entry (CPOE) systems are incorporated into clinical decision support systems (CDSSs) to reduce medication errors and improve patient safety. Automatic alerts generated from CDSSs can directly assist physicians in making useful clinical decisions and can help shape prescribing behavior. Multiple studies reported that approximately 90%-96% of alerts are overridden by physicians, which raises questions about the effectiveness of CDSSs. There is intense interest in developing sophisticated methods to combat alert fatigue, but there is no consensus on the optimal approaches so far.

Objective

Our objective was to develop machine learning prediction models to predict physicians’ responses in order to reduce alert fatigue from disease medication–related CDSSs.

Methods

We collected data from a disease medication–related CDSS from a university teaching hospital in Taiwan. We considered prescriptions that triggered alerts in the CDSS between August 2018 and May 2019. Machine learning models, such as artificial neural network (ANN), random forest (RF), naïve Bayes (NB), gradient boosting (GB), and support vector machine (SVM), were used to develop prediction models. The data were randomly split into training (80%) and testing (20%) datasets.

Results

A total of 6453 prescriptions were used in our model. The ANN machine learning prediction model demonstrated excellent discrimination (area under the receiver operating characteristic curve [AUROC] 0.94; accuracy 0.85), whereas the RF, NB, GB, and SVM models had AUROCs of 0.93, 0.91, 0.91, and 0.80, respectively. The sensitivity and specificity of the ANN model were 0.87 and 0.83, respectively.

Conclusions

In this study, ANN showed substantially better performance in predicting individual physician responses to an alert from a disease medication–related CDSS, as compared to the other models. To our knowledge, this is the first study to use machine learning models to predict physician responses to alerts; furthermore, it can help to develop sophisticated CDSSs in real-world clinical settings.

Digital Health in Physicians' and Pharmacists' Office: A Comparative Study of e-Prescription Systems' Architecture and Digital Security in Eight Countries

e-Prescription systems are key components and drivers of digital health. They can enhance the safety of the patients, and are gaining popularity in health care systems around the world. Yet, there is little knowledge on comparative international analysis of e-Prescription systems' architecture and digital security. We report, in this study, original findings from a comparative analysis of the e-Prescription systems in eight different countries, namely, Canada, United States, United Kingdom, Australia, Spain, Japan, Sweden, and Denmark. We surveyed the databases related to pharmacies, eHealth, e-Prescriptions, and related digital health websites for each country, and their system architectures. We also compared the digital security and privacy protocols in place within and across these digital systems. We evaluated the systems' authentication protocols used by pharmacies to verify patients' identities during the medication dispensing process. Furthermore, we examined the supporting systems/services used to manage patients' medication histories and enhance patients' medication safety. Taken together, we report, in this study, original comparative findings on the limitations and challenges of the surveyed systems as well as in adopting e-Prescription systems. While the present study was conducted before the onset of COVID-19, e-Prescription systems have become highly relevant during the current pandemic and hence, a deeper understanding of the country systems' architecture and digital security that can help design effective strategies against the pandemic. e-Prescription systems can help reduce physical contact and the risk of exposure to the virus, as well as the wait times in pharmacies, thus enhancing patient safety and improving planetary health.

Computerized provider order entry-related medication errors among hospitalized patients: An integrative review

The Institute of Medicine estimates that 7,000 lives are lost yearly as a result of medication errors. Computerized physician and/or provider order entry was one of the proposed solutions to overcome this tragic issue. Despite some promising data about its effectiveness, it has been found that computerized provider order entry may facilitate medication errors.The purpose of this review is to summarize current evidence of computerized provider order entry -related medication errors and address the sociotechnical factors impacting the safe use of computerized provider order entry. By using PubMed and Google Scholar databases, a systematic search was conducted for articles published in English between 2007 and 2019 regarding the unintended consequences of computerized provider order entry and its related medication errors. A total of 288 articles were screened and categorized based on their use within the review. One hundred six articles met our pre-defined inclusion criteria and were read in full, in addition to another 27 articles obtained from references. All included articles were classified into the following categories: rates and statistics on computerized provider order entry -related medication errors, types of computerized provider order entry -related unintended consequences, factors contributing to computerized provider order entry failure, and recommendations based on addressing sociotechnical factors. Identifying major types of computerized provider order entry -related unintended consequences and addressing their causes can help in developing appropriate strategies for safe and effective computerized provider order entry. The interplay between social and technical factors can largely affect its safe implementation and use. This review discusses several factors associated with the unintended consequences of this technology in healthcare settings and presents recommendations for enhancing its effectiveness and safety within the context of sociotechnical factors.

Strategies to reduce insulin prescribing errors in UK hospitals: results from a national survey

AIM

To describe insulin prescribing practice in National Health Service hospitals in the UK and the current use of interventions and strategies to reduce insulin prescribing errors.

METHODS

We sent a cross-sectional questionnaire to chief pharmacists in all National Health Service hospital trusts in the UK in January 2019. Questions concerned the use and functionality of electronic and paper systems used to prescribe subcutaneous insulin, along with features and interventions designed to reduce insulin prescribing errors.

RESULTS

Ninety-five hospital trusts responded (54%). Electronic prescribing of insulin was reported in 40% of hospitals, most of which were teaching hospitals in England. We found a wide variation in the functionality of both electronic prescribing and paper-based systems to enable the safe prescribing of insulin for inpatients. The availability of specialist diabetes pharmacists to support the safe prescribing of insulin was low (29%), but was positively associated with the use of a greater number of insulin prescribing error reduction strategies (P=0.002). The use of specific interventions to improve insulin prescribing quality (e.g. self-administration policies) varied greatly between respondent hospitals.

CONCLUSIONS

There is potential to optimize the functionality of both electronic and paper-based prescribing systems to improve the safe prescribing of insulin in hospitals in the UK. The wide variation in the use of insulin error reduction strategies may be improved by the availability of specialist diabetes pharmacists who can support the implementation of insulin-prescribing interventions.

MHealth and perceived quality of care delivery: a conceptual model and validation

BACKGROUND

The objective of this research is to examine, conceptualize, and empirically validate a model of mobile health (mHealth) impacts on physicians' perceived quality of care delivery (PQoC).

METHODS

Observational quasi-experimental one group posttest-only design was implemented through the empirical testing of the conceptual model with nine hypotheses related to the association of task and technology characteristics, self-efficacy, m-health utilization, task-technology fit (TTF), and their relationships with PQoC. Primary data was collected over a four-month period from acute care physicians in The Ottawa Hospital, Ontario, Canada. The self-reported data was collected by employing a survey and distributed through the internal hospital channels to physicians who adopted iPads for their daily activities.

RESULTS

Physicians' PQoC was found to be positively affected by the level of mHealth utilization and TTF, while the magnitude of the TTF direct effect was two times stronger than utilization. Additionally, self-efficacy has the highest direct and total effect on mHealth utilization; in the formation of TTF, technological characteristics dominate followed by task characteristics.

CONCLUSION

To date, the impact of utilized mHealth on PQoC has neither been richly theorized nor explored in depth. We address this gap in existing literature. Realizing how an organization can improve TTF will lead to better PQoC.