Computerised provider order entry combined with clinical decision support systems to improve medication safety: a narrative review

Clinical-pharmaceutical assessment of medication CDSS alerts: content appropriateness and patient relevance in clinical practice

Background

Clinical pharmacy services and clinical decision support systems (CDSSs) are increasingly implemented to optimize medication safety. However, risks as overalerting can limit these benefits. Therefore, the Meona medication CDSS was interprofessionally evaluated and locally configured prior to implementation at Erlangen University Hospital.

Aim

We aimed to analyze the displayed CDSS alerts and to evaluate the content appropriateness and patient relevance of CDSS alerts in a hospital with established clinical ward pharmacists. Furthermore, we characterized pharmaceutical interventions triggered by CDSS and CDSS-independent interventions.

Methods

Pseudonymized clinical data of 160 patients from four clinical departments were prospectively included once between days 1 and 3 after hospital admission to analyze the frequency, type, and severity of the displayed CDSS alerts. All severe and "duplicate prescription" CDSS alerts were evaluated regarding their content appropriateness and patient relevance by clinical pharmacists using the four-eyes principle. For patient-relevant CDSS alerts, clinical ward pharmacists intervened during weekly ward rounds. All pharmaceutical interventions, including CDSS-independent interventions, were documented in ADKA-DokuPIK by recording reason, acceptance rate, and severity.

Results

In total, 1,799 CDSS alerts (median 9.0/patient) were displayed. Of those, 33.9% (609/1,799) were classified as severe by Meona. Clinical pharmacists validated 647 CDSS alerts (609 severe and 38 "duplicate prescriptions"). Only 82.7% (535/647) were rated as content appropriate, of which 19.6% (105/535) were classified as patient relevant. The clinical ward pharmacists recorded 244 interventions in 150 patients discussed during rounds (1.6/patient). CDSS-independent interventions by clinical ward pharmacists (158/244, 64.8%) were significantly more frequent compared to pharmaceutical interventions triggered by the CDSS (86/244, 35.2%). (p = 0.0002). The acceptance rate of interventions was 92.2% (225/244). The most common severity category was C (error occurred, no harm).

Conclusion

Despite the locally customized medication CDSS, a high number of CDSS alerts were displayed. Interestingly, we still observed content-inappropriate CDSS alerts defined by pharmaceutical validation. The majority of CDSS alerts with appropriate content were rated not patient relevant in clinical practice and could be considered as overalerting. Our results highlight that a CDSS can support healthcare professionals but underline (1) the continuing need for clinical pharmacists to improve medication safety by interpreting CDSS alerts and performing comprehensive medication reviews and (2) the further need for CDSS improvements.

A Qualitative Study Exploring the Acceptability and Usability of the e-Prescribing Risk and Safety Evaluation (ePRaSE) Assessment Within English Hospitals

OBJECTIVES

The e-prescribing risk and safety evaluation (ePRaSE) tool was developed to support the evaluation of hospital e-prescribing (EP) systems. The tool uses fictitious patients alongside previously validated prescribing scenarios to detect whether these systems provide appropriate prescribing advice to users. We sought to evaluate the usability and acceptability of ePRaSE across different EP systems in England.

MATERIALS AND METHODS

NHS hospitals in England with live EP systems were invited to participate. A combination of observations and semi-structured interviews were used to explore participants' perspectives on the acceptability and usability of ePRaSE throughout all stages of the tool development. The data were transcribed verbatim, coded, and analyzed using the Framework Approach.

RESULTS

The study was conducted over 2 periods: April-December 2019 and September 2022-January 2023. Thirty-two health care professionals across 22 different NHS hospitals participated in semi-structured interviews (n=25) and 13 observations (n=20) involving 11 different EP systems in total. The ePRaSE assessment was completed in 2 to 3 hours and participants described the tool as easy to use with clinically relevant prescribing tasks. However, some participants experienced difficulties inputting clinical data, such as laboratory results, due to restricted access to different parts of the electronic health record. Many participants suggested areas for further improvement such as capturing a broader range of implemented clinical decision support and requested more detailed feedback on the performance of their systems.

CONCLUSIONS

EP system users found ePRaSE to be a useful and acceptable tool. Further refinement is desirable, particularly in recording EP system responses and providing detailed results to optimize EP systems for safety benefits.

Digital Health Technology Interventions for Improving Medication Safety: Systematic Review of Economic Evaluations

BACKGROUND

Medication-related harm, including adverse drug events (ADEs) and medication errors, represents a significant iatrogenic burden in clinical care. Digital health technology (DHT) interventions can significantly enhance medication safety outcomes. Although the clinical effectiveness of DHT for medication safety has been relatively well studied, much less is known about the cost-effectiveness of these interventions.

OBJECTIVE

This study aimed to systematically review the economic impact of DHT interventions on medication safety and examine methodological challenges to inform future research directions.

METHODS

A systematic search was conducted across 3 major electronic databases (ie, PubMed, Scopus, and EBSCOhost). The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines were followed for this systematic review. Two independent investigators conducted a full-text review after screening preliminary titles and abstracts. We adopted recommendations from the Panel on Cost-Effectiveness in Health and Medicine for data extraction. A narrative analysis was conducted to synthesize clinical and economic outcomes. The quality of reporting for the included studies was assessed using the CHEERS (Consolidated Health Economic Evaluation Reporting Standards) guidelines.

RESULTS

We included 13 studies that assessed the cost-effectiveness (n=9, 69.2%), cost-benefit (n=3, 23.1%), and cost-utility (n=1, 7.7%) of DHT for medication safety. Of the included studies, more than half (n=7, 53.9%) evaluated a clinical decision support system (CDSS)/computerized provider order entry (CPOE), 4 (30.8%) examined automated medication-dispensing systems, and 2 (15.4%) focused on pharmacist-led outreach programs targeting health care professionals. In 12 (92.3% ) studies, DHT was either cost-effective or cost beneficial compared to standard care. On average, DHT interventions reduced ADEs by 37.12% (range 8.2%-66.5%) and medication errors by 54.38% (range 24%-83%). The key drivers of cost-effectiveness included reductions in outcomes, the proportion of errors resulting in ADEs, and implementation costs. Despite a significant upfront cost, DHT showed a return on investment within 3-4.25 years due to lower cost related with ADE treatment and improved workflow efficiency. In terms of reporting quality, the studies were classified as good (n=10, 76.9%) and moderate (n=3, 23.1%). Key methodological challenges included short follow-up periods, the absence of alert compliance tracking, the lack of ADE and error severity categorization, and omission of indirect costs.

CONCLUSIONS

DHT interventions are economically viable to improve medication safety, with a substantial reduction in ADEs and medication errors. Future studies should prioritize incorporating alert compliance tracking, ADE and error severity classification, and evaluation of indirect costs, thereby increasing clinical benefits and economic viability.

Safety of human-AI cooperative decision-making within intensive care: A physical simulation study

The safety of Artificial Intelligence (AI) systems is as much one of human decision-making as a technological question. In AI-driven decision support systems, particularly in high-stakes settings such as healthcare, ensuring the safety of human-AI interactions is paramount, given the potential risks of following erroneous AI recommendations. To explore this question, we ran a safety-focused clinician-AI interaction study in a physical simulation suite. Physicians were placed in a simulated intensive care ward, with a human nurse (played by an experimenter), an ICU data chart, a high-fidelity patient mannequin and an AI recommender system on a display. Clinicians were asked to prescribe two drugs for the simulated patients suffering from sepsis and wore eye-tracking glasses to allow us to assess where their gaze was directed. We recorded clinician treatment plans before and after they saw the AI treatment recommendations, which could be either 'safe' or 'unsafe'. 92% of clinicians rejected unsafe AI recommendations vs 29% of safe ones. Physicians paid increased attention (+37% gaze fixations) to unsafe AI recommendations vs safe ones. However, visual attention on AI explanations was not greater in unsafe scenarios. Similarly, clinical information (patient monitor, patient chart) did not receive more attention after an unsafe versus safe AI reveal suggesting that the physicians did not look back to these sources of information to investigate why the AI suggestion might be unsafe. Physicians were only successfully persuaded to change their dose by scripted comments from the bedside nurse 5% of the time. Our study emphasises the importance of human oversight in safety-critical AI and the value of evaluating human-AI systems in high-fidelity settings that more closely resemble real world practice.

A simulation study to quantitatively assess the performance of electronic prescribing systems in English NHS Hospital Trusts

Prescribing errors are a source of preventable harm in healthcare, which may be mitigated using Electronic Prescribing (EP) systems. Anyone who routinely prescribes medication could benefit from digitally assisted automated checks to identify whether a prescription should potentially not be allowed (e.g. drug allergy). National Health Service (NHS) Trusts have implemented a range of EP systems; however, their performance has not hitherto been evaluated. We developed the web-based Electronic Prescribing Risk and Safety Evaluation (ePRaSE) tool, which comprises a bank of prescribing scenarios to evaluate the performance of EP systems. We solicited ePRaSE testing: 68 pharmacists from across 45 English NHS Trusts, utilising 13 different EP systems volunteered for the study. We found considerable variability in mitigation performance (systems correctly identifying risk of error when prescribing) across both NHS Trusts and EP systems. Moreover, we found that mitigation performance varied considerably across NHS Trusts using the same EP system, strongly suggesting there are opportunities to optimise performance within systems. The ePRaSE tool is effective in identifying variability in risk management between NHS Trusts and EP systems. Wider use of this tool may facilitate improvements in EP system configurations, thus minimising potential harm from prescribing errors.

Implementation of Computerized Prescriber Order Entry Systems: A Review of Impacts, Barriers, and Facilitators

Objective

This review evaluated the impact of a digitized computerized prescriber order entry (CPOE) system and described barriers and facilitators for introducing a digitized system.

Data Sources

A systematic literature search was conducted in PubMed, Medline, and CINAHL using keywords. Articles in English during the last 10 years were included.

Study Selection and Data Extraction

Study selection was presented using a PRISMA flow diagram. Forty-eight studies were included. Data from the articles were presented to address each of the three objectives.

Data Synthesis

CPOE systems improved the quality of care provided but also introduced new types of errors. Facilitating factors for implementation included leadership, stakeholder engagement, training, and user-centered design. Inadequate training, software design, changes in workload, and workflow disruptions were identified as barriers. Recommendations for implementation included dedicated training of users, user-centered design, a backup for system downtimes, and stakeholder engagement.

Conclusion

Application of knowledge of the facilitators and barriers for the introduction of a CPOE system supports this change-management process within the specific context and augurs for more successful implementation.

The potential for drug incompatibility and its drivers - A hospital wide retrospective descriptive study

OBJECTIVE

Drug incompatibility, a significant subset of medication errors, threaten patient safety during the medication administration phase. Despite the undeniably high prevalence of drug incompatibility, it is currently poorly understood because previous studies are focused predominantly on intensive care unit (ICU) settings. To enhance patient safety, it is crucial to expand our understanding of this issue from a comprehensive viewpoint. This study aims to investigate the prevalence and mechanism of drug incompatibility by analysing hospital-wide prescription and administration data.

METHODS

This retrospective cross-sectional study, conducted at a tertiary academic hospital, included data extracted from the clinical data warehouse of the study institution on patients admitted between January 1, 2021, and May 31, 2021. Potential contacts in drug pairs (PCs) were identified using the study site clinical workflow. Drug incompatibility for each PC was determined by using a commercial drug incompatibility database, the Trissel's™ 2 Clinical Pharmaceutics Database (Trissel's 2 database). Drivers of drug incompatibility were identified, based on a descriptive analysis, after which, multivariate logistic regression was conducted to assess the risk factors for experiencing one or more drug incompatibilities during admission.

RESULTS

Among 30,359 patients (representing 40,061 hospitalisations), 24,270 patients (32,912 hospitalisations) with 764,501 drug prescriptions (1,001,685 IV administrations) were analysed, after checking for eligibility. Based on the rule for determining PCs, 5,813,794 cases of PCs were identified. Among these, 25,108 (0.4 %) cases were incompatible PCs: 391 (1.6 %) PCs occurred during the prescription process and 24,717 (98.4 %) PCs during the administration process. By classifying these results, we identified the following drivers contributing to drug incompatibility: incorrect order factor; incorrect administration factor; and lack of related research. In multivariate analysis, the risk of encountering incompatible PCs was higher for patients who were male, older, with longer lengths of stay, with higher comorbidity, and admitted to medical ICUs.

CONCLUSIONS

We comprehensively described the current state of drug incompatibility by analysing hospital-wide drug prescription and administration data. The results showed that drug incompatibility frequently occurs in clinical settings.

Development, validation, and usability evaluation of machine learning algorithms for predicting personalized red blood cell demand among thoracic surgery patients

INTRODUCTION

Preparing appropriate red blood cells (RBCs) before surgery is crucial for improving both the efficacy of perioperative workflow and patient safety. In particular, thoracic surgery (TS) is a procedure that requires massive transfusion with high variability for each patient. Hence, the precise prediction of RBC requirements for individual patients is becoming increasingly important. This study aimed to 1) develop and validate a machine learning algorithm for personalized RBC predictions for TS patients and 2) assess the usability of a clinical decision support system (CDSS) integrating this artificial intelligence model.

METHODS

Adult patients who underwent TS between January 2016 and October 2021 were included in this study. Multiple models were developed by employing both traditional statistical- and machine-learning approaches. The primary outcome evaluated the model's performance in predicting RBC requirements through root mean square error and adjusted R2. Surgeons and informaticians determined the precision MSBOS-Thoracic Surgery (pMSBOS-TS) algorithm through a consensus process. The usability of the pMSBOS-TS was assessed using the System Usability Scale (SUS) survey with 60 clinicians.

RESULTS

We identified 7,843 cases (6,200 for training and 1,643 for test sets) of TSs. Among the models with variable performance indices, the extreme gradient boosting model was selected as the pMSBOS-TS algorithm. The pMSBOS-TS model showed statistically significant lower root mean square error (mean: 3.203 and 95% confidence interval [CI]: 3.186-3.220) compared to the calculated Maximum Surgical Blood Ordering Schedule (MSBOS) and a higher adjusted R2 (mean: 0.399 and 95% CI: 0.395-0.403) compared to the calculated MSBOS, while requiring approximately 200 fewer packs for RBC preparation compared to the calculated MSBOS. The SUS score of the pMSBOS-TS CDSS was 72.5 points, indicating good acceptability.

CONCLUSIONS

We successfully developed the pMSBOS-TS capable of predicting personalized RBC transfusion requirements for perioperative patients undergoing TS.

Improving thiamine prescribing in alcohol use disorder using electronic decision support in a large urban academic medical center: A pre-post intervention study

INTRODUCTION

Thiamine is the only therapy for prevention and treatment of Wernicke Encephalopathy among patients with Alcohol Use Disorder (AUD). Despite this fact, up to 75 % of inpatients with AUD are not prescribed thiamine during hospitalization. Even fewer patients are prescribed high-dose thiamine which many experts recommend should be standard of care. Previous attempts to improve thiamine prescribing for inpatients have had limited success.

METHODS

We conducted an evaluation of thiamine prescribing in the year before and year after an intervention to increase high-dose thiamine prescribing. Pre-post study analysis occurred on two distinct study cohorts: those with alcohol-related diagnoses and those with elevated alcohol levels. The intervention was new electronic health record-based decision support which encouraged high-dose thiamine when any thiamine order was sought. No educational support was provided. The primary outcome was prescription of high-dose thiamine before versus after intervention. Of those with alcohol-related diagnoses, the monthly percentage of thiamine treatment courses including high-dose thiamine were graphed on a control chart.

RESULTS

We examined 5307 admissions with alcohol-related diagnoses (2285 pre- and 3022 post-intervention) and 698 admissions with elevated alcohol levels (319 pre- and 379 post-intervention). Among admissions with alcohol-related diagnoses, the intervention was associated with a higher proportion of admissions receiving high-dose thiamine prescriptions in the first 24 h (4.7 % vs. 1.1 %, adjusted odds ratio 4.50, CI 2.93 to 6.89, p < 0.001). A similar difference in high-dose thiamine was seen post-intervention among admissions with elevated alcohol levels (14.3 % vs. 2.5 %, adjusted odds ratio 6.43, CI 3.05 to 13.53, p < 0.001). The control chart among those with an alcohol-related diagnosis demonstrated special cause variation: the median percentage of thiamine treatment courses including high-dose thiamine improved from 8.2 % to 13.0 %.

CONCLUSIONS

Electronic decision support without educational interventions increased the use of high-dose thiamine among patients with alcohol-related diagnoses and with elevated alcohol levels during hospitalization. This increase occurred immediately in the month after the intervention and was sustained in the year-long study period after.

Impact of automated pop-up alerts on simultaneous prescriptions of antimicrobial agents and metal cations

BACKGROUND

Antimicrobial agents (AMAs) are essential for treating infections. A part of AMAs chelate with metal cations (MCs), reducing their blood concentrations. That drug-drug interaction could lead to a reduction of therapeutic efficacy and the emergence of drug-resistant bacteria. However, prescriptions ordering concomitant intake (co-intake) of AMAs and MCs are frequently seen in clinical settings. A method for preventing such prescriptions is urgently needed.

METHODS

We implemented pop-up alerts in the hospital's ordering and pharmacy dispensation support system to notify the prescriptions ordering co-intake of AMAs and MCs for physicians and pharmacists, respectively. To assess the effectiveness of the pop-up alerts, we investigated the number of prescriptions ordering co-intake of AMAs and MCs and the number of pharmacist inquiries to prevent co-intake of AMAs and MCs before and after the implementation of pop-up alerts.

RESULTS

Before the implementation of pop-up alerts, 84.5% of prescriptions containing AMA and MCs ordered co-intake of AMAs and MCs. Implementing pop-up alerts time-dependently reduced the proportion of prescriptions ordering co-intake of AMAs and MCs to 43.8% and 29.5% one year and two years later, respectively. The reduction of tetracycline-containing prescriptions was mainly significant. Before the implementation of pop-up alerts, the proportion of prescriptions in which pharmacists prevented co-intake of AMAs and MCs was 3.4%. Implementing pop-up alerts time-dependently increased proportions of such prescriptions to 20.9% and 28.2% one year and two years later.

CONCLUSION

Implementing pop-up alerts reduced prescriptions ordering co-intake of AMAs and MCs and accelerated pharmacists to prevent co-intake of AMAs and MCs. The implementation of dual pop-up alerts in the hospital's ordering and pharmacy dispensation support system could help prevent co-intake of AMAs and MCs.

How intervention studies measure the effectiveness of medication safety-related clinical decision support systems in primary and long-term care: a systematic review

BACKGROUND

Medication errors and associated adverse drug events (ADE) are a major cause of morbidity and mortality worldwide. In recent years, the prevention of medication errors has become a high priority in healthcare systems. In order to improve medication safety, computerized Clinical Decision Support Systems (CDSS) are increasingly being integrated into the medication process. Accordingly, a growing number of studies have investigated the medication safety-related effectiveness of CDSS. However, the outcome measures used are heterogeneous, leading to unclear evidence. The primary aim of this study is to summarize and categorize the outcomes used in interventional studies evaluating the effects of CDSS on medication safety in primary and long-term care.

METHODS

We systematically searched PubMed, Embase, CINAHL, and Cochrane Library for interventional studies evaluating the effects of CDSS targeting medication safety and patient-related outcomes. We extracted methodological characteristics, outcomes and empirical findings from the included studies. Outcomes were assigned to three main categories: process-related, harm-related, and cost-related. Risk of bias was assessed using the Evidence Project risk of bias tool.

RESULTS

Thirty-two studies met the inclusion criteria. Almost all studies (n = 31) used process-related outcomes, followed by harm-related outcomes (n = 11). Only three studies used cost-related outcomes. Most studies used outcomes from only one category and no study used outcomes from all three categories. The definition and operationalization of outcomes varied widely between the included studies, even within outcome categories. Overall, evidence on CDSS effectiveness was mixed. A significant intervention effect was demonstrated by nine of fifteen studies with process-related primary outcomes (60%) but only one out of five studies with harm-related primary outcomes (20%). The included studies faced a number of methodological problems that limit the comparability and generalizability of their results.

CONCLUSIONS

Evidence on the effectiveness of CDSS is currently inconclusive due in part to inconsistent outcome definitions and methodological problems in the literature. Additional high-quality studies are therefore needed to provide a comprehensive account of CDSS effectiveness. These studies should follow established methodological guidelines and recommendations and use a comprehensive set of harm-, process- and cost-related outcomes with agreed-upon and consistent definitions.

PROSPERO REGISTRATION

CRD42023464746.

Interprofessional Evaluation of a Medication Clinical Decision Support System Prior to Implementation

BACKGROUND

 Computerized physician order entry (CPOE) and clinical decision support systems (CDSS) are widespread due to increasing digitalization of hospitals. They can be associated with reduced medication errors and improved patient safety, but also with well-known risks (e.g., overalerting, nonadoption).

OBJECTIVES

 Therefore, we aimed to evaluate a commonly used CDSS containing Medication-Safety-Validators (e.g., drug-drug interactions), which can be locally activated or deactivated, to identify limitations and thereby potentially optimize the use of the CDSS in clinical routine.

METHODS

 Within the implementation process of Meona (commercial CPOE/CDSS) at a German University hospital, we conducted an interprofessional evaluation of the CDSS and its included Medication-Safety-Validators following a defined algorithm: (1) general evaluation, (2) systematic technical and content-related validation, (3) decision of activation or deactivation, and possibly (4) choosing the activation mode (interruptive or passive). We completed the in-depth evaluation for exemplarily chosen Medication-Safety-Validators. Moreover, we performed a survey among 12 German University hospitals using Meona to compare their configurations.

RESULTS

 Based on the evaluation, we deactivated 3 of 10 Medication-Safety-Validators due to technical or content-related limitations. For the seven activated Medication-Safety-Validators, we chose the interruptive option ["PUSH-(&PULL)-modus"] four times (4/7), and a new, on-demand option ["only-PULL-modus"] three times (3/7). The site-specific configuration (activation or deactivation) differed across all participating hospitals in the survey and led to varying medication safety alerts for identical patient cases.

CONCLUSION

 An interprofessional evaluation of CPOE and CDSS prior to implementation in clinical routine is crucial to detect limitations. This can contribute to a sustainable utilization and thereby possibly increase medication safety.

Qualitative study of challenges with recruitment of hospitals into a cluster controlled trial of clinical decision support in Australia

OBJECTIVE

To identify barriers to hospital participation in controlled cluster trials of clinical decision support (CDS) and potential strategies for addressing barriers.

DESIGN

Qualitative descriptive design comprising semistructured interviews.

SETTING

Five hospitals in New South Wales and one hospital in Queensland, Australia.

PARTICIPANTS

Senior hospital staff, including department directors, chief information officers and those working in health informatics teams.

RESULTS

20 senior hospital staff took part. Barriers to hospital-level recruitment primarily related to perceptions of risk associated with not implementing CDS as a control site. Perceived risks included reductions in patient safety, reputational risk and increased likelihood that benefits would not be achieved following electronic medical record (EMR) implementation without CDS alerts in place. Senior staff recommended clear communication of trial information to all relevant stakeholders as a key strategy for boosting hospital-level participation in trials.

CONCLUSION

Hospital participation in controlled cluster trials of CDS is hindered by perceptions that adopting an EMR without CDS is risky for both patients and organisations. The improvements in safety expected to follow CDS implementation makes it challenging and counterintuitive for hospitals to implement EMR without incorporating CDS alerts for the purposes of a research trial. To counteract these barriers, clear communication regarding the evidence base and rationale for a controlled trial is needed.

Ability of machine-learning based clinical decision support system to reduce alert fatigue, wrong-drug errors, and alert users about look alike, sound alike medication

BACKGROUND AND OBJECTIVE

The overall benefits of using clinical decision support systems (CDSSs) can be restrained if physicians inadvertently ignore clinically useful alerts due to "alert fatigue" caused by an excessive number of clinically irrelevant warnings. Moreover, inappropriate drug errors, look-alike/sound-alike (LASA) drug errors, and problem list documentation are common, costly, and potentially harmful. This study sought to evaluate the overall performance of a machine learning-based CDSS (MedGuard) for triggering clinically relevant alerts, acceptance rate, and to intercept inappropriate drug errors as well as LASA drug errors.

METHODS

We conducted a retrospective study that evaluated MedGuard alerts, the alert acceptance rate, and the rate of LASA alerts between July 1, 2019, and June 31, 2021, from outpatient settings at an academic hospital. An expert pharmacist checked the suitability of the alerts, rate of acceptance, wrong-drug errors, and confusing drug pairs.

RESULTS

Over the two-year study period, 1,206,895 prescriptions were ordered and a total of 28,536 alerts were triggered (alert rate: 2.36 %). Of the 28,536 alerts presented to physicians, 13,947 (48.88 %) were accepted. A total of 8,014 prescriptions were changed/modified (28.08 %, 8,014/28,534) with the most common reasons being adding and/or deleting diseases (52.04 %, 4,171/8,014), adding and/or deleting drugs (21.89 %, 1,755/8,014) and others (35.48 %, 2,844/ 8,014). However, the rate of drug error interception was 1.64 % (470 intercepted errors out of 28,536 alerts), which equates to 16.4 intercepted errors per 1000 alerted orders.

CONCLUSION

This study shows that machine learning based CDSS, MedGuard, has an ability to improve patients' safety by triggering clinically valid alerts. This system can also help improve problem list documentation and intercept inappropriate drug errors and LASA drug errors, which can improve medication safety. Moreover, high acceptance of alert rates can help reduce clinician burnout and adverse events.

Automated search methods for identifying wrong patient order entry-a scoping review

Objective

To investigate: (1) what automated search methods are used to identify wrong-patient order entry (WPOE), (2) what data are being captured and how they are being used, (3) the causes of WPOE, and (4) how providers identify their own errors.

Materials and Methods

A systematic scoping review of the empirical literature was performed using the databases CINAHL, Embase, and MEDLINE, covering the period from database inception until 2021. Search terms were related to the use of automated searches for WPOE when using an electronic prescribing system. Data were extracted and thematic analysis was performed to identify patterns or themes within the data.

Results

Fifteen papers were included in the review. Several automated search methods were identified, with the retract-and-reorder (RAR) method and the Void Alert Tool (VAT) the most prevalent. Included studies used automated search methods to identify background error rates in isolation, or in the context of an intervention. Risk factors for WPOE were identified, with technological factors and interruptions deemed the biggest risks. Minimal data on how providers identify their own errors were identified.

Discussion

RAR is the most widely used method to identify WPOE, with a good positive predictive value (PPV) of 76.2%. However, it will not currently identify other error types. The VAT is nonspecific for WPOE, with a mean PPV of 78%-93.1%, but the voiding reason accuracy varies considerably.

Conclusion

Automated search methods are powerful tools to identify WPOE that would otherwise go unnoticed. Further research is required around self-identification of errors.

Evaluating the costs and consequences of computerized clinical decision support systems in hospitals: a scoping review and recommendations for future practice

OBJECTIVE

Sustainable investment in computerized decision support systems (CDSS) requires robust evaluation of their economic impacts compared with current clinical workflows. We reviewed current approaches used to evaluate the costs and consequences of CDSS in hospital settings and presented recommendations to improve the generalizability of future evaluations.

MATERIALS AND METHODS

A scoping review of peer-reviewed research articles published since 2010. Searches were completed in the PubMed, Ovid Medline, Embase, and Scopus databases (last searched February 14, 2023). All studies reported the costs and consequences of a CDSS-based intervention compared with current hospital workflows. Findings were summarized using narrative synthesis. Individual studies were further appraised against the Consolidated Health Economic Evaluation and Reporting (CHEERS) 2022 checklist.

RESULTS

Twenty-nine studies published since 2010 were included. Studies evaluated CDSS for adverse event surveillance (5 studies), antimicrobial stewardship (4 studies), blood product management (8 studies), laboratory testing (7 studies), and medication safety (5 studies). All studies evaluated costs from a hospital perspective but varied based on the valuation of resources affected by CDSS implementation, and the measurement of consequences. We recommend future studies follow guidance from the CHEERS checklist; use study designs that adjust for confounders; consider both the costs of CDSS implementation and adherence; evaluate consequences that are directly or indirectly affected by CDSS-initiated behavior change; examine the impacts of uncertainty and differences in outcomes across patient subgroups.

DISCUSSION AND CONCLUSION

Improving consistency in the conduct and reporting of evaluations will enable detailed comparisons between promising initiatives, and their subsequent uptake by decision-makers.

Detection of Drug-Related Problems through a Clinical Decision Support System Used by a Clinical Pharmacy Team

Clinical decision support systems (CDSSs) are intended to detect drug-related problems in real time and might be of value in healthcare institutions with a clinical pharmacy team. The objective was to report the detection of drug-related problems through a CDSS used by an existing clinical pharmacy team over 22 months. It was a retrospective single-center study. A CDSS was integrated in the clinical pharmacy team in July 2019. The investigating clinical pharmacists evaluated the pharmaceutical relevance and physician acceptance rates for critical alerts (i.e., alerts for drug-related problems arising during on-call periods) and noncritical alerts (i.e., prevention alerts arising during the pharmacist’s normal work day) from the CDSS. Of the 3612 alerts triggered, 1554 (43.0%) were critical, and 594 of these 1554 (38.2%) prompted a pharmacist intervention. Of the 2058 (57.0%) noncritical alerts, 475 of these 2058 (23.1%) prompted a pharmacist intervention. About two-thirds of the total pharmacist interventions (PI) were accepted by physicians; the proportion was 71.2% for critical alerts (i.e., 19 critical alerts per month vs. 12.5 noncritical alerts per month). Some alerts were pharmaceutically irrelevant—mainly due to poor performance by the CDSS. Our results suggest that a CDSS is a useful decision-support tool for a hospital pharmacist’s clinical practice. It can help to prioritize drug-related problems by distinguishing critical and noncritical alerts. However, building an appropriate organizational structure around the CDSS is important for correct operation.

Healthcare Professionals' Expectations of Medical Artificial Intelligence and Strategies for its Clinical Implementation: A Qualitative Study

OBJECTIVES

Although medical artificial intelligence (AI) systems that assist healthcare professionals in critical care settings are expected to improve healthcare, skepticism exists regarding whether their potential has been fully actualized. Therefore, we aimed to conduct a qualitative study with physicians and nurses to understand their needs, expectations, and concerns regarding medical AI; explore their expected responses to recommendations by medical AI that contradicted their judgments; and derive strategies to implement medical AI in practice successfully.

METHODS

Semi-structured interviews were conducted with 15 healthcare professionals working in the emergency room and intensive care unit in a tertiary teaching hospital in Seoul. The data were interpreted using summative content analysis. In total, 26 medical AI topics were extracted from the interviews. Eight were related to treatment recommendation, seven were related to diagnosis prediction, and seven were related to process improvement.

RESULTS

While the participants expressed expectations that medical AI could enhance their patients' outcomes, increase work efficiency, and reduce hospital operating costs, they also mentioned concerns regarding distortions in the workflow, deskilling, alert fatigue, and unsophisticated algorithms. If medical AI decisions contradicted their judgment, most participants would consult other medical staff and thereafter reconsider their initial judgment.

CONCLUSIONS

Healthcare professionals wanted to use medical AI in practice and emphasized that artificial intelligence systems should be trustworthy from the standpoint of healthcare professionals. They also highlighted the importance of alert fatigue management and the integration of AI systems into the workflow.

Short- and long-term effects of an electronic medication management system on paediatric prescribing errors

Electronic medication management (eMM) systems are designed to improve safety, but there is little evidence of their effectiveness in paediatrics. This study assesses the short-term (first 70 days of eMM use) and long-term (one-year) effectiveness of an eMM system to reduce prescribing errors, and their potential and actual harm. We use a stepped-wedge cluster randomised controlled trial (SWCRCT) at a paediatric referral hospital, with eight clusters randomised for eMM implementation. We assess long-term effects from an additional random sample of medication orders one-year post-eMM. In the SWCRCT, errors that are potential adverse drug events (ADEs) are assessed for actual harm. The study comprises 35,260 medication orders for 4821 patients. Results show no significant change in overall prescribing error rates in the first 70 days of eMM use (incident rate ratio [IRR] 1.05 [95%CI 0.92-1.21], but a 62% increase (IRR 1.62 [95%CI 1.28-2.04]) in potential ADEs suggesting immediate risks to safety. One-year post-eMM, errors decline by 36% (IRR 0.64 [95%CI 0.56-0.72]) and high-risk medication errors decrease by 33% (IRR 0.67 [95%CI 0.51-0.88]) compared to pre-eMM. In all periods, dose error rates are more than double that of other error types. Few errors are associated with actual harm, but 71% [95%CI 50-86%] of patients with harm experienced a dose error. In the short-term, eMM implementation shows no improvement in error rates, and an increase in some errors. A year after eMM error rates significantly decline suggesting long-term benefits. eMM optimisation should focus on reducing dose errors due to their high frequency and capacity to cause harm.

Development of an Interoperable and Easily Transferable Clinical Decision Support System Deployment Platform: System Design and Development Study

BACKGROUND

A clinical decision support system (CDSS) is recognized as a technology that enhances clinical efficacy and safety. However, its full potential has not been realized, mainly due to clinical data standards and noninteroperable platforms.

OBJECTIVE

In this paper, we introduce the common data model-based intelligent algorithm network environment (CANE) platform that supports the implementation and deployment of a CDSS.

METHODS

CDSS reasoning engines, usually represented as R or Python objects, are deployed into the CANE platform and converted into C# objects. When a clinician requests CANE-based decision support in the electronic health record (EHR) system, patients' information is transformed into Health Level 7 Fast Healthcare Interoperability Resources (FHIR) format and transmitted to the CANE server inside the hospital firewall. Upon receiving the necessary data, the CANE system's modules perform the following tasks: (1) the preprocessing module converts the FHIRs into the input data required by the specific reasoning engine, (2) the reasoning engine module operates the target algorithms, (3) the integration module communicates with the other institutions' CANE systems to request and transmit a summary report to aid in decision support, and (4) creates a user interface by integrating the summary report and the results calculated by the reasoning engine.

RESULTS

We developed a CANE system such that any algorithm implemented in the system can be directly called through the RESTful application programming interface when it is integrated with an EHR system. Eight algorithms were developed and deployed in the CANE system. Using a knowledge-based algorithm, physicians can screen patients who are prone to sepsis and obtain treatment guides for patients with sepsis with the CANE system. Further, using a nonknowledge-based algorithm, the CANE system supports emergency physicians' clinical decisions about optimum resource allocation by predicting a patient's acuity and prognosis during triage.

CONCLUSIONS

We successfully developed a common data model-based platform that adheres to medical informatics standards and could aid artificial intelligence model deployment using R or Python.

Appropriateness of Alerts and Physicians’ Responses with a Medication-related Clinical Decision Support System (Preprint)

Background

Alert fatigue is unavoidable when many irrelevant alerts are generated in response to a small number of useful alerts. It is necessary to increase the effectiveness of the clinical decision support system (CDSS) by understanding physicians’ responses.

Objective

This study aimed to understand the CDSS and physicians’ behavior by evaluating the clinical appropriateness of alerts and the corresponding physicians’ responses in a medication-related passive alert system.

Methods

Data on medication-related orders, alerts, and patients’ electronic medical records were analyzed. The analyzed data were generated between August 2019 and June 2020 while the patient was in the emergency department. We evaluated the appropriateness of alerts and physicians’ responses for a subset of 382 alert cases and classified them.

Results

Of the 382 alert cases, only 7.3% (n=28) of the alerts were clinically appropriate. Regarding the appropriateness of the physicians’ responses about the alerts, 92.4% (n=353) were deemed appropriate. In the classification of alerts, only 3.4% (n=13) of alerts were successfully triggered, and 2.1% (n=8) were inappropriate in both alert clinical relevance and physician’s response. In this study, the override rate was 92.9% (n=355).

Conclusions

We evaluated the appropriateness of alerts and physicians’ responses through a detailed medical record review of the medication-related passive alert system. An excessive number of unnecessary alerts are generated, because the algorithm operates as a rule base without reflecting the individual condition of the patient. It is important to maximize the value of the CDSS by comprehending physicians’ responses.

The impact of clinical decision support systems on provider behavior in the inpatient setting: A systematic review and meta-analysis

BACKGROUND

Clinical decision support systems (CDSS) are used to improve processes of care. CDSS proliferation may have unintended consequences impacting effectiveness.

OBJECTIVE

To evaluate the effectiveness of CDSS in altering clinician behavior.

DESIGN

Electronic searches were performed in EMBASE, PubMed, and Cochrane Central Register of Control Trials for randomized controlled trials testing the impacted of CDSS on clinician behavior from 2000-2021. Extracted data included study design, CDSS attributed and outcomes, user characteristics, settings, and risk of bias. Eligible studies were analyzed qualitatively to describe CDSS types. Studies with sufficient outcome data were included in the meta-analysis.

SETTING AND PARTICIPANTS

Adult inpatients in the United States.

INTERVENTION

Clinical decision support system versus non-clinical decision support system.

MAIN OUTCOME AND MEASURE

A random-effects model measured the pooled risk difference (RD) and odds ratio of clinicians' adherence to CDSS; subgroup analyses tested differences in CDSS effectiveness over time and by CDSS type.

RESULTS

Qualitative synthesis included 22 studies. Eleven studies reported sufficient outcome data for inclusion in the meta-analysis. CDSS did not result in a statistically significant increase in clinician adoption of desired practicies (RD = 0.04 [95% confidence interval {CI} 0.00, 0.07]). CDSS from 2010-2015 (n = 5) did not increase clinician adoption of desired practice [RD -0.01, (95% CI -0.04, 0.02)].CDSS from 2016-2021 (n = 6) were associated with an increase in targeted practices [RD 0.07 (95% CI0.03, 0.12)], pInteraction = 0.004. EHR [RD 0.04 (95% CI 0.00, 0.08)] vs. non-EHR [RD 0.01 (95% CI -0.01, 0.04)] based CDSS interventions did not result in different adoption of desired practices (pInteraction = 0.27). The meta-analysis did not find an overall positive impact of CDSS on clinician behavior in the inpatient setting.

Ambulatory Medication Safety in Primary Care: A Systematic Review

PURPOSE

To review the literature on medication safety in primary care in the electronic health record era.

METHODS

Included studies measured rates and outcomes of medication safety in patients whose prescriptions were written in primary care clinics with electronic prescribing. Four investigators independently reviewed titles and analyzed abstracts with dual-reviewer review for eligibility, characteristics, and risk of bias.

RESULTS

Of 1464 articles identified, 56 met the inclusion criteria. Forty-three studies were noninterventional and 13 included an intervention. The majority of the studies (30) used their own definition of error. The most common outcomes were potentially inappropriate prescribing/medications (PIPs), adverse drug events (ADEs), and potential prescribing omissions (PPOs). Most of the studies only included high-risk subpopulations (39), usually older adults taking > 4 medications. The rate of PIPs varied widely (0.19% to 98.2%). The rate of ADEs was lower (0.47% to 14.7%). There was poor correlation of PIP and PPO with documented ADEs leading to physical harm.

CONCLUSIONS

This literature is limited by its inconsistent and highly variable outcomes. The majority of medication safety studies in primary care were in high-risk populations and measured potential harms rather than actual harms. Applying algorithms to primary care medication lists significantly overestimates rate of actual harms.

Functional IT Complementarity and Hospital Performance in the United States: A Longitudinal Investigation

For healthcare practitioners and policymakers, one of the most challenging problems is understanding how to implement health information technology (HIT) applications in a way that yields the most positive impacts on quality and cost of care. We identify four clinical HIT functions which we label as order entry and management (OEM), decision support (DS), electronic clinical documentation (ECD), and results viewing (RV). We view OEM and DS as primary clinical functions and ECD and RV as support clinical functions. Our results show that no single combination of applications uniformly improves clinical and experiential quality and reduces cost for all hospitals. Thus, managers must assess which HIT interactions improve which performance metric under which conditions. Our results suggest that synergies can be realized when these systems are implemented simultaneously. Additionally, synergies can occur when support HIT is implemented before primary HIT and irrespective of the order in which primary HITs are implemented. Practitioners should also be aware that the synergistic effects of HITs and their impact on cost and quality are different for chronic and acute diseases. Our key message to top managers is to prioritize different combinations of HIT contingent on the performance variables they are targeting for their hospitals but also to realize that technology may not impact all outcomes.

Modulators Influencing Medication Alert Acceptance: An Explorative Review

OBJECTIVES

Clinical decision support systems (CDSSs) use alerts to enhance medication safety and reduce medication error rates. A major challenge of medication alerts is their low acceptance rate, limiting their potential benefit. A structured overview about modulators influencing alert acceptance is lacking. Therefore, we aimed to review and compile qualitative and quantitative modulators of alert acceptance and organize them in a comprehensive model.

METHODS

In accordance with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guideline, a literature search in PubMed was started in February 2018 and continued until October 2021. From all included articles, qualitative and quantitative parameters and their impact on alert acceptance were extracted. Related parameters were then grouped into factors, allocated to superordinate determinants, and subsequently further allocated into five categories that were already known to influence alert acceptance.

RESULTS

Out of 539 articles, 60 were included. A total of 391 single parameters were extracted (e.g., patients' comorbidity) and grouped into 75 factors (e.g., comorbidity), and 25 determinants (e.g., complexity) were consequently assigned to the predefined five categories, i.e., CDSS, care provider, patient, setting, and involved drug. More than half of all factors were qualitatively assessed (n = 21) or quantitatively inconclusive (n = 19). Furthermore, 33 quantitative factors clearly influenced alert acceptance (positive correlation: e.g., alert type, patients' comorbidity; negative correlation: e.g., number of alerts per care provider, moment of alert display in the workflow). Two factors (alert frequency, laboratory value) showed contradictory effects, meaning that acceptance was significantly influenced both positively and negatively by these factors, depending on the study. Interventional studies have been performed for only 12 factors while all other factors were evaluated descriptively.

CONCLUSION

This review compiles modulators of alert acceptance distinguished by being studied quantitatively or qualitatively and indicates their effect magnitude whenever possible. Additionally, it describes how further research should be designed to comprehensively quantify the effect of alert modulators.

An e-Delphi study to obtain expert consensus on the level of risk associated with preventable e-prescribing events

Aims

We aim to seek expert opinion and gain consensus on the risks associated with a range of prescribing scenarios, preventable using e‐prescribing systems, to inform the development of a simulation tool to evaluate the risk and safety of e‐prescribing systems (ePRaSE).

Methods

We conducted a two‐round e‐Delphi survey where expert participants were asked to score pre‐designed prescribing scenarios using a five‐point Likert scale to ascertain the likelihood of occurrence of the prescribing event, likelihood of occurrence of harm and the severity of the harm.

Results

Twenty‐four experts consented to participate with 15 pand 13 participants completing rounds 1 and 2, respectively. Experts agreed on the level of risk associated with 136 out of 178 clinical scenarios with 131 scenarios categorised as high or extreme risk.

Conclusion

We identified 131 extreme or high‐risk prescribing scenarios that may be prevented using e‐prescribing clinical decision support. The prescribing scenarios represent a variety of categories, with drug–disease contraindications being the most frequent, representing 37 (27%) scenarios, and antimicrobial agents being the most common drug class, representing 28 (21%) of the scenarios. Our e‐Delphi study has achieved expert consensus on the risk associated with a range of clinical scenarios with most of the scenarios categorised as extreme or high risk. These prescribing scenarios represent the breadth of preventable prescribing error categories involving both basic and advanced clinical decision support. We will use the findings of this study to inform the development of the e‐prescribing risk and safety evaluation tool.

Evaluation of two European risk models for predicting medication harm in an Australian patient cohort

PURPOSE

To externally evaluate the performance of two European risk prediction models, for identifying patients at high-risk of medication harm, in an Australian hospital setting.

METHODS

This was a secondary analysis of a pre-existing cohort study described in a recently published study by Falconer et al. (Br J Clin Pharmacol 87(3):1512-1524, 2021) describing the development of a predictive risk model for inpatient medication harm. We retrospectively extracted relevant variables using the electronic health records of general medical and geriatric patients admitted to a quaternary hospital, in Brisbane, over 6 months from July to December 2017. This dataset was used to externally evaluate the two European models, The Brighton Adverse Drug Reaction Risk (BADRI) model by Tangiisuran et al. and a risk model developed by Trivalle et al. The variables were entered into both models and the patients' risk of medication harm was calculated, and compared with actual patient outcomes. Predictive performance was evaluated by measuring area under the receiver operative characteristic (AuROC) curves.

RESULTS

The Australian patient cohort included 1982 patients (median age 74 years), of which 136 (7%) patients experienced ≥ 1 medication harm event(s). External evaluation of the two European models identified that both the BADRI and the Trivalle models had reduced predictive performance in an Australian patient cohort, compared with their original studies (AuROC of 0.63 [95% CI: 0.58-0.68] and 0.60 [95% CI: 0.55-0.65], respectively).

CONCLUSION

Neither model demonstrated sufficient discrimination to warrant further evaluation in our local setting. This is likely a result of variations between the development and the validation cohorts, and the change in healthcare systems over time, and highlights the need for an up-to-date and context-specific risk prediction model.

Physician Knowledge Base: Clinical Decision Support Systems

With the introduction of electronic medical records (EMRs), it has become possible to accumulate massive amounts of qualitative medical data. As such, EMRs have become increasingly used in clinical decision support systems (CDSSs). While CDSSs aim to reduce medical errors normally occurring in the process of treating patients by physicians, technical maturity and the completeness of CDSSs do not meet standards for medical use yet. As data further accumulates, CDSS algorithms must be continuously updated to allow CDSSs to perform their core functions. Doing so, however, requires extensive time and manpower investments. In current practice, computational systems already perform a wide variety of functions in medical settings to allow medical staff to focus on other tasks. However, no prior research has evaluated the potential effectiveness of future CDSSs nor analyzed possibilities for their further development. In this article, we evaluate CDSS technology with the consideration that medical staff also understand the core functions of such systems.

Effect of Computerized Physician Order Entry and Clinical Decision Support System on Adverse Drug Events Prevention in the Emergency Department: A Systematic Review

Objective: An adverse drug event (ADE) is an injury resulting from a medical intervention related to a drug. The emergency department (ED) is a ward vulnerable to more ADEs because of overcrowding. Information technologies such as computerized physician order entry (CPOE) and clinical decision support system (CDSS) may decrease the occurrence of ADEs. This study aims to review research that reported the evaluation of the effectiveness of CPOE and CDSS on lowering the occurrence of ADEs in the ED. Data Sources: PubMed, EMBASE, and Web of Science databases were used to find studies published from 2003 to 2018. The search was conducted in November 2018. Study Selection and Data Extraction: The search resulted in 1700 retrieved articles. After applying inclusion and exclusion criteria, 11 articles were included. Data on the date, country, type of system, medication process stages, study design, participants, sample size, and outcomes were extracted. Data Synthesis: Results showed that CPOE and CDSS may prevent ADEs in the ED through significantly decreasing the rate of errors, ADEs, excessive dose, and inappropriate prescribing (in 54.5% of articles); furthermore, CPOE and CDSS may significantly increase the rate of appropriate prescribing and dosing in compliance with established guidelines (45.5% of articles). Conclusion: This study revealed that the use of CPOE and CDSS can lower the occurrence of ADEs in the ED; however, further randomized controlled trials are needed to address the effect of a CDSS, with basic or advanced features, on the occurrence of ADEs in the ED.

Characteristics of pharmacist's interventions triggered by prescribing errors related to computerised physician order entry in French hospitals: a cross-sectional observational study

OBJECTIVES

Computerised physician order entry (CPOE) systems facilitate the review of medication orders by pharmacists. Reports have emerged that show conception flaws or the misuse of CPOE systems generate prescribing errors. We aimed to characterise pharmacist interventions (PIs) triggered by prescribing errors identified as system-related errors (PISREs) in French hospitals.

DESIGN

This was a cross-sectional observational study based on PIs prospectively documented in the Act-IP observatory database from January 2014 to December 2018.

SETTING

PISREs from 319 French computerised healthcare facilities were analysed.

PARTICIPANTS

Among the 319 French hospitals, 232 (72.7%) performed SRE interventions, involving 652 (51%) pharmacists.

RESULTS

Among the 331 678 PIs recorded, 27 058 were qualified as due to SREs (8.2%). The main drug-related problems associated with PISREs were supratherapeutic (27.5%) and subtherapeutic dosage (17.2%), non-conformity with guidelines/contraindications (22.4%) and improper administration (17.9%). The PI prescriber acceptation rate was 78.9% for SREs vs 67.6% for other types of errors. The PISRE ratio was estimated relative to the total number of PIs. Concerning the certification status of CPOE systems, the PISRE ratio was 9.4% for non-certified systems vs 5.5% for certified systems (p<0.001). The PISRE ratio for senior pharmacists was 9.2% and that for pharmacy residents 5.4% (p<0.001). Concerning prescriptions made by graduate prescribers and those made by residents, the PISRE ratio was 8.4% and 7.8%, respectively (p<0.001).

CONCLUSION

Computer-related prescribing errors are common. The PI acceptance rate by prescribers was higher than that observed for PIs that were not CPOE related. This suggests that physicians consider the potential clinical consequences of SREs for patients to be more frequently serious than interventions unrelated to CPOE. CPOE medication review requires continual pharmacist diligence to catch these errors. The significantly lower PISRE ratio for certified software should prompt patient safety agencies to undertake studies to identify the safest software and discard software that is potentially dangerous.

Reducing and Sustaining Duplicate Medical Record Creation by Usability Testing and System Redesign

OBJECTIVES

Duplicate medical record creation is a common and consequential health care systems error often caused by poor search system usability and inappropriate user training.

METHODS

We conducted two phases of scenario-based usability testing with patient registrars working in areas at risk of generating duplicate medical records. Phase 1 evaluated the existing search system, which led to system redesigns. Phase 2 tested the redesigned system to mitigate potential errors before health system-wide implementation. To evaluate system effectiveness, we compared the monthly potential duplicate medical record rates for preimplementation and postimplementation months.

RESULTS

The existing system could not effectively handle a misspelling, which led to failed search and duplicate medical record creation. Using the existing system, 96% of registrars found commonly spelled patient names whereas only 69% successfully found complicated names. Registrars lacked knowledge and usage of a phonetic matching function to assist in misspelling. The new system consistently captured the correct patient regardless of misspelling, but search returned more potential matches, resulting in, on average, 4 seconds longer to select common names. Potential monthly duplicate medical record rate reduced by 38%, from 4% to 2.3% after implementation of the new system, and has sustained at an average of 2.5% for 2 years.

CONCLUSIONS

Usability testing was an effective method to reveal problems and aid system redesign to deliver a more user friendly system, hence reducing the potential for medical record duplication. Greater standards for usability would ensure that these improvements can be realized before rather than after exposing patients to risks.

Pharmacogenomic testing to support prescribing in primary care: a structured review of implementation models

The application of pharmacogenomics could meaningfully contribute toward medicines optimization within primary care. This review identified 13 studies describing eight implementation models utilizing a multi-gene pharmacogenomic panel within a primary care or community setting. These were small feasibility studies (n <200). They demonstrated importance and feasibility of pre-test counseling, the role of the pharmacist, data integration into the electronic medical record and point-of-care clinical decision support systems (CDSS). Findings were considered alongside existing primary care prescribing practices and implementation frameworks to demonstrate how issues may be addressed by existing nationalized healthcare and primary care infrastructure. Development of point-of-care CDSS should be prioritized; establishing clinical leadership, education programs, defining practitioner roles and responsibilities and addressing commissioning issues will also be crucial.

Physician-Nurse Communication Surrounding Computerized Physician Order Entry Systems From Social and Technical Perspective: An Ethnographic Study

Although computerized physician order entry systems improve order transmission and patient safety, overdependence on these systems can impede users' communication. This ethnographic study explored physician-nurse communication surrounding computerized physician order entry systems using a sociotechnical framework. Fieldwork conducted in a tertiary teaching hospital comprised 89 hours of participant observation, and individual semistructured interviews were held with seven nurses and five physicians. In addition, documents and artifacts were collected. Three core themes emerged. First, computerized physician order entry quality-related issues undermined the work efficiency of physicians and nurses. Specifically, usability was error prone because of cognitive overload, and the system was unable to perform relevant traces and raise alerts, demonstrating poor interoperability. Second, social factors, including insufficient training, unclear responsibilities, and a lack of awareness concerning interdisciplinary communication, compounded communication problems. Last, environmental factors, including noncoterminous spaces and times and insufficient technical support, impeded the resolution of communication problems. Technical and social contextual factors relating to computerized physician order entry systems jointly affected physician-nurse communication. Cognitive issues and insufficient alerts impacted work efficiency the most and were compounded by contextual individual- and team-related factors and environmental factors. Therefore, improved functions of computerized physician order entry systems and interprofessional communication training are required to optimize technical and social aspects of physician-nurse communication.

Medication Safety in Two Intensive Care Units of a Community Teaching Hospital After Electronic Health Record Implementation: Sociotechnical and Human Factors Engineering Considerations

OBJECTIVE

The aim of the study was to assess the impact of Electronic Health Record (EHR) implementation on medication safety in two intensive care units (ICUs).

METHODS

Using a prospective pre-post design, we assessed 1254 consecutive admissions to two ICUs before and after an EHR implementation. Each medication event was evaluated with regard to medication error (error type, medication-management stage) and impact on patient (severity of potential or actual harm).

RESULTS

We identified 4063 medication-related events either pre-implementation (2074 events) or post-implementation (1989 events). Although the overall potential for harm due to medication errors decreased post-implementation only 2 of the 3 error rates were significantly lower post-implementation. After EHR implementation, we observed reductions in rates of medication errors per admission at the stages of transcription (0.13-0, P < 0.001), dispensing (0.49-0.16, P < 0.001), and administration (0.83-0.56, P = 0.011). Within the ordering stage, 4 error types decreased post-implementation (orders with omitted information, error-prone abbreviations, illegible orders, failure to renew orders) and 4 error types increased post-implementation (orders of wrong drug, orders containing a wrong start or stop time, duplicate orders, orders with inappropriate or wrong information). Within the administration stage, we observed a reduction of late administrations and increases in omitted administrations and incorrect documentation.

CONCLUSIONS

Electronic Health Record implementation in two ICUs was associated with both improvement and worsening in rates of specific error types. Further safety improvements require a nuanced understanding of how various error types are influenced by the technology and the sociotechnical work system of the technology implementation. Recommendations based on human factors engineering principles are provided for reducing medication errors.

Can evidence-based decision support tools transform antibiotic management? A systematic review and meta-analyses

OBJECTIVES

To assess the effectiveness of clinical decision support systems (CDSSs) at reducing unnecessary and suboptimal antibiotic prescribing within different healthcare settings.

METHODS

A systematic review of published studies was undertaken with seven databases from database inception to November 2018. A protocol was developed using the PRISMA-P checklist and study selection criteria were determined prior to performing the search. Critical appraisal of studies was undertaken using relevant tools. Meta-analyses were performed using a random-effects model to determine whether CDSS use affected optimal antibiotic management.

RESULTS

Fifty-seven studies were identified that reported on CDSS effectiveness. Most were non-randomized studies with low methodological quality. However, randomized controlled trials of moderate methodological quality were available and assessed separately. The meta-analyses indicated that appropriate antibiotic therapy was twice as likely to occur following the implementation of CDSSs (OR 2.28, 95% CI 1.82-2.86, k = 20). The use of CDSSs was also associated with a relative decrease (18%) in mortality (OR 0.82, 95% CI 0.73-0.91, k = 18). CDSS implementation also decreased the overall volume of antibiotic use, length of hospital stay, duration and cost of therapy. The magnitude of the effect did vary by study design, but the direction of the effect was consistent in favouring CDSSs.

CONCLUSIONS

Decision support tools can be effective to improve antibiotic prescribing, although there is limited evidence available on use in primary care. Our findings suggest that a focus on system requirements and implementation processes would improve CDSS uptake and provide more definitive benefits for antibiotic stewardship.

How effective are electronic medication systems in reducing medication error rates and associated harm among hospital inpatients? A systematic review and meta-analysis

OBJECTIVE

To conduct a systematic review and meta-analysis to assess: 1) changes in medication error rates and associated patient harm following electronic medication system (EMS) implementation; and 2) evidence of system-related medication errors facilitated by the use of an EMS.

MATERIALS AND METHODS

We searched Medline, Scopus, Embase, and CINAHL for studies published between January 2005 and March 2019, comparing medication errors rates with or without assessments of related harm (actual or potential) before and after EMS implementation. EMS was defined as a computer-based system enabling the prescribing, supply, and/or administration of medicines. Study quality was assessed.

RESULTS

There was substantial heterogeneity in outcomes of the 18 included studies. Only 2 were strong quality. Meta-analysis of 5 studies reporting change in actual harm post-EMS showed no reduced risk (RR: 1.22, 95% CI: 0.18-8.38, P = .8) and meta-analysis of 3 studies reporting change in administration errors found a significant reduction in error rates (RR: 0.77, 95% CI: 0.72-0.83, P = .004). Of 10 studies of prescribing error rates, 9 reported a reduction but variable denominators precluded meta-analysis. Twelve studies provided specific examples of system-related medication errors; 5 quantified their occurrence.

DISCUSSION AND CONCLUSION

Despite the wide-scale adoption of EMS in hospitals around the world, the quality of evidence about their effectiveness in medication error and associated harm reduction is variable. Some confidence can be placed in the ability of systems to reduce prescribing error rates. However, much is still unknown about mechanisms which may be most effective in improving medication safety and design features which facilitate new error risks.

What do senior physicians think about AI and clinical decision support systems: Quantitative and qualitative analysis of data from specialty societies

AIMS

The aim was to help physicians engage with NHS and other policymakers about the use, procurement and regulation of artificial intelligence, algorithms and clinical decision support systems (CDSS) in the NHS by identifying the professional benefits of and concerns about these systems.

METHODS

We piloted a three-page survey instrument with closed and open-ended questions on SurveyMonkey, then circulated it to specialty societies via email. Both quantitative and qualitative methods were used to analyse responses.

RESULTS

The results include the current usage of CDSS; identified benefits; concerns about quality; concerns about regulation, professional practice, ethics and liability, as well as actions being taken by the specialty societies to address these; and aspects of CDSS quality that need to be tested.

CONCLUSION

While results confirm many expected benefits and concerns about CDSS, they raise new professional concerns and suggest further actions to explore with partners on behalf of the physician community.

Current Knowledge about Providing Drug-Drug Interaction Services for Patients-A Scoping Review

Drug-drug interactions (DDIs) pose a major problem to patient safety. eHealth solutions have the potential to address this problem and generally improve medication management by providing digital services for health care professionals and patients. Clinical decision support systems (CDSS) to alert physicians or pharmacists about DDIs are common, and there is an extensive body of research about CDSS for professionals. Information about DDIs is commonly requested by patients, but little is known about providing similar support to patients. The aim of this scoping review was to explore and describe current knowledge about providing digital DDI services for patients. Using a broad search strategy and an established framework for scoping reviews, 19 papers were included. The results show that although some patients want to check for DDIs themselves, there are differences between patients, in terms of demands and ability. There are numerous DDI services available, but the existence of large variations regarding service quality implies potential safety issues. The review includes suggestions about design features but also indicates a substantial knowledge gap highlighting the need for further research about how to best design and provide digital DDI to patients without risking patient safety or having other unintended consequences.

Categorization of Medication Safety Errors in Ambulatory Electronic Health Records

Preventable medication errors continue to affect the quality and consistency in the delivery of care. While numerous studies on medication safety have been performed in the inpatient setting, a review of ambulatory patient safety by the American Medical Association found that medication safety errors were the most frequent safety problems in the outpatient arena. The leading cause of ambulatory safety problems, adverse drug events (ADEs), are common, with estimates of more than 2 million ADEs each year in the ambulatory Medicare population alone, and these events are frequently preventable. We conducted an environmental scan that allowed us to create our own categorization schema of medication safety errors in electronic healthcare records (EHRs) found in the outpatient setting and observed which of these were additionally supported in the literature. This study combines data from the California Hospital Patient Safety Organization (CHPSO), with several key articles in the area of medication errors in the EHR era. Method: To best utilize the various EHR ambulatory medication events submitted into CHPSO’s database, we chose to create a framework to bucket the near misses or adverse events (AEs) submitted to the database. This newly created categorization scheme was based on our own drafted categorization labels of events, after a high-level review, and from two leading articles on physician order entry. Additionally, we conducted a literature review of computerized provider order entry (CPOE) medication errors in the ambulatory setting. Within the newly created categorization scheme, we organized the articles based on issues addressed so we could see areas that were supported by the literature and what still needed to be researched. Results: We initially screened the CHPSO database for ambulatory safety events and found 25,417 events. Based on those events, an initial review was completed, and 19,242 events were found in the “Medication or Other Substance” and “Other” categories, in which the EHR appeared to have been a potential contributing factor. This review identified a subset of 2,236 events that were then reviewed. One hundred events were randomly selected for further review to identify common categories. The most common categories in which errors occurred were orders in order sets and plans (n=12) and orders crossing or not crossing encounters (n=12), incorrect order placed on correct patient (n=10), orders missing (n=8), standing orders (n=8), manual data entry errors (n=6), and future orders (n=6). Conclusion: There were several common themes seen in this analysis of ambulatory medication safety errors related to the EHR. Common among them were incorrect orders consisting of examples such as dose errors or ordering the wrong medication. The manual data entry errors consisted of height or weight being entered incorrectly or entering the wrong diagnostic codes. Lastly, different sources of medication safety information demonstrate a diversity of errors in ambulatory medication safety. This confirms the importance of considering more than one source when attempting to comprehensively describe ambulatory medication safety errors.

Clinical decision support systems-based interventions to improve medication outcomes: A systematic literature review on features and effects

Background: Clinical decision support systems (CDSSs) interventions were used to improve the life quality and safety in patients and also to improve practitioner performance, especially in the field of medication. Therefore, the aim of the paper was to summarize the available evidence on the impact, outcomes and significant factors on the implementation of CDSS in the field of medicine. Methods: This study is a systematic literature review. PubMed, Cochrane Library, Web of Science, Scopus, EMBASE, and ProQuest were investigated by 15 February 2017. The inclusion requirements were met by 98 papers, from which 13 had described important factors in the implementation of CDSS, and 86 were medicated-related. We categorized the system in terms of its correlation with medication in which a system was implemented, and our intended results were examined. In this study, the process outcomes (such as; prescription, drug-drug interaction, drug adherence, etc.), patient outcomes, and significant factors affecting the implementation of CDSS were reviewed. Results: We found evidence that the use of medication-related CDSS improves clinical outcomes. Also, significant results were obtained regarding the reduction of prescription errors, and the improvement in quality and safety of medication prescribed. Conclusion: The results of this study show that, although computer systems such as CDSS may cause errors, in most cases, it has helped to improve prescribing, reduce side effects and drug interactions, and improve patient safety. Although these systems have improved the performance of practitioners and processes, there has not been much research on the impact of these systems on patient outcomes.

Remote pharmacy service in primary care: The implementation of a cloud-based pre-prescription review system

BACKGROUND

To reduce the occurrence of inappropriate prescription in primary care through the introduction of a cloud-based pre-prescription review system.

OBJECTIVE

We aimed to describe the implementation of a cloud-based pre-prescription review system in the pharmacy practice of Chinese community health centers (CHCs), which currently have few qualified pharmacists.

PRACTICE DESCRIPTION

The cloud-based pre-prescription review system featured reviews by remote clinical pharmacists and targeted the prevention of inappropriate prescription in primary care.

PRACTICE INNOVATION

This study describes the implementation of remote pharmacy at 22 CHCs in Futian District, Shenzhen, China. A pre-prescription system was developed and deployed in the cloud, which is linked to CHCs, and a consortium of qualified clinical pharmacists located in tertiary hospital. All prescriptions were mandatorily reviewed before printing and payment. First, prescriptions were reviewed using cloud-based rational drug use software. Then any detected potentially inappropriate prescriptions were reviewed by the remote pharmacist. The pharmacist consortium also modified review rules to improve efficiency and accuracy.

EVALUATION METHODS

The frequency and proportions of potentially inappropriate prescriptions identified by the review software and the remote pharmacist consortium were analyzed descriptively.

RESULTS

During the 6-month study period (July 1, 2019-December 31, 2019), 340,117 prescription entries from general practitioners in 22 community health care centers were reviewed. Of these, 6479 (3.0%) unique potential entries were suspended for pharmacist review, of which 3230 (49.9%) needed correction from prescribers in the CHCs. The most common corrections were related to improper administration routes or drug-drug interactions or had no justified indications.

CONCLUSION

Inappropriate prescription is not uncommon in CHCs. The cloud-based prescription prereview model proposed in this study can serve as an important tool for the prevention of inappropriate prescription in primary care. The pre-prescription review system also provided opportunities for pharmacists to participate in the enhancement of patient care in primary care.

Alert Override Patterns With a Medication Clinical Decision Support System in an Academic Emergency Department: Retrospective Descriptive Study

Background

Physicians’ alert overriding behavior is considered to be the most important factor leading to failure of computerized provider order entry (CPOE) combined with a clinical decision support system (CDSS) in achieving its potential adverse drug events prevention effect. Previous studies on this subject have focused on specific diseases or alert types for well-defined targets and particular settings. The emergency department is an optimal environment to examine physicians’ alert overriding behaviors from a broad perspective because patients have a wider range of severity, and many receive interdisciplinary care in this environment. However, less than one-tenth of related studies have targeted this physician behavior in an emergency department setting.

Objective

The aim of this study was to describe alert override patterns with a commercial medication CDSS in an academic emergency department.

Methods

This study was conducted at a tertiary urban academic hospital in the emergency department with an annual census of 80,000 visits. We analyzed data on the patients who visited the emergency department for 18 months and the medical staff who treated them, including the prescription and CPOE alert log. We also performed descriptive analysis and logistic regression for assessing the risk factors for alert overrides.

Results

During the study period, 611 physicians cared for 71,546 patients with 101,186 visits. The emergency department physicians encountered 13.75 alerts during every 100 orders entered. Of the total 102,887 alerts, almost two-thirds (65,616, 63.77%) were overridden. Univariate and multivariate logistic regression analyses identified 21 statistically significant risk factors for emergency department physicians’ alert override behavior.

Conclusions

In this retrospective study, we described the alert override patterns with a medication CDSS in an academic emergency department. We found relatively low overrides and assessed their contributing factors, including physicians’ designation and specialty, patients’ severity and chief complaints, and alert and medication type.

Computerized provider order entry-related medication errors among hospitalized patients: An integrative review

The Institute of Medicine estimates that 7,000 lives are lost yearly as a result of medication errors. Computerized physician and/or provider order entry was one of the proposed solutions to overcome this tragic issue. Despite some promising data about its effectiveness, it has been found that computerized provider order entry may facilitate medication errors.The purpose of this review is to summarize current evidence of computerized provider order entry -related medication errors and address the sociotechnical factors impacting the safe use of computerized provider order entry. By using PubMed and Google Scholar databases, a systematic search was conducted for articles published in English between 2007 and 2019 regarding the unintended consequences of computerized provider order entry and its related medication errors. A total of 288 articles were screened and categorized based on their use within the review. One hundred six articles met our pre-defined inclusion criteria and were read in full, in addition to another 27 articles obtained from references. All included articles were classified into the following categories: rates and statistics on computerized provider order entry -related medication errors, types of computerized provider order entry -related unintended consequences, factors contributing to computerized provider order entry failure, and recommendations based on addressing sociotechnical factors. Identifying major types of computerized provider order entry -related unintended consequences and addressing their causes can help in developing appropriate strategies for safe and effective computerized provider order entry. The interplay between social and technical factors can largely affect its safe implementation and use. This review discusses several factors associated with the unintended consequences of this technology in healthcare settings and presents recommendations for enhancing its effectiveness and safety within the context of sociotechnical factors.

The impact of a multifaceted intervention to reduce potentially inappropriate prescribing among discharged older adults: a before-and-after study

Background

Potentially inappropriate prescribing (PIP) is associated with the incidence of adverse drug reactions, drug-related hospitalization and other negative outcomes in older adults. After hospitalization, older adults might be discharged with several types of PIPs. Studies have found that the lack of healthcare professionals' (HCPs) knowledge regarding PIP is one of the major contributing factors in this issue. The purpose of this study is to investigate the impact of a multifaceted intervention on physicians' and clinical pharmacists' behavior regarding potentially inappropriate medication (PIM) and potential prescribing omission (PPO) among hospitalized older adults.

Methods

This is a before-and-after study that took place in a tertiary Malaysian hospital. Discharge medications of patients ≥65 years old were reviewed to identify PIMs/PPOs using version 2 of the STOPP/START criteria. The prevalence and pattern of PIM/PPO before and after the intervention were compared. The intervention targeted the physicians and clinical pharmacists and it consisted of academic detailing and a newly developed smartphone application (app).

Results

The study involved 240 patients before (control group) and 240 patients after the intervention. The prevalence of PIM was 22% and 27% before and after the intervention, respectively (P = 0.213). The prevalence of PPO in the intervention group was significantly lower than that in the control group (42% Vs. 53.3%); P = 0.014. This difference remained statistically significant after controlling for other variables (P = 0.015). The intervention was effective in reducing the two most common PPOs; the omission of vitamin D supplements in patients with a history of falls (P = 0.001) and the omission of angiotensin converting enzyme inhibitor in patients with coronary artery disease (P = 0.03).

Conclusions

The smartphone app coupled with academic detailing was effective in reducing the prevalence of PPO at discharge. However, it did not significantly affect the prevalence or pattern of PIM.

The Use of a Decision Support System in Swedish Pharmacies to Identify Potential Drug-Related Problems-Effects of a National Intervention Focused on Reviewing Elderly Patients' Prescriptions

In pharmacies in Sweden, a clinical decision support system called Electronic Expert Support (EES) is available to analyse patients' prescriptions for potential drug-related problems. A nationwide intervention was performed in 2018 among all Swedish pharmacy chains to increase the use of EES among patients 75 years or older. The aim of this research was to study the use of EES in connection with the national intervention in order to describe any effects of the intervention, to understand how pharmacists use EES and to identify any barriers and facilitators for the use of EES by pharmacists for elderly patients. Data on the number and categories of EES analyses, alerts, resolved alerts and active pharmacies was provided by the Swedish eHealth Agency. The effects of the intervention were analysed using interrupted time series regression. A web-based questionnaire comprising 20 questions was sent to 1500 pharmacists randomly selected from all pharmacies in Sweden. The study shows that pharmacists use and appreciate EES and that the national intervention had a clear effect during the week of the intervention and seems to have contributed to a faster increase in pharmacists' use of EES during the year to follow. The study also identified several issues or barriers for using EES.

Effects of CPOE-based medication ordering on outcomes: an overview of systematic reviews

BACKGROUND

Computerised provider order entry (CPOE) systems are widely used in clinical settings for the electronic ordering of medications, laboratory tests and radiological therapies. However, evidence regarding effects of CPOE-based medication ordering on clinical and safety outcomes is mixed. We conducted an overview of systematic reviews (SRs) to characterise the cumulative effects of CPOE use for medication ordering in clinical settings.

METHODS

MEDLINE, EMBASE, CINAHL and the Cochrane Library were searched to identify published SRs from inception to 12 February 2018. SRs investigating the effects of the use of CPOE for medication ordering were included. Two reviewers independently extracted data and assessed the methodological quality of included SRs.

RESULTS

Seven SRs covering 118 primary studies were included for review. Pooled studies from the SRs in inpatient settings showed that CPOE use resulted in statistically significant decreases in medication errors and adverse drug events (ADEs); however, there was considerable variation in the magnitude of their relative risk reduction (54%-92% for errors, 35%-53% for ADEs). There was no significant relative risk reduction on hospital mortality or length of stay. Bibliographic analysis showed limited overlap (24%) among studies included across all SRs.

CONCLUSION

SRs on CPOEs included predominantly non-randomised controlled trials and observational studies with varying foci. SRs predominantly focused on inpatient settings and often lacked comparison groups; SRs used inconsistent definitions of outcomes, lacked descriptions regarding the effects on patient harm and did not differentiate among the levels of available decision support. With five of the seven SRs having low to moderate quality, findings from the SRs must be interpreted with caution. We discuss potential directions for future primary studies and SRs of CPOE.

Organizational Characteristics and the Adoption of Electronic Health Records Among Nursing Homes in One Southern State

Electronic health records (EHRs) can improve quality of care and patient safety, as demonstrated in a variety of health care settings. However, greater use of EHRs in nursing homes (NHs) is needed. To understand which NHs have and have not adopted EHR systems, all federally certified NHs in Arkansas (n = 223) were surveyed, with 27.9% responding. Non-responders were similar to responders on all characteristics except for staffing skill mix, with responders having a higher skill mix than non-responders. Two thirds of responding Arkansas NHs reported having an EHR system in use (69.8%), while only a few reported no plans for an EHR system (4.8%). NHs with greater resources and in competitive markets were more likely to implement EHR systems. Full implementation across all NHs may require intervention, which should be explored in future research. In addition, future investigation should consider the level of interoperability of EHR systems that are in place among NHs.