Predicting self-intercepted medication ordering errors using machine learning

Using machine learning or deep learning models in a hospital setting to detect inappropriate prescriptions: a systematic review

OBJECTIVES

The emergence of artificial intelligence (AI) is catching the interest of hospital pharmacists. A massive collection of health data is now available to train AI models and hold the promise of disrupting codes and practices. The objective of this systematic review was to examine the state of the art of machine learning or deep learning models that detect inappropriate hospital medication orders.

METHODS

A systematic review was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement. MEDLINE and Embase databases were searched from inception to May 2023. Studies were included if they reported and described an AI model intended for use by clinical pharmacists in hospitals. Risk of bias was assessed using the Prediction model Risk Of Bias ASsessment Tool (PROBAST).

RESULTS

13 articles were selected after review: 12 studies were judged to have high risk of bias; 11 studies were published between 2020 and 2023; 8 were conducted in North America and Asia; 6 analysed orders and detected inappropriate prescriptions according to patient profiles and medication orders; and 7 detected specific inappropriate prescriptions, such as detecting antibiotic resistance, dosage abnormality in prescriptions, high alert drugs errors from prescriptions or predicting the risk of adverse drug events. Various AI models were used, mainly supervised learning techniques. The training datasets used were very heterogeneous; the length of study varied from 2 weeks to 7 years and the number of prescription orders analysed went from 31 to 5 804 192.

CONCLUSIONS

This systematic review points out that, to date, few original research studies report AI tools based on machine or deep learning in the field of hospital clinical pharmacy. However, these original articles, while preliminary, highlighted the potential value of integrating AI into clinical hospital pharmacy practice.

Risk Management and Patient Safety in the Artificial Intelligence Era: A Systematic Review

BACKGROUND

Healthcare systems represent complex organizations within which multiple factors (physical environment, human factor, technological devices, quality of care) interconnect to form a dense network whose imbalance is potentially able to compromise patient safety. In this scenario, the need for hospitals to expand reactive and proactive clinical risk management programs is easily understood, and artificial intelligence fits well in this context. This systematic review aims to investigate the state of the art regarding the impact of AI on clinical risk management processes. To simplify the analysis of the review outcomes and to motivate future standardized comparisons with any subsequent studies, the findings of the present review will be grouped according to the possibility of applying AI in the prevention of the different incident type groups as defined by the ICPS.

MATERIALS AND METHODS

On 3 November 2023, a systematic review of the literature according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines was carried out using the SCOPUS and Medline (via PubMed) databases. A total of 297 articles were identified. After the selection process, 36 articles were included in the present systematic review.

RESULTS AND DISCUSSION

The studies included in this review allowed for the identification of three main "incident type" domains: clinical process, healthcare-associated infection, and medication. Another relevant application of AI in clinical risk management concerns the topic of incident reporting.

CONCLUSIONS

This review highlighted that AI can be applied transversely in various clinical contexts to enhance patient safety and facilitate the identification of errors. It appears to be a promising tool to improve clinical risk management, although its use requires human supervision and cannot completely replace human skills. To facilitate the analysis of the present review outcome and to enable comparison with future systematic reviews, it was deemed useful to refer to a pre-existing taxonomy for the identification of adverse events. However, the results of the present study highlighted the usefulness of AI not only for risk prevention in clinical practice, but also in improving the use of an essential risk identification tool, which is incident reporting. For this reason, the taxonomy of the areas of application of AI to clinical risk processes should include an additional class relating to risk identification and analysis tools. For this purpose, it was considered convenient to use ICPS classification.

Machine learning models to detect and predict patient safety events using electronic health records: A systematic review

INTRODUCTION

Identifying patient safety events using electronic health records (EHRs) and automated machine learning-based detection methods can help improve the efficiency and quality of healthcare service provision.

OBJECTIVE

This study aimed to systematically review machine learning-based methods and techniques, as well as their results for patient safety event management using EHRs.

METHODS

We reviewed the studies that focused on machine learning techniques, including automatic prediction and detection of patient safety events and medical errors through EHR analysis to manage patient safety events. The data were collected by searching Scopus, PubMed (Medline), Web of Science, EMBASE, and IEEE Xplore databases.

RESULTS

After screening, 41 papers were reviewed. Support vector machine (SVM), random forest, conditional random field (CRF), and bidirectional long short-term memory with conditional random field (BiLSTM-CRF) algorithms were mostly applied to predict, identify, and classify patient safety events using EHRs; however, they had different performances. BiLSTM-CRF was employed in most of the studies to extract and identify concepts, e.g., adverse drug events (ADEs) and adverse drug reactions (ADRs), as well as relationships between drug and severity, drug and ADEs, drug and ADRs. Recurrent neural networks (RNN) and BiLSTM-CRF had the best results in detecting ADEs compared to other patient safety events. Linear classifiers and Naive Bayes (NB) had the highest performance for ADR detection. Logistic regression had the best results in detecting surgical site infections. According to the findings, the quality of articles has non-significantly improved in recent years, but they had low average scores.

CONCLUSIONS

Machine learning can be useful in automatic detection and prediction of patient safety events. However, most of these algorithms have not yet been externally validated or prospectively tested. Therefore, further studies are required to improve the performance of these automated systems.

Association between hospital legal constructions and medical disputes: A multi-center analysis of 130 tertiary hospitals in Hunan Province, China

Background

Medical disputes are common in hospitals and a major challenge for the operations of medical institutions. However, few studies have looked into the association between medical disputes and hospital legal constructions. The purpose of the study was to investigate the relationship between hospital legal constructions and medical disputes, and it also aimed to develop a nomogram to estimate the likelihood of medical disputes.

Methods

Between July and September 2021, 2,716 administrators from 130 hospitals were enrolled for analysis. The study collected seventeen variables for examination. To establish a nomogram, administrators were randomly split into a training group (n = 1,358) and a validation group (n = 1,358) with a 50:50 ratio. The nomogram was developed using data from participants in the training group, and it was validated in the validation group. The nomogram contained significant variables that were linked to medical disputes and were identified by multivariate analysis. The nomogram's predictive performance was assessed utilizing discriminative and calibrating ability. A web calculator was developed to be conducive to model utility.

Results

Medical disputes were observed in 41.53% (1,128/2,716) of participants. Five characteristics, including male gender, higher professional ranks, longer length of service, worse understanding of the hospital charters, and worse construction status of hospital rule of law, were significantly associated with more medical disputes based on the multivariate analysis. As a result, these variables were included in the nomogram development. The AUROC was 0.67 [95% confident interval (CI): 0.64-0.70] in the training group and 0.68 (95% CI: 0.66-0.71) in the validation group. The corresponding calibration slopes were 1.00 and 1.05, respectively, and intercepts were 0.00 and -0.06, respectively. Three risk groups were created among the participants: Those in the high-risk group experienced medical disputes 2.83 times more frequently than those in the low-risk group (P < 0.001).

Conclusion

Medical dispute is prevailing among hospital administrators, and it can be reduced by the effective constructions of hospital rule of law. This study proposes a novel nomogram to estimate the likelihood of medical disputes specifically among administrators in tertiary hospitals, and a web calculator can be available at https://ymgarden.shinyapps.io/Predictionofmedicaldisputes/.

Detecting Unusual Intravenous Infusion Alerting Patterns with Machine Learning Algorithms

OBJECTIVE

To detect unusual infusion alerting patterns using machine learning (ML) algorithms as a first step to advance safer inpatient intravenous administration of high-alert medications.

MATERIALS AND METHODS

We used one year of detailed propofol infusion data from a hospital. Interpretable and clinically relevant variables were feature engineered, and data points were aggregated per calendar day. A univariate (maximum times-limit) moving range (mr) control chart was used to simulate clinicians' common approach to identifying unusual infusion alerting patterns. Three different unsupervised multivariate ML-based anomaly detection algorithms (Local Outlier Factor, Isolation Forest, and k-Nearest Neighbors) were used for the same purpose. Results from the control chart and ML algorithms were compared.

RESULTS

The propofol data had 3,300 infusion alerts, 92% of which were generated during the day shift and seven of which had a times-limit greater than 10. The mr-chart identified 15 alert pattern anomalies. Different thresholds were set to include the top 15 anomalies from each ML algorithm. A total of 31 unique ML anomalies were grouped and ranked by agreeability. All algorithms agreed on 10% of the anomalies, and at least two algorithms agreed on 36%. Each algorithm detected one specific anomaly that the mr-chart did not detect. The anomaly represented a day with 71 propofol alerts (half of which were overridden) generated at an average rate of 1.06 per infusion, whereas the moving alert rate for the week was 0.35 per infusion.

DISCUSSION

These findings show that ML-based algorithms are more robust than control charts in detecting unusual alerting patterns. However, we recommend using a combination of algorithms, as multiple algorithms serve a benchmarking function and allow researchers to focus on data points with the highest algorithm agreeability.

CONCLUSION

Unsupervised ML algorithms can assist clinicians in identifying unusual alert patterns as a first step toward achieving safer infusion practices.

Detecting Unusual Intravenous Infusion Alerting Patterns with Machine Learning Algorithms

OBJECTIVE

To detect unusual infusion alerting patterns using machine learning (ML) algorithms as a first step to advance safer inpatient intravenous administration of high-alert medications.

MATERIALS AND METHODS

We used one year of detailed propofol infusion data from a hospital. Interpretable and clinically relevant variables were feature engineered, and data points were aggregated per calendar day. A univariate (maximum times-limit) moving range (mr) control chart was used to simulate clinicians' common approach to identifying unusual infusion alerting patterns. Three different unsupervised multivariate ML-based anomaly detection algorithms (Local Outlier Factor, Isolation Forest, and k-Nearest Neighbors) were used for the same purpose. Results from the control chart and ML algorithms were compared.

RESULTS

The propofol data had 3,300 infusion alerts, 92% of which were generated during the day shift and seven of which had a times-limit greater than 10. The mr-chart identified 15 alert pattern anomalies. Different thresholds were set to include the top 15 anomalies from each ML algorithm. A total of 31 unique ML anomalies were grouped and ranked by agreeability. All algorithms agreed on 10% of the anomalies, and at least two algorithms agreed on 36%. Each algorithm detected one specific anomaly that the mr-chart did not detect. The anomaly represented a day with 71 propofol alerts (half of which were overridden) generated at an average rate of 1.06 per infusion, whereas the moving alert rate for the week was 0.35 per infusion.

DISCUSSION

These findings show that ML-based algorithms are more robust than control charts in detecting unusual alerting patterns. However, we recommend using a combination of algorithms, as multiple algorithms serve a benchmarking function and allow researchers to focus on data points with the highest algorithm agreeability.

CONCLUSION

Unsupervised ML algorithms can assist clinicians in identifying unusual alert patterns as a first step toward achieving safer infusion practices.