Decision Support Tools within the Electronic Health Record

Interpretation of acid-base metabolism on arterial blood gas samples via machine learning algorithms

BACKGROUND

Arterial blood gas evaluation is crucial for critically ill patients, as it provides essential information about acid-base metabolism and respiratory balance, but evaluation can be complex and time-consuming. Artificial intelligence can perform tasks that require human intelligence, and it is revolutionizing healthcare through technological advancements.

AIM

This study aims to assess arterial blood gas evaluation using artificial intelligence algorithms.

METHODS

The study included 21.541 retrospective arterial blood gas samples, categorized into 15 different classes by experts for evaluating acid-base metabolism status. Six machine learning algorithms were utilized; accuracy, balanced accuracy, sensitivity, specificity, precision, and F1 values of the models were determined; and ROC curves were drawn to assess areas under the curve for each class. Evaluation of which sample was estimated in which class was conducted using the confusion matrices of the models.

RESULTS

The bagging classifier (BC) model achieved the highest balanced accuracy with 99.24%, whereas the XGBoost model reached the highest accuracy with 99.66%. The BC model shows 100% sensitivity for nine classes and 100% specificity for 10 classes, and the model correctly predicted 6438 of 6463 test samples and achieved an accuracy of 99.61%, with an area under the curve > 0.9 in all classes on a class basis.

CONCLUSION

The machine learning models developed exhibited remarkable accuracy, sensitivity, and specificity in predicting the status of acid-base metabolism. However, implementing these models can aid clinicians, freeing up their time for more intricate tasks.

Development and implementation of a clinical decision support system-based quality initiative to reduce central line-associated bloodstream infections

Background

Central venous lines (CVLs) are frequently utilized in critically ill patients and confer a risk of central line-associated bloodstream infections (CLABSIs). CLABSIs are associated with increased mortality, extended hospitalization, and increased costs. Unnecessary CVL utilization contributes to CLABSIs. This initiative sought to implement a clinical decision support system (CDSS) within an electronic health record (EHR) to quantify the prevalence of potentially unnecessary CVLs and improve their timely removal in six adult intensive care units (ICUs).

Methods

Intervention components included: (1) evaluating existing CDSS' effectiveness, (2) clinician education, (3) developing/implementing an EHR-based CDSS to identify potentially unnecessary CVLs, (4) audit/feedback, and (5) reviewing EHR/institutional data to compare rates of removal of potentially unnecessary CVLs, device utilization, and CLABSIs pre- and postimplementation. Data was evaluated with statistical process control charts, chi-square analyses, and incidence rate ratios.

Results

Preimplementation, 25.2% of CVLs were potentially removable, and the mean weekly proportion of these CVLs that were removed within 24 hours was 20.0%. Postimplementation, a greater proportion of potentially unnecessary CVLs were removed (29%, p < 0.0001), CVL utilization decreased, and days between CLABSIs increased. The intervention was most effective in ICUs staffed by pulmonary/critical care physicians, who received monthly audit/feedback, where timely CVL removal increased from a mean of 18.0% to 30.5% (p < 0.0001) and days between CLABSIs increased from 17.3 to 25.7.

Conclusions

A significant proportion of active CVLs were potentially unnecessary. CDSS implementation, in conjunction with audit and feedback, correlated with a sustained increase in timely CVL removal and an increase in days between CLABSIs.

Clinical decision support to improve CBC and differential ordering

OBJECTIVES

Complete blood count and differential (CBC diff) is a common laboratory test that may be overused or misordered, particularly in an inpatient setting. We assessed the ability of a clinical decision support (CDS) alert to decrease unnecessary orders for CBC diff and analyzed its impact in the laboratory.

METHODS

We designed 3 CDS alerts to provide guidance to providers ordering CBC diff on inpatients at frequencies of daily, greater than once daily, or as needed.

RESULTS

The 3 alerts were highly effective in reducing orders for CBC diff at the frequencies targeted by the alert. Overall, test volume for CBC diff decreased by 32% (mean of 5257 tests per month) after implementation of the alerts, with a corresponding decrease of 22% in manual differentials performed (mean of 898 per month). Turnaround time for manual differentials decreased by a mean of 41.5 minutes, with a mean decrease of up to 90 minutes during peak morning hours.

CONCLUSIONS

The 3 CDS alerts successfully decreased inpatient orders for CBC diff and improved the quality of patient care by decreasing turnaround time for manual differentials.

Retrospective evaluation of clinical decision support for within-laboratory optimization of SARS-CoV-2 NAAT workflow

The unprecedented demand for severe acute respiratory syndrome coronavirus 2 (SARS‑CoV‑2) testing led to challenges in prioritizing and processing specimens efficiently. We describe and evaluate a novel workflow using provider- and patient-facing ask at order entry (AOE) questions to generate distinctive icons on specimen labels for within-laboratory clinical decision support (CDS) for specimen triaging. A multidisciplinary committee established target turnaround times (TATs) for SARS-CoV-2 nucleic acid amplification test (NAAT) based on common clinical scenarios. A set of AOE questions was used to collect relevant clinical information that prompted icon generation for triaging SARS-CoV-2 NAAT specimens. We assessed the collect-to-verify TATs among relevant clinical scenarios. Our study included a total of 1,385,813 SARS-CoV-2 NAAT conducted from March 2020 to June 2022. Most testing met the TAT targets established by institutional committees, but deviations from target TATs occurred during periods of high demand and supply shortages. Median TATs for emergency department (ED) and inpatient specimens and ambulatory pre-procedure populations were stable over the pandemic. However, healthcare worker and other ambulatory test TATs varied substantially, depending on testing volume and community transmission rates. Median TAT significantly differed throughout the pandemic for ED and inpatient clinical scenarios, and there were significant differences in TAT among label icon-signified ambulatory clinical scenarios. We describe a novel approach to CDS for triaging specimens within the laboratory. The use of CDS tools could help clinical laboratories prioritize and process specimens efficiently, especially during times of high demand. Further studies are needed to evaluate the impact of our CDS tool on overall laboratory efficiency and patient outcomes. IMPORTANCE We describe a novel approach to clinical decision support (CDS) for triaging specimens within the clinical laboratory for severe acute respiratory syndrome coronavirus 2 (SARS‑CoV‑2) nucleic acid amplification tests (NAAT). The use of our CDS tool could help clinical laboratories prioritize and process specimens efficiently, especially during times of high demand. There were significant differences in the turnaround time for specimens differentiated by icons on specimen labels. Further studies are needed to evaluate the impact of our CDS tool on overall laboratory efficiency and patient outcomes.

Managing Resource Utilization Cost of Laboratory Tests for Patients on Chemotherapy in Johns Hopkins Aramco Healthcare

Introduction

Laboratory testing is a fundamental diagnostic and prognostic tool to ensure the quality of healthcare, treatment, and responses. This study aimed to evaluate the cost of laboratory tests performed for patients undergoing chemotherapy treatment in the oncology treatment center at Johns Hopkins Aramco Healthcare in Saudi Arabia. Additionally, we aimed to reduce the cost of unnecessary laboratory tests in a 1-year period.

Methods

This was a quality improvement study with a quasi-experimental design using DMAIC methodology. The intervention strategy involved educating staff about adhering to the British Columbia Cancer Agency (BCCA) guidelines when ordering laboratory tests for chemotherapy patients, then integrating those guidelines into the electronic health record system. Data were collected for 200 randomly selected cases with 10 different chemotherapy protocols before and after the intervention. A paired t test was used to analyze differences in mean cost for all laboratory tests and unnecessary testing before and after the intervention.

Results

A significant cost reduction was achieved for unnecessary laboratory tests (77%, p < 0.01) when following the BCCA guidelines. In addition, the mean cost of all laboratory tests (including necessary and unnecessary) was significantly reduced by 45.5% (p = 0.023).

Conclusion

Lean thinking in clinical practice, realized by integrating a standardized laboratory test guided by BCCA guidelines into the electronic health record, significantly reduced financial costs within 1 year, thereby enhancing efficient resource utilization in the organization. This quality improvement project may serve to increase awareness of further efforts to improve resource utilization for other oncology treatment protocols.

Capturing Semantic Relationships in Electronic Health Records Using Knowledge Graphs: An Implementation Using MIMIC III Dataset and GraphDB

Electronic health records (EHRs) are an increasingly important source of information for healthcare professionals and researchers. However, EHRs are often fragmented, unstructured, and difficult to analyze due to the heterogeneity of the data sources and the sheer volume of information. Knowledge graphs have emerged as a powerful tool for capturing and representing complex relationships within large datasets. In this study, we explore the use of knowledge graphs to capture and represent complex relationships within EHRs. Specifically, we address the following research question: Can a knowledge graph created using the MIMIC III dataset and GraphDB effectively capture semantic relationships within EHRs and enable more efficient and accurate data analysis? We map the MIMIC III dataset to an ontology using text refinement and Protege; then, we create a knowledge graph using GraphDB and use SPARQL queries to retrieve and analyze information from the graph. Our results demonstrate that knowledge graphs can effectively capture semantic relationships within EHRs, enabling more efficient and accurate data analysis. We provide examples of how our implementation can be used to analyze patient outcomes and identify potential risk factors. Our results demonstrate that knowledge graphs are an effective tool for capturing semantic relationships within EHRs, enabling a more efficient and accurate data analysis. Our implementation provides valuable insights into patient outcomes and potential risk factors, contributing to the growing body of literature on the use of knowledge graphs in healthcare. In particular, our study highlights the potential of knowledge graphs to support decision-making and improve patient outcomes by enabling a more comprehensive and holistic analysis of EHR data. Overall, our research contributes to a better understanding of the value of knowledge graphs in healthcare and lays the foundation for further research in this area.

Electronic Health Record Optimization for Artificial Intelligence

Laboratory clinical decision support (CDS) typically relies on data from the electronic health record (EHR). The implementation of a sustainable, effective laboratory CDS program requires a commitment to standardization and harmonization of key EHR data elements that are the foundation of laboratory CDS. The direct use of artificial intelligence algorithms in CDS programs will be limited unless key elements of the EHR are structured. The identification, curation, maintenance, and preprocessing steps necessary to implement robust laboratory-based algorithms must account for the heterogeneity of data present in a typical EHR.

Use of Clinical Decision Support to Improve the Laboratory Evaluation of Monoclonal Gammopathies

OBJECTIVES

There is considerable variation in ordering practices for the initial laboratory evaluation of monoclonal gammopathies (MGs) despite clear society guidelines to include serum free light chain (sFLC) testing. We assessed the ability of a clinical decision support (CDS) alert to improve guideline compliance and analyzed its clinical impact.

METHODS

We designed and deployed a targeted CDS alert to educate and prompt providers to order an sFLC assay when ordering serum protein electrophoresis (SPEP) testing.

RESULTS

The alert was highly effective at increasing the co-ordering of SPEP and sFLC testing. Preimplementation, 62.8% of all SPEP evaluations included sFLC testing, while nearly 90% of evaluations included an sFLC assay postimplementation. In patients with no prior sFLC testing, analysis of sFLC orders prompted by the alert led to the determination that 28.9% (800/2,769) of these patients had an abnormal κ/λ ratio. In 452 of these patients, the sFLC assay provided the only laboratory evidence of a monoclonal protein. Moreover, within this population, there were numerous instances of new diagnoses of multiple myeloma and other MGs.

CONCLUSIONS

The CDS alert increased compliance with society guidelines and improved the diagnostic evaluation of patients with suspected MGs.

Automated prediction of low ferritin concentrations using a machine learning algorithm

OBJECTIVES

Computational algorithms for the interpretation of laboratory test results can support physicians and specialists in laboratory medicine. The aim of this study was to develop, implement and evaluate a machine learning algorithm that automatically assesses the risk of low body iron storage, reflected by low ferritin plasma levels, in anemic primary care patients using a minimal set of basic laboratory tests, namely complete blood count and C-reactive protein (CRP).

METHODS

Laboratory measurements of anemic primary care patients were used to develop and validate a machine learning algorithm. The performance of the algorithm was compared to twelve specialists in laboratory medicine from three large teaching hospitals, who predicted if patients with anemia have low ferritin levels based on laboratory test reports (complete blood count and CRP). In a second round of assessments the algorithm outcome was provided to the specialists in laboratory medicine as a decision support tool.

RESULTS

Two separate algorithms to predict low ferritin concentrations were developed based on two different chemistry analyzers, with an area under the curve of the ROC of 0.92 (Siemens) and 0.90 (Roche). The specialists in laboratory medicine were less accurate in predicting low ferritin concentrations compared to the algorithms, even when knowing the output of the algorithms as support tool. Implementation of the algorithm in the laboratory system resulted in one new iron deficiency diagnosis on average per day.

CONCLUSIONS

Low ferritin levels in anemic patients can be accurately predicted using a machine learning algorithm based on routine laboratory test results. Moreover, implementation of the algorithm in the laboratory system reduces the number of otherwise unrecognized iron deficiencies.

Clinical Decision Support for Laboratory Testing

BACKGROUND

As technology enables new and increasingly complex laboratory tests, test utilization presents a growing challenge for healthcare systems. Clinical decision support (CDS) refers to digital tools that present providers with clinically relevant information and recommendations, which have been shown to improve test utilization. Nevertheless, individual CDS applications often fail, and implementation remains challenging.

CONTENT

We review common classes of CDS tools grounded in examples from the literature as well as our own institutional experience. In addition, we present a practical framework and specific recommendations for effective CDS implementation.

SUMMARY

CDS encompasses a rich set of tools that have the potential to drive significant improvements in laboratory testing, especially with respect to test utilization. Deploying CDS effectively requires thoughtful design and careful maintenance, and structured processes focused on quality improvement and change management play an important role in achieving these goals.

Real-time monitoring of drug laboratory test interactions: a proof of concept

OBJECTIVES

For the correct interpretation of test results, it is important to be aware of drug-laboratory test interactions (DLTIs). If DLTIs are not taken into account by clinicians, erroneous interpretation of test results may lead to a delayed or incorrect diagnosis, unnecessary diagnostic testing or therapy with possible harm for patients. A DLTI alert accompanying a laboratory test result could be a solution. The aim of this study was to test a multicentre proof of concept of an electronic clinical decision support system (CDSS) for real-time monitoring of DLTIs.

METHODS

CDSS was implemented in three Dutch hospitals. So-called 'clinical rules' were programmed to alert medical specialists for possible DLTIs based on laboratory test results outside the reference range in combination with prescribed drugs. A selection of interactions from the DLTI database of the Dutch society of clinical chemistry and laboratory medicine were integrated in 43 clinical rules, including 24 tests and 25 drugs. During the period of one month all generated DTLI alerts were registered in the laboratory information system.

RESULTS

Approximately 65 DLTI alerts per day were detected in each hospital. Most DLTI alerts were generated in patients from the internal medicine and intensive care departments. The most frequently reported DLTI alerts were potassium-proton pump inhibitors (16%), potassium-beta blockers (11%) and creatine kinase-statins (11%).

CONCLUSIONS

This study shows that it is possible to alert for potential DLTIs in real-time with a CDSS. The CDSS was successfully implemented in three hospitals. Further research must reveal its usefulness in clinical practice.

Development and implementation of a clinical decision support-based initiative to drive intravenous fluid prescribing

OBJECTIVE

Studies suggest superior outcomes with use of intravenous (IV) balanced fluids compared to normal saline (NS). However, significant fluid prescribing variability persists, highlighting the knowledge-to-practice gap. We sought to identify contributors to prescribing variation and utilize a clinical decision support system (CDSS) to increase institutional balanced fluid prescribing.

MATERIALS AND METHODS

This single-center informatics-enabled quality improvement initiative for patients hospitalized or treated in the emergency department included stepwise interventions of 1) identification of design factors within the computerized provider order entry (CPOE) of our electronic health record (EHR) that contribute to preferential NS ordering, 2) clinician education, 3) fluid stocking modifications, 4) re-design and implementation of a CDSS-integrated IV fluid ordering panel, and 5) comparison of fluid prescribing before and after the intervention. EHR-derived prescribing data was analyzed via single interrupted time series.

RESULTS

Pre-intervention (3/2019-9/2019), balanced fluids comprised 33% of isotonic fluid orders, with gradual uptake (1.4%/month) of balanced fluid prescribing. Clinician education (10/2019-2/2020) yielded a modest (4.4%/month, 95% CI 1.6-7.2, p = 0.01) proportional increase in balanced fluid prescribing, while CPOE redesign (3/2020) yielded an immediate (20.7%, 95% CI 17.7-23.6, p < 0.0001) and sustained increase (72% of fluid orders in 12/2020). The intervention proved most effective among those with lower baseline balanced fluids utilization, including emergency medicine (57% increase, 95% CI 0.7-1.8, p < 0.0001) and internal medicine/subspecialties (18% increase, 95% CI 14.4-21.3, p < 0.0001) clinicians and substantially reduced institutional prescribing variation.

CONCLUSION

Integration of CDSS into an EHR yielded a robust and sustained increase in balanced fluid prescribing. This impact far exceeded that of clinician education highlighting the importance of CDSS.

Use of machine learning to predict clinical decision support compliance, reduce alert burden, and evaluate duplicate laboratory test ordering alerts

Objectives

While well-designed clinical decision support (CDS) alerts can improve patient care, utilization management, and population health, excessive alerting may be counterproductive, leading to clinician burden and alert fatigue. We sought to develop machine learning models to predict whether a clinician will accept the advice provided by a CDS alert. Such models could reduce alert burden by targeting CDS alerts to specific cases where they are most likely to be effective.

Materials and Methods

We focused on a set of laboratory test ordering alerts, deployed at 8 hospitals within the Partners Healthcare System. The alerts notified clinicians of duplicate laboratory test orders and advised discontinuation. We captured key attributes surrounding 60 399 alert firings, including clinician and patient variables, and whether the clinician complied with the alert. Using these data, we developed logistic regression models to predict alert compliance.

Results

We identified key factors that predicted alert compliance; for example, clinicians were less likely to comply with duplicate test alerts triggered in patients with a prior abnormal result for the test or in the context of a nonvisit-based encounter (eg, phone call). Likewise, differences in practice patterns between clinicians appeared to impact alert compliance. Our best-performing predictive model achieved an area under the receiver operating characteristic curve (AUC) of 0.82. Incorporating this model into the alerting logic could have averted more than 1900 alerts at a cost of fewer than 200 additional duplicate tests.

Conclusions

Deploying predictive models to target CDS alerts may substantially reduce clinician alert burden while maintaining most or all the CDS benefit.

Clinical usefulness of drug-laboratory test interaction alerts: a multicentre survey

OBJECTIVES

Knowledge of possible drug-laboratory test interactions (DLTIs) is important for the interpretation of laboratory test results. Failure to recognize these interactions may lead to misinterpretation, a delayed or erroneous diagnosis, or unnecessary extra diagnostic tests or therapy, which may harm patients. The aim of this multicentre survey was to evaluate the clinical value of DLTI alerts.

METHODS

A survey was designed with six predefined clinical cases selected from the clinical laboratory practice with a potential DLTI. Physicians from several departments, including internal medicine, cardiology, intensive care, surgery and geriatrics in six participating hospitals were recruited to fill in the survey. The survey addressed their knowledge of DLTIs, motivation to receive an alert and opinion on the potential influence on medical decision making.

RESULTS

A total of 210 physicians completed the survey. Of these respondents 93% had a positive attitude towards receiving DLTI alerts; however, the reported value differed per case and per respondent's background. In each clinical case, medical decision making was influenced as a consequence of the reported DLTI message (ranging from 3 to 45% of respondents per case).

CONCLUSIONS

In this multicentre survey, most physicians stated DLTI messages to be useful in laboratory test interpretation. Medical decision making was influenced by reporting DLTI alerts in each case. Alerts should be adjusted according to the needs and preferences of the receiving physicians.

Iron studies and transferrin, a source of test ordering confusion highly amenable to clinical decision support

INTRODUCTION

An important cause of laboratory test misordering and overutilization is clinician confusion between tests with similar sounding names or similar indications. We identified an area of test ordering confusion with iron studies that involves total iron binding capacity (TIBC), transferrin, and transferrin saturation. We observed concurrent ordering of direct transferrin along with TIBC at many hospitals within our health system and suspected this was unnecessary.

METHODS

We extracted patient test results for transferrin, TIBC and other biomarkers. Using these data, we evaluated both patterns of test utilization and test result concordance. We implemented a clinical decision support (CDS) alert to discourage unnecessary orders for direct transferrin.

RESULTS

Using linear regression, we were able to predict transferrin from either TIBC alone or TIBC with other analytes with a high degree of accuracy, demonstrating that in most cases, direct transferrin in combination with TIBC provides little if any additional diagnostic information beyond TIBC alone. The CDS alert proved highly effective in reducing transferrin test utilization at four different hospitals.

CONCLUSIONS

Concurrent ordering of direct transferrin and TIBC should usually be avoided. Removal of transferrin or TIBC from the test menu or implementation of CDS may improve utilization of these tests.

Essential role of laboratory physicians in transformation of laboratory practice and management to a value-based patient-centric model

The laboratory is a vital part of the continuum of patient care. In fact, there are few programs in the healthcare system that do not rely on ready access and availability of complex diagnostic laboratory services. The existing transactional model of laboratory "medical practice" will not be able to meet the needs of the healthcare system as it rapidly shifts toward value-based care and precision medicine, which demands that practice be based on total system indicators, clinical effectiveness, and patient outcomes. Laboratory "value" will no longer be focused primarily on internal testing quality and efficiencies but rather on the relative cost of diagnostic testing compared to direct improvement in clinical and system outcomes. The medical laboratory as a "business" focused on operational efficiency and cost-controls must transform to become an essential clinical service that is a tightly integrated equal partner in direct patient care. We would argue that this paradigm shift would not be necessary if laboratory services had remained a "patient-centric" medical practice throughout the last few decades. This review is focused on the essential role of laboratory physicians in transforming laboratory practice and management to a value-based patient-centric model. Value-based practice is necessary not only to meet the challenges of the new precision medicine world order but also to bring about sustainable healthcare service delivery.

Validation and Implementation of an Ordering Alert to Improve the Efficiency of Monoclonal Gammopathy Evaluation

OBJECTIVES

To evaluate the use of a provider ordering alert to improve laboratory efficiency and reduce costs.

METHODS

We conducted a retrospective study to assess the use of an institutional reflex panel for monoclonal gammopathy evaluation. We then created a clinical decision support (CDS) alert to educate and encourage providers to change their less-efficient orders to the reflex panel.

RESULTS

Our retrospective analysis demonstrated that an institutional reflex panel could be safely substituted for a less-efficient and higher-cost panel. The implemented CDS alert resulted in 79% of providers changing their high-cost order panel to an order panel based on the reflex algorithm.

CONCLUSIONS

The validated decision support alert demonstrated high levels of provider acceptance and directly led to operational and cost savings within the laboratory. Furthermore, these studies highlight the value of laboratory involvement with CDS efforts to provide agile and targeted provider ordering assistance.