Assessing the accuracy of a computerized decision support system for digoxin dosing in primary care: an observational study

Improvement of Adequate Digoxin Dosage: An Application of Machine Learning Approach

Digoxin is a high-alert medication because of its narrow therapeutic range and high drug-to-drug interactions (DDIs). Approximately 50% of digoxin toxicity cases are preventable, which motivated us to improve the treatment outcomes of digoxin. The objective of this study is to apply machine learning techniques to predict the appropriateness of initial digoxin dosage. A total of 307 inpatients who had their conditions treated with digoxin between 2004 and 2013 at a medical center in Taiwan were collected in the study. Ten independent variables, including demographic information, laboratory data, and whether the patients had CHF were also noted. A patient with serum digoxin concentration being controlled at 0.5-0.9 ng/mL after his/her initial digoxin dosage was defined as having an appropriate use of digoxin; otherwise, a patient was defined as having an inappropriate use of digoxin. Weka 3.7.3, an open source machine learning software, was adopted to develop prediction models. Six machine learning techniques were considered, including decision tree (C4.5), k-nearest neighbors (kNN), classification and regression tree (CART), randomForest (RF), multilayer perceptron (MLP), and logistic regression (LGR). In the non-DDI group, the area under ROC curve (AUC) of RF (0.912) was excellent, followed by that of MLP (0.813), CART (0.791), and C4.5 (0.784); the remaining classifiers performed poorly. For the DDI group, the AUC of RF (0.892) was the best, followed by CART (0.795), MLP (0.777), and C4.5 (0.774); the other classifiers' performances were less than ideal. The decision tree-based approaches and MLP exhibited markedly superior accuracy performance, regardless of DDI status. Although digoxin is a high-alert medication, its initial dose can be accurately determined by using data mining techniques such as decision tree-based and MLP approaches. Developing a dosage decision support system may serve as a supplementary tool for clinicians and also increase drug safety in clinical practice.

Computerized advice on drug dosage to improve prescribing practice

BACKGROUND

Maintaining therapeutic concentrations of drugs with a narrow therapeutic window is a complex task. Several computer systems have been designed to help doctors determine optimum drug dosage. Significant improvements in health care could be achieved if computer advice improved health outcomes and could be implemented in routine practice in a cost-effective fashion. This is an updated version of an earlier Cochrane systematic review, first published in 2001 and updated in 2008.

OBJECTIVES

To assess whether computerized advice on drug dosage has beneficial effects on patient outcomes compared with routine care (empiric dosing without computer assistance).

SEARCH METHODS

The following databases were searched from 1996 to January 2012: EPOC Group Specialized Register, Reference Manager; Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE, Ovid; EMBASE, Ovid; and CINAHL, EbscoHost. A "top up" search was conducted for the period January 2012 to January 2013; these results were screened by the authors and potentially relevant studies are listed in Studies Awaiting Classification. The review authors also searched reference lists of relevant studies and related reviews.

SELECTION CRITERIA

We included randomized controlled trials, non-randomized controlled trials, controlled before-and-after studies and interrupted time series analyses of computerized advice on drug dosage. The participants were healthcare professionals responsible for patient care. The outcomes were any objectively measured change in the health of patients resulting from computerized advice (such as therapeutic drug control, clinical improvement, adverse reactions).

DATA COLLECTION AND ANALYSIS

Two review authors independently extracted data and assessed study quality. We grouped the results from the included studies by drug used and the effect aimed at for aminoglycoside antibiotics, amitriptyline, anaesthetics, insulin, anticoagulants, ovarian stimulation, anti-rejection drugs and theophylline. We combined the effect sizes to give an overall effect for each subgroup of studies, using a random-effects model. We further grouped studies by type of outcome when appropriate (i.e. no evidence of heterogeneity).

MAIN RESULTS

Forty-six comparisons (from 42 trials) were included (as compared with 26 comparisons in the last update) including a wide range of drugs in inpatient and outpatient settings. All were randomized controlled trials except two studies. Interventions usually targeted doctors, although some studies attempted to influence prescriptions by pharmacists and nurses. Drugs evaluated were anticoagulants, insulin, aminoglycoside antibiotics, theophylline, anti-rejection drugs, anaesthetic agents, antidepressants and gonadotropins. Although all studies used reliable outcome measures, their quality was generally low.This update found similar results to the previous update and managed to identify specific therapeutic areas where the computerized advice on drug dosage was beneficial compared with routine care:1. it increased target peak serum concentrations (standardized mean difference (SMD) 0.79, 95% CI 0.46 to 1.13) and the proportion of people with plasma drug concentrations within the therapeutic range after two days (pooled risk ratio (RR) 4.44, 95% CI 1.94 to 10.13) for aminoglycoside antibiotics;2. it led to a physiological parameter more often within the desired range for oral anticoagulants (SMD for percentage of time spent in target international normalized ratio +0.19, 95% CI 0.06 to 0.33) and insulin (SMD for percentage of time in target glucose range: +1.27, 95% CI 0.56 to 1.98);3. it decreased the time to achieve stabilization for oral anticoagulants (SMD -0.56, 95% CI -1.07 to -0.04);4. it decreased the thromboembolism events (rate ratio 0.68, 95% CI 0.49 to 0.94) and tended to decrease bleeding events for anticoagulants although the difference was not significant (rate ratio 0.81, 95% CI 0.60 to 1.08). It tended to decrease unwanted effects for aminoglycoside antibiotics (nephrotoxicity: RR 0.67, 95% CI 0.42 to 1.06) and anti-rejection drugs (cytomegalovirus infections: RR 0.90, 95% CI 0.58 to 1.40);5. it tended to reduce the length of time spent in the hospital although the difference was not significant (SMD -0.15, 95% CI -0.33 to 0.02) and to achieve comparable or better cost-effectiveness ratios than usual care;6. there was no evidence of differences in mortality or other clinical adverse events for insulin (hypoglycaemia), anaesthetic agents, anti-rejection drugs and antidepressants.For all outcomes, statistical heterogeneity quantified by I(2) statistics was moderate to high.

AUTHORS' CONCLUSIONS

This review update suggests that computerized advice for drug dosage has some benefits: it increases the serum concentrations for aminoglycoside antibiotics and improves the proportion of people for which the plasma drug is within the therapeutic range for aminoglycoside antibiotics.It leads to a physiological parameter more often within the desired range for oral anticoagulants and insulin. It decreases the time to achieve stabilization for oral anticoagulants. It tends to decrease unwanted effects for aminoglycoside antibiotics and anti-rejection drugs, and it significantly decreases thromboembolism events for anticoagulants. It tends to reduce the length of hospital stay compared with routine care while comparable or better cost-effectiveness ratios were achieved.However, there was no evidence that decision support had an effect on mortality or other clinical adverse events for insulin (hypoglycaemia), anaesthetic agents, anti-rejection drugs and antidepressants. In addition, there was no evidence to suggest that some decision support technical features (such as its integration into a computer physician order entry system) or aspects of organization of care (such as the setting) could optimize the effect of computerized advice.Taking into account the high risk of bias of, and high heterogeneity between, studies, these results must be interpreted with caution.

Computerized advice on drug dosage to improve prescribing practice

BACKGROUND

Maintaining therapeutic concentrations of drugs with a narrow therapeutic window is a complex task. Several computer systems have been designed to help doctors determine optimum drug dosage. Significant improvements in health care could be achieved if computer advice improved health outcomes and could be implemented in routine practice in a cost effective fashion. This is an updated version of an earlier Cochrane systematic review, by Walton et al, published in 2001.

OBJECTIVES

To assess whether computerised advice on drug dosage has beneficial effects on the process or outcome of health care.

SEARCH STRATEGY

We searched the Cochrane Effective Practice and Organisation of Care Group specialized register (June 1996 to December 2006), MEDLINE (1966 to December 2006), EMBASE (1980 to December 2006), hand searched the journal Therapeutic Drug Monitoring (1979 to March 2007) and the Journal of the American Medical Informatics Association (1996 to March 2007) as well as reference lists from primary articles.

SELECTION CRITERIA

Randomized controlled trials, controlled trials, controlled before and after studies and interrupted time series analyses of computerized advice on drug dosage were included. The participants were health professionals responsible for patient care. The outcomes were: any objectively measured change in the behaviour of the health care provider (such as changes in the dose of drug used); any change in the health of patients resulting from computerized advice (such as adverse reactions to drugs).

DATA COLLECTION AND ANALYSIS

Two reviewers independently extracted data and assessed study quality.

MAIN RESULTS

Twenty-six comparisons (23 articles) were included (as compared to fifteen comparisons in the original review) including a wide range of drugs in inpatient and outpatient settings. Interventions usually targeted doctors although some studies attempted to influence prescriptions by pharmacists and nurses. Although all studies used reliable outcome measures, their quality was generally low. Computerized advice for drug dosage gave significant benefits by:1.increasing the initial dose (standardised mean difference 1.12, 95% CI 0.33 to 1.92)2.increasing serum concentrations (standradised mean difference 1.12, 95% CI 0.43 to 1.82)3.reducing the time to therapeutic stabilisation (standardised mean difference -0.55, 95%CI -1.03 to -0.08)4.reducing the risk of toxic drug level (rate ratio 0.45, 95% CI 0.30 to 0.70)5.reducing the length of hospital stay (standardised mean difference -0.35, 95% CI -0.52 to -0.17).

AUTHORS' CONCLUSIONS

This review suggests that computerized advice for drug dosage has some benefits: it increased the initial dose of drug, increased serum drug concentrations and led to a more rapid therapeutic control. It also reduced the risk of toxic drug levels and the length of time spent in the hospital. However, it had no effect on adverse reactions. In addition, there was no evidence to suggest that some decision support technical features (such as its integration into a computer physician order entry system) or aspects of organization of care (such as the setting) could optimise the effect of computerised advice.