A generalized model for predicting postcompletion errors

Assessment of Unintentional Duplicate Orders by Emergency Department Clinicians Before and After Implementation of a Visual Aid in the Electronic Health Record Ordering System

IMPORTANCE

Electronic health records allow teams of clinicians to simultaneously care for patients, but an unintended consequence is the potential for duplicate orders of tests and medications.

OBJECTIVE

To determine whether a simple visual aid is associated with a reduction in duplicate ordering of tests and medications.

DESIGN, SETTING, AND PARTICIPANTS

This cohort study used an interrupted time series model to analyze 184 694 consecutive patients who visited the emergency department (ED) of an academic hospital with 55 000 ED visits annually. Patient visits occurred 1 year before and after each intervention, as follows: for laboratory orders, from August 13, 2012, to August 13, 2014; for medication orders, from February 3, 2013, to February 3, 2015; and for radiology orders, from December 12, 2013, to December 12, 2015. Data were analyzed from April to September 2019.

EXPOSURE

If an order had previously been placed during the ED visit, a red highlight appeared around the checkbox of that order in the computerized provider order entry system.

MAIN OUTCOMES AND MEASURES

Number of unintentional duplicate laboratory, medication, and radiology orders.

RESULTS

A total of 184 694 patients (mean [SD] age, 51.6 [20.8] years; age range, 0-113.0 years; 99 735 [54.0%] women) who visited the ED were analyzed over the 3 overlapping study periods. After deployment of a noninterruptive nudge in electronic health records, there was an associated 49% decrease in the rate of unintentional duplicate orders for laboratory tests (incidence rate ratio, 0.51; 95% CI, 0.45-0.59), from 4485 to 2731 orders, and an associated 40% decrease in unintentional duplicate orders of radiology tests (incidence rate ratio, 0.60; 95% CI, 0.44-0.82), from 956 to 782 orders. There was not a statistically significant change in unintentional duplicate orders of medications (incidence rate ratio, 1.17; 95% CI, 0.52-2.61), which increased from 225 to 287 orders. The nudge eliminated an estimated 17 936 clicks in our electronic health record.

CONCLUSIONS AND RELEVANCE

In this interrupted time series cohort study, passive visual cues that provided just-in-time decision support were associated with reductions in unintentional duplicate orders for laboratory and radiology tests but not in unintentional duplicate medication orders.

Eye-Tracking Study to Enhance Usability of Molecular Diagnostics Reports in Cancer Precision Medicine

PURPOSE

We conducted usability studies on commercially available molecular diagnostic (MDX) test reports to identify strengths and weaknesses in content and form that drive clinical decision making. Given routine genomic testing in cancer medicine, oncologists must interpret MDX reports as well as evidence concerning clinical utility of biomarkers accurately for treatment or trial selection. This work aims to evaluate effectiveness of MDX reports in facilitating cancer treatment planning.

METHODS

Fourteen clinicians at an academic tertiary care medical facility, with a wide range of experience in oncology and in the use of molecular testing, participated in this study. Three commercially available, widely used, Clinical Laboratory Improvement Amendments (CLIA)-certified, College of American Pathologists (CAP)-accredited test reports (labeled Laboratories A, B, and C) were used. Eye tracking, surveys, and think-aloud protocols were used to collect usability data for these MDX reports focusing on ease of comprehension and actionability.

RESULTS

Clinicians found two primary areas in molecular diagnostic reports most useful for patient care: therapy options with benefit or lack of benefit to patients, including enrolling clinical trials; and pathogenic tumor molecular anomalies detected. Therapeutic implications and therapy classes such as US Food and Drug Administration-approved off-label, on-label, clinical trials were critical for decision making. However, all reports had usability and comprehension issues in these areas and could be improved.

CONCLUSION

Focused usability studies can help drive our understanding of the clinical workflow for use of molecular diagnostic tests in cancer care. This in turn can have major effects on quality of care, outcomes, costs, and patient satisfaction. This study demonstrates the use of specific usability techniques (eye tracking and think-aloud protocols) to help clinical laboratories improve MDX report design in a precision oncology treatment setting.

Human Error as an Emergent Property of Action Selection and Task Place-Holding

OBJECTIVE

A computational process model could explain how the dynamic interaction of human cognitive mechanisms produces each of multiple error types.

BACKGROUND

With increasing capability and complexity of technological systems, the potential severity of consequences of human error is magnified. Interruption greatly increases people's error rates, as does the presence of other information to maintain in an active state.

METHOD

The model executed as a software-instantiated Monte Carlo simulation. It drew on theoretical constructs such as associative spreading activation for prospective memory, explicit rehearsal strategies as a deliberate cognitive operation to aid retrospective memory, and decay.

RESULTS

The model replicated the 30% effect of interruptions on postcompletion error in Ratwani and Trafton's Stock Trader task, the 45% interaction effect on postcompletion error of working memory capacity and working memory load from Byrne and Bovair's Phaser Task, as well as the 5% perseveration and 3% omission effects of interruption from the UNRAVEL Task.

CONCLUSION

Error classes including perseveration, omission, and postcompletion error fall naturally out of the theory.

APPLICATION

The model explains post-interruption error in terms of task state representation and priming for recall of subsequent steps. Its performance suggests that task environments providing more cues to current task state will mitigate error caused by interruption. For example, interfaces could provide labeled progress indicators or facilities for operators to quickly write notes about their task states when interrupted.

Alert dwell time: introduction of a measure to evaluate interruptive clinical decision support alerts

Metrics for evaluating interruptive prescribing alerts have many limitations. Additional methods are needed to identify opportunities to improve alerting systems and prevent alert fatigue. In this study, the authors determined whether alert dwell time-the time elapsed from when an interruptive alert is generated to when it is dismissed-could be calculated by using historical alert data from log files. Drug-drug interaction (DDI) alerts from 3 years of electronic health record data were queried. Alert dwell time was calculated for 25,965 alerts, including 777 unique DDIs. The median alert dwell time was 8 s (range, 1-4913 s). Resident physicians had longer median alert dwell times than other prescribers (P < 001). The 10 most frequent DDI alerts (n = 8759 alerts) had shorter median dwell times than alerts that only occurred once (P < 001). This metric can be used in future research to evaluate the effectiveness and efficiency of interruptive prescribing alerts.