Lessons from another industry: aviation, usability, and medical device design

Reducing Prescribing Errors in Hospitalized Children on the Ketogenic Diet

BACKGROUND

Children on the ketogenic diet must limit carbohydrate intake to maintain ketosis and reduce seizure burden. Patients on ketogenic diet are vulnerable to harm in the hospital setting where carbohydrate-containing medications are commonly prescribed. We developed clinical decision support to reduce inappropriate prescription of carbohydrate-containing medications in hospitalized children on ketogenic diet.

METHODS

A clinical decision support alert was developed through formative and summative usability testing. The alert warned prescribers when they entered an order for a carbohydrate-containing medication in patients on ketogenic diet. The alert was implemented using a quasi-experimental design with sequential crossover from control to intervention at two tertiary care pediatric hospitals within a single health system. The primary outcome was carbohydrate-containing medication orders per patient-day.

RESULTS

During the study period, there were 280 ketogenic diet patient admissions totaling 1219 patient-days. The carbohydrate-containing medication order rate declined from 0.69 to 0.35 orders per patient-day (absolute rate reduction 0.34, 95% confidence interval 0.25-0.43), corresponding to 256 inappropriate orders prevented. The alert fired 398 times and was accepted (i.e., the order was removed) 227 times for an overall acceptance rate of 57%.

CONCLUSIONS

Implementation of a clinical decision support alert at order-entry resulted in a sustained reduction in carbohydrate-containing medication orders for hospitalized patients on ketogenic diet without an increase in alert burden. Clinical decision support developed with user-centered design principles can improve patient safety for children on ketogenic diet by influencing prescriber behavior.

Impact of Design on Medical Device Safety

The growing number of emerging medical technologies and sophistication of modern medical devices (MDs) that improve both survival and quality of life indexes are often challenged by alarming cases of vigilance data cover-up and lack of sufficient pre- and post-authorization controls. Combining Quality with Risk Management processes and implementing them as early as possible in the design of MDs has proven to be an effective strategy to minimize residual risk. This article aims to discuss how the design of MDs interacts with their safety profile and how this dipole of intended performance and safety may be supported by Human Factors Engineering (HFE) throughout the Total Product Life-Cycle (TPLC) of an MD in order to capitalize on medical technologies without exposing users and patients to unnecessary risks.

Formative Usability Testing Reduces Severe Blood Product Ordering Errors

BACKGROUND

Medical errors in blood product orders and administration are common, especially for pediatric patients. A failure modes and effects analysis in our health care system indicated high risk from the electronic blood ordering process.

OBJECTIVES

There are two objectives of this study as follows:(1) To describe differences in the design of the original blood product orders and order sets in the system (original design), new orders and order sets designed by expert committee (DEC), and a third-version developed through user-centered design (UCD).(2) To compare the number and type of ordering errors, task completion rates, time on task, and user preferences between the original design and that developed via UCD.

METHODS

A multidisciplinary expert committee proposed adjustments to existing blood product order sets resulting in the DEC order set. When that order set was tested with front-line users, persistent failure modes were detected, so orders and order sets were redesigned again via formative usability testing. Front-line users in their native clinical workspaces were observed ordering blood in realistic simulated scenarios using a think-aloud protocol. Iterative adjustments were made between participants. In summative testing, participants were randomized to use the original design or UCD for five simulated scenarios. We evaluated differences in ordering errors, time on task, and users' design preference with two-sample t-tests.

RESULTS

Formative usability testing with 27 providers from seven specialties led to 18 changes made to the DEC to produce the UCD. In summative testing, error-free task completion for the original design was 36%, which increased to 66% in UCD (30%, 95% confidence interval [CI]: 3.9-57%; p = 0.03). Time on task did not vary significantly.

CONCLUSION

UCD led to substantially different blood product orders and order sets than DEC. Users made fewer errors when ordering blood products for pediatric patients in simulated scenarios when using the UCD orders and order sets compared with the original design.