Modeling the mind: How do we design effective decision-support?

"Be Really Careful about That": Clinicians' Perceptions of an Intelligence Augmentation Tool for In-Hospital Deterioration Detection

OBJECTIVE

 This study aimed to explore clinicians' perceptions and preferences of prototype intelligence augmentation (IA)-based visualization displays of in-hospital deterioration risk scores to inform future user interface design and implementation in clinical care.

METHODS

 Prototype visualization displays incorporating an IA-based early warning score (EWS) for in-hospital deterioration were developed using cognitive theory and user-centered design principles. The displays featured variations of EWS and clinical data arranged in multipatient and single-patient views. Physician and nurse participants with at least 5 years of clinical experience were recruited to participate in semistructured qualitative interviews focused on understanding their experiences with IA and thoughts and preferences about the prototype displays. A thematic analysis was performed on these data.

RESULTS

 Six themes were identified: (1) clinicians perceive IA as valuable with some caveats related to function and context; (2) individual differences among users influence preferences for customizability; (3) EWS are particularly useful for patient triage; (4) need for patient-centered contextual information to complement EWS; (5) perspectives related to understanding the EWS composition; and (6) design preferences that focus on clarity for interpretation of information.

CONCLUSION

 This study demonstrates clinicians' interest in and reservations about IA tools for clinical deterioration. The findings underscore the importance of understanding clinicians' cognitive needs and framing IA-generated tools as complementary to support them. A clinician focuses on high-level pattern matching information, and clinician's comments related to the power of consistency with typical views (e.g., this is "how I usually see things"), and questions regarding support of score interpretation (e.g., age of the data, questions about what the model "knows") suggest some of the challenges of IA implementation. The findings also identify design implications including the need for contextualizing the EWS for the patient's specific situation, incorporating trend information, and explaining the display purpose for clinical use.

Systematic User-centered Design of a Prototype Clinical Decision Support System for Glaucoma

Purpose

To rigorously develop a prototype clinical decision support (CDS) system to help clinicians determine the appropriate timing for follow-up visual field testing for patients with glaucoma and to identify themes regarding the context of use for glaucoma CDS systems, design requirements, and design solutions to meet these requirements.

Design

Semistructured qualitative interviews and iterative design cycles.

Participants

Clinicians who care for patients with glaucoma, purposefully sampled to ensure a representation of a range of clinical specialties (glaucoma specialist, general ophthalmologist, optometrist) and years in clinical practice.

Methods

Using the established User-Centered Design Process framework, we conducted semistructured interviews with 5 clinicians that addressed the context of use and design requirements for a glaucoma CDS system. We analyzed the interviews using inductive thematic analysis and grounded theory to generate themes regarding the context of use and design requirements. We created design solutions to address these requirements and used iterative design cycles with the clinicians to refine the CDS prototype.

Main Outcome Measures

Themes regarding decision support for determining the timing of visual field testing for patients with glaucoma, CDS design requirements, and CDS design features.

Results

We identified 9 themes that addressed the context of use for the CDS system, 9 design requirements for the prototype CDS system, and 9 design features intended to address these design requirements. Key design requirements included the preservation of clinician autonomy, incorporation of currently used heuristics, compilation of data, and increasing and communicating the level of certainty regarding the decision. After completing 3 iterative design cycles using this preliminary CDS system design solution, the design was satisfactory to the clinicians and was accepted as our prototype glaucoma CDS system.

Conclusions

We used a systematic design process based on the established User-Centered Design Process to rigorously develop a prototype glaucoma CDS system, which will be used as a starting point for a future, large-scale iterative refinement and implementation process. Clinicians who care for patients with glaucoma need CDS systems that preserve clinician autonomy, compile and present data, incorporate currently used heuristics, and increase and communicate the level of certainty regarding the decision.

Financial Disclosures

Proprietary or commercial disclosure may be found after the references.

"Are we there yet?" Ten persistent hazards and inefficiencies with the use of medication administration technology from the perspective of practicing nurses

OBJECTIVES

(1) Characterize persistent hazards and inefficiencies in inpatient medication administration; (2) Explore cognitive attributes of medication administration tasks; and (3) Discuss strategies to reduce medication administration technology-related hazards.

MATERIALS AND METHODS

Interviews were conducted with 32 nurses practicing at 2 urban, eastern and western US health systems. Qualitative analysis using inductive and deductive coding included consensus discussion, iterative review, and coding structure revision. We abstracted hazards and inefficiencies through the lens of risks to patient safety and the cognitive perception-action cycle (PAC).

RESULTS

Persistent safety hazards and inefficiencies related to MAT organized around the PAC cycle included: (1) Compatibility constraints create information silos; (2) Missing action cues; (3) Intermittent communication flow between safety monitoring systems and nurses; (4) Occlusion of important alerts by other, less helpful alerts; (5) Dispersed information: Information required for tasks is not collocated; (6) Inconsistent data organization: Mismatch of the display and the user's mental model; (7) Hidden medication administration technologies (MAT) limitations: Inaccurate beliefs about MAT functionality contribute to overreliance on the technology; (8) Software rigidity causes workarounds; (9) Cumbersome dependencies between technology and the physical environment; and (10) Technology breakdowns require adaptive actions.

DISCUSSION

Errors might persist in medication administration despite successful Bar Code Medication Administration and Electronic Medication Administration Record deployment for reducing errors. Opportunities to improve MAT require a deeper understanding of high-level reasoning in medication administration, including control over the information space, collaboration tools, and decision support.

CONCLUSION

Future medication administration technology should consider a deeper understanding of nursing knowledge work for medication administration.

Call for Decision Support for High-Alert Medication Administration Among Pediatric Nurses: Findings From a Large, Multicenter, Cross-Sectional Survey in China

Background: Children have a higher risk of medication errors (MEs) than adults. The Institute for Safe Medication Practice (ISMP) defined high-alert medications (HAMs) as a group of medications that could cause significant patient harm or even death when they are used in error. Nurses are actively involved in and responsible for patient care, especially in medication administration. This study aimed to estimate the knowledge, decision-making basis and confidence and decision support needs related to HAMs among pediatric nurses in China.

Methods: A web-based, cross-sectional survey was conducted among pediatric nurses who were recruited from 14 member hospitals of the Pediatric Nursing Alliance of National Children’s Medical Center in China using a convenient sampling technique. Data were collected using a self-administered instrument composed of four parts: the demographic characteristics of participants, participants’ knowledge about HAMs, participants’ self-evaluation of the basis of and confidence in decision-making, and decision support needs regarding HAMs. Among the participants, the maximum score for HAM knowledge was 100. All data were entered and analyzed using SPSS 20.0.

Results: A total of 966 nurses participated in this study. Nurses were found to have insufficient knowledge about HAMs, with a median (IQR) of 75.0 (70.0, 80.0), out of a maximum score of 100. Knowledge about HAM administration was significantly higher than that about HAM regulation, with a p value < 0.001. The three lowest-scoring items concerned HAM regulation, and the “Treat fentanyl skin patches as a regulated narcotic” item obtained the lowest score, with only 1/5 of respondents answering it correctly. Most participants reported that their basis for decision-making about HAMs was drug instructions (90.0%) or drug handbooks (81.9%) and evaluated their confidence in decision-making about HAMs as high or relatively high (84.6%). The decision-making difficulties when encountering HAMs focused on most stages of HAM administration, especially the appropriateness of prescriptions, checks, preparation and administration. The vast majority of participants assessed decision support as necessary or very necessary (92.0%), and the most popular options for decision support were computerized clinical decision support systems (46.4%) and real-time online communication with pharmacists (23.9%).

Conclusion: Our study demonstrated the inadequacies in HAM knowledge, the basis and difficulty of decision-making, and decision support needs regarding HAMs in Chinese pediatric nurses. Nurses need greater support in HAM administration, including not only training but also adequate technology, mutually beneficial interprofessional collaboration, and a positive institutional culture.

Feeling and thinking: can theories of human motivation explain how EHR design impacts clinician burnout?

The psychology of motivation can help us understand the impact of electronic health records (EHRs) on clinician burnout both directly and indirectly. Informatics approaches to EHR usability tend to focus on the extrinsic motivation associated with successful completion of clearly defined tasks in clinical workflows. Intrinsic motivation, which includes the need for autonomy, sense-making, creativity, connectedness, and mastery is not well supported by current designs and workflows. This piece examines existing research on the importance of 3 psychological drives in relation to healthcare technology: goal-based decision-making, sense-making, and agency/autonomy. Because these motives are ubiquitous, foundational to human functioning, automatic, and unconscious, they may be overlooked in technological interventions. The results are increased cognitive load, emotional distress, and unfulfilling workplace environments. Ultimately, we hope to stimulate new research on EHR design focused on expanding functionality to support intrinsic motivation, which, in turn, would decrease burnout and improve care.

Special Commentary: Using Clinical Decision Support Systems to Bring Predictive Models to the Glaucoma Clinic

Advances in the field of predictive modeling using artificial intelligence and machine learning have the potential to improve clinical care and outcomes, but only if the results of these models are presented appropriately to clinicians at the time they make decisions for individual patients. Clinical decision support (CDS) systems could be used to accomplish this. Modern CDS systems are computer-based tools designed to improve clinician decision making for individual patients. However, not all CDS systems are effective. Four principles that have been shown in other medical fields to be important for successful CDS system implementation are (1) integration into clinician workflow, (2) user-centered interface design, (3) evaluation of CDS systems and rules, and (4) standards-based development so the tools can be deployed across health systems.

Incorporating Guideline Adherence and Practice Implementation Issues into the Design of Decision Support for Beta-Blocker Titration for Heart Failure

BACKGROUND

The recognition of and response to undertreatment of heart failure (HF) patients can be complicated. A clinical reminder can facilitate use of guideline-concordant β-blocker titration for HF patients with depressed ejection fraction. However, the design must consider the cognitive demands on the providers and the context of the work.

OBJECTIVE

This study's purpose is to develop requirements for a clinical decision support tool (a clinical reminder) by analyzing the cognitive demands of the task along with the factors in the Cabana framework of physician adherence to guidelines, the health information technology (HIT) sociotechnical framework, and the Promoting Action on Research Implementation in Health Services (PARIHS) framework of health services implementation. It utilizes a tool that extracts information from medical records (including ejection fraction in free text reports) to identify qualifying patients at risk of undertreatment.

METHODS

We conducted interviews with 17 primary care providers, 5 PharmDs, and 5 Registered Nurses across three Veterans Health Administration outpatient clinics. The interviews were based on cognitive task analysis (CTA) methods and enhanced through the inclusion of the Cabana, HIT sociotechnical, and PARIHS frameworks. The analysis of the interview data led to the development of requirements and a prototype design for a clinical reminder. We conducted a small pilot usability assessment of the clinical reminder using realistic clinical scenarios.

RESULTS

We identified organizational challenges (such as time pressures and underuse of pharmacists), knowledge issues regarding the guideline, and information needs regarding patient history and treatment status. We based the design of the clinical reminder on how to best address these challenges. The usability assessment indicated the tool could help the decision and titration processes.

CONCLUSION

Through the use of CTA methods enhanced with adherence, sociotechnical, and implementation frameworks, we designed a decision support tool that considers important challenges in the decision and execution of β-blocker titration for qualifying HF patients at risk of undertreatment.