The effect of cognitive aid design on the perceived usability of critical event cognitive aids

An interprofessional cognitive aid to optimise extubation planning for patients with acute spinal cord injury

BACKGROUND

Early management of patients in the intensive care unit after acute spinal cord injury is challenging, particularly for patients with cervical injury and tetraplegia who have high rates of pneumonia and extubation failure.

OBJECTIVES

The primary objective was to evaluate the usability of a new interprofessional cognitive aid to optimise extubation planning for patients with acute tetraplegia. Secondary objectives were to (i) observe and compare clinician behaviour during high-fidelity simulation scenarios and (ii) compare clinician self-ratings of confidence before and after introduction to the cognitive aid.

METHODS

Dual methods design. Twenty-six intensive care clinicians (doctors, nurses, and physiotherapists) completed two cervical spinal cord-injured patient simulation scenarios in a random order. Between their two scenarios, participants were orientated to the new cognitive aid. Simulations were audiovisually recorded and scored by a blinded observer using a standardised checklist. Pre and post simulation questionnaires and semistructured interviews were completed.

RESULTS

The cognitive aid had good usability across all three disciplines (mean score on the System Usability Scale was 74.4). After introduction to the cognitive aid, clinicians had higher confidence with completing an independent respiratory assessment (p < 0.01) and objective outcome measures required for extubation (p < 0.01) and to discuss their objective findings and ongoing management with the multidisciplinary team (p = 0.04). Significantly more factors related to pneumonia (p < 0.001) and extubation readiness (p < 0.01) were identified, and significantly more time was taken to complete the second simulation (p = 0.03). The simulation scenarios were described as realistic, and the cognitive aid was positively perceived.

CONCLUSIONS

The interprofessional cognitive aid had good usability and enhanced intensive care clinicians' assessment, confidence, and communication about the extubation readiness of patients with acute cervical spinal cord injury.

Getting ON-TRAC, a team-centred design study of a reflexivity aid to support resuscitation teams' information sharing

BACKGROUND

Effective information sharing is crucial for emergency care teams to maintain an accurate shared mental model. This study describes the design, simulation-based testing and implementation of a team reflexivity aid to facilitate in-action information sharing during resuscitations.

METHODS

A five-phase team-centred iterative design process was employed. Phase 1 involved a literature review to identify in-action cognitive aids. Phase 2 focused on conceptual design, followed by simulation-based testing and modifications in phase 3. Implementation through simulation-based user training occurred in phase 4 at a large non-university teaching hospital. Phase 5 evaluated the aid among resuscitation team members in the emergency department after one year.

RESULTS

The phase 1 literature review identified 58 cognitive aids, with only 10 designed as 'team aid'. Studies using team information screens found increase team and task performance in simulation-based environments, with no evaluations in authentic workplaces. Phase 2 resulted in a three-section team reflexivity aid, iteratively modified in three rounds of simulation-based testing (N = 30 groups) phase 3 resulted in a team reflexivity aid containing five sections: resuscitation times and intervals, patient history, interventions on a longitudinal timeline, differential diagnosis and a quick review section. Phase 4 consisted of reflexivity aid user training with simulation-based education (N = 60 sessions) and the creation of a digital entry form to store data in the patient's electronic medical record. Evaluation after one year in phase 5, (N = 84) showed perceived improvements in communication (3.82 ± 0.77), documentation (4.25 ± 0.66), cognitive load (3.94 ± 0.68), and team performance (3.80 ± 0.76) on a 5-point Likert scale. Thematic analysis of user feedback identified improvements in both teamwork and taskwork. Teamwork enhancements included better situation awareness, communication and team participation. Taskwork improvements were seen in drug administration and clinical reasoning.

CONCLUSIONS

This study demonstrated the successful development and implementation of a Team Reflexivity Aid for Cardiac arrests using simulation methodology. This task-focused team tool improved perceived team situation awareness, communication, and overall performance. The research highlights the interplay between task- and teamwork in healthcare settings, underscoring the potential for taskwork-oriented tools to benefit team dynamics. These findings warrant further investigation into team-supportive interventions and their impact on resuscitation outcomes.

Why are critical event checklists not always used in the perioperative setting?: A retrospective survey

INTRODUCTION

During surgery and anesthesia, life-threatening critical events, including cardiac arrest, may occur. By facilitating recall of key management steps, suggesting diagnostic possibilities, and providing dose and drug information, cognitive aids may improve clinician performance during such events. In actual clinical practice, however, cognitive aids may be available but inconsistently used. One possibility explaining aid non-use during critical events is a lack of familiarity with how cognitive aids may be helpful. We hypothesized that introduction of critical event cognitive aids along with implementation of cognitive aid resources would change the quantitative incidence of cognitive aid use and qualitative reasons for aid non-use. We surveyed members of an academic anesthesia department before and after implementation of critical event cognitive aid resources.

METHODS

All anesthesia clinicians at a single academic medical center were surveyed. Participants were surveyed both pre- and post-training with a focused program to introduce critical event cognitive aid resources. Incidences of and reasons for cognitive aid use and non-use were collected and analyzed. Survey responses were compared pre- and post-implementation.

RESULTS

The response rate was 64.5%. One-hundred eighty-five reasons for non-use were collected before the focused program and 149 after. Overall, 80% of clinicians had encountered at least one critical event during the study period and use of cognitive aids during all reported events was 7%. Six categories of reasons for non-use were identified: 'Not Available', 'Not Needed', 'No Time', 'Another Person In Charge', 'Used In Another Way', 'No Reason Given'. After implementation, a decrease in the number of respondents who cited availability and who cited 'another person running crisis,' as reasons for non-use was observed (p < 0.001).

CONCLUSIONS

Implementation of cognitive aids for critical events in an academic anesthesia environment improved the perception of cognitive aid availability and decreased the number of subjects who chose to not use the aid due to another person running the crisis response. Looking at the multiple reasons for cognitive aid non-use may guide implementation, training, and design.

A Comparative Analysis of the Impact of Two Different Cognitive Aid Bundle Designs on Adherence to Best Clinical Practice in Simulated Perioperative Emergencies

Background: Stress and human error during perioperative emergency situations can significantly impact patient morbidity and mortality. Previous research has shown that cognitive aid bundles (CABs) minimize critical misses by 75%. This study aimed to compare the effectiveness of two different CAB designs with the same content in reducing missed critical management steps for simulated perioperative emergencies. Methods: A multicenter randomized controlled simulation-based study was conducted including 27 teams, each consisting of three participants; each team performed four simulation scenarios. In the first scenario for each team (Scenario 1), no CAB was used. Scenarios 2 and 3 were randomly allocated to the groups, with either a branched, clustered design (CAB-1) or a linear, step-by-step design (CAB-2) of the cognitive aid. In Scenario 4, the groups used one of the previously mentioned CABs according to their own preference. The primary outcome was the difference in the percentage of missed critical management steps between the two different CABs. Secondary outcomes included user preference for one CAB design and the reduction in percentage of missed critical management steps using any CAB versus no CAB. Results: Twenty-seven teams simulated 108 perioperative emergency situations. The percentage of missed critical management steps was similar between CAB-1 and CAB-2 (27% [interquartile range (IQR) 20-29] versus 29% [IQR 20-35], p = 0.23). However, most participants favored the branched, clustered design CAB-1 (77.8%). Additionally, employing any CAB reduced the percentage of missed critical management steps by 36% (33% missed steps vs. 21% missed steps, p = 0.003). Conclusions: While the two CAB designs did not differ significantly in reducing missed critical management steps, the branched, clustered design was perceived as more user-friendly. Importantly, using any CAB significantly reduced the percentage of missed critical management steps compared to not using a cognitive aid, emphasizing the need for CAB use in the operating room.

Clinical use of an emergency manual by resuscitation teams and impact on performance in the emergency department: a prospective mixed-methods study protocol

INTRODUCTION

Simulation-based studies indicate that crisis checklist use improves management of patients with critical conditions in the emergency department (ED). An interview-based study suggests that use of an emergency manual (EM)-a collection of crisis checklists-improves management of clinical perioperative crises. There is a need for in-depth prospective studies of EM use during clinical practice, evaluating when and how EMs are used and impact on patient management.

METHODS AND ANALYSIS

This 6-month long study prospectively evaluates a digital EM during management of priority 1 patients in the Skåne University Hospital at Lund's ED. Resuscitation teams are encouraged to use the EM after a management plan has been derived ('Do-Confirm'). The documenting nurse activates and reads from the EM, and checklists are displayed on a large screen visible to all team members. Whether the EM is activated, and which sections are displayed, are automatically recorded. Interventions performed thanks to Do-Confirm EM use are registered by the nurse. Fifty cases featuring such interventions are reviewed by specialists in emergency medicine blinded to whether the interventions were performed prior to or after EM use. All interventions are graded as indicated, of neutral relevance or not indicated. The primary outcome measures are the proportions of interventions performed thanks to Do-Confirm EM use graded as indicated, of neutral relevance, and not indicated. A secondary outcome measure is the team's subjective evaluation of the EM's value on a Likert scale of 1-6. Team members can report events related to EM use, and information from these events is extracted through structured interviews.

ETHICS AND DISSEMINATION

The study is approved by the Swedish Ethical Review Authority (Dnr 2022-01896-01). Results will be published in a peer-reviewed journal and abstracts submitted to national and international conferences to disseminate our findings.

TRIAL REGISTRATION NUMBER

NCT05649891.

The use of cognitive aids in the operating room: a systematic review

Background: Cognitive aids (CAs) are clinical tools guiding clinical decision-making during critical events in the operating room. They may counteract the adverse effects of stress on the non-technical skills of the attending clinician(s). Although most clinicians acknowledge the importance of CAs, their uptake in clinical practice seems to be lagging behind. This situation has led us to investigate which features of CAs may enhance their uptake. Therefore, in this systematic review we explored the optimums regarding the 1) timing to consult the CA, 2) person consulting the CA, 3) location of the CA in the operating room, 4) CA design (paper vs. electronic), 5) CA lay-out, 6) reader of the CA and 7) if the use of CAs in the form of decision support tools lead to improved outcome. Methods: Seven PICO-questions guided our literature search in 4 biomedical databases (MEDLINE, Embase, Web of Science and Google Scholar). We selected English-language randomized controlled trials (RCTs), observational studies and expert opinions discussing the use of cognitive aids during life-threatening events in the operating theatre. Articles discussing non-urgent or non-operating room settings were excluded. The quality of evidence was evaluated with the Grading of Recommendations Assessment, Development and Evaluation (GRADE). Results: We found 7 RCTs, 14 observational studies and 6 expert opinions. All trials were conducted in a simulation environment. The person who should trigger the use of a cognitive aid and the optimal timing of its initiation, could not be defined by the current literature. The ideal location of the cognitive aids remains also unclear. A favorable lay-out of an aid should be well-structured, standardized and easily readable. In addition, several potentially beneficial design features are described. RCT’s could not demonstrate a possible superiority of either electronic or paper-based aids. Both have their advantages and disadvantages. Furthermore, electronic decision support tools are potentially associated with an enhanced performance of the clinician. Likewise, the presence of a reader was associated with an improved performance of key steps in the management of a critical event. However, it remains unclear who should fulfill this role. Conclusion: Several features of the design or utilization of CAs may play a role in enhancing the uptake of CAs in clinical practice during the management of a critical event in the operating room. However, robust evidence supporting the use of a certain feature over another is lacking.

A Fundamental Cognitive Taxonomy for Cognition Aids

OBJECTIVE

This study aimed to organize the literature on cognitive aids to allow comparison of findings across studies and link the applied work of aid development to psychological constructs and theories of cognition.

BACKGROUND

Numerous taxonomies have been developed, all of which label cognitive aids via their surface characteristics. This complicates integration of the literature, as a type of aid, such as a checklist, can provide many different forms of support (cf. prospective memory for steps and decision support for alternative diagnoses).

METHOD

In this synthesis of the literature, we address the disparate findings and organize them at their most basic level: Which cognitive processes does the aid need to support? Which processes do they support? Such processes include attention, perception, decision making, memory, and declarative knowledge.

RESULTS

Cognitive aids can be classified into the processes they support. Some studies focused on how an aid supports the cognitive processes demanded by the task (aid function). Other studies focused on supporting the processes needed to utilize the aid (aid usability).

CONCLUSION

Classifying cognitive aids according to the processes they support allows comparison across studies in the literature and a formalized way of planning the design of new cognitive aids. Once the literature is organized, theory-based guidelines and applied examples can be used by cognitive aid researchers and designers.

APPLICATION

Aids can be designed according to the cognitive processes they need to support. Designers can be clear about their focus, either examining how to support specific cognitive processes or improving the usability of the aid.

The timing of cognitive aid access during simulated pediatric intraoperative critical events

BACKGROUND

Many cognitive aids are formatted in a step-by-step fashion with the intent that the aid will be accessed at the beginning of a critical event and that key behaviors will be performed in sequence.

AIMS

We hypothesized that, during simulated pediatric intraoperative critical events, anesthesia clinicians may not use cognitive aids immediately after the onset of a critical event but instead access the aid only after first performing several key behaviors.

MATERIALS AND METHODS

This manuscript is a re-analysis of previously published simulation data. The original study involved 89 clinicians participating in 143 pediatric intraoperative events divided into 6 types: arrhythmia, venous air embolus, hypoxemia, malignant hyperthermia, hypotension, and supraventricular tachycardia. For each trial involving cognitive aid use, we measured the time from event trigger to cognitive aid use, and the number and type of key behaviors performed by simulation participants prior to cognitive aid access.

RESULTS

Cognitive aid use was sought in 66 of 93 trials where it was available. Sufficient data for this analysis were available in 65 trials. The average time from event trigger to first cognitive aid use was 258 seconds. In 62/65 trials (95%), the cognitive aid was accessed after at least one key behavior had already been performed. The time from event trigger to cognitive aid use varied by type of scenario (P = .03, df 5, adjusted H 12.78), with the shortest time for "supraventricular tachycardia" (90 [66,156] seconds (median [IQR]) and the longest time for "hypoxemia" (354 [192,492] seconds).

CONCLUSION

In simulated critical events, anesthesia residents and student nurse anesthetists often consulted a cognitive aid only after first performing at least some key behaviors. Incorporating the possibility of delayed access into critical event cognitive aid design may facilitate the effectiveness of that aid.