Artifacts and collaborative work in healthcare: methodological, theoretical, and technological implications of the tangible

Local, Early, and Precise: Designing a Clinical Decision Support System for Child and Adolescent Mental Health Services

Mental health disorders often develop during childhood and adolescence, causing long term and debilitating impacts at individual and societal levels. Local, early, and precise assessment and evidence-based treatment are key to achieve positive mental health outcomes and to avoid long-term care. Technological advancements, such as computerized Clinical Decision Support Systems (CDSSs), can support practitioners in providing evidence-based care. While previous studies have found CDSS implementation helps to improve aspects of medical care, evidence is limited on its use for child and adolescent mental health care. This paper presents challenges and opportunities for adapting CDSS design and implementation to child and adolescent mental health services (CAMHS). To highlight the complexity of incorporating CDSSs within local CAMHS, we have structured the paper around four components to consider before designing and implementing the CDSS: supporting collaboration among multiple stakeholders involved in care; optimally using health data; accounting for comorbidities; and addressing the temporality of patient care. The proposed perspective is presented within the context of the child and adolescent mental health services in Norway and an ongoing Norwegian innovative research project, the Individualized Digital DEcision Assist System (IDDEAS), for child and adolescent mental health disorders. Attention deficit hyperactivity disorder (ADHD) among children and adolescents serves as the case example. The integration of IDDEAS in Norway intends to yield significantly improved outcomes for children and adolescents with enduring mental health disorders, and ultimately serve as an educational opportunity for future international approaches to such CDSS design and implementation.

What do healthcare professionals need to turn risk models for type 2 diabetes into usable computerized clinical decision support systems? Lessons learned from the MOSAIC project

Background

To understand user needs, system requirements and organizational conditions towards successful design and adoption of Clinical Decision Support Systems for Type 2 Diabetes (T2D) care built on top of computerized risk models.

Methods

The holistic and evidence-based CEHRES Roadmap, used to create eHealth solutions through participatory development approach, persuasive design techniques and business modelling, was adopted in the MOSAIC project to define the sequence of multidisciplinary methods organized in three phases, user needs, implementation and evaluation. The research was qualitative, the total number of participants was ninety, about five-seventeen involved in each round of experiment.

Results

Prediction models for the onset of T2D are built on clinical studies, while for T2D care are derived from healthcare registries. Accordingly, two set of DSSs were defined: the first, T2D Screening, introduces a novel routine; in the second case, T2D Care, DSSs can support managers at population level, and daily practitioners at individual level. In the user needs phase, T2D Screening and solution T2D Care at population level share similar priorities, as both deal with risk-stratification. End-users of T2D Screening and solution T2D Care at individual level prioritize easiness of use and satisfaction, while managers prefer the tools to be available every time and everywhere. In the implementation phase, three Use Cases were defined for T2D Screening, adapting the tool to different settings and granularity of information. Two Use Cases were defined around solutions T2D Care at population and T2D Care at individual, to be used in primary or secondary care. Suitable filtering options were equipped with “attractive” visual analytics to focus the attention of end-users on specific parameters and events. In the evaluation phase, good levels of user experience versus bad level of usability suggest that end-users of T2D Screening perceived the potential, but they are worried about complexity. Usability and user experience were above acceptable thresholds for T2D Care at population and T2D Care at individual.

Conclusions

By using a holistic approach, we have been able to understand user needs, behaviours and interactions and give new insights in the definition of effective Decision Support Systems to deal with the complexity of T2D care.

Electronic supplementary material

The online version of this article (10.1186/s12911-019-0887-8) contains supplementary material, which is available to authorized users.

Development process and patient usability preferences for a touch screen tablet-based questionnaire

We sought to design a touch tablet asthma questionnaire while identifying patient preferences for usability features of such questionnaires. We created an evidence-based prototype and employed rapid-cycle design (semi-structured focus group testing, analysis, corresponding modifications, re-testing) with asthma patients aged ⩾16 years. We analyzed transcripts using deductive and inductive content analysis. Quantitative measures included Likert-type-scale responses, the System Usability Scale, and questionnaire completion times. There were 20 participants across five focus groups (15/20 female, age 49.1 ± 15.6 years). Usability-related themes included (1) "Touch Technology" (hygiene, touch technology familiarity, ease of use) and (2) "Questionnaire Design" (visual characteristics, navigation). Completion time was 11.7 ± 5.9 min. Summative Likert-type scale responses suggested high system usability, as did a System Usability Scale score of 84.2 ± 14.7. In summary, Attention to specific technology- and design-related preferences can result in a highly usable patient-facing touch tablet questionnaire. Our findings can inform touch questionnaire design across other diseases.

Measuring Relevant Information in Health Social Network Conversations and Clinical Diagnosis Cases

The Internet and social media is an enormous source of information. Health social networks and online collaborative environments enable users to create shared content that afterwards can be discussed. The aim of this paper is to present a novel methodology designed for quantifying relevant information provided by different participants in clinical online discussions. The main goal of the methodology is to facilitate the comparison of participant interactions in clinical conversations. A set of key indicators for different aspects of clinical conversations and specific clinical contributions within a discussion have been defined. Particularly, three new indicators have been proposed to make use of biomedical knowledge extraction based on standard terminologies and ontologies. These indicators allow measuring the relevance of information of each participant of the clinical conversation. Proposed indicators have been applied to one discussion extracted from PatientsLikeMe, as well as to two real clinical cases from the Sanar collaborative discussion system. Results obtained from indicators in the tested cases have been compared with clinical expert opinions to check indicators validity. The methodology has been successfully used for describing participant interactions in real clinical cases belonging to a collaborative clinical case discussion tool and from a conversation from a health social network. This work can be applied to assess collaborative diagnoses, discussions among patients, and the participation of students in clinical case discussions. It permits moderators and educators to obtain a quantitatively measure of the contribution of each participant.

Faster clinical response to the onset of adverse events: A wearable metacognitive attention aid for nurse triage of clinical alarms

Objective

This study evaluates the potential for improving patient safety by introducing a metacognitive attention aid that enables clinicians to more easily access and use existing alarm/alert information. It is hypothesized that this introduction will enable clinicians to easily triage alarm/alert events and quickly recognize emergent opportunities to adapt care delivery. The resulting faster response to clinically important alarms/alerts has the potential to prevent adverse events and reduce healthcare costs.

Materials and methods

A randomized within-subjects single-factor clinical experiment was conducted in a high-fidelity 20-bed simulated acute care hospital unit. Sixteen registered nurses, four at a time, cared for five simulated patients each. A two-part highly realistic clinical scenario was used that included representative: tasking; information; and alarms/alerts. The treatment condition introduced an integrated wearable attention aid that leveraged metacognition methods from proven military systems. The primary metric was time for nurses to respond to important alarms/alerts.

Results

Use of the wearable attention aid resulted in a median relative within-subject improvement for individual nurses of 118% (W = 183, p = 0.006). The top quarter of relative improvement was 3,303% faster (mean; 17.76 minutes reduced to 1.33). For all unit sessions, there was an overall 148% median faster response time to important alarms (8.12 minutes reduced to 3.27; U = 2.401, p = 0.016), with 153% median improvement in consistency across nurses (F = 11.670, p = 0.001).

Discussion and conclusion

Existing device-centric alarm/alert notification solutions can require too much time and effort for nurses to access and understand. As a result, nurses may ignore alarms/alerts as they focus on other important work. There has been extensive research on reducing alarm frequency in healthcare. However, alarm safety remains a top problem. Empirical observations reported here highlight the potential of improving patient safety by supporting the meta-work of checking alarms.

Workarounds to Intended Use of Health Information Technology: A Narrative Review of the Human Factors Engineering Literature

Objective To integrate and synthesize insights from recent studies of workarounds to the intended use of health information technology (HIT) by health care professionals. Background Systems are safest when the documentation of how work is done in policies and procedures closely matches what people actually do when they are working. Proactively identifying and managing workarounds to the intended use of technology, including deviations from expected workflows, can improve system safety. Method A narrative review of studies of workarounds with HIT was conducted to identify themes in the literature. Results Three themes were identified: (1) Users circumvented new additional steps in the workflow when using HIT, (2) interdisciplinary team members communicated via HIT in text fields that were intended for other purposes, and (3) locally developed paper-based and manual whiteboard systems were used instead of HIT to support situation awareness of individuals and groups; an example of a locally developed system was handwritten notes about a patient on a piece of paper folded up and carried in a nurse's pocket. Conclusion Workarounds were employed to avoid changes to workflow, enable interdisciplinary communication, coordinate activities, and have real-time portable access to summarized and synthesized information. Application Implications for practice include providing summary overview displays, explicitly supporting role-based communication and coordination through HIT, and reducing the risk to reputation due to electronic monitoring of individual performance.

Changes in surgical team performance and safety climate attitudes following expansion of perioperative services: a repeated-measures study

Objective The aim of the present study was to describe process changes in surgical team performance and team members’ attitudes to safety culture following hospital relocation and expansion of perioperative services. Methods The study was a naturalistic study using structured observations and surveys to assess non-technical skills (NTS; i.e. communication, teamwork, situational awareness, decision making and leadership) in surgery. This interrupted time series design used mixed-linear regression models to examine the effect of phase (before and after hospital relocation) on surgical teams’ NTS and their processes that may affect performance. Differences in self-reported teamwork and safety climate attitudes were also examined. Results In all, 186 procedures (100 before and 81 after hospital relocation) were observed across teams working in general, paediatric, orthopaedic and thoracic surgeries. Interobserver agreement ranged from 86% to 95%. An effect of phase was found, indicating that there were significant improvements after relocation in the use of NTS by the teams observed (P=0.020; 95% confidence interval 1.9–4.7). Conclusions The improvements seen in surgical teams’ NTS performance and safety culture attitudes may be related to the move to a new state-of-the-art perioperative department. What is known about the topic? Patient safety in surgery relies on optimal team performance, underpinned by effective NTS. What does this paper add? The NTS of surgical teams may be improved through ergonomic innovations that promote teams’ shared mental models. What are the implications for practitioners? Effective multidisciplinary teamwork relies on a combination of NTS and ergonomic factors, which inherently contribute to team performance and safety climate attitudes.

Mediating the intersections of organizational routines during the introduction of a health IT system

Public interest in the quality and safety of health care has spurred examination of specific organizational routines believed to yield risk in health care work. Medication administration routines, in particular, have been the subject of numerous improvement projects involving information technology development, and other forms of research and regulation. This study draws from ethnographic observation to examine how the common routine of medication administration intersects with other organizational routines, and why understanding such intersections is important. We present three cases describing intersections between medication administration and other routines, including a pharmacy routine, medication administration on the next shift and management reporting. We found that each intersection had ostensive and performative dimensions; and furthermore, that IT-enabled changes to one routine led to unintended consequences in its intersection with others, resulting in misalignment of ostensive and performative aspects of the intersection. Our analysis focused on the activities of a group of nurses who provide technology use mediation (TUM) before and after the rollout of a new health IT system. This research offers new insights on the intersection of organizational routines, demonstrates the value of analyzing TUM activities to better understand the relationship between IT introduction and changes in routines, and has practical implications for the implementation of technology in complex practice settings.

Human factors in mental healthcare: A work system analysis of a community-based program for older adults with depression and dementia

Mental healthcare is a critical but largely unexplored application domain for human factors/ergonomics. This paper reports on a work system evaluation of a home-based dementia and depression care program for older adults, the Aging Brain Care program. The Workflow Elements Model was used to guide data collection and analysis of 59 h of observation, supplemented by key informant input. We identified four actors, 37 artifacts across seven types, ten action categories, and ten outcomes including improved health and safety. Five themes emerged regarding barriers and facilitators to care delivery in the program: the centrality of relationship building; the use of adaptive workarounds; performance of duplicate work; travel and scheduling challenges; and communication-related factors. Findings offer new insight into how mental healthcare services are delivered in a community-based program and key work-related factors shaping program outcomes.

Cognitive engineering and health informatics: Applications and intersections

Cognitive engineering is an applied field with roots in both cognitive science and engineering that has been used to support design of information displays, decision support, human-automation interaction, and training in numerous high risk domains ranging from nuclear power plant control to transportation and defense systems. Cognitive engineering provides a set of structured, analytic methods for data collection and analysis that intersect with and complement methods of Cognitive Informatics. These methods support discovery of aspects of the work that make performance challenging, as well as the knowledge, skills, and strategies that experts use to meet those challenges. Importantly, cognitive engineering methods provide novel representations that highlight the inherent complexities of the work domain and traceable links between the results of cognitive analyses and actionable design requirements. This article provides an overview of relevant cognitive engineering methods, and illustrates how they have been applied to the design of health information technology (HIT) systems. Additionally, although cognitive engineering methods have been applied in the design of user-centered informatics systems, methods drawn from informatics are not typically incorporated into a cognitive engineering analysis. This article presents a discussion regarding ways in which data-rich methods can inform cognitive engineering.

Mind the gulfs: An analysis of medication-related cognitive artifacts used by older adults with heart failure

Medication management is a patient health-related activity characterized by poor performance in older adults with chronic disease. Interventions focus on educating and motivating the patient with limited long-term effects. Cognitive artifacts facilitate cognitive tasks by making them easier, faster, and more effective and can potentially improve medication management performance. This study examined how older adult patients with heart failure use cognitive artifacts and how representational structure and physical properties facilitated or impeded medication-related tasks and processes. Interview, observation, medical record, and photographic data of and about older patients with heart failure (N = 30) and their informal caregivers (N=14) were content analyzed for cross-cutting themes about patient goals, representations, and actions. Results illustrated patient artifacts designed from a clinical rather than patient perspective, disparate internal and external representations threatening safety, and incomplete information exchange between patients and clinicians. Implications for design were the need for bridging artifacts, automatic information transfer, and cognitive artifacts designed from the perspective of the patient.

Centralized and Decentralized Nurse Station Design: An Examination of Caregiver Communication, Work Activities, and Technology

The healthcare construction boom requires evidence for effective design of nurse stations, including evidence supporting workflow processes, computerization, integration of technology, communication of caregivers, and optimal patient outcomes. This article describes the examination of a traditional centralized nursing station using a total patient care delivery model and minimal computerization and a highly computerized, decentralized nursing station using a team nursing model. Results specific to communication activities, time with patients, number of patient visits per registered nurse, and patient satisfaction with response time are reported.

An Electronic Task Management (ETM) system for after hours hospital work and subsequent socially mediated effects of task completion

Modern Hospitals are under ever increasing efficiency pressures; patient safety and flow are paramount. In the after-hours period, many tasks such as transfers between clinical areas and procedures are delayed because the resources may be limited or poorly distributed compared to in hours. An Electronic Task Management (ETM) system was iteratively designed to support the redesign of the after hours staffing and task distribution model that addressed these delays and improved staff efficiency.

The solution consisted of a task controller program installed on desktop PCs in the clinical areas and similar software on smart phones for the clinical staff (operatives) undertaking the clinical tasks. In a system without clinical leadership and workload transparency, the forced reallocation of tasks to operatives was strongly resisted by the operatives. The development of an interface that allowed workloads of all operatives to be visualized by all other operatives led to a socially mediated, cooperative solution that was readily accepted. The quality of the information sent by the ETM was superior to the previous paging system and workloads were more equitable among operatives with the introduction of the ETM.

Medication-related cognitive artifacts used by older adults with heart failure

OBJECTIVE

To use a human factors perspective to examine how older adult patients with heart failure use cognitive artifacts for medication management.

METHODS

We performed a secondary analysis of data collected from 30 patients and 14 informal caregivers enrolled in a larger study of heart failure self-care. Data included photographs, observation notes, interviews, video recordings, medical record data, and surveys. These data were analyzed using an iterative content analysis.

RESULTS

Findings revealed that medication management was complex, inseparable from other patient activities, distributed across people, time, and place, and complicated by knowledge gaps. We identified fifteen types of cognitive artifacts including medical devices, pillboxes, medication lists, and electronic personal health records used for: 1) measurement/evaluation; 2) tracking/communication; 3) organization/administration; and 4) information/sensemaking. These artifacts were characterized by fit and misfit with the patient's sociotechnical system and demonstrated both advantages and disadvantages. We found that patients often modified or "finished the design" of existing artifacts and relied on "assemblages" of artifacts, routines, and actors to accomplish their self-care goals.

CONCLUSIONS

Cognitive artifacts are useful but sometimes are poorly designed or are not used optimally. If appropriately designed for usability and acceptance, paper-based and computer-based information technologies can improve medication management for individuals living with chronic illness. These technologies can be designed for use by patients, caregivers, and clinicians; should support collaboration and communication between these individuals; can be coupled with home-based and wearable sensor technology; and must fit their users' needs, limitations, abilities, tasks, routines, and contexts of use.

Understanding safety-critical interactions with a home medical device through Distributed Cognition

As healthcare shifts from the hospital to the home, it is becoming increasingly important to understand how patients interact with home medical devices, to inform the safe and patient-friendly design of these devices. Distributed Cognition (DCog) has been a useful theoretical framework for understanding situated interactions in the healthcare domain. However, it has not previously been applied to study interactions with home medical devices. In this study, DCog was applied to understand renal patients' interactions with Home Hemodialysis Technology (HHT), as an example of a home medical device. Data was gathered through ethnographic observations and interviews with 19 renal patients and interviews with seven professionals. Data was analyzed through the principles summarized in the Distributed Cognition for Teamwork methodology. In this paper we focus on the analysis of system activities, information flows, social structures, physical layouts, and artefacts. By explicitly considering different ways in which cognitive processes are distributed, the DCog approach helped to understand patients' interaction strategies, and pointed to design opportunities that could improve patients' experiences of using HHT. The findings highlight the need to design HHT taking into consideration likely scenarios of use in the home and of the broader home context. A setting such as home hemodialysis has the characteristics of a complex and safety-critical socio-technical system, and a DCog approach effectively helps to understand how safety is achieved or compromised in such a system.

Cognitive informatics in biomedicine and healthcare

Cognitive Informatics (CI) is a burgeoning interdisciplinary domain comprising of the cognitive and information sciences that focuses on human information processing, mechanisms and processes within the context of computing and computer applications. Based on a review of articles published in the Journal of Biomedical Informatics (JBI) between January 2001 and March 2014, we identified 57 articles that focused on topics related to cognitive informatics. We found that while the acceptance of CI into the mainstream informatics research literature is relatively recent, its impact has been significant - from characterizing the limits of clinician problem-solving and reasoning behavior, to describing coordination and communication patterns of distributed clinical teams, to developing sustainable and cognitively-plausible interventions for supporting clinician activities. Additionally, we found that most research contributions fell under the topics of decision-making, usability and distributed team activities with a focus on studying behavioral and cognitive aspects of clinical personnel, as they performed their activities or interacted with health information systems. We summarize our findings within the context of the current areas of CI research, future research directions and current and future challenges for CI researchers.

The clinician in the driver's seat: part 2 - intelligent uses of space in a drag/drop user-composable electronic health record

User-composable approaches provide clinicians with the control to design and assemble information elements on screen via drag/drop. They hold considerable promise for enhancing the electronic-health-records (EHRs) user experience. We previously described this novel approach to EHR design and our illustrative system, MedWISE. The purpose of this paper is to describe clinician users' intelligent uses of space during completion of real patient case studies in a laboratory setting using MedWISE. Thirteen clinicians at a quaternary academic medical center used the system to review four real patient cases. We analyzed clinician utterances, behaviors, screen layouts (i.e., interface designs), and their perceptions associated with completing patient case studies. Clinicians effectively used the system to review all cases. Two coding schemata pertaining to human-computer interaction and diagnostic reasoning were used to analyze the data. Users adopted three main interaction strategies: rapidly gathering items on screen and reviewing ('opportunistic selection' approach); creating highly structured screens ('structured' approach); and interacting with small groups of items in sequence as their case review progressed ('dynamic stage' approach). They also used spatial arrangement in ways predicted by theory and research on workplace spatial arrangement. This includes assignment of screen regions for particular purposes (24% of spatial codes), juxtaposition to facilitate calculation or other cognitive tasks ('epistemic action'), and grouping elements with common meanings or relevance to the diagnostic facets of the case (20.3%). A left-to-right progression of orienting materials, data, and action items or reflection space was a commonly observed pattern. Widget selection was based on user assessment of what information was useful or relevant. We developed and tested an illustrative system that gives clinicians greater control of the EHR, and demonstrated its feasibility for case review by typical clinicians. Producing the simplifying inventions, such as user-composable platforms that shift control to the user, may serve to promote productive EHR use and enhance its value as an instrument of patient care.

Teamwork and Collaboration

Safety and quality of health care depend on collaborative efforts of multiprofessional and multidisciplinary teams of care providers. Team research in aviation and the military has produced a wealth of knowledge in terms of concepts and intervention strategies to improve team performance. Research on collaborative work in health care in the past 20 years has uncovered unique characteristics and requirements of teams in hospitals and other health care settings and has provided early assessment of the utility of the theoretical concepts, methodologies, and interventions developed outside health care. In this chapter, we review a set of concepts that have been used in characterizing teams in health care and in improving teamwork. These concepts include the organizational shell to capture the sociotechnical environment in which teams reside as well as nontechnical skills, team leadership, team mental models, and so on. We will review a number of leading interventions to enhance team performance, such as teamwork training (e.g., TeamSTEPPS) and structured communication (e.g., SBAR). Future directions are suggested on better understanding of the interdependencies between teams and their organizational shell, such as standardization of operating procedures and training, and to focus on the patient in terms of teamwork improvement.

Communication spaces

Background and objective

Annotations to physical workspaces such as signs and notes are ubiquitous. When densely annotated, work areas become communication spaces. This study aims to characterize the types and purpose of such annotations.

Methods

A qualitative observational study was undertaken in two wards and the radiology department of a 440-bed metropolitan teaching hospital. Images were purposefully sampled; 39 were analyzed after excluding inferior images.

Results

Annotation functions included signaling identity, location, capability, status, availability, and operation. They encoded data, rules or procedural descriptions. Most aggregated into groups that either created a workflow by referencing each other, supported a common workflow without reference to each other, or were heterogeneous, referring to many workflows. Higher-level assemblies of such groupings were also observed.

Discussion

Annotations make visible the gap between work done and the capability of a space to support work. Annotations are repairs of an environment, improving fitness for purpose, fixing inadequacy in design, or meeting emergent needs. Annotations thus record the missing information needed to undertake tasks, typically added post-implemented. Measuring annotation levels post-implementation could help assess the fit of technology to task. Physical and digital spaces could meet broader user needs by formally supporting user customization, ‘programming through annotation’. Augmented reality systems could also directly support annotation, addressing existing information gaps, and enhancing work with context sensitive annotation.

Conclusions

Communication spaces offer a model of how work unfolds. Annotations make visible local adaptation that makes technology fit for purpose post-implementation and suggest an important role for annotatable information systems and digital augmentation of the physical environment.

Reframing clinical workplace learning using the theory of distributed cognition

In medicine, knowledge is embodied and socially, temporally, spatially, and culturally distributed between actors and their environment. In addition, clinicians increasingly are using technology in their daily work to gain and share knowledge. Despite these characteristics, surprisingly few studies have incorporated the theory of distributed cognition (DCog), which emphasizes how cognition is distributed in a wider system in the form of multimodal representations (e.g., clinical images, speech, gazes, and gestures) between social actors (e.g., doctors and patients) in the physical environment (e.g., with technological instruments and computers). In this article, the authors provide an example of an interaction between medical actors. Using that example, they then introduce the important concepts of the DCog theory, identifying five characteristics of clinical representations-that they are interwoven, co-constructed, redundantly accessed, intersubjectively shared, and substantiated-and discuss their value for learning. By contrasting these DCog perspectives with studies from the field of medical education, the authors argue that researchers should focus future medical education scholarship on the ways in which medical actors use and connect speech, bodily movements (e.g., gestures), and the visual and haptic structures of their own bodies and of artifacts, such as technological instruments and computers, to construct complex, multimodal representations. They also argue that future scholarship should "zoom in" on detailed, moment-by-moment analysis and, at the same time, "zoom out" following the distribution of cognition through an overall system to develop a more integrated view of clinical workplace learning.

Digital video analysis of health professionals' interactions with an electronic whiteboard: a longitudinal, naturalistic study of changes to user interactions

As hospital departments continue to introduce electronic whiteboards in real clinical settings a range of human factor issues have emerged and it has become clear that there is a need for improved methods for designing and testing these systems. In this study, we employed a longitudinal and naturalistic method in the usability evaluation of an electronic whiteboard system. The goal of the evaluation was to explore the extent to which usability issues experienced by users change as they gain more experience with the system. In addition, the paper explores the use of a new approach to collection and analysis of continuous digital video recordings of naturalistic "live" user interactions. The method developed and employed in the study included recording the users' interactions with system during actual use using screen-capturing software and analyzing these recordings for usability issues. In this paper we describe and discuss both the method and the results of the evaluation. We found that the electronic whiteboard system contains system-related usability issues that did not change over time as the clinicians collectively gained more experience with the system. Furthermore, we also found user-related issues that seemed to change as the users gained more experience and we discuss the underlying reasons for these changes. We also found that the method used in the study has certain advantages over traditional usability evaluation methods, including the ability to collect analyze live user data over time. However, challenges and drawbacks to using the method (including the time taken for analysis and logistical issues in doing live recordings) should be considered before utilizing a similar approach. In conclusion we summarize our findings and call for an increased focus on longitudinal and naturalistic evaluations of health information systems and encourage others to apply and refine the method utilized in this study.

Automation and adaptation: Nurses' problem-solving behavior following the implementation of bar coded medication administration technology

The most common change facing nurses today is new technology, particularly bar coded medication administration technology (BCMA). However, there is a dearth of knowledge on how BCMA alters nursing work. This study investigated how BCMA technology affected nursing work, particularly nurses' operational problem-solving behavior. Cognitive systems engineering observations and interviews were conducted after the implementation of BCMA in three nursing units of a freestanding pediatric hospital. Problem-solving behavior, associated problems, and goals, were specifically defined and extracted from observed episodes of care. Three broad themes regarding BCMA's impact on problem solving were identified. First, BCMA allowed nurses to invent new problem-solving behavior to deal with pre-existing problems. Second, BCMA made it difficult or impossible to apply some problem-solving behaviors that were commonly used pre-BCMA, often requiring nurses to use potentially risky workarounds to achieve their goals. Third, BCMA created new problems that nurses were either able to solve using familiar or novel problem-solving behaviors, or unable to solve effectively. Results from this study shed light on hidden hazards and suggest three critical design needs: (1) ecologically valid design; (2) anticipatory control; and (3) basic usability. Principled studies of the actual nature of clinicians' work, including problem solving, are necessary to uncover hidden hazards and to inform health information technology design and redesign.

Debunking health IT usability myths

Poor usability is a threat to patient safety and linked to productivity loss, workflow disruption, user frustration, sub-optimal product use and system de-installations. Although usability is receiving more attention nationally and internationally, myths about usability persist. This editorial debunks five common myths about usability (1) usability only concerns the look and feel of a product and is, therefore, only a minor concern, (2) usability is not measurable, (3) usability stifles innovation, (4) vendors are solely responsible for product usability, and (5) usability methods are not practical for use in healthcare.

Medication errors in residential aged care facilities: a distributed cognition analysis of the information exchange process

BACKGROUND

Medication safety is a pressing concern for residential aged care facilities (RACFs). Retrospective studies in RACF settings identify inadequate communication between RACFs, doctors, hospitals and community pharmacies as the major cause of medication errors. Existing literature offers limited insight about the gaps in the existing information exchange process that may lead to medication errors. The aim of this research was to explicate the cognitive distribution that underlies RACF medication ordering and delivery to identify gaps in medication-related information exchange which lead to medication errors in RACFs.

METHODS

The study was undertaken in three RACFs in Sydney, Australia. Data were generated through ethnographic field work over a period of five months (May-September 2011). Triangulated analysis of data primarily focused on examining the transformation and exchange of information between different media across the process.

RESULTS

The findings of this study highlight the extensive scope and intense nature of information exchange in RACF medication ordering and delivery. Rather than attributing error to individual care providers, the explication of distributed cognition processes enabled the identification of gaps in three information exchange dimensions which potentially contribute to the occurrence of medication errors namely: (1) design of medication charts which complicates order processing and record keeping (2) lack of coordination mechanisms between participants which results in misalignment of local practices (3) reliance on restricted communication bandwidth channels mainly telephone and fax which complicates the information processing requirements. The study demonstrates how the identification of these gaps enhances understanding of medication errors in RACFs.

CONCLUSIONS

Application of the theoretical lens of distributed cognition can assist in enhancing our understanding of medication errors in RACFs through identification of gaps in information exchange. Understanding the dynamics of the cognitive process can inform the design of interventions to manage errors and improve residents' safety.

Mapping out the emergency department disposition decision for high-acuity patients

STUDY OBJECTIVE

There are sparse data on how emergency health professionals make the important decision of emergency department (ED) patient admission or discharge, also known as the disposition decision. This study seeks to create a process map, a visual step-by-step diagram, and highlight error-prone areas for disposition decisions for high-acuity or nonambulatory ED patients.

METHODS

We conducted 6 focus groups at an academic tertiary care ED: residents, social workers and registered nurses, registered nurses only, attending physicians, patient safety committee members, and consensus group from the 5 preceding groups. We asked participants to create a disposition decision process map and identify error-prone areas. We audiotaped, transcribed, and analyzed the sessions for themes, using qualitative techniques.

RESULTS

Forty-two stakeholders with clinical experience from 1 to 30 years participated. We found 9 dominant themes (ordered according to prevalence): triage, ED location of patient assessment, monitoring, diagnosis, departmental busyness, clinical gestalt, response to treatment, social work involvement, and patient and family communication. Groups identified overarching themes such as risk stratification and administrative policy. One group included dynamic elements such as interactions with consultants and handover. Participants described the following contributors to disposition error: triage, diagnostic error, communication error, ED location of patient assessment, and ED crowding.

CONCLUSION

Participants endorsed triage, diagnostic error, communication error, ED location of patient assessment, and ED crowding as the most important contributors to ED disposition decisionmaking errors. Understanding these factors in clinical decisionmaking is fundamental to improving future ED patient safety.

In search of common ground in handoff documentation in an Intensive Care Unit

OBJECTIVE

Handoff is an intra-disciplinary process, yet the flow of critical handoff information spans multiple disciplines. Understanding this information flow is important for the development of computer-based tools that supports the communication and coordination of patient care in a multi-disciplinary and highly specialized critical care setting. We aimed to understand the structure, functionality, and content of nurses' and physicians' handoff artifacts.

DESIGN

We analyzed 22 nurses' and physicians' handoff artifacts from a Cardiothoracic Intensive Care Unit (CTICU) at a large urban medical center. We combined artifact analysis with semantic coding based on our published Interdisciplinary Handoff Information Coding (IHIC) framework for a novel two-step data analysis approach.

RESULTS

We found a high degree of structure and overlap in the content of nursing and physician artifacts. Our findings demonstrated a non-technical, yet sophisticated, system with a high degree of structure for the organization and communication of patient data that functions to coordinate the work of multiple disciplines in a highly specialized unit of patient care.

LIMITATIONS

This study took place in one CTICU. Further work is needed to determine the generalizability of the results.

CONCLUSIONS

Our findings indicate that the development of semi-structured patient-centered interdisciplinary handoff tools with discipline specific views customized for specialty settings may effectively support handoff communication and patient safety.

Estimating the success of e-health collaborative services: the THEMIS framework

This study proposes a prototype framework (THEMIS) for estimating algebraically the success (S) of the electronic health collaborative services (e-HCS) and examines two hypotheses: first, that the S estimation of an e-HCS, developed by a third-party vendor, demands a 'shrunk formative model' and second that causal relationships between the involved dimensions (FFP, CO, COSTS) do exist, and their parameters affect the S - from weakly to strongly and vice-versa. A formative model was shrunk to generate three causal dimensions ('Collaborators Objections', 'Costs', 'Fitness for Purpose'). Then, the new framework (THEMIS) was enriched with a causal loop diagram, a prototype scoring method, (termed 'polarisation method') and 42 questions. In order to investigate the feasibility of the THEMIS framework, we estimated the S of 15 e-HCSs and the algebraic outcomes (E(S)) were compared - one by one - with usage categories produced by a commercial software. Our findings supported the initial hypotheses. The S was estimated with accuracy; for the e-HCSs with a weak E(S) the commercial software verified that they remained idle several times during the 11-month evaluation period, whereas the e-HCS with a strong E(S) the commercial software verified that they were used frequently. Frameworks, such as the one proposed, which are based on both qualitative and quantitative methods, may provide significant support on the S estimation field.

The effects of a hands-free communication device system in a surgical suite

This case report describes a qualitative investigation into how a Hands-free Communication Device (HCD) system impacted communication among anesthesia staff in a pediatric surgical suite. The authors recruited a purposive sample that included anesthesiologists, certified registered nurse anesthetists, circulating nurses, a charge nurse, and a postanesthesia care unit nurse. Data were collected using semistructured interviews and observations, then analyzed using a constant comparison approach. The results corroborate and enrich themes that were discovered in a previous qualitative study of HCD systems: (1) communication access, (2) control, (3) training, (4) environment and infrastructure. The results also generated new subthemes and themes: (1) technical control, (2) choosing communication channels, and (3) reliability. The authors conclude that HCD systems profoundly impacted communication in a largely positive way, although reliability of the technology remained an issue. The authors' findings contribute a valuable insight into the growing body of knowledge about implementation and use of HCD systems.

Does the process map influence the outcome of quality improvement work? A comparison of a sequential flow diagram and a hierarchical task analysis diagram

BACKGROUND

Many quality and safety improvement methods in healthcare rely on a complete and accurate map of the process. Process mapping in healthcare is often achieved using a sequential flow diagram, but there is little guidance available in the literature about the most effective type of process map to use. Moreover there is evidence that the organisation of information in an external representation affects reasoning and decision making. This exploratory study examined whether the type of process map - sequential or hierarchical - affects healthcare practitioners' judgments.

METHODS

A sequential and a hierarchical process map of a community-based anti coagulation clinic were produced based on data obtained from interviews, talk-throughs, attendance at a training session and examination of protocols and policies. Clinic practitioners were asked to specify the parts of the process that they judged to contain quality and safety concerns. The process maps were then shown to them in counter-balanced order and they were asked to circle on the diagrams the parts of the process where they had the greatest quality and safety concerns. A structured interview was then conducted, in which they were asked about various aspects of the diagrams.

RESULTS

Quality and safety concerns cited by practitioners differed depending on whether they were or were not looking at a process map, and whether they were looking at a sequential diagram or a hierarchical diagram. More concerns were identified using the hierarchical diagram compared with the sequential diagram and more concerns were identified in relation to clinical work than administrative work. Participants' preference for the sequential or hierarchical diagram depended on the context in which they would be using it. The difficulties of determining the boundaries for the analysis and the granularity required were highlighted.

CONCLUSIONS

The results indicated that the layout of a process map does influence perceptions of quality and safety problems in a process. In quality improvement work it is important to carefully consider the type of process map to be used and to consider using more than one map to ensure that different aspects of the process are captured.

Clinical review: checklists - translating evidence into practice

Checklists are common tools used in many industries. Unfortunately, their adoption in the field of medicine has been limited to equipment operations or part of specific algorithms. Yet they have tremendous potential to improve patient outcomes by democratizing knowledge and helping ensure that all patients receive evidence-based best practices and safe high-quality care. Checklist adoption has been slowed by a variety of factors, including provider resistance, delays in knowledge dissemination and integration, limited methodology to guide development and maintenance, and lack of effective technical strategies to make them available and easy to use. In this article, we explore some of the principles and possible strategies to further develop and encourage the implementation of checklists into medical practice. We describe different types of checklists using examples and explore the benefits they offer to improve care. We suggest methods to create checklists and offer suggestions for how we might apply them, using some examples from our own experience, and finally, offer some possible directions for future research.

Decoding the perioperative process breakdowns: a theoretical model and implications for system design

BACKGROUND

Breakdowns in communication and coordination are situations of mismatch between actual and expected conditions in joint activities. Breakdowns have been identified as the leading cause of adverse events in healthcare, especially in the Operating Room environment. As a result, researchers have started to examine breakdowns in healthcare as emergent dynamics of teamwork. However, the occurrence and consequences of breakdowns related to inter-team processes are yet to be addressed at a fine level of detail. In this paper we seek understanding of breakdowns at the systemic level, and its relevance to design.

OBJECTIVES

The objective of this study is to bring forward an in-depth understanding of the impact of breakdowns on the surgical process by expanding the focus of analysis beyond teamwork dynamics, to the level of hospital system processes. This study also aims to examine the implications of such understanding of breakdowns for the design of clinical systems.

METHODS

Properties of breakdowns and repairs were inductively derived, and developed into a formal coding scheme, which was applied over a set of observed breakdowns from an elective surgery unit in a North American hospital. Systematic content analysis was employed to quantify qualitative data spanning 79 h of observations, followed by statistical hypotheses testing for relationships between variables of breakdowns and repairs.

MEASURES

Breakdown type, theme, tangibility, coordination scale, breakdown lifetime, repair strategy, and repair cost.

RESULTS

The results reveal that properties of breakdowns determine properties of repairs. The majority of breakdowns were outside the scope of teamwork--at the inter-team coordination level. The results also demonstrate that breakdowns usually propagate downstream in the surgical process, affecting the work of multiple teams, and the longer they propagate the higher the communication cost associated with the respective repair. The implications are two-fold: in terms of theory we develop a conceptual framework of breakdowns in perioperative work, and in terms of system design we propose a design framework informed by the acquired understanding of breakdowns.

CONCLUSIONS

This study achieved an initial understanding of the deep features of breakdowns from a process-oriented perspective, which allowed us to build the groundwork for a theoretical model of breakdowns in perioperative activities and to propose a design approach that tackles breakdowns during early stages of system development. The direct association between breakdowns and repairs can be exploited in both IT-system design and organizational design. The patterns of repair work can inform design so as to provide clinicians with the types of information that will prevent breakdowns from occurring or to mitigate the impact of breakdowns. The results reveal that preventing breakdown propagation should be a prime target in surgical applications design.

The effects of clinical information presentation on physicians' and nurses' decision-making in ICUs

This research evaluated physicians' agreement about patients' diagnoses and nurses' ability to detect patient change using traditional charts (TC) and a work domain analysis-based paper prototype (PP) and also sought to determine whether differences persisted when the PP was represented as an electronic prototype (EP). Nurses' change detection improved using the PP and EP compared to TC (PP vs TC, t((df=6))=1.94, p<0.03; EP vs TC, t((df=6))=3.14, p<0.01) and detection was better using the EP compared with the PP (t((df=6))=5.96, p<0.001). Physicians were more likely to agree about failed physiological systems using the EP compared with the PP (t((df=10))=3.14, p<0.01), but agreement about patient diagnoses was higher using the PP compared with the EP (t((df=10))=2.23; p<0.02). These results are attributed to information grouping around physiological functions and the direct association of cause-and-effect relations in clinical information design.

Development and evaluation of nursing user interface screens using multiple methods

Building upon the foundation of the Structured Narrative Electronic Health Record (EHR) model, we applied theory-based (combined Technology Acceptance Model and Task-Technology Fit Model) and user-centered methods to explore nurses' perceptions of functional requirements for an electronic nursing documentation system, design user interface screens reflective of the nurses' perspectives, and assess nurses' perceptions of the usability of the prototype user interface screens. The methods resulted in user interface screens that were perceived to be easy to use, potentially useful, and well-matched to nursing documentation tasks associated with Nursing Admission Assessment, Blood Administration, and Nursing Discharge Summary. The methods applied in this research may serve as a guide for others wishing to implement user-centered processes to develop or extend EHR systems. In addition, some of the insights obtained in this study may be informative to the development of safe and efficient user interface screens for nursing document templates in EHRs.

A comparison of usability methods for testing interactive health technologies: methodological aspects and empirical evidence

OBJECTIVE

Usability evaluation is now widely recognized as critical to the success of interactive health care applications. However, the broad range of usability inspection and testing methods available may make it difficult to decide on a usability assessment plan. To guide novices in the human-computer interaction field, we provide an overview of the methodological and empirical research available on the three usability inspection and testing methods most often used.

METHODS

We describe two 'expert-based' and one 'user-based' usability method: (1) the heuristic evaluation, (2) the cognitive walkthrough, and (3) the think aloud.

RESULTS

All three usability evaluation methods are applied in laboratory settings. Heuristic evaluation is a relatively efficient usability evaluation method with a high benefit-cost ratio, but requires high skills and usability experience of the evaluators to produce reliable results. The cognitive walkthrough is a more structured approach than the heuristic evaluation with a stronger focus on the learnability of a computer application. Major drawbacks of the cognitive walkthrough are the required level of detail of task and user background descriptions for an adequate application of the latest version of the technique. The think aloud is a very direct method to gain deep insight in the problems end users encounter in interaction with a system but data analyses is extensive and requires a high level of expertise both in the cognitive ergonomics and in computer system application domain.

DISCUSSION AND CONCLUSIONS

Each of the three usability evaluation methods has shown its usefulness, has its own advantages and disadvantages; no single method has revealed any significant results indicating that it is singularly effective in all circumstances. A combination of different techniques that compliment one another should preferably be used as their collective application will be more powerful than applied in isolation. Innovative mobile and automated solutions to support end-user testing have emerged making combined approaches of laboratory, field and remote usability evaluations of new health care applications more feasible.

Challenges to effective crisis management: using information and communication technologies to coordinate emergency medical services and emergency department teams

OBJECTIVE

The purpose of this study is to identify the major challenges to coordination between emergency department (ED) teams and emergency medical services (EMS) teams.

DESIGN

We conducted a series of focus groups involving both ED and EMS team members using a crisis scenario as the basis of the focus group discussion. We also collected organizational workflow data.

RESULTS

We identified three major challenges to coordination between ED and EMS teams including ineffectiveness of current information and communication technologies, lack of common ground, and breakdowns in information flow.

DISCUSSION

The three challenges highlight the importance of designing systems from socio-technical perspective. In particular, these inter-team coordination systems must support socio-technical issues such as awareness, context, and workflow between the two teams.

[Changes in physicians' attitudes to computerized ambulatory medical record systems: a longitudinal qualitative study]

OBJECTIVES

To explore physicians' beliefs about a computerized ambulatory medical record system at different stages of its implementation.

METHODS

We performed a longitudinal qualitative in-depth interview study (July 2001 to December 2003) in the Hospital Italiano, Buenos Aires, Argentina. Semi-structured interviews were conducted in 20 primary care cardiologists purposively selected before, during and after the system's implementation process (10 interviews per stage). The interviews were independently analyzed by 2 researchers, who jointly designed an agreed category list.

RESULTS

Both before and during the first stage of the implementation process, the physicians expected that that the system would improve healthcare-related administration and increase accessibility to individual data. However, they did not foresee that the system's shared information could modify the clinical aspects of patient care. By the end of the implementation process, the physicians realized that the system provided them with a broader perspective on their patients, which in turn improved their own professional performance. Throughout the implementation, the physicians were against using the computer while the patient was present. This opposition prevented them from regarding the system as part of the medical consultation and from considering data from the system as direct patient-related signs.

CONCLUSIONS

The system's implementation modified the physicians' views on computerized ambulatory medical records, as they eventually considered them as an ancillary tool to clinical activity. The value assigned to the system depends on its relevance within the institutional framework.