Use, Perceived Usability, and Barriers to Implementation of a Patient Safety Dashboard Integrated within a Vendor EHR

Engaging Multidisciplinary Clinical Users in the Design of an Artificial Intelligence-Powered Graphical User Interface for Intensive Care Unit Instability Decision Support

BACKGROUND

Critical instability forecast and treatment can be optimized by artificial intelligence (AI)-enabled clinical decision support. It is important that the user-facing display of AI output facilitates clinical thinking and workflow for all disciplines involved in bedside care.

OBJECTIVES

Our objective is to engage multidisciplinary users (physicians, nurse practitioners, physician assistants) in the development of a graphical user interface (GUI) to present an AI-derived risk score.

METHODS

Intensive care unit (ICU) clinicians participated in focus groups seeking input on instability risk forecast presented in a prototype GUI. Two stratified rounds (three focus groups [only nurses, only providers, then combined]) were moderated by a focus group methodologist. After round 1, GUI design changes were made and presented in round 2. Focus groups were recorded, transcribed, and deidentified transcripts independently coded by three researchers. Codes were coalesced into emerging themes.

RESULTS

Twenty-three ICU clinicians participated (11 nurses, 12 medical providers [3 mid-level and 9 physicians]). Six themes emerged: (1) analytics transparency, (2) graphical interpretability, (3) impact on practice, (4) value of trend synthesis of dynamic patient data, (5) decisional weight (weighing AI output during decision-making), and (6) display location (usability, concerns for patient/family GUI view). Nurses emphasized having GUI objective information to support communication and optimal GUI location. While providers emphasized need for recommendation interpretability and concern for impairing trainee critical thinking. All disciplines valued synthesized views of vital signs, interventions, and risk trends but were skeptical of placing decisional weight on AI output until proven trustworthy.

CONCLUSION

Gaining input from all clinical users is important to consider when designing AI-derived GUIs. Results highlight that health care intelligent decisional support systems technologies need to be transparent on how they work, easy to read and interpret, cause little disruption to current workflow, as well as decisional support components need to be used as an adjunct to human decision-making.

Identifying barriers and facilitators to successful implementation of computerized clinical decision support systems in hospitals: a NASSS framework-informed scoping review

BACKGROUND

Successful implementation and utilization of Computerized Clinical Decision Support Systems (CDSS) in hospitals is complex and challenging. Implementation science, and in particular the Nonadoption, Abandonment, Scale-up, Spread and Sustainability (NASSS) framework, may offer a systematic approach for identifying and addressing these challenges. This review aimed to identify, categorize, and describe barriers and facilitators to CDSS implementation in hospital settings and map them to the NASSS framework. Exploring the applicability of the NASSS framework to CDSS implementation was a secondary aim.

METHODS

Electronic database searches were conducted (21 July 2020; updated 5 April 2022) in Ovid MEDLINE, Embase, Scopus, PyscInfo, and CINAHL. Original research studies reporting on measured or perceived barriers and/or facilitators to implementation and adoption of CDSS in hospital settings, or attitudes of healthcare professionals towards CDSS were included. Articles with a primary focus on CDSS development were excluded. No language or date restrictions were applied. We used qualitative content analysis to identify determinants and organize them into higher-order themes, which were then reflexively mapped to the NASSS framework.

RESULTS

Forty-four publications were included. These comprised a range of study designs, geographic locations, participants, technology types, CDSS functions, and clinical contexts of implementation. A total of 227 individual barriers and 130 individual facilitators were identified across the included studies. The most commonly reported influences on implementation were fit of CDSS with workflows (19 studies), the usefulness of the CDSS output in practice (17 studies), CDSS technical dependencies and design (16 studies), trust of users in the CDSS input data and evidence base (15 studies), and the contextual fit of the CDSS with the user's role or clinical setting (14 studies). Most determinants could be appropriately categorized into domains of the NASSS framework with barriers and facilitators in the "Technology," "Organization," and "Adopters" domains most frequently reported. No determinants were assigned to the "Embedding and Adaptation Over Time" domain.

CONCLUSIONS

This review identified the most common determinants which could be targeted for modification to either remove barriers or facilitate the adoption and use of CDSS within hospitals. Greater adoption of implementation theory should be encouraged to support CDSS implementation.

Developing, pilot testing, and refining requirements for 3 EHR-integrated interventions to improve diagnostic safety in acute care: a user-centered approach

Objective

To describe a user-centered approach to develop, pilot test, and refine requirements for 3 electronic health record (EHR)-integrated interventions that target key diagnostic process failures in hospitalized patients.

Materials and Methods

Three interventions were prioritized for development: a Diagnostic Safety Column (DSC) within an EHR-integrated dashboard to identify at-risk patients; a Diagnostic Time-Out (DTO) for clinicians to reassess the working diagnosis; and a Patient Diagnosis Questionnaire (PDQ) to gather patient concerns about the diagnostic process. Initial requirements were refined from analysis of test cases with elevated risk predicted by DSC logic compared to risk perceived by a clinician working group; DTO testing sessions with clinicians; PDQ responses from patients; and focus groups with clinicians and patient advisors using storyboarding to model the integrated interventions. Mixed methods analysis of participant responses was used to identify final requirements and potential implementation barriers.

Results

Final requirements from analysis of 10 test cases predicted by the DSC, 18 clinician DTO participants, and 39 PDQ responses included the following: DSC configurable parameters (variables, weights) to adjust baseline risk estimates in real-time based on new clinical data collected during hospitalization; more concise DTO wording and flexibility for clinicians to conduct the DTO with or without the patient present; and integration of PDQ responses into the DSC to ensure closed-looped communication with clinicians. Analysis of focus groups confirmed that tight integration of the interventions with the EHR would be necessary to prompt clinicians to reconsider the working diagnosis in cases with elevated diagnostic error (DE) risk or uncertainty. Potential implementation barriers included alert fatigue and distrust of the risk algorithm (DSC); time constraints, redundancies, and concerns about disclosing uncertainty to patients (DTO); and patient disagreement with the care team's diagnosis (PDQ).

Discussion

A user-centered approach led to evolution of requirements for 3 interventions targeting key diagnostic process failures in hospitalized patients at risk for DE.

Conclusions

We identify challenges and offer lessons from our user-centered design process.

Novel Note Templates to Enhance Signal and Reduce Noise in Medical Documentation: a Prospective Improvement Study

BACKGROUND

The introduction of electronic workflows has allowed for the flow of raw un-contextualized clinical data into medical documentation. As a result, many electronic notes have become replete of "noise" and deplete of clinically significant "signals". There is an urgent need to develop and implement innovative approaches in electronic clinical documentation that improve note quality and reduce unnecessary bloating.

OBJECTIVE

To describe the development and impact of a novel set of templates designed to change the flow of information in medical documentation.

METHODS

This is a multi-hospital nonrandomized prospective improvement study conducted on the Inpatient General Internal Medicine Service across three hospital campuses at the New York University (NYU) Langone Health System. A group of physician leaders representing each campus met biweekly for six months. The output of these meetings included 1) a conceptualization of the note bloat problem as a dysfunction in information flow 2) a set of guiding principles for organizational documentation improvement 3) the design and build of novel electronic templates that reduced the flow of extraneous information into provider notes by providing link outs to best practice data visualizations and 4) a documentation improvement curriculum for inpatient medicine providers. Prior to go-live, pragmatic usability testing was performed with the new progress note template, and the overall user experience measured using the System Usability Scale (SUS). Primary outcomes measures after go-live include template utilization rate and note length in characters.

RESULTS

In usability testing amongst 22 medicine providers, the new progress note template averaged a usability score of 90.6/100 on the System Usability Scale. 77% of providers strongly agreed that the new template was easy to use. 68% strongly agreed that they would like to use the template frequently. In the three months after template implementation, General Internal Medicine providers wrote 65% of all inpatient notes with the new templates. During this period of time the organization saw a 46%, 47%, and 32% reduction in note length for general medicine progress notes, consults, and H&Ps, respectively, when compared to a baseline measurement period prior to interventions.

CONCLUSIONS

A bundled intervention that included deployment of novel templates for inpatient general medicine providers significantly reduced average note length on the clinical service. Templates designed to reduce the flow of extraneous information into provider notes performed well during usability testing, and these templates were rapidly adopted across all hospital campuses. Further research is needed to assess the impact of novel templates on note quality, provider efficiency and patient outcomes.

CLINICALTRIAL

Triptychon: Usability evaluation and implementation of a web-based application for patients' lab and vital parameters

Background

A major challenge in healthcare is the interpretation of the constantly increasing amount of clinical data of interest to inpatients for diagnosis and therapy. It is vital to accurately structure and represent data from different sources to help clinicians make informed decisions.

Objective

We evaluated the usability of our tool 'Triptychon' - a three-part visualisation dashboard of essential patients' medical data provided by a direct overview of their hospitalisation information, laboratory, and vital parameters over time.

Methods

The study followed a cohort of 20 participants using the mixed-methods approach, including interviews and the usability questionnaires, Health Information Technology Usability Evaluation Scale (Health-ITUES), and User Experience Questionnaire (UEQ). The participant's interactions with the dashboard were also observed. A thematic analysis approach was applied to analyse qualitative data and the quantitative data's task completion time and success rates.

Results

The usability evaluation of the visualisation dashboard revealed issues relating to the terminology used in the user interface and colour coding in its left and middle panels. The Health-ITUES score was 3.72 (standard deviation (SD) = 1.0), and the UEQ score was 1.6 (SD = 0.74). The study demonstrated improvements in intuitive dashboard use and overall satisfaction with using the dashboard daily.

Conclusion

The Triptychon dashboard is a promising new tool for medical data presentation. We identified design and layout issues of the dashboard for improving its usability in routine clinical practice. According to users' feedback, the three panels on the dashboard provided a holistic view of a patient's hospital stay.

Usability Evaluation of Dashboards: A Systematic Literature Review of Tools

Introduction

In recent years, the use of dashboards in healthcare has been considered an effective approach for the visual presentation of information to support clinical and administrative decisions. Effective and efficient use of dashboards in clinical and managerial processes requires a framework for the design and development of tools based on usability principles.

Objectives

The present study is aimed at investigating the existing questionnaires used for the usability evaluation framework of dashboards and at presenting more specific usability criteria for evaluating dashboards.

Methods

This systematic review was conducted using PubMed, Web of Science, and Scopus, without any time restrictions. The final search of articles was performed on September 2, 2022. Data collection was performed using a data extraction form, and the content of selected studies was analyzed based on the dashboard usability criteria.

Results

After reviewing the full text of relevant articles, a total of 29 studies were selected according to the inclusion criteria. Regarding the questionnaires used in the selected studies, researcher-made questionnaires were used in five studies, while 25 studies applied previously used questionnaires. The most widely used questionnaires were the System Usability Scale (SUS), Technology Acceptance Model (TAM), Situation Awareness Rating Technique (SART), Questionnaire for User Interaction Satisfaction (QUIS), Unified Theory of Acceptance and Use of Technology (UTAUT), and Health Information Technology Usability Evaluation Scale (Health-ITUES), respectively. Finally, dashboard evaluation criteria, including usefulness, operability, learnability, ease of use, suitability for tasks, improvement of situational awareness, satisfaction, user interface, content, and system capabilities, were suggested.

Conclusion

General questionnaires that were not specifically designed for dashboard evaluation were mainly used in reviewed studies. The current study suggested specific criteria for measuring the usability of dashboards. When selecting the usability evaluation criteria for dashboards, it is important to pay attention to the evaluation objectives, dashboard features and capabilities, and context of use.

Real-time analysis and display of quantitative measures to track and improve clinical workflow

Purpose

Radiotherapy treatment planning is a complex process with multiple, dependent steps involving an interdisciplinary patient care team. Effective communication and real‐time tracking of resources and care path activities are key for clinical efficiency and patient safety.

Materials and Methods

We designed and implemented a secure, interactive web‐based dashboard for patient care path, clinical workflow, and resource utilization management. The dashboard enables visualization of resource utilization and tracks progress in a patient's care path from the time of acquisition of the planning CT to the time of treatment in real‐time. It integrates with the departmental electronic medical records (EMR) system without the creation and maintenance of a separate database or duplication of work by clinical staff. Performance measures of workflow were calculated.

Results

The dashboard implements a standardized clinical workflow and dynamically consolidates real‐time information queried from multiple tables in the EMR database over the following views: (1) CT Sims summarizes patient appointment information on the CT simulator and patient load; (2) Linac Sims summarizes patient appointment times, setup history, and notes, and patient load; (3) Task Status lists the clinical tasks associated with a treatment plan, their due date, status and ownership, and patient appointment details; (4) Documents provides the status of all documents in the patients' charts; and (5) Diagnoses and Interventions summarizes prescription information, imaging instructions and whether the plan was approved for treatment. Real‐time assessment and quantification of progress and delays in a patient's treatment start were achieved.

Conclusions

This study indicates it is feasible to develop and implement a dashboard, tailored to the needs of an interdisciplinary team, which derives and integrates information from the EMR database for real‐time analysis and display of resource utilization and clinical workflow in radiation oncology. The framework developed facilitates informed, data‐driven decisions on clinical workflow management as we seek to optimize clinical efficiency and patient safety.

Developing the VA Geriatric Scholars Programs' Clinical Dashboards Using the PDSA Framework for Quality Improvement

Background  Involving clinician end users in the development process of clinical dashboards is important to ensure that user needs are adequately met prior to releasing the dashboard for use. The challenge with following this approach is that clinician end users can undergo periodic turnover, meaning, the clinicians that played a role in the initial development process may not be the same individuals that use the dashboard in future.

Objectives  Here, we summarize our Plan, Do, Study, Act (PDSA)-guided clinical dashboard development process for the VA Geriatric Scholars Program (GSP) and the value of continuous, iterative development. We summarize dashboard adaptations that resulted from two PDSA cycles of improvement for the potentially inappropriate medication dashboard (PIMD), one of many Geriatric Scholars clinical dashboards. We also present the evaluative performance of the PIMD.

Methods  Evaluation of the PIMD was performed using the system usability scale (SUS) and through review of user interaction logs. Routine end users that were Geriatric Scholars and had evidence of 5 or more dashboard views were invited to complete an electronic form that contained the 10-item SUS.

Results  The proportion of Geriatric Scholars that utilized the PIMD increased for each iterative dashboard version that was produced as a byproduct from feedback (31.0% in 2017 to 60.2% in 2019). The overall usability of the PIMD among routine users was found to be above average (SUS score: 75.2 [95% CI 70.5–79.8]) in comparison to the recommended standard of acceptability (SUS score: 68)

Conclusion  The solicitation of feedback during dashboard orientations led to iterative adaptations of the PIMD that broadened its intended use. The presented PDSA-guided process to clinical dashboard development for the VA GSP can serve as a valuable framework for development teams seeking to produce well-adopted and usable health information technology (IT) innovations.

Dashboard Design to Identify and Balance Competing Risk of Multiple Hospital-Acquired Conditions

BACKGROUND

Hospital-acquired conditions (HACs) are common, costly, and national patient safety priority. Catheter-associated urinary tract infections (CAUTIs), hospital-acquired pressure injury (HAPI), and falls are common HACs. Clinicians assess each HAC risk independent of other conditions. Prevention strategies often focus on the reduction of a single HAC rather than considering how actions to prevent one condition could have unintended consequences for another HAC.

OBJECTIVES

The objective of this study is to design an empirical framework to identify, assess, and quantify the risks of multiple HACs (MHACs) related to competing single-HAC interventions.

METHODS

This study was an Institutional Review Board approved, and the proof of concept study evaluated MHAC Competing Risk Dashboard to enhance clinicians' management combining the risks of CAUTI, HAPI, and falls. The empirical model informing this study focused on the removal of an indwelling urinary catheter to reduce CAUTI, which may impact HAPI and falls. A multisite database was developed to understand and quantify competing risks of HACs; a predictive model dashboard was designed and clinical utility of a high-fidelity dashboard was qualitatively tested. Five hospital systems provided data for the predictive model prototype; three served as sites for testing and feedback on the dashboard design and usefulness. The participatory study design involved think-aloud methods as the clinician explored the dashboard. Individual interviews provided an understanding of clinician's perspective regarding ease of use and utility.

RESULTS

Twenty-five clinicians were interviewed. Clinicians favored a dashboard gauge design composed of green, yellow, and red segments to depict MHAC risk associated with the removal of an indwelling urinary catheter to reduce CAUTI and possible adverse effects on HAPI and falls.

CONCLUSION

Participants endorsed the utility of a visual dashboard guiding clinical decisions for MHAC risks preferring common stoplight color understanding. Clinicians did not want mandatory alerts for tool integration into the electronic health record. More research is needed to understand MHAC and tools to guide clinician decisions.

Toward a Learning Health Care System: A Systematic Review and Evidence-Based Conceptual Framework for Implementation of Clinical Analytics in a Digital Hospital

Objective  A learning health care system (LHS) uses routinely collected data to continuously monitor and improve health care outcomes. Little is reported on the challenges and methods used to implement the analytics underpinning an LHS. Our aim was to systematically review the literature for reports of real-time clinical analytics implementation in digital hospitals and to use these findings to synthesize a conceptual framework for LHS implementation.

Methods  Embase, PubMed, and Web of Science databases were searched for clinical analytics derived from electronic health records in adult inpatient and emergency department settings between 2015 and 2021. Evidence was coded from the final study selection that related to (1) dashboard implementation challenges, (2) methods to overcome implementation challenges, and (3) dashboard assessment and impact. The evidences obtained, together with evidence extracted from relevant prior reviews, were mapped to an existing digital health transformation model to derive a conceptual framework for LHS analytics implementation.

Results  A total of 238 candidate articles were reviewed and 14 met inclusion criteria. From the selected studies, we extracted 37 implementation challenges and 64 methods employed to overcome such challenges. We identified common approaches for evaluating the implementation of clinical dashboards. Six studies assessed clinical process outcomes and only four studies evaluated patient health outcomes. A conceptual framework for implementing the analytics of an LHS was developed.

Conclusion  Health care organizations face diverse challenges when trying to implement real-time data analytics. These challenges have shifted over the past decade. While prior reviews identified fundamental information problems, such as data size and complexity, our review uncovered more postpilot challenges, such as supporting diverse users, workflows, and user-interface screens. Our review identified practical methods to overcome these challenges which have been incorporated into a conceptual framework. It is hoped this framework will support health care organizations deploying near-real-time clinical dashboards and progress toward an LHS.

Visualizing Opioid-Use Variation in a Pediatric Perioperative Dashboard

BACKGROUND

Anesthesiologists integrate numerous variables to determine an opioid dose that manages patient nociception and pain while minimizing adverse effects. Clinical dashboards that enable physicians to compare themselves to their peers can reduce unnecessary variation in patient care and improve outcomes. However, due to the complexity of anesthetic dosing decisions, comparative visualizations of opioid-use patterns are complicated by case-mix differences between providers.

OBJECTIVES

This single-institution case study describes the development of a pediatric anesthesia dashboard and demonstrates how advanced computational techniques can facilitate nuanced normalization techniques, enabling meaningful comparisons of complex clinical data.

METHODS

We engaged perioperative-care stakeholders at a tertiary care pediatric hospital to determine patient and surgical variables relevant to anesthesia decision-making and to identify end-user requirements for an opioid-use visualization tool. Case data were extracted, aggregated, and standardized. We performed multivariable machine learning to identify and understand key variables. We integrated interview findings and computational algorithms into an interactive dashboard with normalized comparisons, followed by an iterative process of improvement and implementation.

RESULTS

The dashboard design process identified two mechanisms-interactive data filtration and machine-learning-based normalization-that enable rigorous monitoring of opioid utilization with meaningful case-mix adjustment. When deployed with real data encompassing 24,332 surgical cases, our dashboard identified both high and low opioid-use outliers with associated clinical outcomes data.

CONCLUSION

A tool that gives anesthesiologists timely data on their practice patterns while adjusting for case-mix differences empowers physicians to track changes and variation in opioid administration over time. Such a tool can successfully trigger conversation amongst stakeholders in support of continuous improvement efforts. Clinical analytics dashboards can enable physicians to better understand their practice and provide motivation to change behavior, ultimately addressing unnecessary variation in high impact medication use and minimizing adverse effects.

Situational Awareness of Opioid Consumption: The Missing Link to Reducing Dependence After Surgery?

A tool for collecting and analyzing morphine milligram equivalents (MMEs) can be used to overcome barriers to situational awareness around opioid utilization in the setting of multimodal pain management. Our software application (App) has facilitated data collection, analysis, and benchmarking in a manner that is not logistically feasible using manual methods. Real-time postoperative tracking of MME over the course of an episode of care can be prohibitively labor-intensive, and teams must have practical strategies to overcome this obstacle. In view of the link between the magnitude of opioid prescriptions at discharge and persistent opioid use after cardiac surgery, we believe that improving situational awareness among the patient care team is a vital first step in reducing opioid dependence after cardiac surgery.

OUP accepted manuscript

Objectives

(1) Systematically review the literature on computerized audit and feedback (e-A&F) systems in healthcare. (2) Compare features of current systems against e-A&F best practices. (3) Generate hypotheses on how e-A&F systems may impact patient care and outcomes.

Methods

We searched MEDLINE (Ovid), EMBASE (Ovid), and CINAHL (Ebsco) databases to December 31, 2020. Two reviewers independently performed selection, extraction, and quality appraisal (Mixed Methods Appraisal Tool). System features were compared with 18 best practices derived from Clinical Performance Feedback Intervention Theory. We then used realist concepts to generate hypotheses on mechanisms of e-A&F impact. Results are reported in accordance with the PRISMA statement.

Results

Our search yielded 4301 unique articles. We included 88 studies evaluating 65 e-A&F systems, spanning a diverse range of clinical areas, including medical, surgical, general practice, etc. Systems adopted a median of 8 best practices (interquartile range 6–10), with 32 systems providing near real-time feedback data and 20 systems incorporating action planning. High-confidence hypotheses suggested that favorable e-A&F systems prompted specific actions, particularly enabled by timely and role-specific feedback (including patient lists and individual performance data) and embedded action plans, in order to improve system usage, care quality, and patient outcomes.

Conclusions

e-A&F systems continue to be developed for many clinical applications. Yet, several systems still lack basic features recommended by best practice, such as timely feedback and action planning. Systems should focus on actionability, by providing real-time data for feedback that is specific to user roles, with embedded action plans.

Protocol Registration

PROSPERO CRD42016048695.

User-centered design of a scalable, electronic health record-integrated remote symptom monitoring intervention for patients with asthma and providers in primary care

OBJECTIVE

To determine user and electronic health records (EHR) integration requirements for a scalable remote symptom monitoring intervention for asthma patients and their providers.

METHODS

Guided by the Non-Adoption, Abandonment, Scale-up, Spread, and Sustainability (NASSS) framework, we conducted a user-centered design process involving English- and Spanish-speaking patients and providers affiliated with an academic medical center. We conducted a secondary analysis of interview transcripts from our prior study, new design sessions with patients and primary care providers (PCPs), and a survey of PCPs. We determined EHR integration requirements as part of the asthma app design and development process.

RESULTS

Analysis of 26 transcripts (21 patients, 5 providers) from the prior study, 21 new design sessions (15 patients, 6 providers), and survey responses from 55 PCPs (71% of 78) identified requirements. Patient-facing requirements included: 1- or 5-item symptom questionnaires each week, depending on asthma control; option to request a callback; ability to enter notes, triggers, and peak flows; and tips pushed via the app prior to a clinic visit. PCP-facing requirements included a clinician-facing dashboard accessible from the EHR and an EHR inbox message preceding the visit. PCP preferences diverged regarding graphical presentations of patient-reported outcomes (PROs). Nurse-facing requirements included callback requests sent as an EHR inbox message. Requirements were consistent for English- and Spanish-speaking patients. EHR integration required use of custom application programming interfaces (APIs).

CONCLUSION

Using the NASSS framework to guide our user-centered design process, we identified patient and provider requirements for scaling an EHR-integrated remote symptom monitoring intervention in primary care. These requirements met the needs of patients and providers. Additional standards for PRO displays and EHR inbox APIs are needed to facilitate spread.

Clinical Information Systems Research in the Pandemic Year 2020

Objective: In this synopsis, we give an overview of recent research and propose a selection of best papers published in 2020 in the field of Clinical Information Systems (CIS).

Method: As CIS section editors, we annually apply a systematic process to retrieve articles for the International Medical Informatics Association Yearbook of Medical Informatics. For seven years now, we use the same query to find relevant publications in the CIS field. Each year we retrieve more than 2,400 papers which we categorize in a multi-pass review to distill a preselection of 15 candidate papers. External reviewers and yearbook editors then assess the selected candidate papers. Based on the review results, the IMIA Yearbook editorial board chooses up to four best publications for the section at a selection meeting. To get an overview of the content of the retrieved articles, we use text mining and term co-occurrence mapping techniques.

Results: We carried out the query in mid-January 2021 and retrieved a deduplicated result set of 2,787 articles from 1,135 different journals. We nominated 15 papers as candidates and finally selected four of them as the best papers in the CIS section. As in the previous years, the content analysis of the articles revealed the broad spectrum of topics covered by CIS research. Thus, this year we could observe a significant impact of COVID-19 on CIS research.

Conclusions: The trends in CIS research, as seen in recent years, continue to be observable. What was very visible was the impact of the Corona Virus Disease 2019 (COVID-19) pandemic, which has affected not only our lives but also CIS.

Human Factors and Organizational Issues in Health Informatics: Innovations and Opportunities

OBJECTIVE

Human factors and ergonomics (HF/E) frameworks and methods are becoming embedded in the health informatics community. There is now broad recognition that health informatics tools must account for the diverse needs, characteristics, and abilities of end users, as well as their context of use. The objective of this review is to synthesize the current nature and scope of HF/E integration into the health informatics community.

METHODS

Because the focus of this synthesis is on understanding the current integration of the HF/E and health informatics research communities, we manually reviewed all manuscripts published in primary HF/E and health informatics journals during 2020.

RESULTS

HF/E-focused health informatics studies included in this synthesis focused heavily on EHR customizations, specifically clinical decision support customizations and customized data displays, and on mobile health innovations. While HF/E methods aimed to jointly improve end user safety, performance, and satisfaction, most HF/E-focused health informatics studies measured only end user satisfaction.

CONCLUSION

HF/E-focused health informatics researchers need to identify and communicate methodological standards specific to health informatics, to better synthesize findings across resource intensive HF/E-focused health informatics studies. Important gaps in the HF/E design and evaluation process should be addressed in future work, including support for technology development platforms and training programs so that health informatics designers are as diverse as end users.

Interdisciplinary collaboration between nursing and engineering in health care: A scoping review

BACKGROUND

Due to the rapid advancements in precision medicine and artificial intelligence, interdisciplinary collaborations between nursing and engineering have emerged. Although engineering is vital in solving complex nursing problems and advancing healthcare, the collaboration between the two fields has not been fully elucidated.

OBJECTIVES

To identify the study areas of interdisciplinary collaboration between nursing and engineering in health care, particularly focusing on the role of nurses in the collaboration.

METHODS

In this study, a scoping review using the Preferred Reporting Items for Systematic Reviews and Meta-Analysis Extension for Scoping Reviews was performed. A comprehensive search for published literature was conducted using the PubMed, Cumulative Index to Nursing and Allied Health Literature, Scopus, Embase, Web of Science, ScienceDirect, Institute of Electrical and Electronics Engineers Digital Library, and Association for Computing Machinery Digital Library from inception to November 22, 2020. Data screening and extraction were performed independently by two reviewers. Any discrepancies in results were resolved through discussions or in consultation with a third reviewer. Data were analyzed by descriptive statistics and content analysis. Results were visualized in an interdisciplinary collaboration model.

RESULTS

We identified 6,752 studies through the literature search, and 60 studies met the inclusion criteria. The study areas of interdisciplinary collaboration concentrated on patient safety (n = 18), symptom monitoring and health management (n = 18), information system and nursing human resource management (n = 16), health education (n = 5), and nurse-patient communication (n = 3). The roles of nurses in the interdisciplinary collaboration were divided into four themes: requirement analyst (n = 21), designer (n = 22), tester(n = 37) and evaluator (n = 49). Based on these results, an interdisciplinary collaboration model was constructed.

CONCLUSIONS

Interdisciplinary collaborations between nursing and engineering promote nursing innovation and practice. However, these collaborations are still emerging and in the early stages. In the future, nurses should be more involved in the early stages of solving healthcare problems, particularly in the requirement analysis and designing phases. Furthermore, there is an urgent need to develop interprofessional education, strengthen nursing connections with the healthcare engineering industry, and provide more platforms and resources to bring nursing and engineering disciplines together.

Implementation of a stepped wedge cluster randomized trial to evaluate a hospital mobility program

Background

Stepped wedge cluster randomized trials (SW-CRT) are increasingly used to evaluate new clinical programs, yet there is limited guidance on practical aspects of applying this design. We report our early experiences conducting a SW-CRT to examine an inpatient mobility program (STRIDE) in the Veterans Health Administration (VHA). We provide recommendations for future research using this design to evaluate clinical programs.

Methods

Based on data from study records and reflections from the investigator team, we describe and assess the design and initial stages of a SW-CRT, from site recruitment to program launch in 8 VHA hospitals.

Results

Site recruitment consisted of thirty 1-h conference calls with representatives from 22 individual VAs who expressed interest in implementing STRIDE. Of these, 8 hospitals were enrolled and randomly assigned in two stratified blocks (4 hospitals per block) to a STRIDE launch date. Block 1 randomization occurred in July 2017 with first STRIDE launch in December 2017; block 2 randomization occurred in April 2018 with first STRIDE launch in January 2019. The primary study outcome of discharge destination will be assessed using routinely collected data in the electronic health record (EHR). Within randomized blocks, two hospitals per sequence launched STRIDE approximately every 3 months with primary outcome assessment paused during the 3-month time period of program launch. All sites received 6–8 implementation support calls, according to a pre-specified schedule, from the time of recruitment to program launch, and all 8 sites successfully launched within their assigned 3-month window. Seven of the eight sites initially started with a limited roll out (for example on one ward) or modified version of STRIDE (for example, using existing staff to conduct walks until new positions were filled).

Conclusions

Future studies should incorporate sufficient time for site recruitment and carefully consider the following to inform design of SW-CRTs to evaluate rollout of a new clinical program: (1) whether a blocked randomization fits study needs, (2) the amount of time and implementation support sites will need to start their programs, and (3) whether clinical programs are likely to include a “ramp-up” period. Successful execution of SW-CRT designs requires both adherence to rigorous design principles and also careful consideration of logistical requirements for timing of program roll out.

Trial registration

ClinicalsTrials.gov NCT03300336. Prospectively registered on 3 October 2017.

The Effect of Electronic Health Record Usability Redesign on Annual Screening Rates in an Ambulatory Setting

OBJECTIVES

Improving the usability of electronic health records (EHR) continues to be a focus of clinicians, vendors, researchers, and regulatory bodies. To understand the impact of usability redesign of an existing, site-configurable feature, we evaluated the user interface (UI) used to screen for depression, alcohol and drug misuse, fall risk, and the existence of advance directive information in ambulatory settings.

METHODS

As part of a quality improvement project, based on heuristic analysis, the existing UI was redesigned. Using an iterative, user-centered design process, several usability defects were corrected. Summative usability testing was performed as part of the product development and implementation cycle. Clinical quality measures reflecting rolling 12-month rates of screening were examined over 8 months prior to the implementation of the redesigned UI and 9 months after implementation.

RESULTS

Summative usability testing demonstrated improvements in task time, error rates, and System Usability Scale scores. Interrupted time series analysis demonstrated significant improvements in all screening rates after implementation of the redesigned UI compared with the original implementation.

CONCLUSION

User-centered redesign of an existing site-specific UI may lead to significant improvements in measures of usability and quality of patient care.

Interactive Digital Health Tools to Engage Patients and Caregivers in Discharge Preparation: Implementation Study

BACKGROUND

Poor discharge preparation during hospitalization may lead to adverse events after discharge. Checklists and videos that systematically engage patients in preparing for discharge have the potential to improve safety, especially when integrated into clinician workflow via the electronic health record (EHR).

OBJECTIVE

This study aims to evaluate the implementation of a suite of digital health tools integrated with the EHR to engage hospitalized patients, caregivers, and their care team in preparing for discharge.

METHODS

We used the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework to identify pertinent research questions related to implementation. We iteratively refined patient and clinician-facing intervention components using a participatory process involving end users and institutional stakeholders. The intervention was implemented at a large academic medical center from December 2017 to July 2018. Patients who agreed to participate were coached to watch a discharge video, complete a checklist assessing discharge readiness, and request postdischarge text messaging with a physician 24 to 48 hours before their expected discharge date, which was displayed via a patient portal and bedside display. Clinicians could view concerns reported by patients based on their checklist responses in real time via a safety dashboard integrated with the EHR and choose to open a secure messaging thread with the patient for up to 7 days after discharge. We used mixed methods to evaluate our implementation experience.

RESULTS

Of 752 patient admissions, 510 (67.8%) patients or caregivers participated: 416 (55.3%) watched the video and completed the checklist, and 94 (12.5%) completed the checklist alone. On average, 4.24 concerns were reported per each of the 510 checklist submissions, most commonly about medications (664/2164, 30.7%) and follow-up (656/2164, 30.3%). Of the 510 completed checklists, a member of the care team accessed the safety dashboard to view 210 (41.2%) patient-reported concerns. For 422 patient admissions where postdischarge messaging was available, 141 (33.4%) patients requested this service; of these, a physician initiated secure messaging for 3 (2.1%) discharges. Most patient survey participants perceived that the intervention promoted self-management and communication with their care team. Patient interview participants endorsed gaps in communication with their care team and thought that the video and checklist would be useful closer toward discharge. Clinicians participating in focus groups perceived the value for patients but suggested that low awareness and variable workflow regarding the intervention, lack of technical optimization, and inconsistent clinician leadership limited the use of clinician-facing components.

CONCLUSIONS

A suite of EHR-integrated digital health tools to engage patients, caregivers, and clinicians in discharge preparation during hospitalization was feasible, acceptable, and valuable; however, important challenges were identified during implementation. We offer strategies to address implementation barriers and promote adoption of these tools.

TRIAL REGISTRATION

ClinicalTrials.gov NCT03116074; https://clinicaltrials.gov/ct2/show/NCT03116074.