Development, implementation and preliminary evaluation of clinical dashboards in a department of anesthesia

Quality Dashboards in Hospital Settings: A Systematic Review With Implications for Nurses

BACKGROUND

Dashboards visually display quality and safety data to aid nurses in making informed decisions.

PURPOSE

This systematic review evaluated quality improvement (QI) dashboard characteristics associated with interventions to improve patient outcomes and positive end-user evaluation.

METHODS

Literature was searched from 2012 to 2022 in PubMed, CINAHL, Scopus, MEDLINE, and Google Scholar.

RESULTS

Sixteen articles were included. Varied dashboard characteristics were noted, with mixed patient outcomes and end-user responses. Graphs and tabular presentations were associated with improved patient outcomes, whereas graphs were associated with end-user satisfaction. Benchmarks were noted with improved patient outcomes but not end-user satisfaction. Interactive dashboards were important for end users and improved patient outcomes.

CONCLUSION

Nurses can find dashboards helpful in guiding QI projects. Dashboards may include graphs and/or tables, benchmarks, and interactivity but should be useful, usable, and aligned to unit needs. Future research should focus on the use of quality dashboards in nursing practice.

Dashboard of Short-Term Postoperative Patient Outcomes for Anesthesiologists: Development and Preliminary Evaluation

BACKGROUND

Anesthesiologists require an understanding of their patients' outcomes to evaluate their performance and improve their practice. Traditionally, anesthesiologists had limited information about their surgical outpatients' outcomes due to minimal contact post discharge. Leveraging digital health innovations for analyzing personal and population outcomes may improve perioperative care. BC Children's Hospital's postoperative follow-up registry for outpatient surgeries collects short-term outcomes such as pain, nausea, and vomiting. Yet, these data were previously not available to anesthesiologists.

OBJECTIVE

This quality improvement study aimed to visualize postoperative outcome data to allow anesthesiologists to reflect on their care and compare their performance with their peers.

METHODS

The postoperative follow-up registry contains nurse-reported postoperative outcomes, including opioid and antiemetic administration in the postanesthetic care unit (PACU), and family-reported outcomes, including pain, nausea, and vomiting, within 24 hours post discharge. Dashboards were iteratively co-designed with 5 anesthesiologists, and a department-wide usability survey gathered anesthesiologists' feedback on the dashboards, allowing further design improvements. A final dashboard version has been deployed, with data updated weekly.

RESULTS

The dashboard contains three sections: (1) 24-hour outcomes, (2) PACU outcomes, and (3) a practice profile containing individual anesthesiologist's case mix, grouped by age groups, sex, and surgical service. At the time of evaluation, the dashboard included 24-hour data from 7877 cases collected from September 2020 to February 2023 and PACU data from 8716 cases collected from April 2021 to February 2023. The co-design process and usability evaluation indicated that anesthesiologists preferred simpler designs for data summaries but also required the ability to explore details of specific outcomes and cases if needed. Anesthesiologists considered security and confidentiality to be key features of the design and most deemed the dashboard information useful and potentially beneficial for their practice.

CONCLUSIONS

We designed and deployed a dynamic, personalized dashboard for anesthesiologists to review their outpatients' short-term postoperative outcomes. This dashboard facilitates personal reflection on individual practice in the context of peer and departmental performance and, hence, the opportunity to evaluate iterative practice changes. Further work is required to establish their effect on improving individual and department performance and patient outcomes.

Anesthesia quality indicators to measure and improve your practice: a modified delphi study

BACKGROUND

Implementation of the new competency-based post-graduate medical education curriculum has renewed the push by medical regulatory bodies in Canada to strongly advocate and/or mandate continuous quality improvement (cQI) for all physicians. Electronic anesthesia information management systems contain vast amounts of information yet it is unclear how this information could be used to promote cQI for practicing anesthesiologists. The aim of this study was to create a refined list of meaningful anesthesia quality indicators to assist anesthesiologists in the process of continuous self-assessment and feedback of their practice.

METHODS

An initial list of quality indicators was created though a literature search. A modified-Delphi (mDelphi) method was used to rank these indicators and achieve consensus on those indicators considered to be most relevant. Fourteen anesthesiologists representing different regions across Canada participated in the panel.

RESULTS

The initial list contained 132 items and through 3 rounds of mDelphi the panelists selected 56 items from the list that they believed to be top priority. In the fourth round, a subset of 20 of these indicators were ranked as highest priority. The list included items related to process, structure and outcome.

CONCLUSION

This ranked list of anesthesia quality indicators from this modified Delphi study could aid clinicians in their individual practice assessments for continuous quality improvement mandated by Canadian medical regulatory bodies. Feasibility and usability of these quality indicators, and the significance of process versus outcome measures in assessment, are areas of future research.

Development and usage of an anesthesia data warehouse: lessons learnt from a 10-year project

This paper describes the development and implementation of an anesthesia data warehouse in the Lille University Hospital. We share the lessons learned from a ten-year project and provide guidance for the implementation of such a project. Our clinical data warehouse is mainly fed with data collected by the anesthesia information management system and hospital discharge reports. The data warehouse stores historical and accurate data with an accuracy level of the day for administrative data, and of the second for monitoring data. Datamarts complete the architecture and provide secondary computed data and indicators, in order to execute queries faster and easily. Between 2010 and 2021, 636 784 anesthesia records were integrated for 353 152 patients. We reported the main concerns and barriers during the development of this project and we provided 8 tips to handle them. We have implemented our data warehouse into the OMOP common data model as a complementary downstream data model. The next step of the project will be to disseminate the use of the OMOP data model for anesthesia and critical care, and drive the trend towards federated learning to enhance collaborations and multicenter studies.

Implementation and use of a national electronic dashboard to guide COVID-19 clinical management in Fiji

Problem

From April to September 2021, Fiji experienced a second wave of coronavirus disease (COVID-19) precipitated by the Delta variant of concern, prompting a need to strengthen existing data management of positive COVID-19 cases.

Context

With COVID-19 cases peaking at 1405 a day and many hospital admissions, the need to develop a better way to visualize data became clear.

Action

The Fiji Ministry of Health and Medical Services, the World Health Organization and the United Nations Office for the Coordination of Humanitarian Affairs collaborated to develop an online clinical dashboard to support better visualization of case management data.

Outcome

The dashboard was used across Fiji at national, divisional and local levels for COVID-19 management. At the national level, it provided real-time reports describing the surge pattern, severity and management of COVID-19 cases across the country during daily incident management team meetings. At the divisional level, it gave the divisional directors access to timely information about hospital and community isolation of cases. At the hospital level, the dashboard allowed managers to monitor trends in isolated cases and use of oxygen resources.

Discussion

The dashboard replaced previous paper-based reporting of statistics with delivery of trends and real-time data. The team that developed the tool were situated in different locations and did not meet physically, demonstrating the ease of implementing this online tool in a resource-constrained setting. The dashboard is easy to use and could be used in other Pacific island countries and areas.

Visual Analytics to Leverage Anesthesia Electronic Health Record

BACKGROUND

Visual analytics is the science of analytical reasoning supported by interactive visual interfaces called dashboards. In this report, we describe our experience addressing the challenges in visual analytics of anesthesia electronic health record (EHR) data using a commercially available business intelligence (BI) platform. As a primary outcome, we discuss some performance metrics of the dashboards, and as a secondary outcome, we outline some operational enhancements and financial savings associated with deploying the dashboards.

METHODS

Data were transferred from the EHR to our departmental servers using several parallel processes. A custom structured query language (SQL) query was written to extract the relevant data fields and to clean the data. Tableau was used to design multiple dashboards for clinical operation, performance improvement, and business management.

RESULTS

Before deployment of the dashboards, detailed case counts and attributions were available for the operating rooms (ORs) from perioperative services; however, the same level of detail was not available for non-OR locations. Deployment of the yearly case count dashboards provided near-real-time case count information from both central and non-OR locations among multiple campuses, which was not previously available. The visual presentation of monthly data for each year allowed us to recognize seasonality in case volumes and adjust our supply chain to prevent shortages. The dashboards highlighted the systemwide volume of cases in our endoscopy suites, which allowed us to target these supplies for pricing negotiations, with an estimated annual cost savings of $250,000. Our central venous pressure (CVP) dashboard enabled us to provide individual practitioner feedback, thus increasing our monthly CVP checklist compliance from approximately 92% to 99%.

CONCLUSIONS

The customization and visualization of EHR data are both possible and worthwhile for the leveraging of information into easily comprehensible and actionable data for the improvement of health care provision and practice management. Limitations inherent to EHR data presentation make this customization necessary, and continued open access to the underlying data set is essential.

Health, Security and Fire Safety Process Optimisation Using Intelligence at the Edge

The proliferation of sensors to capture parametric measures or event data over a myriad of networking topologies is growing exponentially to improve our daily lives. Large amounts of data must be shared on constrained network infrastructure, increasing delays and loss of valuable real-time information. Our research presents a solution for the health, security, safety, and fire domains to obtain temporally synchronous, credible and high-resolution data from sensors to maintain the temporal hierarchy of reported events. We developed a multisensor fusion framework with energy conservation via domain-specific "wake up" triggers that turn on low-power model-driven microcontrollers using machine learning (TinyML) models. We investigated optimisation techniques using anomaly detection modes to deliver real-time insights in demanding life-saving situations. Using energy-efficient methods to analyse sensor data at the point of creation, we facilitated a pathway to provide sensor customisation at the "edge", where and when it is most needed. We present the application and generalised results in a real-life health care scenario and explain its application and benefits in other named researched domains.

Dashboards in Health Care Settings: Protocol for a Scoping Review

Background

Health care organizations increasingly depend on business intelligence tools, including “dashboards,” to capture, analyze, and present data on performance metrics. Ideally, dashboards allow users to quickly visualize actionable data to inform and optimize clinical and organizational performance. In reality, dashboards are typically embedded in complex health care organizations with massive data streams and end users with distinct needs. Thus, designing effective dashboards is a challenging task and theoretical underpinnings of health care dashboards are poorly characterized; even the concept of the dashboard remains ill-defined. Researchers, informaticists, clinical managers, and health care administrators will benefit from a clearer understanding of how dashboards have been developed, implemented, and evaluated, and how the design, end user, and context influence their uptake and effectiveness.

Objective

This scoping review first aims to survey the vast published literature of “dashboards” to describe where, why, and for whom they are used in health care settings, as well as how they are developed, implemented, and evaluated. Further, we will examine how dashboard design and content is informed by intended purpose and end users.

Methods

In July 2020, we searched MEDLINE, Embase, Web of Science, and the Cochrane Library for peer-reviewed literature using a targeted strategy developed with a research librarian and retrieved 5188 results. Following deduplication, 3306 studies were screened in duplicate for title and abstract. Any abstracts mentioning a health care dashboard were retrieved in full text and are undergoing duplicate review for eligibility. Articles will be included for data extraction and analysis if they describe the development, implementation, or evaluation of a dashboard that was successfully used in routine workflow. Articles will be excluded if they were published before 2015, the full text is unavailable, they are in a non-English language, or they describe dashboards used for public health tracking, in settings where direct patient care is not provided, or in undergraduate medical education. Any discrepancies in eligibility determination will be adjudicated by a third reviewer. We chose to focus on articles published after 2015 and those that describe dashboards that were successfully used in routine practice to identify the most recent and relevant literature to support future dashboard development in the rapidly evolving field of health care informatics.

Results

All articles have undergone dual review for title and abstract, with a total of 2019 articles mentioning use of a health care dashboard retrieved in full text for further review. We are currently reviewing all full-text articles in duplicate. We aim to publish findings by mid-2022. Findings will be reported following guidance from the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) checklist.

Conclusions

This scoping review will provide stakeholders with an overview of existing dashboard tools, highlighting the ways in which dashboards have been developed, implemented, and evaluated in different settings and for different end user groups, and identify potential research gaps. Findings will guide efforts to design and use dashboards in the health care sector more effectively.

International Registered Report Identifier (IRRID)

DERR1-10.2196/34894

Visualizing Opioid-Use Variation in a Pediatric Perioperative Dashboard

BACKGROUND

Anesthesiologists integrate numerous variables to determine an opioid dose that manages patient nociception and pain while minimizing adverse effects. Clinical dashboards that enable physicians to compare themselves to their peers can reduce unnecessary variation in patient care and improve outcomes. However, due to the complexity of anesthetic dosing decisions, comparative visualizations of opioid-use patterns are complicated by case-mix differences between providers.

OBJECTIVES

This single-institution case study describes the development of a pediatric anesthesia dashboard and demonstrates how advanced computational techniques can facilitate nuanced normalization techniques, enabling meaningful comparisons of complex clinical data.

METHODS

We engaged perioperative-care stakeholders at a tertiary care pediatric hospital to determine patient and surgical variables relevant to anesthesia decision-making and to identify end-user requirements for an opioid-use visualization tool. Case data were extracted, aggregated, and standardized. We performed multivariable machine learning to identify and understand key variables. We integrated interview findings and computational algorithms into an interactive dashboard with normalized comparisons, followed by an iterative process of improvement and implementation.

RESULTS

The dashboard design process identified two mechanisms-interactive data filtration and machine-learning-based normalization-that enable rigorous monitoring of opioid utilization with meaningful case-mix adjustment. When deployed with real data encompassing 24,332 surgical cases, our dashboard identified both high and low opioid-use outliers with associated clinical outcomes data.

CONCLUSION

A tool that gives anesthesiologists timely data on their practice patterns while adjusting for case-mix differences empowers physicians to track changes and variation in opioid administration over time. Such a tool can successfully trigger conversation amongst stakeholders in support of continuous improvement efforts. Clinical analytics dashboards can enable physicians to better understand their practice and provide motivation to change behavior, ultimately addressing unnecessary variation in high impact medication use and minimizing adverse effects.

State-of-the-art Dashboards on Clinical Indicator Data to Support Reflection on Practice: Scoping Review

Background

There is an increasing interest in using routinely collected eHealth data to support reflective practice and long-term professional learning. Studies have evaluated the impact of dashboards on clinician decision-making, task completion time, user satisfaction, and adherence to clinical guidelines.

Objective

This scoping review aims to summarize the literature on dashboards based on patient administrative, medical, and surgical data for clinicians to support reflective practice.

Methods

A scoping review was conducted using the Arksey and O’Malley framework. A search was conducted in 5 electronic databases (MEDLINE, Embase, Scopus, ACM Digital Library, and Web of Science) to identify studies that met the inclusion criteria. Study selection and characterization were performed by 2 independent reviewers (BB and CP). One reviewer extracted the data that were analyzed descriptively to map the available evidence.

Results

A total of 18 dashboards from 8 countries were assessed. Purposes for the dashboards were designed for performance improvement (10/18, 56%), to support quality and safety initiatives (6/18, 33%), and management and operations (4/18, 22%). Data visualizations were primarily designed for team use (12/18, 67%) rather than individual clinicians (4/18, 22%). Evaluation methods varied among asking the clinicians directly (11/18, 61%), observing user behavior through clinical indicators and use log data (14/18, 78%), and usability testing (4/18, 22%). The studies reported high scores on standard usability questionnaires, favorable surveys, and interview feedback. Improvements to underlying clinical indicators were observed in 78% (7/9) of the studies, whereas 22% (2/9) of the studies reported no significant changes in performance.

Conclusions

This scoping review maps the current literature landscape on dashboards based on routinely collected clinical indicator data. Although there were common data visualization techniques and clinical indicators used across studies, there was diversity in the design of the dashboards and their evaluation. There was a lack of detail regarding the design processes documented for reproducibility. We identified a lack of interface features to support clinicians in making sense of and reflecting on their personal performance data.

Transforming Anesthesia Data Into the Observational Medical Outcomes Partnership Common Data Model: Development and Usability Study

BACKGROUND

Electronic health records (EHRs, such as those created by an anesthesia management system) generate a large amount of data that can notably be reused for clinical audits and scientific research. The sharing of these data and tools is generally affected by the lack of system interoperability. To overcome these issues, Observational Health Data Sciences and Informatics (OHDSI) developed the Observational Medical Outcomes Partnership (OMOP) common data model (CDM) to standardize EHR data and promote large-scale observational and longitudinal research. Anesthesia data have not previously been mapped into the OMOP CDM.

OBJECTIVE

The primary objective was to transform anesthesia data into the OMOP CDM. The secondary objective was to provide vocabularies, queries, and dashboards that might promote the exploitation and sharing of anesthesia data through the CDM.

METHODS

Using our local anesthesia data warehouse, a group of 5 experts from 5 different medical centers identified local concepts related to anesthesia. The concepts were then matched with standard concepts in the OHDSI vocabularies. We performed structural mapping between the design of our local anesthesia data warehouse and the OMOP CDM tables and fields. To validate the implementation of anesthesia data into the OMOP CDM, we developed a set of queries and dashboards.

RESULTS

We identified 522 concepts related to anesthesia care. They were classified as demographics, units, measurements, operating room steps, drugs, periods of interest, and features. After semantic mapping, 353 (67.7%) of these anesthesia concepts were mapped to OHDSI concepts. Further, 169 (32.3%) concepts related to periods and features were added to the OHDSI vocabularies. Then, 8 OMOP CDM tables were implemented with anesthesia data and 2 new tables (EPISODE and FEATURE) were added to store secondarily computed data. We integrated data from 5,72,609 operations and provided the code for a set of 8 queries and 4 dashboards related to anesthesia care.

CONCLUSIONS

Generic data concerning demographics, drugs, units, measurements, and operating room steps were already available in OHDSI vocabularies. However, most of the intraoperative concepts (the duration of specific steps, an episode of hypotension, etc) were not present in OHDSI vocabularies. The OMOP mapping provided here enables anesthesia data reuse.

How Do We Use Dashboards to Enhance Quality in Cardiac Anesthesia?

The use of clinical dashboards has expanded significantly in healthcare in recent years in a variety of settings. The ability to analyze data related to quality metrics in one screen is highly desirable for cardiac anesthesiologists, as they have considerable influence on important clinical outcomes. Building a robust quality program within cardiac anesthesia relies on consistent access and review of quality outcome measures, process measures, and operational measures through a clinical dashboard. Signals and trends in these measures may be compared to other cardiac surgical programs to analyze gaps and areas for quality improvement efforts. In this article, the authors describe how they designed a clinical cardiac anesthesia dashboard for quality efforts at their institution.