Impact of random safety analyses on structure, process and outcome indicators: multicentre study

Assessing the impact of real-time random safety audits through full propensity score matching on reliable data from the clinical information system

BACKGROUND

Evidence-based care processes are not always applied at the bedside in critically ill patients. Numerous studies have assessed the impact of checklists and related strategies on the process of care and patient outcomes. We aimed to evaluate the effects of real-time random safety audits on process-of-care and outcome variables in critical care patients.

METHODS

This prospective study used data from the clinical information system to evaluate the impact of real-time random safety audits targeting 32 safety measures in two intensive care units during a 9-month period. We compared endpoints between patients attended with safety audits and those not attended with safety audits. The primary endpoint was mortality, measured by Cox hazard regression after full propensity-score matching. Secondary endpoints were the impact on adherence to process-of-care measures and on quality indicators.

RESULTS

We included 871 patients; 228 of these were attended in ≥ 1 real-time random safety audits. Safety audits were carried out on 390 patient-days; most improvements in the process of care were observed in safety measures related to mechanical ventilation, renal function and therapies, nutrition, and clinical information system. Although the group of patients attended in safety audits had more severe disease at ICU admission [APACHE II score 21 (16-27) vs. 20 (15-25), p = 0.023]; included a higher proportion of surgical patients [37.3 % vs. 26.4 %, p = 0.003] and a higher proportion of mechanically ventilated patients [72.8 % vs. 40.3 %, p < 0.001]; averaged more days on mechanical ventilation, central venous catheter, and urinary catheter; and had a longer ICU stay [12.5 (5.5-23.3) vs. 2.9 (1.7-5.9), p < 0.001], ICU mortality did not differ significantly between groups (19.3 % vs. 18.8 % in the group without safety rounds). After full propensity-score matching, Cox hazard regression analysis showed real-time random safety audits were associated with a lower risk of mortality throughout the ICU stay (HR 0.31; 95 %CI 0.20-0.47).

CONCLUSIONS

Real-time random safety audits are associated with a reduction in the risk of ICU mortality. Exploiting data from the clinical information system is useful in assessing the impact of them on the care process, quality indicators, and mortality.

Equipment preparedness for neonatal resuscitation in neonatal intensive care unit in resource limited setting: cross-sectional study

Background

Adverse healthcare's events are a critical issue worldwide, neonatal intensive care unit adverse events are a considerable issue. It is important that we recognize the basic equipment needed to address these circumstances. The aim of this study is to asses' equipment preparedness for neonatal resuscitation in the neonatal intensive care unit.

Method

A hospital-based, cross-sectional study was conducted on 210 neonates admitted to neonatal intensive care unit at comprehensive specialized hospital from 26/03/2022 to 26/05/2022. The data were collected using Checklist prepared from Neonatal resuscitation: current evidence and guidelines. The data obtained were summed up and presented as descriptive statistics using the Microsoft Excel and were analyzed using SPSS version 25. The result reported in text and table form.

Result

In this study there was 12.72% complete equipment preparation (without defect) in 210 cases. From the total, there was minor defect in 52.8% cases, and 34.45% cases had serious defect. Serious defects were more frequently detected in the equipment preparation (42.46%), resuscitation medications (12.5%), and radiant warmer set-up (40%).

Conclusion and recommendation

Overall equipment preparation for neonatal resuscitation was insufficient, and quality of equipment preparation for neonatal resuscitation and stabilization needs to be improved. To enhance equipment preparedness in the neonatal intensive care unit staff should establish uniform guidelines.

The Association Between Prepandemic ICU Performance and Mortality Variation in COVID-19: A Multicenter Cohort Study of 35,619 Critically Ill Patients

BACKGROUND

During the COVID-19 pandemic, ICUs remained under stress and observed elevated mortality rates and high variations of outcomes. A knowledge gap exists regarding whether an ICU performing best during nonpandemic times would still perform better when under high pressure compared with the least performing ICUs.

RESEARCH QUESTION

Does prepandemic ICU performance explain the risk-adjusted mortality variability for critically ill patients with COVID-19?

STUDY DESIGN AND METHODS

This study examined a cohort of adults with real-time polymerase chain reaction-confirmed COVID-19 admitted to 156 ICUs in 35 hospitals from February 16, 2020, through December 31, 2021, in Brazil. We evaluated crude and adjusted in-hospital mortality variability of patients with COVID-19 in the ICU during the pandemic. Association of baseline (prepandemic) ICU performance and in-hospital mortality was examined using a variable life-adjusted display (VLAD) during the pandemic and a multivariable mixed regression model adjusted by clinical characteristics, interaction of performance with the year of admission, and mechanical ventilation at admission.

RESULTS

Thirty-five thousand six hundred nineteen patients with confirmed COVID-19 were evaluated. The median age was 52 years, median Simplified Acute Physiology Score 3 was 42, and 18% underwent invasive mechanical ventilation. In-hospital mortality was 13% and 54% for those receiving invasive mechanical ventilation. Adjusted in-hospital mortality ranged from 3.6% to 63.2%. VLAD in the most efficient ICUs was higher than the overall median in 18% of weeks, whereas VLAD was 62% and 84% in the underachieving and least efficient groups, respectively. The least efficient baseline ICU performance group was associated independently with increased mortality (OR, 2.30; 95% CI, 1.45-3.62) after adjusting for patient characteristics, disease severity, and pandemic surge.

INTERPRETATION

ICUs caring for patients with COVID-19 presented substantial variation in risk-adjusted mortality. ICUs with better baseline (prepandemic) performance showed reduced mortality and less variability. Our findings suggest that achieving ICU efficiency by targeting improvement in organizational aspects of ICUs may impact outcomes, and therefore should be a part of the preparedness for future pandemics.

Design of a safety round model for intensive care units

INTRODUCTION

Safety Rounds (SR) are an operational tool that allow knowing adherence to good practices, help identify risks and incidents in patient safety (PS), allowing improvement actions to be implemented. The objective of this work was the design of a procedure to perform SR in an Intensive Care Unit (ICU).

METHODS

Preparation of a checklist for the development of SR in the ICU through the nominal group technique, with the participation of managers, middle managers and professionals from different disciplines and categories. In the first place, a group of experts agreed, based on the recommendations on good practices in PS, the definition of items, their coding, the criteria for compliance and the impact of non-compliance. Subsequently, its viability was determined through a cross-sectional study through the piloting of two SRs to adjust the items in real clinical practice conditions.

RESULTS

A specific SR model for ICUs has been obtained through a checklist. The group of experts prepared a first list made up of 39 items of 6 essential dimensions and defined the method of implementation. Mean time to complete the two SRs was 85 min, including the briefing and subsequent debriefing. After the validation pilot, the dimensions were reduced to 5, 3 items were deleted, 2 items were transferred to another dimension and 3 items related to nosocomial infections and informed consent were modified. In addition, the data sources, the compliance criteria and their relative weight were redefined. The final list was considered useful and relevant to improve practice.

CONCLUSIONS

Through a consensus methodology, a checklist has been built to be used in the RS of an ICU. This model can serve as a basis for its use in healthcare services with similar characteristics.

Secondary use of data extracted from a clinical information system to assess the adherence of tidal volume and its impact on outcomes

OBJECTIVES

To extract data from clinical information systems to automatically calculate high-resolution quality indicators to assess adherence to recommendations for low tidal volume.

DESIGN

We devised two indicators: the percentage of time under mechanical ventilation with excessive tidal volume (>8mL/kg predicted body weight) and the percentage of patients who received appropriate tidal volume (≤8mL/kg PBW) at least 80% of the time under mechanical ventilation. We developed an algorithm to automatically calculate these indicators from clinical information system data and analyzed associations between them and patients' characteristics and outcomes.

SETTINGS

This study has been carried out in our 30-bed polyvalent intensive care unit between January 1, 2014 and November 30, 2019.

PATIENTS

All patients admitted to intensive care unit ventilated >72h were included.

INTERVENTION

Use data collected automatically from the clinical information systems to assess adherence to tidal volume recommendations and its outcomes.

MAIN VARIABLES OF INTEREST

Mechanical ventilation days, ICU length of stay and mortality.

RESULTS

Of all admitted patients, 340 met the inclusion criteria. Median percentage of time under mechanical ventilation with excessive tidal volume was 70% (23%-93%); only 22.3% of patients received appropriate tidal volume at least 80% of the time. Receiving appropriate tidal volume was associated with shorter duration of mechanical ventilation and intensive care unit stay. Patients receiving appropriate tidal volume were mostly male, younger, taller, and less severely ill. Adjusted intensive care unit mortality did not differ according to percentage of time with excessive tidal volume or to receiving appropriate tidal volume at least 80% of the time.

CONCLUSIONS

Automatic calculation of process-of-care indicators from clinical information systems high-resolution data can provide an accurate and continuous measure of adherence to recommendations. Adherence to tidal volume recommendations was associated with shorter duration of mechanical ventilation and intensive care unit stay.

Analysis of adherence to an early mobilization protocol in an intensive care unit: Data collected prospectively over a period of three years by the clinical information system

OBJECTIVE

To determinate the adherence and barriers of our early mobilization protocol in patients who had received mechanical ventilation >48h in routine daily practice through clinical information system during all Intensive Care Unit (ICU) stay.

DESIGN

Observational and prospective cohort study.

SETTING

Polyvalent ICU over a three-year period (2017-2019).

PATIENTS

Adult patients on mechanical ventilation >48h who met the inclusion criteria for the early mobilization protocol.

INTERVENTIONS

None.

MAIN VARIABLES OF INTEREST

Demographics, adherence to the protocol and putative hidden adherence, total number of mobilizations, barriers, artificial airway/ventilatory support at each mobilization level and adverse events.

RESULTS

We analyzed 3269 stay-days from 388 patients with median age of 63 (51-72) years, median APACHE II 23 (18-29) and median ICU stay of 10.1 (6.2-16.5) days. Adherence to the protocol was 56.6% (1850 stay-days), but patients were mobilized in only 32.2% (1472) of all stay-days. The putative hidden adherence was 15.6% (509 stay-days) which would increase adherence to 72.2%. The most common reasons for not mobilizing patients were failure to meeting the criteria for clinical stability in 241 (42%) stay-days and unavailability of physiotherapists in 190 (33%) stay-days. Adverse events occurred in only 6 (0.4%) stay-days.

CONCLUSIONS

Data form Clinical Information System showed although adherence was high, patients were mobilized in only one-third of all stay-days. Knowing the specific reason why patient were not mobilized in each stay-day allow to develop concrete decisions to increase the number of mobilizations.

Cross-cultural adaptation of the SCORE survey and evaluation of the impact of Real-Time Random Safety Audits in organizational culture: A multicenter study

OBJECTIVE

To establish a cross-cultural adaptation of the Safety, Communication, Operational Reliability, and Engagement (SCORE) survey and to use this instrument to evaluate the impact of a safety intervention.

DESIGN

Cross-cultural adaptation and before-and-after evaluation study.

SETTING

5 ICU.

PARTICIPANTS

Medical residents, attending physicians, and nurses at those ICU.

INTERVENTIONS

Adaptation of the SCORE survey to Spanish culture. The adapted survey was used to assess all safety-culture-related domains before and one-year after implementing the use of a safety tool, Real-Time Random Safety Audits (in Spanish: Análisis Aleatorios de Seguridad en Tiempo Real, AASTRE).

MAIN OUTCOME MEASURE

Adaptabiliy of the Spanish version of SCORE survey in the ICU setting and evaluation of the effect of AASTRE on their domains.

RESULTS

The cross-cultural adaptation was adequate. Post-AASTRE survey scores [mean (standard deviation, SD)] were significantly better in the domains learning environment [50.55 (SD 20.62) vs 60.76 (SD 23.66), p<.0001], perception of local leadership [47.98 (SD 23.57) vs 62.82 (SD 27.46), p<.0001], teamwork climate [51.19 (SD 18.55) vs 55.89 (SD 20.25), p=.031], safety climate [45.07 (SD 17.60) vs 50.36 (SD 19.65), p=.01], participation decision making [3 (SD 0.82) vs 3.65 (SD 0.87), p<.0001] and advancement in the organization [3.21 (SD 0.77) vs 4.04 (SD 0.77), p<.0001]. However, post-AASTRE scores were significantly worse in the domains workload and burnout climate.

CONCLUSIONS

The cross-cultural adaptation of the SCORE survey into Spanish is a useful tool for ICUs. The application of the AASTRE is associated with improvements in six SCORE domains, including the safety climate.

Real-Time Safety Audits of Neonatal Delivery Room Resuscitation Areas: Are We Sufficiently Prepared?

OBJECTIVE

This study aimed to use real-time safety audits to establish whether preparation of the equipment required for the stabilization and resuscitation of newborns in the delivery room areas is adequate.

STUDY DESIGN

This was a descriptive, multicenter study performed at five-level III-A neonatal units in Madrid, Spain. For 1 year, one researcher from each center performed random real-time safety audits (RRTSAs), on different days and during different shifts, of at least three neonatal stabilization areas, either in the delivery room or in the operating room used for caesarean sections. Three factors in each area were reviewed: the set-up of the radiant warmer, the materials, and medication available. The global audit was considered without defect when no errors were detected in any of the audited factors. Possible differences in the results were analyzed as a function of the study month, day of the week, or shift during which the audit had been performed.

RESULTS

A total of 852 audits were performed. No defects were detected in any of the three factors analyzed in the 534 (62.7%, 95% confidence interval [CI]: 59.3-65.9) cases. Slight defects were detected in 98 (11.5%, 95% CI: 9.4-13.8) cases and serious defects capable of producing adverse events in the newborn during resuscitation were found in 220 (25.8%, 95% CI: 22.9-28.9) cases. No statistically significant differences in the results were found according to the day of the week or time during which the audits were performed. However, the percentage of RRTSAs without defect increased as the study period progressed (first quarter 38.1% vs. the last quarter 84.2%; p < 0.001).

CONCLUSION

The percentage of adequately prepared resuscitation areas was low. RRTSAs made it possible to detect errors in the correct availability of the neonatal stabilization areas and improved their preparation by preventing errors from being perpetuated over time.

KEY POINTS

· RRTSAs are a tool for improving clinical safety.. · The use of RRTSAs in perinatal care is very uncommon.. · RRTSAs improve the preparation of newborn CPR areas..

Cross-cultural adaptation of the SCORE survey and evaluation of the impact of Real-Time Random Safety Audits in organizational culture: A multicenter study

OBJECTIVE

To establish a cross-cultural adaptation of the Safety, Communication, Operational Reliability, and Engagement (SCORE) survey and to use this instrument to evaluate the impact of a safety intervention.

DESIGN

Cross-cultural adaptation and before-and-after evaluation study.

SETTING

5 ICU.

PARTICIPANTS

Medical residents, attending physicians, and nurses at those ICU.

INTERVENTIONS

Adaptation of the SCORE survey to Spanish culture. The adapted survey was used to assess all safety-culture-related domains before and one-year after implementing the use of a safety tool, Real-Time Random Safety Audits (in Spanish: Análisis Aleatorios de Seguridad en Tiempo Real, AASTRE).

MAIN OUTCOME MEASURE

Adaptabiliy of the Spanish version of SCORE survey in the ICU setting and evaluation of the effect of AASTRE on their domains.

RESULTS

The cross-cultural adaptation was adequate. Post-AASTRE survey scores [mean (standard deviation, SD)] were significantly better in the domains learning environment [50.55 (SD 20.62) vs 60.76 (SD 23.66), p<.0001], perception of local leadership [47.98 (SD 23.57) vs 62.82 (SD 27.46), p<.0001], teamwork climate [51.19 (SD 18.55) vs 55.89 (SD 20.25), p=.031], safety climate [45.07 (SD 17.60) vs 50.36 (SD 19.65), p=.01], participation decision making [3 (SD 0.82) vs 3.65 (SD 0.87), p<.0001] and advancement in the organization [3.21 (SD 0.77) vs 4.04 (SD 0.77), p<.0001]. However, post-AASTRE scores were significantly worse in the domains workload and burnout climate.

CONCLUSIONS

The cross-cultural adaptation of the SCORE survey into Spanish is a useful tool for ICUs. The application of the AASTRE is associated with improvements in six SCORE domains, including the safety climate.

Application, verification and correction from an elaborate checklist with some of the recommendations («do and do not do») of the SEMICYUC working groups

OBJECTIVE

Based on some of the recommendations of the SEMICYUC working groups, we developed a checklist and applied it in 2 periods, analyzing their behavior as a tool for improving safety.

DESIGN

A comparative pre- and post-intervention longitudinal study was carried out.

SETTING

The Intensive Care Unit (ICU) of a 400-bed university hospital.

PATIENTS

Random cases series in 2 periods separated by 6 months.

INTERVENTIONS

We developed a checklist with 24 selected indicators that were randomly applied to 50 patients. Verification was conducted by a professional not related to care (prompter). We analyzed the results and compliance index and carried out corrective measures with training. With 6 months of preparation, we again applied the random checklist to 50 patients (post-intervention period) and compared the compliance indexes between the two timepoints.

RESULTS

There were no differences in demographic characteristics or evolution between the periods. The compliance index at baseline was 0.86±0.12 versus 0.91±0.52 in the post-intervention period (P=.023). An acceptable compliance index was obtained with the 24 indicators, though at baseline the compliance index was<0.85 for 5 recommendations. These detected non-compliances were worked upon through training in the second phase of the study. The post-intervention checklist evidenced improvement in compliance with the recommendations.

CONCLUSIONS

The checklist used to assess compliance with a selection of recommendations of the SEMICYUC applied and moderated by a prompter was seen to be a useful instrument allowing us to identify points for improvement in the management of Intensive Care Unit patients, increasing the quality and safety of care.

Going home to die from critical care: A case study

Much of the activity in critical care is complex but repetitive. In order to standardize care and maintain safety, delivery of care is often directed by protocols and care bundles. This case study will reflect on an instance where care transcended the standard protocol-directed path to be more individualized, creative and compassionate. Acts like these can be unique for the practitioners involved and require an element of positive risk taking, which happened here. It will look at the decision-making, planning and risk involved in preparing for a terminally ill patient, who was inotrope and high-flow oxygen dependent, to go home to have treatment withdrawn there instead of in the hospital. This was to fulfil his wish to die at home. In unpicking the circumstances where this positive risk taking led to the desired outcome and the relationship between safety, uncertainty and risk, three themes arose. These were the journey to safe uncertainty; decision-making with uncertain outcomes; and the importance of robust human factors, particularly effective communication and inter-professional teamwork. If positive risk taking can result in enhanced outcomes for the patient, then the question of how this behaviour can be fostered and encouraged must be addressed.

Safety culture in intensive care internationally and in Australia: A narrative review of the literature

OBJECTIVE

Assessment of safety culture in health care is of particular relevance in the complex intensive care setting, where the effects of human error can have catastrophic consequences. The aim of this review was to examine the literature on safety culture in intensive care units (ICUs) and specifically, to explore the state of knowledge regarding safety culture in the context of Australian ICUs.

METHODS

A search was conducted of key databases for studies published in English between January 2008 and December 2017 using terms 'safety culture', 'safety climate', 'safety attitude', 'intensive care', 'ICU' and 'critical care'. Studies were included if they presented original research, utilised the teamwork and safety climate factors of a quantitative survey tool to assess safety culture, the sample population included participants working in an adult intensive care, and the findings were reported in the context of intensive care.

RESULTS

Of the 36 studies identified, two were conducted in Australia. The studies demonstrate a rapid expansion in safety culture assessment globally. Three levels of safety culture application in intensive care were identified, including safety culture assessment, effect of an intervention on safety culture, and evaluation of the association between safety culture and structural, process and outcomes measures. The use of targeted safety culture domains is emerging. Common findings included variation in perceptions of safety culture between ICUs, unit and hospital management, and professional groups.

CONCLUSION

Though the assessment of safety culture in ICUs has been an area of prolific research internationally over the past ten years, the Australian context is limited and could be advanced through further research, including the effect on safety culture of interventions, and to establish the association between safety culture and patient safety outcomes. Longitudinal studies to demonstrate sustained intervention effects on safety culture should be considered.

Validation of the ICU-DaMa tool for automatically extracting variables for minimum dataset and quality indicators: The importance of data quality assessment

BACKGROUND

Big data analytics promise insights into healthcare processes and management, improving outcomes while reducing costs. However, data quality is a major challenge for reliable results. Business process discovery techniques and an associated data model were used to develop data management tool, ICU-DaMa, for extracting variables essential for overseeing the quality of care in the intensive care unit (ICU).

OBJECTIVE

To determine the feasibility of using ICU-DaMa to automatically extract variables for the minimum dataset and ICU quality indicators from the clinical information system (CIS).

METHODS

The Wilcoxon signed-rank test and Fisher's exact test were used to compare the values extracted from the CIS with ICU-DaMa for 25 variables from all patients attended in a polyvalent ICU during a two-month period against the gold standard of values manually extracted by two trained physicians. Discrepancies with the gold standard were classified into plausibility, conformance, and completeness errors.

RESULTS

Data from 149 patients were included. Although there were no significant differences between the automatic method and the manual method, we detected differences in values for five variables, including one plausibility error and two conformance and completeness errors. Plausibility: 1) Sex, ICU-DaMa incorrectly classified one male patient as female (error generated by the Hospital's Admissions Department). Conformance: 2) Reason for isolation, ICU-DaMa failed to detect a human error in which a professional misclassified a patient's isolation. 3) Brain death, ICU-DaMa failed to detect another human error in which a professional likely entered two mutually exclusive values related to the death of the patient (brain death and controlled donation after circulatory death). Completeness: 4) Destination at ICU discharge, ICU-DaMa incorrectly classified two patients due to a professional failing to fill out the patient discharge form when thepatients died. 5) Length of continuous renal replacement therapy, data were missing for one patient because the CRRT device was not connected to the CIS.

CONCLUSIONS

Automatic generation of minimum dataset and ICU quality indicators using ICU-DaMa is feasible. The discrepancies were identified and can be corrected by improving CIS ergonomics, training healthcare professionals in the culture of the quality of information, and using tools for detecting and correcting data errors.

Protocols for the obvious: Where does it start, and stop?

Protocols can be helpful in specific situations and may have show benefits in clinical trials. So-called evidence based protocols and checklists frequently remind clinicians to do the obvious, but may also contain as part of a bundle, elements that are not based on the best current evidence. However, so called quality improvement programs frequently call for implementation of the total bundle. We think this is basically wrong and warn against that practice.