Competence in Decision Making: Setting Performance Standards for Critical Care

Effect of a competence based medical education program on training quality in Intensive Care Medicine. COBALIDATION TRIAL

OBJECTIVES

The main objective of this study was to evaluate whether the implementation of CoBaTrICE (Competency-Based Training in Intensive Care Medicine in Europe) provides higher levels of competency in comparison with the current official time-based program in Intensive Care Medicine in Spain. Secondary objectives were: 1) To determine the percentage of critical essential performance elements (CEPE) accomplished, 2) To determine compliance with workplace-based assessments (wba).

DESIGN

Multicenter cluster randomized trial.

SETTING

Thirteen Spanish ICU Departments.

PARTICIPANTS

Thirty-six residents INTERVENTION: The implementation of CoBaTrICE included: (1) Training the trainers; (2) Wba; (3) The use of an electronic portfolio. The level of competency achieved by each participant was determined by a simulation-based Objective Structured Clinical Exam (OSCE) performed at the end of the 5th year of training period.

MAIN VARIABLES OF INTEREST

Total scoring in the five scenarios, CEPE completed, level of competency (1-5) achieved.

RESULTS

A total of 119 performances from 26 residents (17 from CoBaTrICE group and 9 from control group) were analyzed in the OSCE. CoBaTrICE residents´ achieved higher levels of competency [2 (1-5) vs. 2 (1-3), p = 0.07) and higher percentages of CEPE´s accomplishment than the control group (78% vs. 71%, p = 0.09).

CONCLUSIONS

The CoBaTrICE group showed a better performance trend in comparison to the control group, but the differences were not statistically significant. Since the number of Wba performed was low, additional research is needed to determine the potential superiority of CoBaTrICE.

Data Representation Structure to Support Clinical Decision-Making in the Pediatric Intensive Care Unit: Interview Study and Preliminary Decision Support Interface Design

BACKGROUND

Clinical decision-making is a complex cognitive process that relies on the interpretation of a large variety of data from different sources and involves the use of knowledge bases and scientific recommendations. The representation of clinical data plays a key role in the speed and efficiency of its interpretation. In addition, the increasing use of clinical decision support systems (CDSSs) provides assistance to clinicians in their practice, allowing them to improve patient outcomes. In the pediatric intensive care unit (PICU), clinicians must process high volumes of data and deal with ever-growing workloads. As they use multiple systems daily to assess patients' status and to adjust the health care plan, including electronic health records (EHR), clinical systems (eg, laboratory, imaging and pharmacy), and connected devices (eg, bedside monitors, mechanical ventilators, intravenous pumps, and syringes), clinicians rely mostly on their judgment and ability to trace relevant data for decision-making. In these circumstances, the lack of optimal data structure and adapted visual representation hinder clinician's cognitive processes and clinical decision-making skills.

OBJECTIVE

In this study, we designed a prototype to optimize the representation of clinical data collected from existing sources (eg, EHR, clinical systems, and devices) via a structure that supports the integration of a home-developed CDSS in the PICU. This study was based on analyzing end user needs and their clinical workflow.

METHODS

First, we observed clinical activities in a PICU to secure a better understanding of the workflow in terms of staff tasks and their use of EHR on a typical work shift. Second, we conducted interviews with 11 clinicians from different staff categories (eg, intensivists, fellows, nurses, and nurse practitioners) to compile their needs for decision support. Third, we structured the data to design a prototype that illustrates the proposed representation. We used a brain injury care scenario to validate the relevance of integrated data and the utility of main functionalities in a clinical context. Fourth, we held design meetings with 5 clinicians to present, revise, and adapt the prototype to meet their needs.

RESULTS

We created a structure with 3 levels of abstraction-unit level, patient level, and system level-to optimize clinical data representation and display for efficient patient assessment and to provide a flexible platform to host the internally developed CDSS. Subsequently, we designed a preliminary prototype based on this structure.

CONCLUSIONS

The data representation structure allows prioritizing patients via criticality indicators, assessing their conditions using a personalized dashboard, and monitoring their courses based on the evolution of clinical values. Further research is required to define and model the concepts of criticality, problem recognition, and evolution. Furthermore, feasibility tests will be conducted to ensure user satisfaction.

Competency assessment of residents of Intensive Care Medicine through a simulation-based objective structured clinical evaluation (OSCE). A multicenter observational study

OBJECTIVES

The current official model of training in Intensive Care Medicine (ICM) in Spain is based on exposure to experiences through clinical rotations. The main objective was to determine the level of competency (I novice to V independent practitioner) achieved by the residents at the end of the 3rd year of training (R3) in ICM through a simulation-based OSCE. Secondary objectives were: (1) To identify gaps in performance, and (2) To investigate the reliability and feasibility of conducting simulation-based assessment at multiple sites.

DESIGN

Observational multicenter study.

SETTING

Thirteen Spanish ICU Departments.

PARTICIPANTS

Thirty six R3.

INTERVENTION

The participants performed on five, 15-min, high-fidelity crisis scenarios in four simulation centers. The performances were video recorded for later scoring by trained raters.

MAIN VARIABLES OF INTEREST

Via a Delphi technique, an independent panel of expert intensivists identified critical essential performance elements (CEPE) for each scenario to define the levels of competency.

RESULTS

A total of 176 performances were analyzed. The internal consistency of the check-lists were adequate (KR-20 range 0.64-0.79). Inter-rater reliability was strong [median Intraclass Correlation Coefficient across scenarios: 0.89 (0.65-0.97)]. Competency levels achieved by R3 were: Level I (18.8%), II (35.2%), III (42.6%), IV/V (3.4%). Overall, a great heterogeneity in performance was observed.

CONCLUSION

The expected level of competency after one year in the ICU was achieved only in half of the performances. A more evidence-based educational approach is needed. Multiple center simulation-based assessment showed feasibility and reliability as an evaluation method of competency.

TRIAL REGISTRATION

COBALIDATION. NCT04278976. (https://register.

CLINICALTRIALS

gov).