Simulation-based assessment to evaluate cognitive performance in an anesthesiology residency program

Ultrasound-Guided Lumbar Plexus-Sciatic Nerve Blocks Versus Epidurals for Orthopaedic Surgeries: A Study to Compare the Competency of Novice Anaesthesiology Residents in a High-Volume Level 1 Trauma Centre

INTRODUCTION

The relative merits of peripheral nerve blocks (PNB) over central neuraxial anaesthesia and the advantages of the above two techniques over general anaesthesia for surgical interventions of the lower limb are well established. The competency of anaesthetic trainees in a high-volume level 1 trauma centre in administering dual ultrasound and nerve stimulator-guided lumbar plexus-sciatic nerve block (DUNLuPS) vs. epidural anaesthesia (EA) was compared by reporting the adequacy of anaesthesia with the two techniques, time taken for the performance of block, time of onset of sensory block (TOSB), and time of onset of motor block (TOMB).

MATERIALS AND METHODS

This prospective, randomized, study enrolled 92 patients aged 18-80 years with lower limb fractures admitted in trauma triage and scheduled for surgery. The patients were randomly allocated equally into the EA group and the DUNLuPS group. A total of 20 anaesthesia trainees in the third year of residency with clinical experience of more than 15 independent lumbar plexus-sciatic nerve blocks were included in the study. A volume of 20 ml of 0.5% ropivacaine was administered in the lumbar plexus (Shamrock technique) but the volume used for sciatic nerve (subgluteal approach) was varied so that the cumulative dose did not exceed 3 mg/kg. For each block, the onset of nerve blockade was assessed every five minutes, and the assessments continued for an additional 30 minutes after the nerve blocks were finished.

RESULTS

Clinical characteristics and adequacy of anaesthesia were comparable, i.e., 95.65% and 93.47% success in the EA (n = 46) and DUNLuPS (n = 46) groups, respectively. Performance time was significantly more in the DUNLuPS group but followed by significantly less TOSB and TOMB. The time for the first analgesic request was 351.63 ± 148.70 minutes in the DUNLuPS group and 147.60 ± 52.65 minutes in the EA group (p < 0.0001).

CONCLUSION

Both EA and DUNLuPS provide effective and comparable intra-operative anaesthesia for orthopaedic lower limb surgeries (OLLS) when administered by residents with more than two years of experience (third year of residency) in ultrasound-guided regional nerve blocks in a high-volume level 1 trauma centre. Statistically significant differences in the block performance characteristics had no clinical advantage as it was compensated by the faster onset time in the DUNLuPS group. Post-operative pain management was better in the DUNLuPS group, so the practice and conduct of anaesthesia for trauma patients should focus on the establishment of "block rooms" and timely training of residents in the former so that the advantages can be extended to the patient population.

Training Anesthesiology Residents to Care for the Traumatically Injured in the United States

Training and education for trauma anesthesiology have been predicated on 2 primary pathways: learning through peripheral "complex, massive transfusion cases"-an assumption that is flawed due to the unique demands, skills, and knowledge of trauma anesthesiology-or learning through experiential education, which is also incomplete due to its unpredictable and variable exposure. Residents may receive training from senior physicians who may not maintain a trauma-focused continuing medical education. Further compounding the issue is the lack of fellowship-trained clinicians and standardized curricula. The American Board of Anesthesiology (ABA) provides a section for trauma education in its Initial Certification in Anesthesiology Content Outline. However, many trauma-related topics also fall under other subspecialties, and the outline excludes "nontechnical" skills. This article focuses on the training of anesthesiology residents and proposes a tier-based approach to teaching the ABA outline by including lectures, simulation, problem-based learning discussions, and case-based discussions that are proctored in conducive environments by knowledgeable facilitators.

Simulation-Based Assessment Identifies Longitudinal Changes in Cognitive Skills in an Anesthesiology Residency Training Program

OBJECTIVES

We describe observed improvements in nontechnical or "higher-order" deficiencies and cognitive performance skills in an anesthesia residency cohort for a 1-year time interval. Our main objectives were to evaluate higher-order, cognitive performance and to demonstrate that simulation can effectively serve as an assessment of cognitive skills and can help detect "higher-order" deficiencies, which are not as well identified through more traditional assessment tools. We hypothesized that simulation can identify longitudinal changes in cognitive skills and that cognitive performance deficiencies can then be remediated over time.

METHODS

We used 50 scenarios evaluating 35 residents during 2 subsequent years, and 18 of those 35 residents were evaluated in both years (post graduate years 3 then 4) in the same or similar scenarios. Individual basic knowledge and cognitive performance during simulation-based scenarios were assessed using a 20- to 27-item scenario-specific checklist. Items were labeled as basic knowledge/technical (lower-order cognition) or advanced cognitive/nontechnical (higher-order cognition). Identical or similar scenarios were repeated annually by a subset of 18 residents during 2 successive academic years. For every scenario and item, we calculated group error scenario rate (frequency) and individual (resident) item success. Grouped individuals' success rates are calculated as mean (SD), and item success grade and group error rates are calculated and presented as proportions. For all analyses, α level is 0.05.

RESULTS

Overall PGY4 residents' error rates were lower and success rates higher for the cognitive items compared with technical item performance in the operating room and resuscitation domains. In all 3 clinical domains, the cognitive error rate by PGY4 residents was fairly low (0.00-0.22) and the cognitive success rate by PGY4 residents was high (0.83-1.00) and significantly better compared with previous annual assessments (P < 0.05). Overall, there was an annual decrease in error rates for 2 years, primarily driven by decreases in cognitive errors. The most commonly observed cognitive error types remained anchoring, availability bias, premature closure, and confirmation bias.

CONCLUSIONS

Simulation-based assessments can highlight cognitive performance areas of relative strength, weakness, and progress in a resident or resident cohort. We believe that they can therefore be used to inform curriculum development including activities that require higher-level cognitive processing.

The OSPE

OBJECTIVES

Resident assessment tends to consist of multiple-choice examinations, even in nuanced areas, such as quality assurance. Internal medicine and many other specialties use objective structured clinical examinations, or OSCEs, to evaluate residents. We adapted the OSCE for pathology, termed the Objective Structured Pathology Examination (OSPE).

METHODS

The OSPE was used to evaluate first- and second-year residents over 2 years. The simulation included an anatomic pathology sign-out session, where the resident could be evaluated on diagnostic skills and knowledge of key information for cancer staging reports, as well as simulated frozen-section analysis, where the resident could be evaluated on communication skills with a "surgeon." The OSPE also included smaller cases with challenging quality issues, such as mismatched slides or gross description irregularities. All cases were scored based on the Pathology Milestones created by the Accreditation Council for Graduate Medical Education.

RESULTS

Using this OSPE, we were able to demonstrate that simulated experiences can be an appropriate tool for standardized evaluation of pathology residents.

CONCLUSIONS

Yearly evaluation using the OSPE could be used to track the progress of both individual residents and the residency program as a whole, identifying problem areas for which further educational content can be developed.

Lack of Association between Blood Pressure Management by Anesthesia Residents and Competence Committee Evaluations or In-training Exam Performance: A Cohort Analysis

BACKGROUND

Prompt treatment of severe blood pressure instability requires both cognitive and technical skill. The ability to anticipate and respond to episodes of hemodynamic instability should improve with training. The authors tested the hypothesis that the duration of severe hypotension during anesthesia administered by residents correlates with concurrent adjusted overall performance evaluations by the Clinical Competence Committee and subsequent in-training exam scores.

METHODS

The authors obtained data on 70 first- and second-year anesthesia residents at the Cleveland Clinic. Analysis was restricted to adults having noncardiac surgery with general anesthesia. Outcome variables were in-training exam scores and subjective evaluations of resident performance ranked in quintiles. The primary predictor was cumulative systolic arterial pressure less than 70 mmHg. Secondary predictors were administration of vasopressors, frequency of hypotension, average duration of hypotensive episodes, and blood pressure variability.

RESULTS

The primary statistical approach was mixed-effects modeling, adjusted for potential confounders. The authors considered 15,216 anesthesia care episodes. A total of 1,807 hypotensive episodes were observed, lasting an average of 32 ± 20 min (SD) per 100 h of anesthesia, with 68% being followed by vasopressor administration. The duration of severe hypotension (systolic pressure less than 70 mmHg) was associated with neither Competence Committee evaluations nor in-training exam scores. There was also no association between secondary blood pressure predictors and either Competence Committee evaluations or in-training exam results.

CONCLUSIONS

There was no association between any of the five blood pressure management characteristics and either in-training exam scores or clinical competence evaluations. However, it remains possible that the measures of physiologic control, as assessed from electronic anesthesia records, evaluate useful but different aspects of anesthesiologist performance.