Using Artificial Intelligence to Gauge Competency on a Novel Laparoscopic Training System

OBJECTIVE

Laparoscopic surgical skill assessment and machine learning are often inaccessible to low-and-middle-income countries (LMIC). Our team developed a low-cost laparoscopic training system to teach and assess psychomotor skills required in laparoscopic salpingostomy in LMICs. We performed video review using AI to assess global surgical techniques. The objective of this study was to assess the validity of artificial intelligence (AI) generated scoring measures of laparoscopic simulation videos by comparing the accuracy of AI results to human-generated scores.

DESIGN

Seventy-four surgical simulation videos were collected and graded by human participants using a modified OSATS (Objective Structured Assessment of Technical Skills). The videos were then analyzed via AI using 3 different time and distance-based calculations of the laparoscopic instruments including path length, dimensionless jerk, and standard deviation of tool position. Predicted scores were generated using 5-fold cross validation and K-Nearest-Neighbors to train classifiers.

SETTING

Surgical novices and experts from a variety of hospitals in Ethiopia, Cameroon, Kenya, and the United States contributed 74 laparoscopic salpingostomy simulation videos.

RESULTS

Complete accuracy of AI compared to human assessment ranged from 65-77%. There were no statistical differences in rank mean scores for 3 domains, Flow of Operation, Respect for Tissue, and Economy of Motion, while there were significant differences in ratings for Instrument Handling, Overall Performance, and the total summed score of all 5 domains (Summed). Estimated effect sizes were all less than 0.11, indicating very small practical effect. Estimated intraclass correlation coefficient (ICC) of Summed was 0.72 indicating moderate correlation between AI and Human scores.

CONCLUSIONS

Video review using AI technology of global characteristics was similar to that of human review in our laparoscopic training system. Machine learning may help fill an educational gap in LMICs where direct apprenticeship may not be feasible.

Early intranasal medication administration in out-of-hospital cardiac arrest: Two randomized simulation trials

Objective

Intranasal medications have been proposed as adjuncts to out-of-hospital cardiac arrest (OHCA) care. We sought to quantify the effects of intranasal medication administration (INMA) in OHCA workflows.

Methods

We conducted separate randomized OHCA simulation trials with lay rescuers (LRs) and first responders (FRs). Participants were randomized to groups performing hands-only cardiopulmonary resuscitation (CPR)/automated external defibrillator with or without INMA during the second analysis phase. Time to compression following the second shock (CPR2) was the primary outcome and compression quality (chest compression rate (CCR) and fraction (CCF)) was the secondary outcome. We fit linear regression models adjusted for CPR training in the LR group and service years in the FR group.

Results

Among LRs, INMA was associated with a significant increase in CPR2 (mean diff. 44.1 s, 95% CI: 14.9, 73.3), which persisted after adjustment (p = 0.005). We observed a significant decrease in CCR (INMA 95.1 compressions per min (cpm) vs control 104.2 cpm, mean diff. -9.1 cpm, 95% CI -16.6, -1.6) and CCF (INMA 62.4% vs control 69.8%, mean diff. -7.5%, 95% CI -12.0, -2.9). Among FRs, we found no significant CPR2 delays (mean diff. -2.1 s, 95% CI -15.9, 11.7), which persisted after adjustment (p = 0.704), or difference in quality (CCR INMA 115.5 cpm vs control 120.8 cpm, mean diff. -5.3 cpm, 95% CI -12.6, 2.0; CCF INMA 79.6% vs control 81.2% mean diff. -1.6%, 95% CI -7.4, 4.3%).

Conclusions

INMA in LR resuscitation was associated with diminished resuscitation performance. INMA by FR did not impede key times or quality.

The impact of brief mindfulness meditation on anxiety, cognitive load, and teamwork in emergency simulation training: A randomized controlled trial

BACKGROUND

Emergency simulation training may impose a heavy psychological burden on health professions students (HPS), evoking anxiety, and overwhelming their cognitive load. Mindfulness may have a positive impact on HPS' anxiety, cognitive load, and teamwork during simulation training. The effects of integration of a brief pre-briefing mindfulness meditation (MM) practice into emergency simulation training, on anxiety, cognitive load, and teamwork remain understudied.

METHODS

From November 2021 to June 2022, 70 undergraduate HPS participated in this pilot randomized controlled study. Participants were randomly assigned to the experimental group (EG, n = 24) that completed a brief in-person mindfulness meditation training program (MMTP) (2 h/week during 4 weeks) or a control group (CG, n = 46). Each group was comprised of teams (2 medical students and 1-2 nursing students). Measures of anxiety, mindfulness, cognitive load, and teamwork were collected at 4-time points: baseline, after the completion of the brief MMTP (for the EG only), at the first simulation assessment within one month later, and the second (6-month follow-up) simulation assessment. Student's t and Wilcoxon Mann-Whiney tests were used to compare the groups' measures and changes over time. A one-way repeated measures ANOVA was conducted to assess anxiety and mindfulness scores' changes over time among EG participants.

RESULTS

52 participants completed all time-point assessments. EG participants recorded better state-anxiety, cognitive load, and teamwork scores than the CG participants in both simulation assessments. However, no statistically significant differences were recorded between the EG and the CG in trait-anxiety and mindfulness at the 6-month follow-up assessment.

CONCLUSION

Brief pre-briefing MM practice mitigates HPS' state-anxiety, and promotes cognitive load, and teamwork during emergency simulation training. Simulation instructors should consider MM practice in the emergency healthcare simulation instructional design.

Evidence supporting performance measures of laparoscopic appendectomy through a novel surgical proficiency assessment tool and low-cost laparoscopic training system

BACKGROUND

Laparoscopic training remains inaccessible for surgeons in low- and middle-income countries, limiting its widespread adoption. We developed a novel tool for assessment of laparoscopic appendectomy skills through ALL-SAFE, a low-cost laparoscopy training system.

METHODS

This pilot study in Ethiopia, Cameroon, and the USA assessed appendectomy skills using the ALL-SAFE training system. Performance measures were captured using the ALL-SAFE verification of proficiency tool (APPY-VOP), consisting of a checklist, modified Objective Structured Assessment of Technical Skills (m-OSATS), and final rating. Twenty participants, including novice (n = 11), intermediate (n = 8), and expert (n = 1), completed an online module covering appendicitis management and psychomotor skills in laparoscopic appendectomy. After viewing an expert skills demonstration video, participants recorded their performance within ALL-SAFE. Using the APPY-VOP, participants rated their own and three peer videos. We used the Kruskal-Wallis test and a Many-Facet Rasch Model to evaluate (i) capacity of APPY-VOP to differentiate performance levels, (ii) correlation among three APPY-VOP components, and (iii) rating differences across groups.

RESULTS

Checklist scores increased from novice (M = 21.02) to intermediate (M = 23.64) and expert (M = 28.25), with differentiation between experts and novices, P = 0.005. All five m-OSATS domains and global summed, total summed, and final rating discriminated across all performance levels (P < 0.001). APPY-VOP final ratings adequately discriminated Competent (M = 2.0), Borderline (N = 1.8), and Not Competent (M = 1.4) performances, Χ2 (2,85) = 32.3, P = 0.001. There was a positive correlation between ALL-SAFE checklist and m-OSATS summed scores, r(83) = 0.63, P < 0.001. Comparison of ratings suggested no differences across expertise levels (P = 0.69) or location (P = 0.66).

CONCLUSION

APPY-VOP effectively discriminated between novice and expert performance in laparoscopic appendectomy skills in a simulated setting. Scoring alignment across raters suggests consistent evaluation, independent of expertise. These results support the use of APPY-VOP among all skill levels inside a peer rating system. Future studies will focus on correlating proficiency to clinical practice and scaling ALL-SAFE to other settings.

Effectiveness of Lay Bystander Hands-Only Cardiopulmonary Resuscitation on a Mattress versus the Floor: A Randomized Cross-Over Trial

STUDY OBJECTIVE

Bystander cardiopulmonary resuscitation increases the likelihood of out-of-hospital cardiac arrest survival by more than two-fold. A common barrier to the prompt initiation of compressions is moving victims to the floor, but compression quality on a "floor" versus a "mattress" has not been tested among lay bystanders.

METHODS

We conducted a prospective, randomized, cross-over trial comparing lay bystander compression quality using a manikin on a bed versus the floor. Participants included adults without professional health care training. We randomized participants to the order of manikin placement, either on a mattress or on the floor. For both, participants were instructed to perform 2 minutes of chest compressions on a cardiopulmonary resuscitation Simon manikin Gaumard (Gaumard Scientific, Miami, FL). The primary outcome was mean compression depth (cm) over 2 minutes. We fit a linear regression model adjusted for scenario order, age, sex, and body mass index with robust standard errors to account for repeated measures and reported mean differences with 95% confidence intervals (CIs).

RESULTS

Our sample of 80 adults was 66% female with a mean age of 50.5 years (SD 18.2). The mean compression depth on the mattress was 2.9 cm (SD 2.3) and 3.5 cm (SD 2.2) on the floor, a mean difference of 0.58 cm (95% CI 0.18, 0.98). Compression depth fell below the 5 to 6 cm depth recommended by the American Heart Association on both surfaces. In the adjusted model, the mean depth was greater when the manikin was on the floor than the mattress (adjusted mean difference 0.62 cm; 95% CI 0.23 to 1.01), and mean depth was less for females than males (adjusted mean difference -1.42 cm, 95% CI -2.59, -0.25). In addition, the difference in compression depth was larger for female participants (mean difference 0.94 cm; 95% CI 0.54, 1.34) than for male participants (mean difference -0.01 cm; 95% CI -0.80, 0.78), and the interaction was statistically significant (P = .04).

CONCLUSION

The mean compression depth was significantly smaller on the mattress and with female bystanders. Further research is needed to understand the benefit of moving out-of-hospital cardiac arrest victims to the floor relative to the detrimental effect of delaying chest compressions.

The Fundamentals of Vaginal Surgery pilot study: developing, validating, and setting proficiency scores for a vaginal surgical skills simulation system

BACKGROUND

Surgical training in the simulation lab can develop basic skills that translate to the operating room. Standardized, basic skills programs that are supported by validated assessment measures exist for open, laparoscopic, and endoscopic surgery; however, there is yet to be a nationally recognized and widely implemented basic skills program specifically for vaginal surgery.

OBJECTIVE

Develop a vaginal surgical simulation system; evaluate robust validity evidence for the simulation system and its related performance measures; and establish a proficiency score that discriminates between novice and experienced vaginal surgeon performance.

STUDY DESIGN

In this 3-phased study, we developed the Fundamentals of Vaginal Surgery simulation system consisting of (1) the Fundamentals of Vaginal Surgery Trainer, a task trainer; (2) a validated regimen of tasks to be performed on the trainer; and (3) performance measures to determine proficiency. In Phase I, we developed the task trainer and selected surgical tasks by performing a needs assessment and hierarchical task analyses, with review and consensus from an expert panel. In Phase II, we conducted a national survey of vaginal surgeons to collect validity evidence regarding test content, response process, and internal structure relevant to the simulation system. In Phase III, we compared performance of novice (first and second year residents) and experienced (third and fourth year residents, fellows, and faculty) surgeons on the simulation system to evaluate relevant relationships to other variables and consequences. Performance measures were analyzed to set a proficiency score that would discriminate between novice and expert (faculty) vaginal surgical performance.

RESULTS

A novel task trainer and 6 basic vaginal surgical skills were developed in Phase I. In Phase II, the survey responses of 48 participants (27 faculty surgeons, 6 fellows, and 14 residents) were evaluated on the dimensions of test content, response process, and internal structure. To support evidence of test content, the participants deemed the task trainer and surgical tasks representative of intended surgical field and supportive of typical surgical actions (mean scores, 3.8-4.4/5). For response process, rater-data analysis revealed high rating variability regarding prototype color. This early evidence confirmed the value of a white prototype. For internal structure, there was high agreement among rater groups (obstetricians and gynecologists generalists vs Female Pelvic Medicine and Reconstructive Surgery specialists: interclass correlation coefficient range, 0.59-0.91; learners vs faculty interclass correlation coefficient range, 0.64-1.0). There were no differences in ratings across institution type, surgeon volume, expertise (P>.14). In Phase III, we analyzed performance from 23 participants (15 [65%] obstetricians and gynecologists residents, 3 [13%] fellows, and 5 [22%] Female Pelvic Medicine and Reconstructive Surgery faculty). Experienced surgeons scored significantly higher than novice surgeons (median, 467.5; interquartile range, [402.5-542.5] vs median, 261.5; interquartile range, [211.5-351.0]; P<.001). Based on these data, setting a proficiency score threshold at 400 results in 0% (0/6) novices attaining the score, with 100% (5/5) experts exceeding it.

CONCLUSION

We present validity evidence relevant to all 5 sources which supports the use of this novel simulation system for basic vaginal surgical skills. To complement the system, a proficiency score of 400 was established to discriminate between novices and experts.

Meeting the New Joint Commission's Maternal Safety Requirements: A Quantitative Blood Loss Train-the-Trainer Program for Improved Process and Outcomes

INTRODUCTION

Recently, the American College of Obstetricians and Gynecologists Committee on Obstetric Practice encouraged the use of quantitative and cumulative assessment of maternal blood loss (QBL) as part of safety bundles for prevention of postpartum hemorrhage. The aim of this study was to determine whether our train-the-trainer QBL program improved process and patient outcomes.

METHODS

In this pre-post interventional study, 26 "champions" participated in the QBL train-the-trainer program in preparation for patient safety bundle rollout. The multipart program included at least 3 simulated C-section hemorrhage scenarios and rapid-cycle debriefing. Champions used the curriculum for broad staff training for 2 months before safety bundle implementation.Six months each of preintervention and postintervention data were collected from medical records of all qualified cases (N = 4413). Process outcomes were evaluated by comparing number of women who received QBL. Patient outcomes were evaluated by comparing blood transfusion rates and number of morbid cases (≥4 packed red blood cell units). χ2 test was used for analyses.

RESULTS

Review of medical records indicated that 99.8% of 2218 women received QBL posttraining compared with 16.6% of 2195 women receiving QBL before training (P = 0.0001.) Transfusion rates increased from 54.47 to 69.25 units/1000 mothers. The rate of morbid cases (≥4 red blood cell units) dropped from 2.13 to 0.48 cases/1000 mothers.

CONCLUSIONS

Our findings suggest the train-the-trainer QBL program was associated with improved process and decreased morbidity when implemented with a hemorrhage safety bundle. Further research is required to examine components of simulation-based QBL training methods and implementation to evaluate their impact on clinical outcomes.

A Cognitive Aid for Neonatal Epinephrine Dosing

OBJECTIVES

Errors are common when preparing epinephrine for neonatal resuscitation. Epinephrine is available in two concentrations (1 mg/mL and 1 mg/10 mL) and requires weight-based calculations, which increases the risk of dosing errors. We developed a printed cognitive aid to assist with dose preparation. We hypothesized that the cognitive aid would result in a 25% difference in errors in preparing the dose of epinephrine during simulated neonatal resuscitation.

METHODS

Nurses (N = 100) in a large academic and community hospital were randomly assigned to calculate the intended dose and prepare epinephrine for neonatal resuscitation with or without the cognitive aid. Scenarios were video recorded and timed. Secondary outcomes included errors in the written intended dose, errors in choosing the correct epinephrine concentration, and time required to prepare the final dose. Proportions were compared by using Fisher's exact test. Variables influencing dosing errors were investigated by using logistic regression.

RESULTS

Using the cognitive aid significantly decreased the proportion of doses prepared incorrectly (24% vs 50%; P = .01). The aid also decreased errors in choosing the correct epinephrine concentration (12% vs 44%; P < .001), but there was no difference in the written intended dose or the time to prepare the dose. Years of experience, self-perceived math comfort, and anxiety were not predictive of dosing errors.

CONCLUSIONS

A simple cognitive aid decreased epinephrine dosing errors during simulated neonatal resuscitation but did not improve efficiency. Despite the effectiveness of the cognitive aid, errors were not completely eliminated. This is a serious safety risk for newborns and requires additional interventions.

Volume Versus Mass Dosing of Epinephrine for Neonatal Resuscitation: A Randomized Trial

BACKGROUND

Intravenous epinephrine for neonatal resuscitation requires weight-based calculations. Epinephrine is available in 2 different concentrations, increasing the risk of dosing errors. Expert panels have conflicting recommendations for the ordering method. The Neonatal Resuscitation Program recommends the volume (milliliters per kilogram) method, whereas the Institute for Safe Medication Practices recommends the mass (milligrams per kilogram) method. In this study, we aim to determine if the mass method is more accurate and efficient than the volume method.

METHODS

In a randomized crossover simulation study, 70 NICU and pediatric emergency department nurses calculated the intended dose then prepared epinephrine using both the mass and volume methods. Both epinephrine concentrations were available. Scenarios were video recorded and timed. The primary outcome was the proportion of epinephrine doses prepared correctly. Variables associated with correct dosing were analyzed by using logistic regression.

RESULTS

Of 136 total doses, 77 (57%) were prepared correctly. The correct intended dose was calculated more frequently by using the mass method (82% vs 68%; risk difference 15%; 95% confidence interval 3% to 26%), but there was no difference in the proportion of doses that were actually prepared correctly (53% of mass method doses versus 60% of volume method doses; risk difference -7%; 95% confidence interval -24% to 9%). There was no difference between methods in the time required to prepare the dose. Selecting the correct epinephrine concentration was the only variable associated with correct dosing.

CONCLUSIONS

The mass method was neither more accurate nor more efficient. Nurses made frequent errors when using both methods. This is a serious patient safety risk. Additional educational and medication safety interventions are urgently needed.

Development of a high fidelity subglottic stenosis simulator for laryngotracheal reconstruction rehearsal using 3D printing

INTRODUCTION

Laryngotracheal reconstruction (LTR) with cartilage graft augmentation is an effective treatment for subglottic stenosis and a critical advanced procedure for Pediatric Otolaryngologists. Trainees almost exclusively learn this procedure intraoperatively on children due to the lack of adequate pediatric training models. An enhanced and accelerated educational experience may be possible if trainees can rehearse the key portions of the procedure on a simulation model.

OBJECTIVE

To design and manufacture a low-cost, high fidelity surgical simulation model of subglottic stenosis for LTR.

METHODS

This simulator is composed of two component models: rib cartilage and trachea. Additive manufacturing techniques, including Computer Aided Design and Three Dimensional (3D) printing, were utilized to create the simulator. Three expert Pediatric Otolaryngologists rated the functionality and realism of the simulator using Likert scale survey data.

RESULTS

The use of CAD and 3D printing techniques allowed for realistic, reproducible surgical simulation of key aspects of LTR. The validation evidence indicated good to excellent means across the five domains relevant to the simulator's fidelity and usability (M = 3.47 to 4.00) out of a maximum of 4 points. Lowest rated items were consistent with expert comments suggesting minor simulator improvements. Time of production is approximately 20 h from print to post-processing, and consumable material costs per model are $2.60 USD.

CONCLUSIONS

This subglottic stenosis airway simulator facilitated Laryngotracheal Reconstruction rehearsal and is a promising training tool for pediatric otolaryngologists. Our methods allow patient-specific, pre-surgical rehearsal for complex airway scenarios that could benefit the experienced airway surgeon and trainees alike. Future research aims to validate this device's utility for accelerating attainment of proficiency and improving surgical outcomes.

This is Not Brain Surgery: Increasing Neurosurgical Knowledge and Retention in Medical Students Through Usage of a Video-Based Education Curriculum

INTRODUCTION

The fundamental elements of neurosurgical care are essential for all medical students regardless of their ultimate clinical specialty. However, the high-pressured neurosurgical environment leaves limited time for teaching medical students. One potential solution is the use of video-based education modules, but the effectiveness and usage of these modules in surgery is unclear.

METHODS

A 2-wk asynchronous, VBE (video based education) curriculum of four key neurosurgical topics as outlined in the Congress of Neurological Surgeons' curriculum for medical students (Intracranial Hemorrhage, Neuro-Imaging, Hydrocephalus, and Glasgow Coma Scale) was created and implemented for surgery clerkship students (n = 65). On day 1 of the study, each student was randomly assigned to 1 of 2 pairs, given a pretest for knowledge and self-efficacy on all 4 topics, and provided with the link for 2 of the 4 VBE modules. On day 14, a post-test for knowledge and self-efficacy on all four topics was completed. Usage analytics were employed to track views of assigned content.

RESULTS

Students who watched the modules (n = 53) increased their knowledge (+11.0%, P = .001) and self-efficacy (+1.37, P = .001) from pre- to post. Students who did not watch the modules (n = 12) showed no change in knowledge (58.8% vs 58.3%, P = NS), but a significant increase in their self-efficacy ratings (+1.42, P = .009). Learning analytics revealed that vast majority of learners (81%) engaged with the curriculum and watched their assigned videos once or on multiple occasions. 19% of learners did not engage with the curriculum, citing their heavy workload as a primary reason.

CONCLUSION

This study shows that a focused, asynchronous, VBE curriculum in neurosurgery has a significantly positive effect on knowledge and self-efficacy scores amongst medical students. Future studies will investigate how to improve learner compliance and better understand the gap in knowledge improvement vs self-efficacy.

Simulation training enables emergency medicine providers to rapidly and safely initiate extracorporeal cardiopulmonary resuscitation (ECPR) in a simulated cardiac arrest scenario

BACKGROUND

Extracorporeal cardiopulmonaryresuscitation (ECPR) is emerging as a viable rescue strategy for refractory out-of-hospital cardiac arrest. In the U.S., limited training of emergency medicine providers is a barrier to widespread implementation.

AIMS

Test the hypothesis that emergency medicine physicians and nurses can acquire and retain the skills to rapidly and safely initiate ECPR using high-fidelity simulation.

STUDY DESIGN

Prospective interventional study.

SETTING

U.S. tertiary academic medical center.

SUBJECTS

Emergency medicine physicians and nurses with no prior ECPR/ECMO experience.

METHODS

Teams of three physicians and three nurses underwent a two-day ECPR training course including didactics, hands-on training, and simulation. Teams were videotaped initiating ECPR in a high-fidelity simulation scenario before and after simulation training. The primary outcome was the proportion of simulations in which full ECPR support was achieved within 30 min of patient arrival.

RESULTS

Five teams completed the entire study. Full ECPR support was achieved within 30 min of patient arrival in 11/15, 15/15, and 15/15 attempts at baseline (B), post-testing (PT) and 3-month post-testing (3-PT), respectively (p = 0.06). Intervals (mean ± sd) required to achieve full ECPR support at B, PT, and 3-PT were 25.8±5.3, 17.2±4.6, and 19.2±1.9 min respectively (p < 0.05 for B vs. PT and 3-PT).

CONCLUSION

High fidelity simulation training is effective in preparing emergency medicine physicians and nurses to rapidly and safely initiate ECPR in a simulated cardiac arrest scenario, and should be considered when implementing an ED-based ECPR program.

Simulation-Based Medical Emergencies Education for Dental Students: A Three-Year Evaluation

This aim of this study was to develop and evaluate a simulation program for dental students to supplement a lecture-based medical emergencies course. Students' self-reported knowledge, experience, confidence, and ability regarding medical emergencies were assessed as program outcomes. For three years (in 2014, 2015, and 2016), all second-year students (N=333) at one U.S. dental school were randomly assigned to groups of 15 and participated in 15 simulated clinical scenarios. All students completed a 21-item pre-post survey and rated their knowledge, experience, and confidence using simulated emergencies. Following the intervention, students' ability to complete critical actions was also peer-assessed using a ten-item checklist. Four open-ended questions were included on the post-intervention survey for acquisition of additional data. For all years, students' self-reported measurements significantly improved with high practical impact (p≤0.001, g=|0.62, 3.93|), with the exception of calling 911 (knowledge). Peer-rated performance indicated the students were deficient (<75% success) in the following: inhaler use, dose of local anesthetic, dose of epinephrine, and EpiPen use. Content analysis of students' comments pointed to areas that need improvement but found high satisfaction with the program. These findings indicate that this program improved students' knowledge, experience, and confidence using simulated medical emergencies.

Evaluating a Solely Mechanical Articulating Laparoscopic Device: A Prospective Randomized Crossover Study

BACKGROUND

The FlexDex^® (FD) is a solely mechanical articulating device that combines the functionality of robotic surgery with the relative low cost and simplicity of laparoscopy. We sought to evaluate the performance of first-time FD users while performing a simple suture task at locations of varying degrees of difficulty.

STUDY DESIGN

A prospective, randomized crossover study was performed comparing the FD to standard laparoscopy (SL). Two specific groups were evaluated; Group 1 consisted of complete novices, and Group 2 consisted of surgical trainees. Participants performed a simple suture with both FD and SL locations of varying degrees of difficulty (Easy, Moderate, and Hard). The following outcomes were evaluated: Instrument Function and Ergonomics (Comfort/Ergonomics survey), Task Difficulty (National Aeronautics and Space Administration Task Load Index [NASA-TLX]), Task Performance Quality (Objective Structured Assessment of Technical Skills [OSATS]), and Time (seconds).

RESULTS

Twenty-two participants were enrolled with 12 participants in Group 1 and 10 participants in Group 2. Group 1-FD participants experienced overall less shoulder strain (1.2 ± 0.40 versus 1.9 ± 0.90, P = .01), and Group 2-FD participants experienced less shoulder (2.5 ± 0.66 versus 4.0 ± 0.50, P = .01), back (1.1 ± 0.32 versus 1.9 ± 0.74, P = .01), and forearm strain (1.9 ± 0.88 versus 2.5 ± 1.1, P = .04). Group 1 participants using the FD experienced higher mental demand (73 ± 17 versus 48 ± 27, P < .01) and perceived effort (70 ± 20 versus 54 ± 23, P < .001). Both Group 1 and Group 2 FD participants performed tasks at the Hard location more effectively. Both Group 1 (70 versus 87, P = .21) and Group 2 (53 versus 60, P = .55) performed tasks at the Hard location in similar times, while Group 1 (80 versus 177, P = .03) and Group 2 (33 versus 70, P = .001) performed tasks at the Easy location in shorter times using SL.

CONCLUSIONS

This study demonstrates the first assessment of the FD, a mechanically articulating laparoscopic tool. First-time FD users demonstrated improved ergonomics and effectiveness suturing at difficult locations. Future studies will focus on comparison to robotic surgery and translation into clinical applications.

An approach to value-based simulator selection: The creation and evaluation of the simulator value index tool

BACKGROUND

Currently there is no reliable, standardized mechanism to support health care professionals during the evaluation of and procurement processes for simulators. A tool founded on best practices could facilitate simulator purchase processes.

METHODS

In a 3-phase process, we identified top factors considered during the simulator purchase process through expert consensus (n = 127), created the Simulator Value Index (SVI) tool, evaluated targeted validity evidence, and evaluated the practical value of this SVI. A web-based survey was sent to simulation professionals. Participants (n = 79) used the SVI and provided feedback. We evaluated the practical value of 4 tool variations by calculating their sensitivity to predict a preferred simulator.

RESULTS

Seventeen top factors were identified and ranked. The top 2 were technical stability/reliability of the simulator and customer service, with no practical differences in rank across institution or stakeholder role. Full SVI variations predicted successfully the preferred simulator with good (87%) sensitivity, whereas the sensitivity of variations in cost and customer service and cost and technical stability decreased (≤54%). The majority (73%) of participants agreed that the SVI was helpful at guiding simulator purchase decisions, and 88% agreed the SVI tool would help facilitate discussion with peers and leadership.

CONCLUSION

Our findings indicate the SVI supports the process of simulator purchase using a standardized framework. Sensitivity of the tool improved when factors extend beyond traditionally targeted factors. We propose the tool will facilitate discussion amongst simulation professionals dealing with simulation, provide essential information for finance and procurement professionals, and improve the long-term value of simulation solutions. Limitations and application of the tool are discussed.

Assessing anesthesiology residents' out-of-the-operating-room (OOOR) emergent airway management

BACKGROUND

At many academic institutions, anesthesiology residents are responsible for managing emergent intubations outside of the operating room (OOOR), with complications estimated to be as high as 39%. In order to create an OOOR training curriculum, we evaluated residents' familiarity with the content and correct adherence to the American Society of Anesthesiologists' Difficult Airway Algorithm (ASA DAA).

METHODS

Residents completed a pre-simulation multiple-choice survey measuring their understanding and use of the DAA. Residents then managed an emergent, difficult OOOR intubation in the simulation center, where two trained reviewers assessed performance using checklists. Post-simulation, the residents completed a survey rating their behaviors during the simulation. The primary outcome was comprehension and adherence to the DAA as assessed by survey responses and behavior in the simulation.

RESULTS

Sixty-three residents completed both surveys and the simulation. Post-survey responses indicated a shift toward decreased self-perceived familiarity with the DAA content compared to pre-survey responses. During the simulation, 22 (35%) residents were unsuccessful with intubation. Of these, 46% placed an LMA and 46% prepared for cricothyroidotomy. Nineteen residents did not attempt intubation. Of these, only 31% considered LMA placement, and 26% initiated cricothyroidotomy.

CONCLUSIONS

Many anesthesiology residency training programs permit resident autonomy in managing emergent intubations OOOR. Residents self-reported familiarity with the content of and adherence to the DAA was higher than that observed during the simulation. Curriculum focused on comprehension of the DAA, as well as improving communication with higher-level physicians and specialists, may improve outcomes during OOORs.

Preliminary Evaluation of a Novel Rigid Bronchoscopy Simulator

PURPOSE

Emergent retrieval of airway foreign bodies (AFBs) in children remains a priority skill set for pediatric surgeons. In the setting of low procedural volume, simulation-based education with deliberate practice is essential to ensure trainees reach expected surgical competency. The purposes of this work were to (1) create a realistic rigid bronchoscopy for AFB retrieval simulation model and (2) to evaluate preliminary validity evidence of a novel simulator for the use of training and assessing pediatric surgical trainees' rigid bronchoscopy skills.

METHODS

After institutional review board exemption determination, 18 participants performed AFB retrieval of two different objects on a novel simulator that represented an 18-month-old pediatric tracheobronchial airway. Participants reported their experience and comfort level, and rated the simulator across two domains-Authenticity and their Ability to perform tasks. Authenticity was measured by 23 items across five subdomains (Visual Attributes, Materials' Response, Realism of Experience, Value and Relevance, and Global Value). Participants who had previously performed ≥10 rigid bronchoscopies were categorized as "experienced," while those reporting <10 were considered "novice." Validity evidence relevant to test content and internal structure was evaluated using a many-facet Rasch model.

RESULTS

Novice surgeons (n = 12) had previously performed a mean of 2.7 (±2.0) rigid bronchoscopies, compared to 15.4 (±7.7) by experienced surgeons (n = 6). For both models, the Value and Relevance subdomain received the highest ratings (observed average [OA] = 3.9, while Materials' Response received the lowest (OA <3.0). Participants' Global Value rating for this model was consistent with "requires minor improvements before it can be considered for use in rigid bronchoscopy training."

CONCLUSIONS

We successfully designed, assembled, and evaluated a novel pediatric rigid bronchoscopy model for AFB retrieval. The model was considered as relevant to educational needs and valuable as a testing and training tool. With recommended improvements, the model could be used for implementation with a Mastery Learning curriculum.

Preliminary Evaluation of a Laparoscopic Common Bile Duct Simulator for Pediatric Surgical Education

PURPOSE

Laparoscopic common bile duct exploration (LCBDE) decreases overall costs and length of stay in patients with choledocolithiasis. However, utilization of LCBDE remains low. We sought to evaluate a previously developed general surgery LCBDE simulator among a cohort of pediatric surgical trainees. The study purpose was to evaluate the content validity of an LCBDE simulator to support or refute its use in pediatric surgery education.

MATERIALS AND METHODS

After IRB exempt determination, 30 participants performed a transcystic LCBDE using a previously developed simulator and evaluated the simulator using a self-reported 28-item instrument. The instrument consisted of two primary domains (Quality and Ability to Perform) that were rated using twenty-five 4-point rating scales and one 4-point global rating scale. Validity evidence relevant to test content was evaluated using a many-facet Rasch model. Interitem consistency was estimated using Cronbach's alpha. P < .05 was considered statistically significant.

RESULTS

The highest combined observed averages were for the Value subdomain (OA = 3.79), whereas the lowest ratings were for the Physical/visual attributes subdomain (OA = 3.19). The averaged global rating was 3.14, consistent with this simulator can be considered for use in pediatric LCBDE training, but could be improved slightly. Rasch indices were favorable and supported evidence relevant to test content. Interitem consistency estimates were also favorable, with α values of 0.94 and 0.56 for Qualities and Ability, respectively.

CONCLUSIONS

Overall, participants rated the LCBDE simulator highly valuable for pediatric surgical education and felt that it could be used as an educational tool with minor modifications.

Simulator and 2 tools: Validation of performance measures from a novel neurosurgery simulation model using the current Standards framework

BACKGROUND

Ventriculostomy is a common neurosurgical procedure with a relatively steep learning curve. A low-cost, high-fidelity simulator paired with procedure-specific performance measures would provide a safe environment to teach ventriculostomy procedural skills. The same validated simulation model could also allow for assessment of trainees' proficiencies with measures that align with Accreditation Council for Graduate Medical Education milestones. This study extends previous work to evaluate validity evidence from the simulator, its newly developed performance assessment, the Ventricolostomy Procedural Assessment Tool, and the Objective Structured Assessment for Technical Skills.

METHODS

After Institutional Review Board exemption, performance data were collected from 11 novice and 3 expert neurosurgeons (n = 14). Participants self-reported their ability to perform tasks on the simulator using the Ventricolostomy Procedural Assessment Tool, an 11-item, step-wise instrument with 5-point rating scales ranging from 1 (unable to perform) to 5 (performs easily and smoothly). De-identified operative performances were videotaped and independently rated by 3 neurosurgeons, using the Ventricolostomy Procedural Assessment Tool and Objective Structured Assessment for Technical Skills. We evaluated multiple sources of validity evidence (2014 Standards) to examine psychometric quality of the measures and to test our assumption that the tools could discriminate between novice and expert performances adequately. We used a multifacet Rasch model and traditional indices, such as Cronbach alpha, intraclass correlation, and Wilcoxon signed-rank test estimates.

RESULTS

Validity evidence relevant to test content and response processes was supported adequately. Evidence of internal structure was supported by high interitem consistency (n = 0.95) and inter-rater agreement for most Ventricolostomy Procedural Assessment Tool items (Intraclass correlation coefficient = [0.00, 0.91]) and all Objective Structured Assessment for Technical Skills items (Intraclass correlation coefficient = [0.80, 0.93]). Overall, novices performed at a lower level than experts on both scales (P < .05), supporting evidence relevant to relationships to other variables. Deeper analysis of novice/expert ratings indicated novices attained lower performances ratings for all Ventricolostomy Procedural Assessment Tool and Objective Structured Assessment for Technical Skills items, but statistical significance was only achieved for the Objective Structured Assessment for Technical Skills items (P < .01). Rater bias estimates were favorable, supporting evidence relevant to consequences of testing.

CONCLUSION

Despite a small sample, favorable evidence using current Standards supports the use of the novel simulator and both tools combined for skills training and performance assessment, but challenges (potential threats to validity) should be considered prior to implementation.

A physical simulator for endoscopic endonasal drilling techniques: technical note

In this paper, the authors present a physical model developed to teach surgeons the requisite drilling techniques when using an endoscopic endonasal approach (EEA) to the skull base. EEA is increasingly used for treating pathologies of the ventral and ventrolateral cranial base. Endonasal drilling is a unique skill in terms of the instruments used, the long reach required, and the restricted angulation, and gaining competency requires much practice. Based on the successful experience in creating custom simulators, the authors used 3D printing to build an EEA training model from post-processed thin-cut head CT scans, formulating the materials to provide realistic haptic feedback and endoscope handling. They performed a preliminary assessment at 2 institutions to evaluate content validity of the simulator as the first step of the validation process. Overall results were positive, particularly in terms of bony landmarks and haptic response, though minor refinements were suggested prior to use as a training device.

Ongoing evaluation of video-based assessment of proctors' scoring of the fundamentals of laparoscopic surgery manual skills examination

BACKGROUND

In continued efforts to maintain standards required for high-stakes examination administration, Society of American Gastrointestinal and Endoscopic Surgeons' Fundamentals of Laparoscopic Surgery (FLS) program requires all prospective test proctors to complete the New FLS Proctor Workshop. As the FLS proctor pool and training evolves, the influence of experience on performance is not clear. In the previous work, we proposed that asynchronous web-based performances might be used to train proctors. In this study, we assessed the accuracy of proctors' ratings from asynchronous web-based performances by comparing the sensitivity and the specificity of ratings from proctors having varied experience levels.

METHODS

A sample of 42 (26% of all registered proctors) FLS proctors (recently trained novice, n = 15; intermediate, n = 20; and expert, n = 7) viewed 2 videotaped, laparoscopic-view FLS performances via the web. The first performance (error) contained 1 predetermined critical performance error in each of the 5 tasks (5 total errors), whereas the second performance (no error) contained no critical errors. For both the performances, participants noted time to complete each of the 5 tasks and any critical errors they observed (dichotomously scored). Participants also completed a demographic section that captured years of experience as a proctor. Using the Kruskal-Wallis test, we compared new trainee, intermediate, and expert proctors' recorded task times for both the performances. The sensitivity and the specificity values were also independently calculated using ratings from the error and the no error performances and then compared using the same test.

RESULTS

There were no differences in recorded times across proctor groups for any of the tasks; p = 0.21 and 0.94. Rating sensitivity was 79% (novice), 75% (intermediate), and 83% (expert), with no significant differences across experience levels; p > 0.46. Following removal of the peg transfer task that had particularly low sensitivity, the sensitivity for the remaining 4 tasks increased [88% and 92%]. The specificity was 93% (novice), 86% (intermediate), and 100% (expert); p = 0.046. Comments from less-experienced proctors indicated that their focus on noncritical performance issues, such as "loop not on black line" and "air knot," were consistent with decreased specificity.

CONCLUSIONS

Favorable results of this pilot study suggest that web-based FLS performances may be used for assessing FLS proctors' rating quality with adequate sensitivity and specificity. Decreased rating specificity from intermediate proctors indicates that these proctors would benefit from increased testing frequencies, updated training on current FLS proctoring standards, and ongoing performance review.

Fundamentals of Laparoscopic Surgery manual test: is videotaped performance assessment an option?

BACKGROUND

In efforts to maintain standards required to evaluate the high-stakes assessment, Society of American Gastrointestinal and Endoscopic Surgeons Fundamentals of Laparoscopic Surgery (FLS) requires all new proctors to complete the train-the-proctor workshop. As the pool of FLS proctors expands, new methods to streamline training and quality assurance programs should be considered. We propose that videotaped performances of the FLS manual tasks may be an alternative proxy to live assessment for training of new proctors, but evaluation of proctors' measures from videotaped FLS performances is required before implementation.

METHODS

A 2-phased research consisted of capturing newly trained proctors' (n = 20) ratings of 3 similar FLS performances across 3 stations-live (Live), videotaped-laparoscopic only (Lap Only) view, and videotaped-dual (Dual) views, during the 2012 Society of American Gastrointestinal and Endoscopic Surgeons FLS train-the-proctor workshop. A month later, a sample of proctors (n = 9) viewed videotaped versions of live FLS performances originally observed during the workshop. Captured metrics include recognition of a predefined critical error for each task (dichotomously scored and summed) and time to complete each of the 5 tasks. Analysis of variance compared the proctors' summed ratings for similar performances across Live, Lap Only, and Dual views, whereas paired t test compared recorded times of Lap Only vs Dual views, Live vs web ratings, and proctors' recorded times across the Lap Only and Dual views.

RESULTS

There were neither differences in ratings across Live, Lap Only, and Dual views (p = 0.49) nor in recorded times for performances viewed across Lap Only and Dual viewing options (p = 0.29 and 0.76, respectively). Mean summed performance ratings observed live (4.6) were higher than those observed via the web (4.0), although not significant (p = 0.051). There were no differences in recorded times for identical performances across Live and web observations (p = 0.18 and 0.69, respectively), although findings were limited by sample size for some tasks.

CONCLUSIONS

In spite of limitations, favorable results of this preliminary study supports use of videotaped FLS performances for streamlining training mechanisms for FLS proctors, and developing best practices in standard setting and long-term evaluation of the of FLS proctors.

Preliminary evaluation of a novel thoracoscopic infant lobectomy simulator

PURPOSE

Thoracoscopic lobectomy in infants requires advanced minimally invasive skills. Simulation-based education has the potential to improve complex procedural skills without exposing the patient to undue risks. The study purposes were (1) to create a size-appropriate infant lobectomy simulator and (2) to evaluate validity evidence to support or refute its use in surgical education.

MATERIALS AND METHODS

In this Institutional Review Board-exempt study, a size-appropriate rib cage for a 3-month-old infant was created. Fetal bovine tissue completed the simulator. Thirty-three participants performed the simulated thoracoscopic lobectomy. Participants completed a self-report, 26-item instrument consisting of 25 4-point rating scales (from 1=not realistic to 4=highly realistic) and a one 4-point Global Rating Scale. Validity evidence relevant to test content and response processes was evaluated using the many-facet Rasch model, and evidence of internal structure (inter-item consistency) was estimated using Cronbach's alpha.

RESULTS

Experienced surgeons (observed average=3.6) had slightly higher overall rating than novice surgeons (observed average=3.4, P=.001). The highest combined observed averages were for the domain Physical Attributes (3.7), whereas the lowest ratings were for the domains Realism of Experience and Ability to Perform Tasks (3.4). The global rating was 2.9, consistent with "this simulator can be considered for use in infant lobectomy training, but could be improved slightly." Inter-item consistency for items used to evaluate the simulator's quality was high (α=0.90).

CONCLUSIONS

With ratings consistent with high physical attributes and realism, we successfully created an infant lobectomy simulator, and preliminary evidence relevant to test content, response processes, and internal structure was supported. Participants rated the model as realistic, relevant to clinical practice, and valuable as a learning tool. Minor improvements were suggested prior to its full implementation as an educational and testing tool.

Evaluation of Three Sources of Validity Evidence for a Synthetic Thoracoscopic Esophageal Atresia/Tracheoesophageal Fistula Repair Simulator

PURPOSE

Thoracoscopic esophageal atresia (EA)/tracheoesophageal fistula (TEF) repair is technically challenging. We have previously reported our experiences with a high-fidelity hybrid model for simulation-based educational instruction in thoracoscopic EA/TEF, including the high cost of the tissue for these models. The purposes of this study were (1) to create a low-cost synthetic tissue EA/TEF repair simulation model and (2) to evaluate the content validity of the synthetic tissue simulator.

MATERIALS AND METHODS

Review of the literature and computed tomography images were used to create computer-aided drawings (CAD) for a synthetic, size-appropriate EA/TEF tissue insert. The inverse of the CAD image was then printed in six different sections to create a mold that could be filled with platinum-cured silicone. The silicone EA/TEF insert was then placed in a previously described neonatal thorax and covered with synthetic skin. Following institutional review board-exempt determination, 47 participants performed some or all of a simulated thoracoscopic EA/TEF during two separate international meetings (International Pediatric Endosurgery Group [IPEG] and World Federation of Associations of Pediatric Surgeons [WOFAPS]). Participants were identified as "experts," having 6-50 self-reported thoracoscopic EA/TEF repairs, and "novice," having 0-5 self-reported thoracoscopic EA/TEF repairs. Participants completed a self-report, six-domain, 24-item instrument consisting of 23 5-point rating scales and one 4-point Global Rating Scale. Validity evidence relevant to test content and response processes was evaluated using the many-facet Rasch model, and evidence of internal structure (interitem consistency) was estimated using Cronbach's alpha.

RESULTS

A review of the participants' ratings indicates there were no overall differences across sites (IPEG versus WOFAPS, P=.84) or experience (expert versus novice, P=.17). The highest observed averages were 4.4 (Value of Simulator as a Training Tool), 4.3 (Physical Attributes-chest circumference, chest depth, and intercostal space), and 4.3 (Realism of Experience-fistula location). The lowest observed averages were 3.5 (Ability to Perform-closure of fistula), 3.7 (Ability to Perform-acquisition target trocar sites), 3.8 (Physical Attributes-landmark visualization), 3.8 (Ability to Perform-anastomosis and dissection of upper pouch), and 3.9 (Realism of Materials-skin). The Global Rating Scale was 2.9, coinciding with a response of "this simulator can be considered for use in neonatal TEF repair training, but could be improved slightly." Material costs for the synthetic EA/TEF inserts were less than $2 U.S. per insert.

CONCLUSIONS

We have successfully created a low-cost synthetic EA/TEF tissue insert for use in a neonatal thoracoscopic EA/TEF repair simulator. Analysis of the participants' ratings of the synthetic EA/TEF simulation model indicates that it has value and can be used to train pediatric surgeons, especially those early in their learning curve, to begin to perform a thoracoscopic EA/TEF repair. Areas for model improvement were identified, and these areas will be the focus for future modifications to the synthetic EA/TEF repair simulator.

Testing of the Patients' Insights and Views of Teamwork (PIVOT) Survey: a validity study

OBJECTIVE

To explore patient observations of teamwork-related behaviors such as inter-team communication through a newly designed survey.

METHODS

In this cross-sectional study, 101 patients (N=86) and caregivers (N=15) recruited from the emergency department (ED) of an urban, academic medical center (>85,000 visits/year) completed the 16-item Patients' Insights and Views Observing Teams (PIVOT) Survey. We evaluated validity evidence through descriptive statistics and analysis including a Many-facet Rasch model to determine associations between questionnaire items and sociodemographic characteristics.

RESULTS

Participant responses provided evidence survey items performed well and reflected patients' awareness of team behaviors such as inter-team communication, coordination, and keeping teammates informed. Also, participants responded about the consistency of information from team members and knowing what people's jobs were on the team. Rasch analysis largely supported that the PIVOT items reflected the intended content area and adequacy of ratings scales supporting evidence of response processes. High internal consistency (Cronbach alpha, r=.87) supported evidence of internal structure. As expected, response patterns differed by ED visit acuity level and length of stay.

CONCLUSIONS

The PIVOT survey offered a means to collect patient and caregiver observations of health care teams.

PRACTICE IMPLICATIONS

PIVOT survey responses may contribute to evaluation of teamwork behaviors.

Design and development of a novel thoracoscopic tracheoesophageal fistula repair simulator

Thoracoscopic repair of esophageal atresia with tracheoesophageal fistula (EA/TEF) is a technically challenging surgical procedure. This congenital anomaly is rare; therefore, training opportunities for surgical trainees are limited. There are currently no validated simulation tools available to help train pediatric surgery trainees. The simulator that was developed is a low-cost, reusable model. It simulates the right side of a term neonate chest and contains a tissue block that has been surgically modified to replicate the anatomy of EA/TEF.

Can skills coaches be used to assess resident performance in the skills laboratory?

INTRODUCTION

The purpose of this study was to compare faculty ratings between live versus video-recorded resident performances and faculty versus skills coaches' ratings of video-recorded resident performances.

METHODS

PGY1 residents were observed, video-recorded, and rated during a Verification of Proficiency examination on 4 stations (ie, suturing, laparotomy, central line, and cricothyroidotomy). One surgeon and 2 trained skills coaches independently rated each video-recorded performance (N = 25). The chi-square test was used to compare checklist ratings. Analysis of variance was used to compare global ratings. Intraclass correlations were used to evaluate inter-rater agreement.

RESULTS

There were no statistical differences in faculty checklist ratings for live versus video-recorded performances (P > .05), and we found a nearly perfect interrater agreement, intraclass correlation coefficient (ICC) = 0.99 (P < .001). When comparing faculty versus skills coaches' ratings on video-recorded performances, we found no differences for the global or checklist ratings. Inter-rater agreement was moderately high for the global ratings, ICC = 0.71 (P <. 0.01, 95% confidence interval 0.23-0.96), and nearly perfect for the checklist ratings, ICC = 0.99 (P < .001, 95% confidence interval 0.94-1.00).

CONCLUSION

When assessing residents' performances, use of video-recorded performance ratings and skills coaches may be viable alternatives to live ratings performed by surgical faculty.

Fundamentals of laparoscopic surgery (FLS) manual skills assessment: surgeon vs nonsurgeon raters

OBJECTIVE

The American Board of Surgery has recently started requiring completion of the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) fundamentals of laparoscopic surgery (FLS) program for board certification in general surgery. Although most SAGES Testing Centers utilize nonsurgeons as FLS proctors, the effectiveness of using nonsurgeons as FLS proctors has not been evaluated.

METHODS

Surgeons and nonsurgeons attending FLS proctor training workshops were studied. Participants reviewed training materials before course attendance. Subjects watched a videotaped FLS performance containing 9 "critical" errors, which participants were asked to identify. This assessment was repeated after hands-on training.

RESULTS

Thirteen surgeon and 17 nonsurgeon subjects participated. At baseline, surgeons detected 66% of errors, vs 65% for nonsurgeons, with no statistical difference between groups. Analysis of individual tasks also showed no difference between groups, except for intracorporeal knot-tying (p = 0.049). Both groups improved after training (p < 0.01), with surgeons detecting 81% of errors vs 83% for nonsurgeons (no difference in overall or task-specific ratings).

CONCLUSIONS

This study suggests that trained nonsurgeons may be as effective as surgeon proctors in detecting errors associated with the FLS manual test. This finding supports the utility of using trained nonsurgeons as FLS proctors as surgical training programs face increasing economic constraints.