Twelve tips for developing simulation-based mastery learning clinical skills checklists

Simulation-based mastery learning is a powerful educational paradigm that leads to high levels of performance through a combination of strict standards, deliberate practice, formative feedback, and rigorous assessment. Successful mastery learning curricula often require well-designed checklists that produce reliable data that contribute to valid decisions. The following twelve tips are intended to help educators create defensible and effective clinical skills checklists for use in mastery learning curricula. These tips focus on defining the scope of a checklist using established principles of curriculum development, crafting the checklist based on a literature review and expert input, revising and testing the checklist, and recruiting judges to set a minimum passing standard. While this article has a particular focus on mastery learning, with the exception of the tips related to standard setting, the general principles discussed apply to the development of any clinical skills checklist.

Where in the world: Mapping medical student learning using the Social and Structural Determinants of Health Curriculum Assessment Tool (SSDH CAT)

INTRODUCTION

Addressing the Social and Structural Determinants of Health (SSDH) is a primary strategy for attaining health equity. Teaching and learning about SSDH has increased across medical schools throughout the world; however, the published literature describing these efforts continues to be limited and many unknowns persist including what should be taught and by whom, what teaching methods and settings should be used, and how medical learners should be assessed.

MATERIALS AND METHODS

Based on published studies, input from experts in the field, and elements from the framework developed by the National Academy of Medicine, we created a universal Social and Structural Determinants of Health Curriculum Assessment Tool (SSDH CAT) to assist medical educators to assess existing SSDH curricular content, ascertain critical gaps, and categorize educational methods, delivery, and assessment techniques and tools that could help inform curricular enhancements to advance the goal of training a health care workforce focused on taking action to achieve health equity. To test the usefulness of the tool, we applied the SSDH CAT to map SSDH-related curriculum at a US-based medical school.

RESULTS

By applying the SSDH CAT to our undergraduate medical school curriculum, we recognized that our SSDH curriculum relied too heavily on lectures, emphasized knowledge without sufficient skill building, and lacked objective assessment measures. As a result of our curricular review, we added more skill-based activities such as using evidence-based tools for screening patients for social needs, and created and implemented a universal, longitudinal, experiential community health curriculum.

DISCUSSION

We created a universal SSDH CAT and applied it to assess and improve our medical school's SSDH curriculum. The SSDH CAT provides a starting point for other medical schools to assess their SSDH content as a strategy to improve teaching and learning about health equity, and to inspire students to act on the SSDH.

Simulation-based Mastery Learning Improves Emergency Medicine Residents' Ability to Perform Temporary Transvenous Cardiac Pacing

INTRODUCTION

Temporary transvenous cardiac pacing (TVP) is a critical intervention that emergency physicians perform infrequently in clinical practice. Prior simulation studies revealed that emergency medicine (EM) residents and board-certified emergency physicians perform TVP poorly during checklist-based assessments. Our objective in this report was to describe the design and implementation of a simulation-based mastery learning (SBML) curriculum and evaluate its impact on EM residents' ability to perform TVP.

METHODS

An expert panel of emergency physicians and cardiologists set a minimum passing standard (MPS) for a previously developed 30-item TVP checklist using the Mastery Angoff approach. Emergency medicine residents were assessed using this checklist and a high-fidelity TVP task trainer. Residents who did not meet the MPS during baseline testing viewed a procedure video and completed a 30-minute individual deliberate practice session before retesting. Residents who did not meet the MPS during initial post-testing completed additional deliberate practice and assessment until meeting or exceeding the MPS.

RESULTS

The expert panel set an MPS of correctly performing 28 (93.3%) checklist items. Fifty-seven EM residents participated. Mean checklist scores improved from 13.4 (95% CI 11.8-15.0) during baseline testing to 27.5 (95% CI 26.9-28.1) during initial post-testing (P < 0.01). No residents met the MPS at baseline testing. The 21 (36.8%) residents who did not meet the MPS during initial post-testing all met or exceeded the MPS after completing one additional 30-minute deliberate practice session.

CONCLUSION

Emergency medicine residents demonstrated significantly improved TVP performance with reduced variability in checklist scores after completing a simulation-based mastery learning curriculum.

From Concept to Publication: Effectiveness of the International Network for Simulation-Based Pediatric Innovation, Research, and Education Project Development Process at Generating Simulation Scholarship

BACKGROUND

As simulation matures, it is critical to develop pathways for researchers. A recent analysis, however, demonstrates a low conversion rate between abstract and peer-reviewed journal publication in our field. The International Network for Simulation-based Pediatric Innovation, Research, and Education has used the ALERT Presentation process for the past decade as a means of accelerating research. In this study, we analyze the scholarly products attributable to ALERT Presentations.

METHODS

Surveys were distributed to all International Network for Simulation-based Pediatric Innovation, Research, and Education Advanced Look Exploratory Research Template (ALERT) Presentation first authors from January 2011 through January 2020. Presenters were asked to provide information on abstracts, grants, journal publications, and book chapters related to their ALERT Presentation, as well as basic demographic information. A structured literature search was conducted for those ALERT Presentations whose authors did not return a survey. The resulting database was descriptively analyzed, and statistical correlations between demographic variables and scholarship were examined.

RESULTS

One hundred sixty-five new ALERT presentations were presented over 10 years. We identified 361 associated scholarly works (170 conference abstracts, 125 peer-reviewed journal publications, 65 grants, and 1 book chapter). Sixty-one percent (101 of 165) of ALERT Presentations produced at least 1 item of scholarship, and 59% (34 of 58) of ALERT Presentations that resulted in at least 1 abstract also led to at least 1 peer-reviewed journal article. Presenter gender was associated with likelihood of journal publication.

CONCLUSIONS

The ALERT Presentation process is an effective approach for facilitating the development of projects that result in disseminated scholarship. Wider adoption may benefit other simulation and education research networks.

Assessment and Evaluation in Social Determinants of Health Education: a National Survey of US Medical Schools and Physician Assistant Programs

BACKGROUND

Social determinants of health (SDOH) curricular content in medical schools and physician assistant programs are increasing. However, there is little understanding of current practice in SDOH learner assessment and program evaluation, or what the best practices are.

OBJECTIVE

Our study aim was to describe the current landscape of assessment and evaluation at US medical schools and physician assistant programs as a first step in developing best practices in SDOH education.

DESIGN

We conducted a national survey of SDOH educators from July to December 2020. The 55-item online survey covered learner assessment methods, program evaluation, faculty training, and barriers to effective assessment and evaluation. Results were analyzed using descriptive statistics.

PARTICIPANTS

One hundred six SDOH educators representing 26% of medical schools and 23% of PA programs in the USA completed the survey.

KEY RESULTS

Most programs reported using a variety of SDOH learner assessment methods. Faculty and self were the most common assessors of learners' SDOH knowledge, attitudes, and skills. Common barriers to effective learner assessment were lack of agreement on "SDOH competency" and lack of faculty training in assessment. Programs reported using evaluation results to refine curricular content, identify the need for new content, and improve assessment strategies.

CONCLUSIONS

We identified a heterogeneity of SDOH assessment and evaluation practices among programs, as well as gaps and barriers in their educational practices. Specific guidance from accrediting bodies and professional organizations and agreement on SDOH competency as well as providing faculty with time, resources, and training will improve assessment and evaluation practice and ensure SDOH education is effective for students, patients, and communities.

Expert Consensus Guidelines for Assessing Students on the Social Determinants of Health

PHENOMENON

Assessment and evaluation guidelines inform programmatic changes necessary for educational effectiveness. Presently, no widely accepted guidelines exist for educators to assess learners and evaluate programs regarding social determinants of health (SDOH) during physician and physician assistant (PA) education. We sought to garner expert consensus about effective SDOH learner assessment and program evaluation, so as to make recommendations for best practices related to SDOH education.

APPROACH

We used a Delphi approach to conduct our study (September 2019 to December 2020). To administer our Delphi survey, we followed a three-step process: 1) literature review, 2) focus groups and semi-structured interviews, 3) question development and refinement. The final survey contained 72 items that addressed SDOH content areas, assessment methods, assessors, assessment integration, and program evaluation. Survey participants included 14 SDOH experts at US medical schools and PA programs. The survey was circulated for three rounds seeking consensus, and when respondents reached consensus on a particular question, that question was removed from subsequent rounds.

FINDINGS

The geographically diverse sample of experts reached consensus on many aspects of SDOH assessment and evaluation. The experts selected three important areas to assess learners' knowledge, skills, and attitudes about SDOH. They identified assessment methods that were "essential", "useful, but not essential", and "not necessary." The essential assessment methods are performance rating scales for knowledge and attitudes and skill-based assessments. They favored faculty and patients as assessors, as well as learner self-assessment, over assessments conducted by other health professionals. Questions about separation versus incorporation of SDOH assessment with other educational assessment did not yield consensus opinion. The experts reached consensus on priority outcome measures to evaluate a school's SDOH program which included student attitudes toward SDOH, Competence-Based Assessment Scales, and the percentage of graduates involved in health equity initiatives.

INSIGHTS

Based on the Delphi survey results, we make five recommendations that medical and PA educators can apply now when designing learner assessments and evaluating SDOH programming. These recommendations include what should be assessed, using what methods, who should do the assessments, and how they should be incorporated into the curriculum. This expert consensus should guide future development of an assessment and evaluation toolkit to optimize SDOH education and clinical practice.

Supplemental data for this article is available online at https://doi.org/10.1080/10401334.2022.2045490 .

Varying Estimates of Sepsis among Adults Presenting to US Emergency Departments: Estimates from a National Dataset from 2002-2018

Background

A variety of approaches to defining sepsis using administrative datasets have been previously reported. We aimed to compare estimates, demographics, treatment factors, outcomes and longitudinal trends of patients identified with sepsis in United States emergency departments (EDs) using differing sets of sepsis criteria.

Methods

We performed a cross-sectional study using the National Healthcare Ambulatory Medical Care Survey, a complex survey of nonfederal US ED encounters between 2002 to 2018. We obtained survey-weighted population-adjusted encounters of sepsis using the following criteria: explicit sepsis, severe sepsis, and quick Sequential Organ Failure Assessment (qSOFA) score combined with the presence of infection.

Results

Age-adjusted for US adults, 18.6, 16.1 and 8.9 encounters per 10 000 population were identified when using the explicit, severe sepsis and qSOFA definitions, respectively. A higher proportion of the explicit cohort was hospitalized and had blood cultures performed, compared to cohorts ascertained using severe sepsis and qSOFA criteria, though confidence intervals overlapped. Antibiotic use was highest in encounters meeting qSOFA criteria. When inspecting unweighted encounters meeting each set of criteria, there was minimal overlap, with only 3% meeting all three. Encounters meeting the explicit and severe sepsis criteria were increasing over time.

Conclusion

The explicit, severe sepsis and qSOFA criteria generated similar annual rates of presentation when applied to US ED encounters, with some evidence of the explicit sepsis cohort being higher acuity. There was minimal overlap of cases and instability in estimates when assessed longitudinally. Our findings inform research efforts to accurately identify sepsis among ED encounters using administrative data.

Outcomes and Factors Associated With Prehospital Treatment of Pediatric Anaphylaxis

OBJECTIVE

Anaphylaxis is a potentially life-threatening reaction requiring prompt treatment with intramuscular epinephrine (EPI). We sought to describe presenting features of pediatric anaphylaxis and compare patient characteristics and outcomes of children treated with prehospital EPI with those untreated.

METHODS

We abstracted data from emergency department (ED) records for children meeting the National Institute of Allergy & Infectious Disease criteria for anaphylaxis (2015-2017) in one tertiary care children's hospital. We analyzed associations between patient characteristics and outcomes and receipt of prehospital EPI using descriptive statistics and multivariate logistic regression.

RESULTS

Of 414 children presenting with anaphylaxis, 39.4% received IM EPI and 62.1% received antihistamines before hospital arrival. Children with Medicaid received pre-emergency department EPI less frequently than did children with private insurance (24.5% vs 43.8%, P = 0.001). Factors positively associated with prehospital EPI administration were history of food allergy (odds ratio [OR], 4.4 [95% confidence interval {CI}, 2.4-8.2]) or arrival by emergency medical services (OR, 8.0 [95% CI, 4.2-15.0]). Medicaid insurance was associated with decreased odds of prehospital EPI (OR, 0.33 [95% CI, 0.16-0.66]) and prehospital H1-antihistamine use (OR, 0.30 [95% CI, 0.17-0.56]). Prehospital EPI treatment was also associated with decreased rates of observation (37% vs 63%), inpatient admission (38% vs 62%), and intensive care unit admission (20% vs 80%) compared with no pretreatment (P = 0.03).

CONCLUSIONS

Prehospital treatment with EPI remains low, and barriers to optimal treatment are more pronounced in children with public insurance. Prehospital treatment with EPI was associated with decreased morbidity including hospitalization and intensive care unit admission.

Impact of Personal Protective Equipment on the Performance of Emergency Pediatric Tasks

OBJECTIVES

Personal protective equipment (PPE) is worn by health care providers (HCPs) to protect against hazardous exposures. Studies of HCPs performing critical resuscitation tasks in PPE have yielded mixed results and have not evaluated performance in care of children. We evaluated the impacts of PPE on timeliness or success of emergency procedures performed by pediatric HCPs.

METHODS

This prospective study was conducted at 2 tertiary children's hospitals. For session 1, HCPs (medical doctors and registered nurses) wore normal attire; for session 2, they wore full-shroud PPE garb with 2 glove types: Ebola level or chemical. During each session, they performed clinical tasks on a patient simulator: intubation, bag-valve mask ventilation, venous catheter (IV) placement, push-pull fluid bolus, and defibrillation. Differences in completion time per task were compared.

RESULTS

There were no significant differences in medical doctor completion time across sessions. For registered nurses, there was a significant difference between baseline and PPE sessions for both defibrillation and IV placement tasks. Registered nurses were faster to defibrillate in Ebola PPE and slower when wearing chemical PPE (median difference, -3.5 vs 2 seconds, respectively; P < 0.01). Registered nurse IV placement took longer in Ebola and chemical PPE (5.5 vs 42 seconds, respectively; P < 0.01). After the PPE session, participants were significantly less likely to indicate that full-body PPE interfered with procedures, was claustrophobic, or slowed them down.

CONCLUSIONS

Personal protective equipment did not affect procedure timeliness or success on a simulated child, with the exception of IV placement. Further study is needed to investigate PPE's impact on procedures performed in a clinical care context.

Development of a Rigorously Designed Procedural Checklist for Assessment of Emergency Medicine Resident Performance of Temporary Transvenous Cardiac Pacing

OBJECTIVES

Temporary transvenous cardiac pacing (TVP) is a potentially lifesaving intervention included in the list of essential core procedures for emergency medicine (EM) training; however, opportunities to perform TVP during residency cannot be guaranteed. EM graduates report feeling subjectively underprepared for this procedure, but objective performance data are lacking. Checklist-based simulated assessment is an increasing focus of competency-based medical education, particularly for invasive procedures like TVP. The objectives of this paper were as follows: first, to enlist a multidisciplinary team of experts to create an assessment tool for TVP using best practices in checklist development; second, to determine the reliability of checklist scoring; and third, to assess EM residents' baseline ability to perform TVP using a dedicated task trainer.

METHODS

This study was conducted at a single 4-year EM residency. A panel of emergency physicians and cardiologists designed a TVP checklist using a modified Delphi approach. After consensus was achieved on a final checklist, EM residents were assessed using a dedicated TVP task trainer. Inter-rater reliability was determined using Cohen's kappa coefficient. Resident performance was determined by number of correctly performed checklist items.

RESULTS

The expert panel achieved consensus on a 30-item checklist after three rounds of revisions. The Cohen's kappa coefficient for the overall checklist score was 0.87, with individual checklist items ranging from 0.63 to 1.00. In total, 58 residents were assessed with a mean score of 13.5 of 30 checklist items. Scores increased with each year of training.

CONCLUSIONS

This study details the rigorous development of a TVP checklist designed by a multidisciplinary team of experts. Checklist scores demonstrated strong inter-rater reliability. The overall poor performance of this cohort suggests the current approach to TVP training does not provide sufficient preparation for EM residents. Competency-based techniques, such as simulation-based mastery learning, should be explored.

Impact of Personal Protective Equipment on Pediatric Cardiopulmonary Resuscitation Performance: A Controlled Trial

OBJECTIVES

This study aimed to determine whether personal protective equipment (PPE) results in deterioration in chest compression (CC) quality and greater fatigue for administering health care providers (HCPs).

METHODS

In this multicenter study, HCPs completed 2 sessions. In session 1 (baseline), HCPs wore normal attire; in session 2, HCPs donned full PPE. During each session, they performed 5 minutes of uninterrupted CCs on a child manikin. Chest compression rate, depth, and release velocity were reported in ten 30-second epochs. Change in CC parameters and self-reported fatigue were measured between the start and 2- and 5-minute epochs.

RESULTS

We enrolled 108 HCPs (prehospital and in-hospital providers). The median CC rate did not change significantly between epochs 1 and 10 during baseline sessions. Median CC depth and release velocity decreased for 5 minutes with PPE. There were no significant differences in CC parameters between baseline and PPE sessions in any provider group. Median fatigue scores during baseline sessions were 2 (at start), 4 (at 2 minutes), and 6 (at 5 minutes). There was a significantly higher median fatigue score between 0 and 5 minutes in both study sessions and in all groups. Fatigue scores were significantly higher for providers wearing PPE compared with baseline specifically among prehospital providers.

CONCLUSIONS

During a clinically appropriate 2-minute period, neither CC quality nor self-reported fatigue worsened to a significant degree in providers wearing PPE. Our data suggest that Pediatric Basic Life Support recommendations for CC providers to switch every 2 minutes need not be altered with PPE use.

Expert Consensus on Inclusion of the Social Determinants of Health in Undergraduate Medical Education Curricula

PURPOSE

Accreditation bodies have mandated teaching social determinants of health (SDH) to medical students, but there has been limited guidance for educators on what or how to teach, and how to evaluate students' competence. To fill this gap, this study aimed to develop an SDH curricular consensus guide for teaching SDH to medical students.

METHOD

In 2017, the authors used a modified Delphi technique to survey an expert panel of educators, researchers, students, and community advocates about knowledge, skills, and attitudes (KSA) and logistics regarding SDH teaching and assessment. They identified the panel and ranked a comprehensive list of topics based on a scoping review of SDH education studies and discussions with key informants. A total of 57 experts were invited.

RESULTS

Twenty-two and 12 panelists participated in Delphi rounds 1 and 2, respectively. The highest-ranked items regarding KSA were "Appreciation that the SDH are some of the root causes of health outcomes and health inequities" and "How to work effectively with community health workers." The panel achieved consensus that SDH should constitute 29% of the total curriculum and be taught continuously throughout the curriculum. Multiple-choice tests were ranked lowest as an assessment method, and patient feedback was ranked highest. Panelists noted that SDH content must be a part of standardized exams to be prioritized by faculty and students.

CONCLUSIONS

An expert panel endorsed essential curricular content, teaching methods, and evaluation approaches that can be used to help guide medical educators regarding SDH curriculum development.

Teaching the Social Determinants of Health in Undergraduate Medical Education: a Scoping Review

BACKGROUND

To provide optimal care, medical students should understand that the social determinants of health (SDH) impact their patients' well-being. Those charged with teaching SDH to future physicians, however, face a paucity of curricular guidance.

OBJECTIVE

This review's objective is to map key characteristics from publications about teaching SDH to students in undergraduate medical education (UME).

METHODS

In 2016, the authors searched PubMed, Embase, Web of Science, the Cochrane and ERIC databases, bibliographies, and MedEdPORTAL for articles published between January 2010 and November 2016. Four reviewers screened articles for eligibility then extracted and analyzed data descriptively. Scoping review methodology was used to map key concepts and curricular logistics as well as educator and student characteristics.

RESULTS

The authors screened 3571 unique articles of which 22 were included in the final review. Many articles focused on community engagement (15). Experiential learning was a common instructional strategy (17) and typically took the form of community or clinic-based learning. Nearly half (10) of the manuscripts described school-wide curricula, of which only three spanned a full year. The majority of assessment was self-reported (20) and often related to affective change. Few studies objectively assessed learner outcomes (2).

CONCLUSIONS

The abundance of initial articles screened highlights the growing interest in SDH in medical education. The small number of selected articles with sufficient detail for abstraction demonstrates limited SDH curricular dissemination. A lack of accepted tools or practices that limit development of robust learner or program evaluation was noted. Future research should focus on identifying and evaluating effective instructional and assessment methodologies to address this gap, exploring additional innovative teaching frameworks, and examining the specific contexts and characteristics of marginalized and underserved populations and their coverage in medical education.

Using the METRICS model for defining routes to scholarship in healthcare simulation

INTRODUCTION

In this paper, we explored the utility and value of the METRICS model for modeling scholarship in healthcare simulation by: (1) describing the distribution of articles in four healthcare simulation journals across the seven areas of METRICS scholarship; and (2) appraising patterns of scholarship expressed in three programs of simulation scholarship and reflecting on how these patterns potentially influence the pursuit of future scholarly activities.

METHODS

Two raters reviewed abstracts of papers published between January 2015 and August 2017 in four healthcare simulation journals and coded them using METRICS. Descriptive statistics were calculated for scholarship type and distribution across journals. Twenty-eight articles from three scholars were reviewed, with patterns of scholarship within articles mapped to METRICS. Descriptive synthesis was constructed through discussion between two reviewers.

RESULTS

A total of 432 articles from four journals were reviewed. The three most commonly published areas of scholarship were: 32.2% (139/432) evaluation, 18.8% (81/432) innovation, and 15.3% (66/432) conceptual. The METRICS model was able to represent different kinds of scholarship expressed in all of the papers reviewed and across programs of research. Reflecting on patterns of scholarship within their scholarly programs was helpful for research in planning future directions.

CONCLUSIONS

The METRICS model for scholarship can describe a wide range of patterns of simulation scholarship within individual articles, programs of research, or across journals.

An institution-wide approach to submission, review, and funding of simulation-based curricula

This article describes the development, implementation, and modification of an institutional process to evaluate and fund graduate medical education simulation curricula. The goals of this activity were to (a) establish a standardized mechanism for proposal submission and evaluation, (b) identify simulation-based medical education (SBME) curricula that would benefit from mentored improvement before implementation, and (c) ensure that funding decisions were fair and defensible. Our intent was to develop a process that was grounded in sound educational principles, allowed for efficient administrative oversight, ensured approved courses were high quality, encouraged simulation education research and scholarship, and provided opportunities for medical specialties that had not previously used SBME to receive mentoring and faculty development.

An Objective Structured Clinical Examination to Improve Formative Assessment for Senior Pediatrics Residents

BACKGROUND

Residency programs are developing new methods to assess resident competence and to improve the quality of formative assessment and feedback to trainees. Simulation is a valuable tool for giving formative feedback to residents.

OBJECTIVE

To develop an objective structured clinical examination (OSCE) to improve formative assessment of senior pediatrics residents.

METHODS

We developed a multistation examination using various simulation formats to assess the skills of senior pediatrics residents in communication and acute resuscitation. We measured several logistical factors (staffing and program costs) to determine the feasibility of such a program.

RESULTS

Thirty-one residents participated in the assessment program over a 3-month period. Residents received formative feedback comparing their performance to both a standard task checklist and to peers' performance. The program required 16 faculty members per session, and had a cost of $624 per resident.

CONCLUSIONS

A concentrated assessment program using simulation can be a valuable tool to assess residents' skills in communication and acute resuscitation and provide directed formative feedback. However, such a program requires considerable financial and staffing resources.

Mastery of Status Epilepticus Management via Simulation-Based Learning for Pediatrics Residents

BACKGROUND

Management of status epilepticus (SE) in the pediatric population is highly time-sensitive. Failure to follow a standard management algorithm may be due to ineffective provider education, and can lead to unfavorable outcomes.

OBJECTIVE

To design a learning module using high-fidelity simulation technology to teach mastery achievement of a hospital algorithm for managing SE.

METHODS

Thirty pediatrics interns were enrolled. Using the Angoff method, an expert panel developed the minimal passing score, which defined mastery. Scoring of simulated performance was done by 2 observers. Sessions were digitally recorded. After the pretest, participants were debriefed on the algorithm and required to repeat the simulation. If mastery (minimal passing score) was not achieved, debriefing and the simulation were repeated until mastery was met. Once mastery was met, participants graded their comfort level in managing SE.

RESULTS

No participants achieved mastery at pretest. After debriefing and deliberate simulator training, all (n=30) achieved mastery of the algorithm: 30% achieved mastery after 1 posttest, 63% after a second, and 6.7% after a third. The Krippendorff α was 0.94, indicating strong interrater agreement. Participants reported more self-efficacy in managing SE, a preference for simulation-based education for learning practice-based algorithms of critical conditions, and highly rated the educational intervention.

CONCLUSIONS

A simulation-based mastery learning program using deliberate practice dramatically improves pediatrics residents' execution of a SE management protocol. Participants enjoyed and benefited from simulation education. Future applications include improving adherence to other hospital protocols.

Impact of simulation training on time to initiation of cardiopulmonary resuscitation for first-year pediatrics residents

BACKGROUND

Pediatrics residents have few opportunities to perform cardiopulmonary resuscitation (CPR). Enhancing the quality of CPR is a key factor to improving outcomes for cardiopulmonary arrest in children and requires effective training strategies.

OBJECTIVE

To evaluate the effectiveness of a simulation-based intervention to reduce first-year pediatrics residents' time for 3 critical actions in CPR: (1) call for help, (2) initiate bag-mask ventilation, and (3) initiate chest compressions.

METHODS

A prospective study involving 31 first-year pediatrics residents at a children's hospital assigned to an early or late (control) intervention group. Residents underwent baseline assessment followed by repeat evaluations at 3 and 6 months. Time to critical actions was scored by video review. A 90-minute educational intervention focused on skill practice was conducted following baseline evaluation for the early-intervention group and following 3-month evaluation for the late-intervention group. Primary outcome was change in time to initiating the 3 critical actions. Change in time was analyzed by comparison of Kaplan-Meier curves, using the log-rank test. A 10% sample was timed by a second rater. Agreement was assessed using intraclass correlation (ICC).

RESULTS

There was a statistically significant reduction in time for all 3 critical actions between baseline and 3-month evaluation in the early intervention group; this was not observed in the late (control) group. Rater agreement was excellent (ICC ≥ 0.99).

CONCLUSIONS

A simulation-based educational intervention significantly reduced time to initiation of CPR for first-year pediatrics residents. Simulation training facilitated acquisition of critical CPR skills that have the potential to impact patient outcome.

Incidence and trends of pediatric ovarian torsion hospitalizations in the United States, 2000-2006

OBJECTIVE

There is significant variation in the literature regarding the characteristics that are associated with pediatric ovarian torsion and its management. National data regarding the demographics and management of pediatric ovarian torsion are lacking. Our objective was to describe the epidemiology of pediatric ovarian torsion and the rate of oophorectomy by using nationally representative data. Demographic factors and hospital characteristics that are associated with rates of oophorectomy were also explored.

METHODS

This was a cohort analysis of the Healthcare Cost and Utilization Project Kids' Inpatient Database (KID) 2000, 2003, and 2006. All females aged 1 to 20 years who were hospitalized with ovarian torsion in states participating in KID 2000, KID 2003, and KID 2006 representing 900, 1224, and 1232 ovarian torsion-related hospitalizations, respectively, were included. Primary outcome measures included the incidence of ovarian torsion and rate of associated oophorectomy. Multivariable regression was used to control for patient and hospital characteristics.

RESULTS

Among females aged 1 to 20 years, there were 1232 cases of ovarian torsion in KID 2006, an estimated incidence of 4.9 per 100000. A total of 713 (58%) were treated with oophorectomy. The rate of ovarian torsion-associated oophorectomy remained unchanged from 2000 to 2006. The adjusted odds of having an oophorectomy decreased by 0.95 for every increasing year of age. Residing in a lower quartile of household income by zip code increased the adjusted odds of oophorectomy. A diagnosis of benign neoplasm increased the adjusted odds of oophorectomy by 2.16. Fewer than 0.5% of ovarian torsion hospitalizations were associated with malignant neoplasm.

CONCLUSIONS

Nationally representative hospital data indicate that ovarian torsion is uncommon but occurs in all ages and is typically associated with normal ovaries or benign lesions. Improved awareness of the epidemiology may help to guide management. Ongoing analysis to identify factors that are associated with successful conservative management is warranted.

Development and evaluation of a simulation-based pediatric emergency medicine curriculum

PURPOSE

The infrequency of severe childhood illness limits opportunities for emergency medicine (EM) providers to learn from real-world experience. Simulation offers an evidence-based educational approach to develop and practice clinical skills.

METHOD

This was a two-phase, randomized trial with a wait-list control condition. The development phase (2005-2006) involved systematic curriculum and rating checklist creation, producing a six-case, simulation-based curriculum linked to three evaluation cases.In the validation phase (2006-2007), the authors randomized 69 residents from two EM residencies to either an intervention group that received the curriculum one month before the first assessment of all participants or a wait-list control group that received the identical curriculum three months later. A final assessment of all residents followed one month after that. Two raters evaluated all residents. Primary outcome measures are percentages of items completed correctly. The authors assessed rater agreement using intraclass correlation (ICC) and compared group performance using mixed-model analysis of variance.

RESULTS

ICCs surpassed 0.78. The instructional intervention produced a statistically significant effect for two of three evaluation cases for the validation phase of the study, a case x occasion interaction. Training year was significantly associated with better performance. In a multivariate analysis, training year and session correlated with score, but study group did not.

CONCLUSIONS

A one-day, simulation-based pediatric EM curriculum produced limited results. The evaluation approach is reasonable and reproducible for the population studied. Instructional dose strength and factors may have limited curriculum effectiveness. Focused, frequent, and effortful instructional interventions are necessary to achieve substantial performance improvements.

End-tidal carbon dioxide measurements in children with acute asthma

OBJECTIVES

A noninvasive method to assess ventilation may aid in management of children with acute asthma. The purpose of this study was to evaluate the association between end-tidal carbon dioxide (EtCO2) values and disease severity among children with acute asthma.

METHODS

This was a prospective, blinded, observational study of children 3-17 years old treated for acute asthma in a pediatric emergency department (ED). EtCO2 measurements were taken before the initiation of therapy and after each nebulization treatment (maximum of three). Peak expiratory flow rate (PEFR), Pediatric Asthma Severity Score (PASS), oxygen saturation, and disposition were recorded. Treating physicians, unaware of the EtCO2 results, made all treatment decisions, including disposition.

RESULTS

One hundred children were enrolled. The mean initial EtCO2 value was 35 mm Hg (95% confidence interval = 34.3 to 36.1 mm Hg). The mean disposition EtCO2 value was 33.3 mm Hg (95% confidence interval = 32.6 to 34.4 mm Hg). PEFR measures were completed on 43 patients and PASS recorded on 100 patients. There was an overall trend toward lower EtCO2 values during treatment (p < 0.01). Sixteen patients were admitted. Initial EtCO2 values were lower among children admitted to the hospital (35.6 mm Hg vs. 32.9 mm Hg; Mann-Whitney U test; p < 0.02). EtCO2 values at disposition did not differ between groups based on PEFR, PASS, or hospital admission.

CONCLUSIONS

Noninvasive bedside measurement of EtCO2 values among children with acute asthma is feasible. EtCO2 values did not distinguish children with mild disease from those with more severe disease. Further data are needed to clarify the association between EtCO2 values and other indicators of disease severity, particularly in children with more severe disease.

Development and evaluation of high-fidelity simulation case scenarios for pediatric resident education

OBJECTIVE

Pediatric residency programs need objective methods of trainee assessment. Patient simulation can contribute to objective evaluation of acute care event management skills. We describe the development and validation of 4 simulation case scenarios for pediatric resident evaluation.

METHODS

We created 4 pediatric simulation cases: apnea, asthma, supraventricular tachycardia, and sepsis. Each case contains a scenario and an unweighted checklist. Case and checklist development began by reaching expert consensus about case content followed by 92 pilot simulation sessions used for content revision and rater training. After development, 54 first-and second-year pediatric residents participated in 108 simulation test cases to assess the validity of data from these tools for our population. We report outcomes for interrater reliability, discriminant validity, and the impact of potential confounding factors on validity estimates.

RESULTS

Interrater reliability (kappa) ranged from 0.75 to 0.87. There were statistically and educationally significant differences in summary scores between first-and second-year residents for 3 of the 4 cases. Neither previous simulation exposure nor the order in which the cases were performed were found to be significant factors by multivariate analysis.

CONCLUSIONS

Simulation can be used to reliably measure and discriminate resident competencies in acute care management. Rigorous measurement development work is difficult and time-consuming. Done correctly, measurement development yields tangible and lasting benefits for trainees, faculty, and residency programs. Development studies that use systematic procedures and large trainee samples at multiple sites are the best approach to creating measurement tools that yield valid data.

Emergency and critical care pediatrics: use of medical simulation for training in acute pediatric emergencies

PURPOSE OF THE REVIEW

Recent data suggest that pediatric trainees receive insufficient training to manage acute pediatric emergencies. This review addresses the use of medical simulation as a way for medical learners to acquire and maintain skills needed to manage pediatric resuscitations.

RECENT FINDINGS

Recent work highlights the value of deliberate practice in the context of simulated medical environments to promote development of medical expertise. Several studies demonstrate the benefit of simulation-based training for improved skill acquisition in advanced life support, emergency airway management, and nontechnical skills. Work in other fields of medicine supports the integration of simulation into pediatric training programs.

SUMMARY

Medical simulation holds great promise to enhance existing pediatric training curricula by increasing skills and expertise in resuscitation. Future research is needed to identify best methods of pediatric simulation-based training.

Assessment of a computer-aided instructional program for the pediatric emergency department

Computer aided instruction (CAI) software is becoming commonplace in medical education. Our experience with CAI programs in our pediatric ED raised concerns about the time commitment some of these programs require. We developed a just-in-time learning program, the Virtual Preceptor (VP) and evaluated this program for use in a busy clinical environment. Forty-three of 47 pediatric residents used the VP at least once. Interns used the program 2 (1/2) times more often than upper level residents. Of 321 topics available in 18 subject categories, 153 (48%) were selected at least once. Content was rated as appropriate by 72% of users. 95% of residents would use the program again. Although no resident felt the program itself took too long to use, 51% said they were too busy to use the VP. Time of use and level of training may be important factors in CAI use in the pediatric ED environment.

Acute hemorrhagic complication of diagnostic lumbar puncture

OBJECTIVE

To present a case of an epidural hematoma after lumbar puncture in a pediatric patient without known risk factors for such a complication and to review the literature regarding this complication.

DESIGN

Case report, review of the literature, and discussion.

DATA SOURCES

A review of MEDLINE (1966-1998) for keywords "lumbar puncture" and "hemorrhage" or "hematoma" was conducted, and each bibliography was reviewed for other sources extending to 1911. Articles describing a case of spinal hematoma after a lumbar puncture for any procedure were included.

RESULTS

A 5-year-old boy underwent a lumbar puncture for evaluation of lethargy and fever, and subsequently developed marked back pain and severe pain on flexion of his legs. Magnetic resonance imaging revealed an epidural blood collection. The patient's symptoms resolved over the next few days in association with steroid administration. Multiple reports of epidural and subdural hematomas were found on literature review, most occurring in the setting of coagulation abnormalities. These reports involve lumbar puncture in anesthetic, interventional, and diagnostic settings.

CONCLUSION

Lumbar puncture is a frequently employed procedure. Known complications include epidural, subdural, and subarachnoid hemorrhage, usually in the setting of abnormal coagulation. The case presented is unusual in that the patient is a child and lacks any known risk factors for a hemorrhagic complication. Such a complication appears to be rare; only five of the 64 cases discovered in the literature review occurred following this diagnostic procedure in patients without known risk factors.

Quantifying the literature of computer-aided instruction in medical education

PURPOSE

To characterize the literature about computer-aided instruction (CAI) as it relates to medical education.

METHOD

A descriptive study using the Medline and ERIC databases, reviewing articles pertaining to CAI for medical professionals. The literature was evaluated with respect to type of article, journal, language, and year of publication.

RESULTS

The search yielded 2,840 citations, 92% of which were in English. The number of journals with at least one citation was 747. Less than 5% of the 5,147 authors had three or more articles published in the CAI literature. Of the citations with abstracts, 60% were demonstrations of a CAI application, 11% were media-comparative studies, and 13% were analyses of the CAI field. While the pace of article publication increased markedly over time, the percentages of article types did not change significantly over time. Less than 10% of CAI articles appeared in core medical journals.

CONCLUSION

Medical CAI is an increasingly popular topic of research and publication. However, these studies appear in journals with smaller circulations, are predominantly demonstration articles, and are generally written by authors with two or fewer publications. Evaluation articles remain less common. A series of analytic articles has appeared offering substantive suggestions for better research design. These suggestions appear to have gone unheeded. CAI investigators need be more aware of the gaps in the existing body of CAI publications, and the inherent difficulties of this type of research, if this literature is to move beyond this early stage of development.