Development of Neurological Emergency Simulations for Assessment: Content Evidence and Response Process

OBJECTIVE

To document two sources of validity evidence for simulation-based assessment in neurological emergencies.

BACKGROUND

A critical aspect of education is development of evaluation techniques that assess learner's performance in settings that reflect actual clinical practice. Simulation-based evaluation affords the opportunity to standardize evaluations but requires validation.

METHODS

We identified topics from the Neurocritical Care Society's Emergency Neurological Life Support (ENLS) training, cross-referenced with the American Academy of Neurology's core clerkship curriculum. We used a modified Delphi method to develop simulations for assessment in neurocritical care. We constructed checklists of action items and communication skills, merging ENLS checklists with relevant clinical guidelines. We also utilized global rating scales, rated one (novice) through five (expert) for each case. Participants included neurology sub-interns, neurology residents, neurosurgery interns, non-neurology critical care fellows, neurocritical care fellows, and neurology attending physicians.

RESULTS

Ten evaluative simulation cases were developed. To date, 64 participants have taken part in 274 evaluative simulation scenarios. The participants were very satisfied with the cases (Likert scale 1-7, not at all satisfied-very satisfied, median 7, interquartile range (IQR) 7-7), found them to be very realistic (Likert scale 1-7, not at all realistic-very realistic, median 6, IQR 6-7), and appropriately difficult (Likert scale 1-7, much too easy-much too difficult, median 4, IQR 4-5). Interrater reliability was acceptable for both checklist action items (kappa = 0.64) and global rating scales (Pearson correlation r = .70).

CONCLUSIONS

We demonstrated two sources of validity in ten simulation cases for assessment in neurological emergencies.

Rapid transfer of knowledge for multiple sclerosis clinical care during COVID-19: ECHO MS

•

The COVID-19 pandemic has added new complexity to multiple sclerosis (MS) care.

•

The Project ECHO model provides a unique method for MS clinical care providers to rapidly share knowledge and expertise that can be particularly beneficial during COVID-19.

•

MS clinical care providers were able to participate in a COVID-19-focused version of ECHO MS and reported positive impacts on their knowledge, attitude and skills for providing MS care during COVID-19.

Background

Healthcare providers caring for people with multiple sclerosis (MS) have had significant concerns about the intersection of MS and COVID-19. As a result, there has been an urgency to understand and share information about how to best provide MS clinical care during COVID-19. The Project ECHO model is well-suited for this challenge, as it provides a uniquely efficient and effective approach to sharing information in real-time using real cases. We report on the translation of the Project ECHO model for the rapid sharing of knowledge among MS clinical providers during COVID-19.

Methods

The ECHO MS COVID-19 Response Clinic was a videoconference-based education and case consultation program offered to providers in the U.S. who care for individuals with MS. The Response Clinic was offered as four sessions, each delivered by three regional hubs. Data were collected on participation and the self-reported impact of the program.

Results

A total of 132 unique providers participated in the Response Clinic, which consisted of 11 didactic modules and 43 case consultations. Participant providers overwhelmingly indicated that the program improved their knowledge, attitude, and skills for providing healthcare for people with MS during the COVID-19 pandemic.

Discussion

The Project ECHO model was successfully adapted to serve the needs of the MS community during COVID-19, suggesting the program could be continued or could be expanded to other disease areas for a similar purpose. More research is needed to objectively measure the impact of the program on patient outcomes.

New antiepileptic drug safety information is not transmitted systematically and accepted by U.S. neurologists

We surveyed U.S. neurologists in order to evaluate their knowledge of, and sources for, recent FDA safety warnings regarding antiepileptic drugs (AEDs) and whether they incorporate this information into their practices. Survey respondents (N=505) were predominantly board-certified American Academy of Neurology members. Approximately 20% of respondent neurologists were not aware of warnings about four drug safety risks: suicidality with newer AEDs, increased birth defect risks from in utero divalproex exposure, impaired cognitive development from in utero divalproex exposure, and the requirement of haplotype screening in patients of Asian descent starting carbamazepine. Most respondents were aware of a recommendation for haplotype screening, yet did not routinely perform the safety screening, and 18 reported patients that had hypersensitivity reactions to carbamazepine. Respondents learned about drug safety risks from varied sources; only notifications from specialty organizations were associated with accurate knowledge of drug safety warnings. Most surveyed neurologists would prefer implementing "a formal warning process via specialty organizations" with e-mails of updated product insert warnings.