Simulation for Milestone Assessment: Use of a Longitudinal Curriculum for Pediatric Residents

Simulation-based summative assessment in healthcare: an overview of key principles for practice

BACKGROUND

Healthcare curricula need summative assessments relevant to and representative of clinical situations to best select and train learners. Simulation provides multiple benefits with a growing literature base proving its utility for training in a formative context. Advancing to the next step, "the use of simulation for summative assessment" requires rigorous and evidence-based development because any summative assessment is high stakes for participants, trainers, and programs. The first step of this process is to identify the baseline from which we can start.

METHODS

First, using a modified nominal group technique, a task force of 34 panelists defined topics to clarify the why, how, what, when, and who for using simulation-based summative assessment (SBSA). Second, each topic was explored by a group of panelists based on state-of-the-art literature reviews technique with a snowball method to identify further references. Our goal was to identify current knowledge and potential recommendations for future directions. Results were cross-checked among groups and reviewed by an independent expert committee.

RESULTS

Seven topics were selected by the task force: "What can be assessed in simulation?", "Assessment tools for SBSA", "Consequences of undergoing the SBSA process", "Scenarios for SBSA", "Debriefing, video, and research for SBSA", "Trainers for SBSA", and "Implementation of SBSA in healthcare". Together, these seven explorations provide an overview of what is known and can be done with relative certainty, and what is unknown and probably needs further investigation. Based on this work, we highlighted the trustworthiness of different summative assessment-related conclusions, the remaining important problems and questions, and their consequences for participants and institutions of how SBSA is conducted.

CONCLUSION

Our results identified among the seven topics one area with robust evidence in the literature ("What can be assessed in simulation?"), three areas with evidence that require guidance by expert opinion ("Assessment tools for SBSA", "Scenarios for SBSA", "Implementation of SBSA in healthcare"), and three areas with weak or emerging evidence ("Consequences of undergoing the SBSA process", "Debriefing for SBSA", "Trainers for SBSA"). Using SBSA holds much promise, with increasing demand for this application. Due to the important stakes involved, it must be rigorously conducted and supervised. Guidelines for good practice should be formalized to help with conduct and implementation. We believe this baseline can direct future investigation and the development of guidelines.

Alternative Strategy Adopted in Practical Assessment for Pathology Postgraduate Certification Examination Amidst COVID-19 Nationwide Lockdown: An Indian Experience with a Novel Hybrid Method of Assessment

BACKGROUND

A major concern and challenge faced by the educational institution during coronavirus disease (COVID-19) pandemic and consequent lockdown is the timely conduct of the summative assessment. Unlike cognitive assessment, real-time practical assessment through online mode, without compromising principles of assessment is difficult. This study was performed to analyze an alternative hybrid approach adopted for the postgraduate practical summative assessment in pathology.

MATERIALS AND METHODS

Evaluation of the process of summative assessment was done where internal and external examiners synchronously assessed examinees real-time on-site (face to face) and off-site (remotely through live videoconferencing) respectively. A pre-validated questionnaire on a novel approach for assessment and feedback (with close-ended on 5-point Likert scale and open-ended questions) were administered to nine participants comprising three final year postgraduates in pathology, four examiners (two internal and two externals) and two technical experts from the Department of Pathology. Overall scores obtained by all examinees were recorded. A pre-validated questionnaire and feedback were administered to the participants to assess the novel approach to assessment.

RESULTS

The mean performance score obtained by examinees was 63.5% in the summative assessment. Feedback analysis revealed that 85% of participants strongly agreed and the rest (15%) agreed that essential minor deviations introduced in this novel mode of assessment did not compromise basic principles and goals of assessment. Quality of assessment through this mode of assessment was graded as good to excellent. All participants responded that this mode of examination may be applied in the future under similar circumstances.

CONCLUSION

This study concludes that an alternative novel hybrid method of real-time synchronous assessment with on-site and off-site examiners may be a feasible and successful model for conducting summative assessment and can be practiced in time of need.