Nursing students' evaluation of a new feedback and reflection tool for use in high-fidelity simulation - Formative assessment of clinical skills. A descriptive quantitative research design

Developing student nurses' evaluative judgement in clinical practice tertiary education: A systematic scoping review of teaching and assessment methods

OBJECTIVES

This review aimed to systematically scope undergraduate or postgraduate tertiary higher education nursing students' clinical practice teaching and assessment methods to identify features that align with promoting students' evaluative judgement.

INTRODUCTION

Evaluative judgement is a new concept to nursing tertiary education. Currently, there are no published reviews of evaluative judgement in nursing clinical practice education. This review aims to assist nursing educators to operationalise the concept of evaluative judgement in clinical practice education. As such the starting point was to determine features of evaluative judgement in current clinical teaching and assessment designs.

INCLUSION CRITERIA

Peer reviewed qualitative or quantitative studies that have evaluated teaching and/or assessment of tertiary (university/higher education) pre-registration (undergraduate) or post-registration (postgraduate) nursing students' clinical practice.

METHODS

The systematic scoping review was prospectively registered systematic review (OSF DOI 10.17605/OSF.IO/PYWZ6) reported using PRISMA guidelines. A systematic search of five databases (Medline, Scopus, Web of Science, ProQuest, CINAHL) was conducted, limited from 1989 onwards and in English. Two reviewers independently screened titles and abstracts, then full text, with disagreements resolved with a third independent author. Data were extracted, including the frequency and methods of developing students' evaluative judgement across the categories of discerning quality, judgement process, calibration and feedback. A narrative synthesis was performed.

RESULTS

Seventy-one studies were included (n=53 teaching, n=18 assessment). Most of the included studies, included some, but not all, of the features to develop nursing students' evaluative judgment. For teaching methods, the most identified evaluative judgement features in the included studies were discerning quality (n=47), feedback (n=41) and judgement process (n=21). Only three studies included a method of calibration. For the assessment methods, feedback (n=16), discerning quality (n=15), judgement process (n=9) and calibration (n=4) were included. Many clinical practice teaching and assessment methods in nursing included features that develop students' evaluative judgement, with methods relating to discerning quality and feedback well embedded. Further adjustments are required to include methods to assist students to judge and calibrate their own performance.

CONCLUSION

This systematic scoping review identified that evaluative judgement in current nursing clinical teaching and assessment is not an overt aim. With minor adjustment to teaching and assessment design, nursing students could be better supported to develop their ability to judge the value of their own work.

Effects of the project on enhancement of teaching skills in gerontic nursing practice of Indonesian nursing lecturer and clinical nurse preceptor

The Indonesia health care services require knowledgeable and skilled nurses as professional service providers, especially in a gerontic nursing area. Moreover, widening the gap between academic and clinical practice is also an issue, which affects the care service quality. In these circumstances, the project to enhance the educational skills of nursing lecturers and clinical nurse preceptors in gerontic nursing practice was started in 2020. Japanese nursing education experts provided guidance on the principle theory of nursing practice in basic nursing education and conducted workshops to develop nursing practice materials for 10 Indonesian trainees. The project provided 3 webinars for more than 100 Indonesian nursing lecturers and preceptors each time, 18 workshops and developed 2 virtual reality (VR) materials, 8 supervisions of the trial class and 5 lectures onsite and online. This study aimed to clarify the effects of the project evaluated in the process and its impact using a questionnaire survey and interviews conducted. The results of the questionnaire survey for students showed that 20% of their knowledge and skills in gerontic nursing, increased and the extent of understanding and satisfaction was high. Lecurers and preceptors perceived improvement in their teaching skills, especially in theoretical thinking and evidence-based teaching methods. Notably, "the evaluation from others" would be a facilitating factor of the project. The project effects were clarified and achieved the goal and objectives of the project.

A systematic review of effective quality feedback measurement tools used in clinical skills assessment

BACKGROUND

Objective Structured Clinical Examination (OSCE) is a valid tool to assess the clinical skills of medical students. Feedback after OSCE is essential for student improvement and safe clinical practice. Many examiners do not provide helpful or insightful feedback in the text space provided after OSCE stations, which may adversely affect learning outcomes. The aim of this systematic review was to identify the best determinants for quality written feedback in the field of medicine.   Methods: PubMed, Medline, Embase, CINHAL, Scopus, and Web of Science were searched for relevant literature up to February 2021. We included studies that described the quality of good/effective feedback in clinical skills assessment in the field of medicine. Four independent reviewers extracted determinants used to assess the quality of written feedback. The percentage agreement and kappa coefficients were calculated for each determinant. The ROBINS-I (Risk Of Bias In Non-randomized Studies of Interventions) tool was used to assess the risk of bias.

RESULTS

14 studies were included in this systematic review. 10 determinants were identified for assessing feedback. The determinants with the highest agreement among reviewers were specific, described gap, balanced, constructive and behavioural; with kappa values of 0.79, 0.45, 0.33, 0.33 and 0.26 respectively. All other determinants had low agreement (kappa values below 0.22) indicating that even though they have been used in the literature, they might not be applicable for good quality feedback. The risk of bias was low or moderate overall.

CONCLUSIONS

This work suggests that good quality written feedback should be specific, balanced, and constructive in nature, and should describe the gap in student learning as well as observed behavioural actions in the exams.  Integrating these determinants in OSCE assessment will help guide and support educators for providing effective feedback for the learner.

Ongoing training and peer feedback in simulation-based learning for local faculty development: A participation action research study

BACKGROUND

Simulation-based learning (SBL) is often used in healthcare education. Professional development has been identified as crucial to the success of SBL. Effective, high-quality SBL requires facilitators who are multiskilled and have a range of SBL-related knowledge, skills and attitudes, which require time and practice to acquire. However, investment in facilitators' competence is often limited, particularly at smaller institutions without an associated simulation centre.

OBJECTIVES

The purpose of this study is to describe how a smaller university college with limited resources and limited facilitation experience has initiated continuing professional development and how this continuing professional development has contributed to maintaining and developing SBL facilitators' competence.

METHOD

Participatory action research has been used to improve the practice of SBL facilitators at a university college in Norway. The evaluations and reflections of 10 facilitators engaged in professional development and 44 national simulation conference participants have been analysed by way of Vaismoradi's qualitative content analysis.

RESULTS

A culture of participation and engagement and a clear professional development structure are both of crucial importance in the implementation and maintenance of continuing professional development in SBL. When these are present, not only does facilitation become more transparent, but facilitators become more aware of their own strengths and weaknesses, manage to address these and perceive an improvement in their confidence and competence.

CONCLUSIONS

Facilitators at smaller institutions without an associated simulation centre can improve their competence and confidence in SBL beyond the initial course, despite the absence of experienced mentors. The results indicate the importance of engaging in ongoing training and self-reflection based on peer feedback, the facilitators' own experience and up-to-date literature. Implementing and maintaining professional development at smaller institutions requires a clear structure, clear expectations and a culture of participation and development.

Nursing students' experiences of clinical assessment at a university in South Africa

Background

Nursing education includes both classroom and clinical teaching. The clinical teaching was explored through this research. The successful training of the undergraduate nursing students can be attributed to effective clinical teaching and supervision and is determined by both training requirements and services provided. Although there have been several researches on clinical supervision, there is still a dearth of information of the realities of supervision regarding assessment of undergraduate nursing students. The authors' original thesis formed the foundation of this manuscript.

Aim

This study aimed to explore and describe nursing students at the undergraduate level experiences regarding clinical supervision.

Setting

The research was conducted at a nursing school at a South African university.

Method

After ethical clearance, focus group interviews were conducted to explore undergraduate nursing students' experiences of clinical supervision using a descriptive design and qualitative approach. Two qualified practitioners in the field collected the data. A purposive method was utilised to select nine participants from each year's level of education. Enrolled undergraduate nursing students at the institution under study formed the inclusion criteria. Utilising content analysis, the interviews were analysed.

Results

The findings confirmed the students' experiences of clinical supervision and voicing their concerns regarding clinical assessment versus a developmental training; clinical teaching, learning and assessment and formative assessment procedures.

Conclusion

A responsive clinical supervision system to strategically respond to the needs of undergraduate nursing students will aid in developmental training and assessment.

Contribution

Understanding of the realities of clinical teaching and supervision regarding clinical assessment and development of undergraduate nursing students.

Multimodal In-training Examination in an Emergency Medicine Residency Training Program: A Longitudinal Observational Study

Background

In-training examination (ITE) has been widely adopted as an assessment tool to measure residents' competency. We incorporated different formats of assessments into the emergency medicine (EM) residency training program to form a multimodal, multistation ITE. This study was conducted to examine the cost and effectiveness of its different testing formats.

Methods

We conducted a longitudinal study in a tertiary teaching hospital in Taiwan. Nine EM residents were enrolled and followed for 4 years, and the biannual ITE scores were recorded and analyzed. Each ITE consisted of 8–10 stations and was categorized into four formats: multiple-choice question (MCQ), question and answer (QA), oral examination (OE), and high-fidelity simulation (HFS) formats. The learner satisfaction, validity, reliability, and costs were analyzed.

Results

486 station scores were recorded during the 4 years. The numbers of MCQ, OE, QA, and HFS stations were 45 (9.26%), 90 (18.5%), 198 (40.7%), and 135 (27.8%), respectively. The overall Cronbach's alpha reached 0.968, indicating good overall internal consistency. The correlation with EM board examination was highest for HFS (ρ = 0.657). The average costs of an MCQ station, an OE station, and an HFS station were ~3, 14, and 21 times that of a QA station.

Conclusions

Multi-dimensional assessment contributes to good reliability. HFS correlates best with the final training exam score but is also the most expensive format among ITEs. Increased testing domains with various formats improve ITE's overall reliability. Program directors must understand each test format's strengths and limitations to bring forth the best combination of exams under the local context.

A systematic review of effective quality feedback measurement tools used in clinical skills assessment

Background: Objective Structured Clinical Examination (OSCE) is a valid tool to assess the clinical skills of medical students. Feedback after OSCE is essential for student improvement and safe clinical practice. Many examiners do not provide helpful or insightful feedback in the text space provided after OSCE stations, which may adversely affect learning outcomes. The aim of this systematic review was to identify the best determinants for quality written feedback in the field of medicine.   Methods: PubMed, Medline, Embase, CINHAL, Scopus, and Web of Science were searched for relevant literature up to February 2021. We included studies that described the quality of good/effective feedback in clinical skills assessment in the field of medicine. Four independent reviewers extracted determinants used to assess the quality of written feedback. The percentage agreement and kappa coefficients were calculated for each determinant. The ROBINS-I (Risk Of Bias In Non-randomized Studies of Interventions) tool was used to assess the risk of bias. Results: 14 studies were included in this systematic review. 10 determinants were identified for assessing feedback. The determinants with the highest agreement among reviewers were specific, described gap, balanced, constructive and behavioural; with kappa values of 0.79, 0.45, 0.33, 0.33 and 0.26 respectively. All other determinants had low agreement (kappa values below 0.22) indicating that even though they have been used in the literature, they might not be applicable for good quality feedback. The risk of bias was low or moderate overall. Conclusions: This work suggests that good quality written feedback should be specific, balanced, and constructive in nature, and should describe the gap in student learning as well as observed behavioural actions in the exams.  Integrating these determinants in OSCE assessment will help guide and support educators for providing effective feedback for the learner.

The Use of Guided Reflection in Simulation-Based Education With Prelicensure Nursing Students: An Integrative Review

BACKGROUND

This integrative review synthesized the findings of studies to determine the uses and outcomes of guided reflection in simulation-based education (SBE) with prelicensure nursing students.

METHOD

A search of Cumulative Index to Nursing and Allied Health (CINAHL) Plus, Education Resources Information Center (ERIC), Education Research Complete (ERC), and PsychINFO using the key words reflection, guided reflection, simulation (nursing education or nursing student), traditional, and baccalaureate identified 18 articles.

RESULTS

Findings suggest facilitation of guided reflection in SBE has a positive effect on leadership development, clinical judgment, student satisfaction, and critical thought. Development of a tool to accurately measure guided reflection is warranted.

CONCLUSION

Current evidence on the use of guided reflection in SBE calls for a major paradigm shift in higher education. Guided SBE experiences enhance participant knowledge, and guided reflection should be used as an innovative teaching strategy in SBE. [J Nurs Educ. 2022;61(2):73-79.].

Clinical nursing competency assessment: a scoping review

Objective

To ensure that only competent graduates are licensed to practice nursing, councils conduct licensing examinations, which may include among others clinical competency assessment. This review explored current practices in clinical competency assessment of nursing students as part of a larger study aimed at developing an evidence-based, context-specific framework for clinical competency assessment in a sub-Saharan African (SSA) country.

Methods

A scoping guided by the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) was conducted.

Results

Findings from 28 out of 1151 studies identified from Scopus, PubMed, CINAHL, Wiley Online Library, and ProQuest were included and synthesized. Results show that a good assessment system must be valid, reliable, transparent, feasible, fair, objective, and must provide feedback and continually improve to have an educational impact. Clinical competency assessment systems must be developed on sound empirical evidence, pilot tested, and involve thorough training and evaluation of the examiners. Continuous evaluation of the assessment system is also essential to ensure the quality and relevance of the assessment system. Only one of the included studies was conducted in Africa.

Conclusions

The paucity of clinical competency assessment research in sub-Saharan Africa may lead to benchmarking assessment systems on research conducted outside the context. Sub-Saharan Africa has a set of circumstances that demand a context-specific clinical competency assessment framework to guide clinical competency assessment.

Nursing Students' and Preceptors' Experiences with Using an Assessment Tool for Feedback and Reflection in Supervision of Clinical Skills: A Qualitative Pilot Study

Background

There is a need to improve students' learning in clinical practice. Undergraduate students need guidance when it comes to transferring knowledge from the classroom to clinical practice in community health services. Competence Development of Practical Procedures (COPPs), a simulation assessment tool, was used to explore students' and preceptors' experiences with feedback and reflection during the supervision of clinical skills in real practice.

Method

This was a pilot study with a qualitative exploratory and descriptive research design. Four students in their first year of a bachelor's programme in nursing and four preceptors participated. Data were collected from eight clinical skills performance assessments, audio recordings of supervision, and open-ended questionnaires. Data were systematized, categorized, and analysed using qualitative content analysis. Findings. Participants' experiences were divided into five categories: “learning environment, an atmosphere of respect, acceptance, and encouragement,” “students' reflections on their own personal learning,” “students' reflections on various care situations,” and “students' and preceptors' assessment and feedback.” Participants found COPPs easy to use and providing structure for assessment, feedback, and reflection during supervision. Concepts related to learning clinical skills became visible for both students and preceptors and helped students assess their performance of clinical skills. Through verbalization and reflection in supervision, participants established a consensus around what students knew and what they needed to learn.

Conclusions

The students and preceptors experienced the tool as a supportive structure to enhance feedback and reflection for the learning of clinical skills in municipal healthcare services. COPPs filled a gap in practice by providing a language for students and preceptors to articulate their knowledge and increasing students' awareness of what constitutes a good performance. The tool supported the coherence of concepts, enhanced clinical reasoning, and promoted deeper thinking and reflection, and the students gained insight into their own needs related to learning clinical skills.

A Mobile-Assisted Peer Assessment Approach for Evidence-Based Nursing Education

Students who learn evidence-based nursing can assist the healthcare team to make proper medical decisions and provide patients with valuable advice, thus optimizing the quality of patient care in specific situations. In clinical work, nursing staff members participate in decision making by searching for relevant empirical nursing literature, a basic ability required to enter clinical practice. In traditional instruction, nursing students are taught the Problem, Intervention, Comparison, and Outcome method to learn to use library resources and gather empirical nursing knowledge for decision making. However, it is a challenge for most students to have sufficient practice to make decisions correctly and to have opportunities to perceive medical cases from diverse perspectives. Therefore, we propose a peer assessment-based Problem, Intervention, Comparison, and Outcome approach to help nursing students locate correct evidence and make appropriate decisions about patient care. We performed an experiment with our approach in a nursing university training program. The experimental results reveal that the subjects learning with the proposed approach show better evidence-based nursing knowledge, learning attitude, and critical thinking ability than those learning with the traditional approach.

Formative peer assessment in higher healthcare education programmes: a scoping review

OBJECTIVES

Formative peer assessment focuses on learning and development of the student learning process. This implies that students are taking responsibility for assessing the work of their peers by giving and receiving feedback to each other. The aim was to compile research about formative peer assessment presented in higher healthcare education, focusing on the rationale, the interventions, the experiences of students and teachers and the outcomes of formative assessment interventions.

DESIGN

A scoping review.

DATA SOURCES

Searches were conducted until May 2019 in PubMed, Cumulative Index to Nursing and Allied Health Literature, Education Research Complete and Education Research Centre. Grey literature was searched in Library Search, Google Scholar and Science Direct.

ELIGIBILITY CRITERIA

Studies addressing formative peer assessment in higher education, focusing on medicine, nursing, midwifery, dentistry, physical or occupational therapy and radiology published in peer-reviewed articles or in grey literature.

DATA EXTRACTIONS AND SYNTHESIS

Out of 1452 studies, 37 met the inclusion criteria and were critically appraised using relevant Critical Appraisal Skills Programme, Joanna Briggs Institute and Mixed Methods Appraisal Tool tools. The pertinent data were analysed using thematic analysis.

RESULT

The critical appraisal resulted in 18 included studies with high and moderate quality. The rationale for using formative peer assessment relates to giving and receiving constructive feedback as a means to promote learning. The experience and outcome of formative peer assessment interventions from the perspective of students and teachers are presented within three themes: (1) organisation and structure of the formative peer assessment activities, (2) personal attributes and consequences for oneself and relationships and (3) experience and outcome of feedback and learning.

CONCLUSION

Healthcare education must consider preparing and introducing students to collaborative learning, and thus develop well-designed learning activities aligned with the learning outcomes. Since peer collaboration seems to affect students' and teachers' experiences of formative peer assessment, empirical investigations exploring collaboration between students are of utmost importance.

Association Between Formative Assessment and Academic Performance for Undergraduate Medical Students in a Chinese Clinical Skills Training Course

BACKGROUND Previous research suggests that formative assessment (FA) enhances learning outcomes, but few studies have evaluated its impact on clinical skills training in China. We conducted this study in a clinical skills integral curriculum to further explore the educational value of FA. MATERIAL AND METHODS Sixty undergraduates from the Second Clinical Medical School of the Southern Medical University in 2016 were selected as the experimental group (consecutive FA), and 50 undergraduates in 2015 were selected as the control term (only final summative assessment, SA). Undergraduates in the FA group completed the after-class questionnaire at each lesson. Teachers, teaching content, assessment objectives, and topics are the same in both groups. RESULTS The results of single-factor covariance (ANCOVA) analysis and Cochran-Mantel-Haenszel (CMH) analysis demonstrated that students of the FA group obtained better performance and higher success rates in summative examination than in the SA group. The students with relatively poor grades benefited more from FA, while the performance of students with higher grades was similar between the FA group and SA group. According to the results of questionnaire for students, the satisfaction of students with the course increased gradually, from 84.4% to 93.0%. CONCLUSIONS Proper use of FA is associated with better learning outcomes for students, especially for those with poorer grades. Our results, together with previous research, indicated that the use of FA may be of great benefit to students' academic performance and satisfaction with the clinical skills training curriculum.

The Learning Effectiveness of High-Fidelity Simulation Teaching Among Chinese Nursing Students: A Mixed-Methods Study

BACKGROUND

High-fidelity simulation (HFS) is an interactive and complex experiential learning pedagogy. Given the limited and inconclusive evidence on the effectiveness of HFS in terms of improving student learning outcomes, a more thorough understanding of students' learning experiences and effects of HFS may inform the improvement of nursing training.

PURPOSE

The aim of this study was to examine the learning effectiveness score of HFS, its influencing factors, and the learning experience of nursing students.

METHODS

A convergent parallel mixed-methods research design was adopted. Five hundred thirty-three third-year undergraduate nursing students completed the Simulation Learning Effectiveness Inventory. Semistructured interviews were used to elicit the opinions of 22 participants regarding their participation in the HFS experience.

RESULTS

The quantitative findings showed a moderately high learning effectiveness of HFS among Chinese undergraduate nursing students (121.81 ± 14.93). The learning effectiveness for equipment resources (15.02 ± 2.38), course arrangement (11.18 ± 1.73), and confidence (18.56 ± 3.67) was relatively low. Extroversion and mixed personality (β = 0.14 and 0.10) and "dislike" or "general like" of the course (β = -0.45 and -0.33) were found to influence learning effectiveness (F = 54.79, p < .001, adjusted R2 = .29). In addition, the qualitative findings indicated that the participants felt positively regarding the "debriefing," "clinical abilities," and "problem solving" dimensions of the training.

CONCLUSIONS/IMPLICATIONS FOR PRACTICE

The focus of the education process and curriculum design of HFS activities should be on improving course arrangement, equipment resources, and students' confidence while paying attention to nursing students' personality traits and course preferences.

The Portfolio as a Tool for Mentoring in Nursing Students: A Scoping Review

BACKGROUND

One of the processes leading to education and evaluation promotion in nursing is mentoring. Moreover, using a portfolio leads to promotion of the mentoring process. Due to the importance of mentoring in nursing, its strengthening by the portfolio, the weakness of mentoring in nursing, and the lack of widespread use of portfolio in nursing education, the aim of this study was to determine portfolio content, which can help the mentoring performance.

MATERIALS AND METHODS

The present scoping review study was conducted in 2017. Articles were searched using the keywords portfolio, mentoring, nursing education, logbook, teaching, learning, nursing, education, and evaluation. The search was conducted in SID, IranMedex, IranDoc, Magiran, PubMed, Google Scholar, Scopus, Web of Science, and ScienceDirect databases. Studies published in Persian and English from 1997 to 2017 were reviewed. The studies were extracted based on the inclusion and exclusion criteria.

RESULTS

Of the 1070 articles searched, 6 were extracted for the final assessment. The results indicated that a portfolio consists of a title page, content list, introduction, record of experience, mentor's feedback and comments, self-evaluation, and educational activities. Furthermore, nursing process is required to help conduct mentoring among nursing students.

CONCLUSIONS

Recording experiences, feedback, etc., in the portfolios of nursing students can be helpful in conducting mentoring for student's education. It is suggested that educational managers perform mentoring in educational environments using the contents suggested in this study.