Are script concordance tests suitable for the assessment of undergraduate students? A multicenter comparative study

The Impact of Script Concordance Testing on Clinical Decision-Making in Paramedic Education

This study investigates the effectiveness of the Script Concordance Test (SCT) in enhancing clinical reasoning skills within paramedic education. Focusing on the Medical University of Lublin, we evaluated the SCT's application across two cohorts of paramedic students, aiming to understand its potential to improve decision-making skills in emergency scenarios. Our approach, informed by Van der Vleuten's assessment framework, revealed that while the SCT's correlation with traditional methods like multiple-choice questions (MCQs) was limited, its formative nature significantly contributed to improved performance in summative assessments. These findings suggest that the SCT can be an effective tool in paramedic training, particularly in strengthening cognitive abilities critical for emergency responses. The study underscores the importance of incorporating innovative assessment tools like SCTs in paramedic curricula, not only to enhance clinical reasoning but also to prepare students for effective emergency responses. Our research contributes to the ongoing efforts in refining paramedic education and highlights the need for versatile assessment strategies in preparing future healthcare professionals for diverse clinical challenges.

Development and Psychometric Testing of the Clinical Reasoning Scale Among Nursing Students Enrolled in Three Types of Programs in Taiwan

BACKGROUND

There is no instrument currently available to assess the essential nursing competency of clinical reasoning (CR).

PURPOSE

The purpose of this study was to develop and test the psychometric properties of CR assessment instrument appropriate for use with nursing students across different types of programs.

METHODS

H. M. Huang et al.'s (2018) Framework of Competencies of Clinical Reasoning for Nursing Students was used to guide this study. Two rounds of Delphi study and confirmatory factor analysis (CFA) were conducted to test content and construct validity. Internal consistency was tested for reliability.

RESULTS

The four-domain, 16-item Likert-scale Clinical Reasoning Scale (CRS) was developed. One thousand five hundred four nursing students currently enrolled in three different types of nursing programs completed the CRS. The content validity index was .85-1.0, the CFA indicated goodness of fit, and the Cronbach's α score range was .78-.89.

CONCLUSION

The CRS is a valid and reliable tool for assessing CR in nursing students in different types of nursing program.

Clinical reasoning in undergraduate paramedicine: utilisation of a script concordance test

INTRODUCTION

Clinical reasoning is a complex cognitive and metacognitive process paramount to patient care in paramedic practice. While universally recognised as an essential component of practice, clinical reasoning has been historically difficult to assess in health care professions. Is the Script Concordance Test (SCT) an achievable and reliable option to test clinical reasoning in undergraduate paramedic students?

METHODS

This was a single institution observational cohort study designed to use the SCT to measure clinical reasoning in paramedic students. Clinical vignettes were constructed across a range of concepts with varying shades of clinical ambiguity. A reference panel mean scores of the test were compared to that of students. Test responses were graded with the aggregate scoring method with scores awarded for both partially and fully correct responses.

RESULTS

Eighty-three student paramedic participants (mean age: 21.8 (3.5) years, 54 (65%) female, 27 (33%) male and 2 (2%) non-binary) completed the SCT. The difference between the reference group mean score of 80 (5) and student mean of score of 65.6 (8.4) was statistically significant (p < 0.001).

DISCUSSION

Clinical reasoning skills are not easily acquired as they are a culmination of education, experience and the ability to apply this in the context to a specific patient. The SCT has shown to be reliable and effective in measuring clinical reasoning in undergraduate paramedics as it has in other health professions such as nursing and medicine. More investigation is required to establish effective pedogeological techniques to optimise clinical reasoning in student and novice paramedics who are devoid of experience.

Prueba de concordancia de guiones para entrenar el razonamiento clínico en estudiantes de fonoaudiología

La prueba de concordancia de guiones (PCG) ha sido utilizada en el entrenamiento y evaluación del razonamiento clínico (RC) como una estrategia innovadora en la formación de profesionales. Sin embargo, no se dispone de evidencia de su aplicación en el pregrado de fonoaudiología. El objetivo de esta investigación fue analizar el desempeño y la percepción de estudiantes de fonoaudiología con respecto al uso de scripts. Se diseñó un piloto pre-experimental y multicéntrico, complementado con tres grupos focales. Las variables cuantitativas continuas fueron resumidas a través de medias y desviación estándar. La comparación entre grupos se ejecutó con Anova one way y la prueba post hoc de Bonferroni, considerando un nivel de significancia p<.05. La fase cualitativa incorporó un análisis de contenido mediante la codificación abierta de textos y la identificación e interpretación de familias de significado emergentes. El rendimiento promedio de los estudiantes fue de 4.03 (DS= 0.35), observándose un incremento en el rendimiento de RC durante el semestre (p= 0.03). La percepción de los estudiantes resulto positiva y se identificó cuatro familias de significado relacionadas con: razonamiento clínico, oportunidades de mejora implementación de la estrategia y retroalimentación docente. A modo de conclusión, la incorporación de scripts en estudiantes de pregrado de fonoaudiología es factible, incrementa el rendimiento y apoya el desarrollo del RC.

Script concordance test acceptability and utility for assessing medical students' clinical reasoning: a user's survey and an institutional prospective evaluation of students' scores

Script Concordance Testing (SCT) is a method for clinical reasoning assessment in the field of health-care training. Our aim was to assess SCT acceptability and utility with a survey and an institutional prospective evaluation of students' scores.With a user's online survey, we collected the opinions and satisfaction data of all graduate students and teachers involved in the SCT setting. We performed a prospective analysis comparing the scores obtained with SCT to those obtained with the national standard evaluation modality.General opinions about SCT were mostly negative. Students tended to express more negative opinions and perceptions. There was a lower proportion of negative responses in the teachers' satisfaction survey. The proportion of neutral responses was higher for teachers. There was a higher proportion of positive positions towards all questions among teachers. PCC scores significantly increased each year, but SCT scores increased only between the first and second tests. PCC scores were found significantly higher than SCT scores for the second and third tests. Medical students' and teachers' global opinion on SCT was negative. At the beginning SCT scores were found quite similar to PCC scores. There was a higher progression for PCC scores through time.

Teaching emergency situations during a psychiatry residency programme using a blended learning approach: a pilot study

BACKGROUND

Emergency psychiatry is an essential component in the training of psychiatry residents who are required to make patient-centred orientation decisions. This training calls for specific knowledge as well as skills and attitudes requiring experience. Kolb introduced a theory on experiential learning which suggested that effective learners should have four types of abilities: concrete experience, reflective observation, abstract conceptualisation and active experimentation. We aimed to evaluate a resident training programme that we designed for use in an emergency psychiatry setting based on the experimental learning theory.

METHODS

We designed a four-step training programme for all first-year psychiatry residents: (i) theoretical teaching of psychiatric emergency knowledge, (ii) concrete experience of ability teaching involving an initial simulation session based on three scenarios corresponding to clinical situations frequently encountered in emergency psychiatry (suicidal crisis, hypomania and depressive episodes), (iii) reflective observation and abstract conceptualisation teaching based on videos and clinical interview commentary by a senior psychiatrist for the same three scenarios, (iv) active experimentation teaching during a second simulation session based on the same three frequently encountered clinical situations but with different scenarios. Training-related knowledge acquisition was assessed after the second simulation session based on a multiple-choice quiz (MCQ), short-answer questions and a script concordance test (SCT). The satisfaction questionnaire was assessed after the resident had completed his/her initial session in order to evaluate the relevance of teaching in clinical practice. The descriptive analyses were described using the mean (+/- standard deviation). The comparative analyses were conducted with the Wilcoxon or Student's t tests depending on data distribution.

RESULTS

The residents' mean MCQ and short-answer question scores and SCT were 7.25/10 (SD = 1.2) 8.33/10 (SD = 1.4), 77.5/100 (SD = 15.8), respectively. The satisfaction questionnaire revealed that 67 % of residents found the teaching consistent.

CONCLUSION

We designed a blended learning programme that associated, classical theoretical learning to acquire the basic concepts, a learning with simulation training to experiment the clinical situations and a video support to improve learning of interview skills and memory recall. The residents indicate that this training was adequate to prepare them to be on duty. However, despite this encouraging point, this program needs further studies to attest of its efficiency.