A first report of East Asian students' perception of progress testing: a focus group study

Progress testing of an objective structured clinical examination during undergraduate clinical clerkship: a mixed-methods pilot study

BACKGROUND

Progress testing is an assessment method in which an examination reflecting competencies at graduation is regularly administered to students over multiple years, thereby facilitating self-directed learning. However, the significance of the objective structured clinical examination as a progress test in undergraduate education, needs to be determined. This study provides evidence of the role of the objective structured clinical examination for progress testing and optimal scoring methods for assessing students in different academic years.

METHODS

We conducted a sequential explanatory mixed-methods pilot study. Participants were assessed using the Item Rating Scale, the year-adjusted Global Rating Scale, and the Training Level Rating Scale. The characteristics of each scale were compared quantitatively. In addition, the influence of the objective structured clinical examination as a progress test on learning attitudes was examined. Qualitative data from a post-examination questionnaire were analyzed, using content analysis to explore influences on self-directed learning.

RESULTS

Sixth and fifth year clinical students (n = 235) took the objective structured clinical examination progress test. The total Item Rating Scales were recorded (%) as 59.03 ± 5.27 and 52.64 ± 5.08 (p < 0.01); Training Level Rating Scale was 3.94 ± 0.39 vs 3.22 ± 0.42 (p < 0.01); and the year-adjusted Global Rating Scale was 4.25 ± 0.44 vs 4.32 ± 0.52 (no significant difference), for the sixth and fifth year students, respectively. The correlations across stations and the reliability of each station were satisfactory. Four categories were identified in the qualitative analysis: "motivation to learn during the clinical clerkship was promoted," "dissatisfied with being asked about things they had not experienced," "confusion about being unable to use conventional test-taking strategies," and "insufficient understanding of competencies at graduation." The scores indicated significant differences in performance according to training year.

CONCLUSIONS

This study provides evidence that the objective structured clinical examination can be used as a progress testing tool for undergraduate clinical clerkships. Further enhancement of training opportunities and dissemination of performance competency goals in clerkship curricula are required if we intend to promote self-directed learning through progress testing.

Written-Based Progress Testing: A Scoping Review

PURPOSE

Progress testing is an increasingly popular form of assessment in which a comprehensive test is administered to learners repeatedly over time. To inform potential users, this scoping review aimed to document barriers, facilitators, and potential outcomes of the use of written progress tests in higher education.

METHOD

The authors followed Arksey and O'Malley's scoping review methodology to identify and summarize the literature on progress testing. They searched 6 databases (Academic Search Complete, CINAHL, ERIC, Education Source, MEDLINE, and PsycINFO) on 2 occasions (May 22, 2018, and April 21, 2020) and included articles written in English or French and pertaining to written progress tests in higher education. Two authors screened articles for the inclusion criteria (90% agreement), then data extraction was performed by pairs of authors. Using a snowball approach, the authors also screened additional articles identified from the included reference lists. They completed a thematic analysis through an iterative process.

RESULTS

A total of 104 articles were included. The majority of progress tests used a multiple-choice and/or true-or-false question format (95, 91.3%) and were administered 4 times a year (38, 36.5%). The most documented source of validity evidence was internal consistency (38, 36.5%). Four major themes were identified: (1) barriers and challenges to the implementation of progress testing (e.g., need for additional resources); (2) established collaboration as a facilitator of progress testing implementation; (3) factors that increase the acceptance of progress testing (e.g., formative use); and (4) outcomes and consequences of progress test use (e.g., progress testing contributes to an increase in knowledge).

CONCLUSIONS

Progress testing appears to have a positive impact on learning, and there is significant validity evidence to support its use. Although progress testing is resource- and time-intensive, strategies such as collaboration with other institutions may facilitate its use.

Perceptions of medical students in Pakistan, KSA, and the US regarding the significance of case-based learning

Objective

This study aims to determine the perceptions of medical students in Pakistan, KSA, and the US regarding the significance of case-based learning (CBL).

Methods

For this cross-sectional study, data were collected by administering an online questionnaire to students in medical schools across Pakistan, KSA, and the US.

Results

A total of 344 medical students participated in this study, the great majority of whom agree that CBL paves the way for developing a sound understanding of the core subject, provides insight into real-life experiences, helps them transform from fact memorisers into problem solvers, and keeps them engaged during sessions, which motivates them to attend more of these. A comparison of respondents from Pakistan and KSA shows that CBL promotes deep learning and fostered their critical thinking; however, there was a difference in perception in some categories, including CBL as a tool used for grasping key concepts (p = 0.004), providing insight into real-life experiences (p = 0.001), offering a platform for self-directed learning (p = 0.000), nurturing collaborative abilities (p = 0.004), and maintaining students’ engagement (p = 0.002).

Conclusion

Our study shows that the selected cohort of medical students perceive CBL as an effective learning tool, as the majority feel overwhelmingly positive towards it. This study thus proposes the introduction of clinical exposure for medical students early in MBBS programmes, which will help promote collaborative skills and self-directed learning among them.

A collaborative two-stage examination in biomedical sciences: Positive impact on feedback and peer collaboration

Examinations present an intensely focused opportunity for student learning yet opportunities for collaboration, communication, and feedbacks are often wasted. Two-stage examinations where students first take examinations individually and then repeat the examination in small groups hold promise to address this issue. Here, we pilot and evaluate a two-stage examination within an advanced undergraduate biomedical sciences course. We evaluated this innovation by triangulation of data from a questionnaire, semi-structured student interviews, as well as, comparison of student grades between stages of examination across higher- and lower-performing student groups. Quantitative data from the structured questionnaire showed that a majority of students perceived the collaborative stage of two-stage examinations successful in promoting peer collaboration and communication. Furthermore, there was deepened conceptual understanding and provision of immediate feedback. The two-stage examination did not, however, ameliorate students' test anxiety. Qualitative data from semi-structured student interviews were consistent with quantitative data to show that a two-stage examination provides positive impact particularly on immediate feedback, peer collaboration and communication but contributed to sustained test anxiety possibly due to negative experiences during group interactions. Both lower- and higher-performing students showed improvement in the collaborative stage as compared to the preceding individual stage of two-stage examination. This would suggest possible benefits of two-stage examination for learning for all student abilities. This study discusses the advantages and pitfalls of two-stage examinations for biomedical sciences and will guide informed recommendations for subsequent implementation elsewhere.

Active learning of medical students in Taiwan: a realist evaluation

BACKGROUND

Active learning is defined as any instructional method that engages students in the learning process. Cultural differences in learning patterns can play an important role in engagement with active learning. We aimed to examine process models of active learning to understand what works, for whom and why.

METHODS

Forty-eight sixth- and seventh-year medical students with experience of active learning methods were purposively selected to participate in ten group interviews. Interactions around active learning were analysed using a realist evaluation framework to unpack the 'context-mechanism-outcome' (CMO) configurations.

RESULTS

Three core CMO configurations, including cultural, training and individual domains, were identified. In the cultural context of a strong hierarchical culture, the mechanisms of fear prompted students to be silent (outcome) and dare not give their opinions. In the training context of teacher-student familiarity alongside teachers' guidance, the mechanisms of learning motivation, self-regulation and enthusiasm were triggered, prompting positive learning outcomes and competencies (outcome). In the individual context of learning how to learn actively at an early stage within the medical learning environment, the mechanisms of internalisation, professional identity and stress resulted in recognising active learning and advanced preparation (outcomes).

CONCLUSIONS

We identified three CMO configurations of Taiwanese medical students' active learning. The connections among hierarchical culture, fear, teachers' guidance, motivation, the medical environment and professional identity have been shown to affect the complex interactions of learning outcomes. Fear derived from a hierarchical culture is a concern as it is a significant and specific contextual factor, often sparking fear with negative outcomes.

Realist approach to evaluating an interprofessional education program for medical students in clinical practice at a community hospital

Introduction: We examined the interrelationships between context, mechanism, and outcome using a realist approach following the introduction of interprofessional education (IPE) to clinical practice for medical students in the community.Methods: Through participant observation and interviews, a working hypothesis was developed. To evaluate IPE in clinical practice, medical students' reports were thematically analyzed, and configuration on contexts, mechanisms, and outcomes were identified using a realist approach.Results: Influential contexts were medical students' experience of clinical practice and learning characteristics, the capacity of other professionals, interprofessional relationships, and characteristics of the community hospital. One key mechanism was observational learning. Others were self-regulated learning, legitimate peripheral participation, experiential learning, contact hypotheses, awareness of social structure, and cognitive empathy. As faculties supported these key mechanisms, medical students became aware of the legitimacy of community-oriented primary care, noting the roles of physicians who support patients' and/or their family's life in collaboration with other professionals, and reflecting the necessity of shifting from physician-centered perspectives. As a result, medical students deepened their empathic understanding for other professionals.Conclusion: Faculties should develop IPE programs in clinical practice based on the 'mechanism', 'context', 'outcome' pattern and 'context-mechanism-outcome' configuration in primary care settings.

Impact of Progress testing on the learning experiences of students in medicine, dentistry and dental therapy

AIMS

To investigate the impact of progress testing on the learning experiences of undergraduate students in three programs namely, medicine, dentistry and dental therapy.

METHODS

Participants were invited to respond to an online questionnaire to share their perceptions and experiences of progress testing. Responses were recorded anonymously, but data on their program, year of study, age, gender, and ethnicity were also captured on a voluntary basis.

RESULTS

A total of 167 participants completed the questionnaire yielding a response rate of 27.2% (n = 167). These included 96 BMBS students (27.4%), 56 BDS students (24.7%), and 15 BScDTH students (39.5%). A 3 -Program (BMBS, BDS, BScDTH) by 8-Topic (A-H) mixed analysis of variance (ANOVA) was conducted on the questionnaire responses. This revealed statistically significant main effects of Program and Topic, as well as a statistically significant interaction between the two (i.e. the pattern of topic differences was different across programs).

CONCLUSIONS

Undergraduate students in medicine, dentistry, and dental therapy and hygiene regarded PT as a useful assessment to support their learning needs. However, in comparison to students in dentistry and dental therapy and hygiene, the perceptions of medical students were less positive in several aspects of PT.

Conversion of extrinsic into intrinsic motivation and computer based testing (CBT)

BACKGROUND

Because computers are used in many aspects of today's life, it seems necessary to include them in teaching and assessment processes.

METHOD

The aims of this cross-sectional study were to construct a multidimensional valid scale, to identify the factors that influenced the nature of student motivation on Computer Based Testing (CBT), to recognize how students self-regulated their activities around CBT, and to describe the efficiency of autonomous versus controlled situations on motivation. The study was carried out among 246 Iranian Paramedical Students of Tabriz Medical Sciences University, Tabriz, Iran; 2013-2014. The researchers prepared a questionnaire, based on the Self-Determination Theory (SDT), containing 26 items with a five-point Likert scale. It was prepared according to a previous valid questionnaire and by sharing opinions with some students and five professors. The factor analysis was done to perform instructional and exploratory factor analysis.

RESULTS

The Kaiser-Meyer-Olkin(KMO) measure was performed and variables were correlated highly enough to provide a reasonable basis for factor analysis. The selected 4 factors determined a 60.28% of the variance; autonomy 26.37%, stimulation 14.11%, relatedness10.71%, and competency 9.10%.

CONCLUSION

A questionnaire was prepared and validated, based on SDT variables. The results indicated that autonomous extrinsic motivation correlated positively with intrinsic motivation and CBT. There was a general positive attitude towards computer-based testing among students. As students became intrinsically motivated through the promotion of autonomous regulation, CBT was recommended as a proper test mode.

Contextual attributes promote or hinder self-regulated learning: A qualitative study contrasting rural physicians with undergraduate learners in Japan

BACKGROUND AND OBJECTIVES

Previous studies support the notion that East Asian medical students do not possess sufficient self-regulation for postgraduate clinical training. However, some East Asian physicians who are employed in geographically isolated and educationally underserved rural settings can self-regulate their study during the early phase of their postgraduate career. To explore the contextual attributes that contribute to self-regulated learning (SRL), we examined the differences in self-regulation between learning as an undergraduate and in a rural context in East Asia.

METHODS

We conducted interviews and diary data collection among rural physicians (n = 10) and undergraduates (n = 11) in Japan who undertook self-study of unfamiliar diseases. We analyzed three domains of Zimmerman's definition of SRL: learning behaviors, motivation, and metacognition using constructivist grounded theory.

RESULTS

Rural physicians recognized their identity as unique, and as professionals with a central role of handling diseases in the local community by conducting self-study. They simultaneously found themselves being at risk of providing inappropriate aid if their self-study was insufficient. They developed strategic learning strategies to cope with this high-stakes task. Undergraduates had a fear of being left behind and preferred to remain as one of the crowd with students in the same school year. Accordingly, they copied the methods of other students for self-study and used monotonous and homogeneous strategies.

CONCLUSIONS

Different learning contexts do not keep East Asian learners from being self-regulated. Awareness of their unique identity leads them to view learning tasks as high-stakes, and to initiate learning strategies in a self-regulated manner. Teacher-centered education systems cause students to identify themselves as one of the crowd, and tasks as low-stakes, and to accordingly employ non-self-regulated strategies.

Percepção e desempenho de graduandos de enfermagem em avaliação de metodologias ativas

Resumo Objetivo: Conhecer o desempenho e a percepção dos estudantes de enfermagem na prova de progressão e verificar a existência de relação entre o desempenho e a série em curso e entre o desempenho e o grau de dificuldade da prova. Métodos: Trata-se de um estudo descritivo de abordagem quantitativa, utilizou-se um instrumento constituído por questões sócio demográficas, classificação do grau de dificuldade da prova e duas perguntas sobre vantagens e desvantagens da prova de progressão. Resultados: A população constituiu-se de 78 estudantes. Apresentaram idade média de 24,4 anos e predomínio do sexo feminino, 89,7%. Os estudantes que consideraram a prova fácil tiveram desempenho maior (p=0.036), os estudantes do segundo ano destacaram-se em relação ao primeiro (p=0,014). Consideraram vantagem: conteúdo da prova 32,0%, progressão e desempenho 25,7%, questões de múltipla escolha 23,1%, e preparo para concursos 7,7%. As desvantagens, 39,7% prova muito extensa, 26,9% ausência de conhecimento, 15,3% temática e estrutura inadequadas e 6,4%pouco tempo para realização da prova. Conclusão: A Prova de Progressão avalia o desempenho do estudante longitudinalmente, identifica potencialidades e fragilidades curriculares, além de avaliar a instituição. Compreende um dos instrumentos avaliativos no uso de metodologias ativas de ensino e deve ser incentivada quanto à sua aplicabilidade nos cursos de graduação.