How cognitive engagement fluctuates during a team-based learning session and how it predicts academic achievement

Student engagement and learning outcomes: an empirical study applying a four-dimensional framework

INTRODUCTION

This study applies Reeve's four-dimensional student engagement framework to a medical education context to elucidate the relationship between behavioral, emotional, cognitive, and agentic engagement and learning outcomes. Meanwhile, we categorize learning outcomes in knowledge and skills, and added taxonomies to the cognitive education objectives for the knowledge part, including memorization, comprehension, and application.

METHODS

We used the China Medical Student Survey to investigate student engagement, and combined it with the Clinical Medicine Proficiency Test for Medical Schools results as a standardized measurement of learning outcomes. We performed multivariate regression analyses to delve into the effectiveness of different types of student engagement. Moreover, we evaluated the moderating roles of gender and the National College Entrance Examination (NCEE) within the relationships between student engagement and learning outcomes.

RESULTS

We observed that emotional engagement is most effective in promoting learning outcomes in basic medical knowledge and basic clinical skills. Emotional engagement and cognitive engagement could effectively contribute to learning outcomes in all three aspects of basic medical knowledge. In contrast, behavioral and agentic engagement showed negative effects on learning outcomes. Besides, we found that the results of the NCEE played a positive moderating role.

CONCLUSION

This study provides robust evidence for the effectiveness of emotional engagement and cognitive engagement in promoting learning outcomes. Whereas behavioral and agentic engagement may not be good predictors of learning outcomes in macro-level general competence tests. We suggest a combined effort by students and institutions to promote student engagement and bridge the distance between general competency tests and daily learning activities.

Construct validity of a questionnaire for measuring student engagement in problem-based learning tutorials

BACKGROUND

Student engagement is student investment of time and energy in academic and non-academic experiences that include learning, teaching, research, governance, and community activities. Although previous studies provided some evidence of measuring student engagement in PBL tutorials, there are no existing quantitative studies in which cognitive, behavioral, and emotional engagement of students in PBL tutorials is measured. Therefore, this study aims to develop and examine the construct validity of a questionnaire for measuring cognitive, behavioral, and emotional engagement of students in PBL tutorials.

METHODS

A 15-item questionnaire was developed guided by a previously published conceptual framework of student engagement. Focus group discussion (n = 12) with medical education experts was then conducted and the questionnaire was piloted with medical students. The questionnaire was then distributed to year 2 and 3 medical students (n = 176) in problem-based tutorial groups at the end of an integrated course, where PBL is the main strategy of learning. The validity of the internal structure of the questionnaire was tested by confirmatory factor analysis using structural equation modeling assuming five different models. Predictive validity evidence of the questionnaire was studied by examining the correlations between students' engagement and academic achievement.

RESULTS

Confirmatory factor analysis indicates a good fit between the measurement and structural model of an 11-item questionnaire composed of a three-factor structure: behavioral engagement (3 items), emotional engagement (4 items), and cognitive engagement (4 items). Models in which the three latent factors were considered semi-independent provided the best fit. The construct reliabilities of behavioral, cognitive, and emotional factors were 0.82, 0.82, and 0.76, respectively. We failed however to find significant relationships between academic achievement and engagement.

CONCLUSIONS

We found a strong evidence to support the construct validity of a three-factor structure of student engagement in PBL tutorial questionnaire. Further studies are required to test the validity of this instrument in other educational settings. The predictive validity is another area needing further scrutiny.

Student engagement in health professions education: AMEE Guide No. 152

This guide aims to support our colleagues to have comprehensive understanding of student engagement in health professions education. Despite the universal agreement about the significance of student engagement, there is lack of uniformity in conceptualizing and operationalizing this emerging construct. We review the theoretical basis explaining student engagement from three main perspectives: behavioral, psychological, and socio-cultural. In addition, we propose a contemporary and comprehensive framework for the student engagement in higher education, which is applicable to health professions education contexts. Drawing from this framework, we explain the conceptualization of the construct and its preceding factors, mediators, dimensions, spheres, and outcomes of student engagement. The proposed framework introduces student 'engagement through partnerships' as a novel component compared with the existing models of student engagement in higher education. This way, we are proposing a mixed model that not only considers the student as a 'customer' but also as a 'partner' in education. Engagement of students through partnerships include four areas: (1) provision of the education program, (2) scholarly research, (3) governance and quality assurance, and (4) community activities. This guide will provide practical applications on how to improve student engagement in health professions education. Finally, we highlight the current gaps in areas of research in the student engagement literature and suggested plans for future directions.[Box: see text].

Using the design-thinking method to develop and validate a peer evaluation scale for team-based learning (PES-TBL) for nursing students

BACKGROUND

Peer evaluation in team-based learning provides meaningful information about the learning process and dynamics. Despite the importance of peer evaluation in learner-centred learning, there is a lack of valid and reliable scales that reflect students' experiences in the classroom.

OBJECTIVES

To develop and validate a peer evaluation scale for team-based learning.

DESIGN

A cross-sectional methodological study.

SETTING

Four universities located in Gwangju, Republic of Korea.

PARTICIPANTS

Eight nursing students exposed to team-based learning voluntarily participated in a design-thinking project, and eight nursing professors were selected for content validity. For the validation of the Scale, 722 nursing students were randomly selected.

METHODS

The design-thinking method was implemented to develop the Scale, and a questionnaire was used to assess the Scale's construct validity and reliability. The construct validity was examined in a split-half analysis with exploratory and confirmatory factor analyses. Cronbach's alpha, McDonald's omega, and composite reliability were investigated for the peer evaluation scale for team-based learning.

RESULTS

A 12-item tool, with each item using a 5-point scale for peer evaluation, was developed through the empathise, define, ideate, prototype, and test stages of the design-thinking method. Exploratory factor analysis identified three factors from the 12 items: responsibility, initiative, and collaboration. Confirmatory factor analysis demonstrated that the tool had acceptable convergent and discriminant validity, thus confirming good construct validity. All values for reliability were >0.70.

CONCLUSIONS

This study was noteworthy in that it employed the design-thinking method to reflect learners' opinions in developing a peer evaluation instrument. Moreover, the study demonstrated adequate evidence of reliability and validity. Consequently, the developed Scale can be effectively applied to team-based learning assessments for nursing students.

Measurement of student engagement in health professions education: a review of literature

Student engagement is a complex multidimensional construct that has attained great interest in health professions education (HPE). Definition and conceptualization of student engagement is an important step that should drive the development of the instruments for its measurement. We have recently proposed a comprehensive framework for student engagement in HPE with a definition of engagement as student investment of time and energy in academic and non-academic experiences that include learning, teaching, research, governance, and community activities. The dimensions of student engagement in this framework included the cognitive, affective, behavioral, agentic, and socio-cultural. Guided by the student engagement framework, this non-systematic review aims to identify, critically appraise, and summarize the existing methods for measuring student engagement in HPE. Extrapolating from higher education literature, we attempted to link the theoretical perspectives of student engagement with the published methods of its measurement in HPE context. In addition, we have described the different methods of measuring student engagement including self-report surveys, real time measures, direct observation, interviews/focus groups, and the use of multiple instruments. The span of engagement dimensions measured by self-report surveys ranges from one to five dimensions. However, measurement of agentic and sociocultural dimensions of engagement in HPE is still limited and further research is required. We have also reflected on the existing methods of measuring engagement of students as active partners in HPE. The review also describes the advantages, limitations, and psychometric properties of each method for measuring student engagement. We ended the review with a guiding conclusion on how to develop and select an instrument for measuring student engagement in HPE. Finally, we addressed the gaps in the literature about measuring engagement of HPE students and future research plans.

Individual differences in selective attention and engagement shape students’ learning from visual cues and instructor presence during online lessons

Although some researchers recommend minimizing extraneous visual information in multimedia lessons, others have demonstrated that features such as visual cues and instructor videos can enhance learning. However, variability in selective attention skills may influence students’ ability to benefit from these additional features. This study investigated links between college students’ selective attention skills and their learning from video lessons that varied in the use of visual cues and the instructor video. Learning outcomes depended on both the visual features available and students’ effort and selective attention skills. Among students who reported increased effort during the lessons, those with more efficient selective attention benefited most when a single additional feature (i.e., either visual cues or the instructor video) was used. All students, regardless of attention skills, benefited when both visual cues and the instructor were combined. These findings suggest that learning during multimedia lessons may depend on the visual features of the lessons and the student’s effort and attention skills.

Does your group matter? How group function impacts educational outcomes in problem-based learning: a scoping review

BACKGROUND

Problem-based learning (PBL) is a common instructional method in undergraduate health professions training. Group interactions with and within PBL curricula may influence learning outcomes, yet few studies have synthesized the existing evidence. This scoping review summarized the literature examining the influence of group function on individual student PBL outcomes. Following Kirkpatrick's framework, experiential, academic, and behavioral outcomes were considered. The impacts of three aspects of group function were explored: (1) Group Composition (identities and diversity), (2) Group Processes (conduct and climate, motivation and confidence, and facilitation), and (3) PBL Processes (tutorial activities).

METHODS

A literature search was conducted using Medline, CINAHL, and APA PsychInfo from 1980-2021, with the help of a librarian. English-language empirical studies and reviews that related group function to learning outcome, as defined, in undergraduate health professions PBL curricula were included. Relevant references from included articles were also added if eligibility criteria were met. The methods, results, discussions, and limitations of the sample were summarized narratively.

RESULTS

The final sample (n = 48) varied greatly in context, design, and results. Most studies examined junior medical students (n = 32), used questionnaires for data collection (n = 29), and reported immediate cross-sectional outcomes (n = 34). Group Processes was the most frequently examined aspect of group function (n = 29), followed by Group Composition (n = 26) and PBL Processes (n = 12). The relationships between group function and outcomes were not consistent across studies. PBL experiences were generally highly rated, but favorable student experiences were not reliable indicators of better academic or behavioral outcomes. Conversely, problematic group behaviors were not predictors of poorer grades. Common confounders of outcome measurements included exam pressure and self-study.

CONCLUSIONS

The main findings of the review suggested that (1) group function is more predictive of experiential than academic or behavioral PBL outcomes, and (2) different Kirkpatrick levels of outcomes are not highly correlated to each other. More research is needed to understand the complexity of group function in PBL tutorials under variable study contexts and better inform curricular training and design. Standardized tools for measuring PBL group function may be required for more conclusive findings.

Professional Skills Teaching within Veterinary Education and Possible Future Directions

Once ignored for their potential to take up precious time within the veterinary curriculum at the expense of hard science and technical competencies, professional skills such as ethical conduct, professional conduct, and communication skills are now considered essential in the creation of successful and employable graduates. Despite the requirement to include professional skills in veterinary curricula, limited communication among colleges and inconsistent documentation of curricular developments have led to a wide range of teaching and assessment methods with no consistent standards existing among colleges. Integration of professional competency teaching into the general curriculum is lauded widely, but barriers such as faculty buy-in have kept many colleges from moving toward a standard in which professional competencies are integrated into the general curriculum. The aim of this article is to provide veterinary educators and curriculum designers with an understanding of the rationale for including professional skills teaching within the curriculum while also presenting currently used, as well as recommended, strategies for effective instruction of professional skills.

Inserting an Erroneous Element in the Answer Key of Grouped Team Readiness Assurance Test in Team-Based Learning: A Randomized Controlled Trial in Clinical Toxicology

PURPOSE

Team-based learning (TBL) is an interactive instructional strategy designed to enhance student engagement. Few studies reported that TBL needs to be more interactive, concerns were raised regarding the insufficient instructor feedback, and students feel that TBL gets less attractive with time. One method of boosting students' interaction and performance is adding an element of challenge or positive stress. Therefore, we hypothesized that inserting an erroneous answer in the answer key would generate an observable improvement in the selected outcomes in terms of better satisfaction, interaction, interest in learning, better academic performance, and better development of competencies compared to traditional TBL.

METHODS

This randomized controlled trial aimed to determine if inserting an erroneous element in the answer key of a grouped team readiness assurance test (GRAT) would update TBL and whether this intervention would enhance students' performance and satisfaction.

RESULTS

A total of 55 medical students were recruited (88.7% response rate). Erroneous elements were inserted in the answer key of the experimental group and students enrolled in traditional TBL were considered as controls. Compared to the control group (p < 0.001), the experimental group revealed significantly higher academic performances in GRAT and team evaluation test (TET). Analysis of students' perception of the implemented TBL revealed better perception among the experimental group (33.7 ± 6.4) than the control group (30.1 ± 7.0). Moreover, significantly higher team dynamics were reported among the experimental group than the control group (33.0 ± 6.3 and 27.8 ± 7.6 for both groups, respectively; p = 0.005). The reported advantages were in-depth understanding, easier information retrieval, and development of problem-solving skills. Students considered time and effort as their main limitations.

CONCLUSION

Adding a few erroneous answers in the GRAT is well perceived by students, enhances their learning competencies and overcomes some TBL challenges.

Effect of Seating Arrangement on Class Engagement in Team-based Learning: a Quasi-Experimental Study

INTRODUCTION

This study investigated the effects of seating distance and orientation on engagement in novice and experienced learners in a large classroom explicitly designed for team-based learning (TBL). Learning what affects TBL engagement may improve its implementation.

METHODS

Participants were novice first-year and experienced second-year undergraduate medical students in Singapore (male = 103, female = 57). Their age ranged from 18 to 23 (M = 19.5, SD = 1.06). This quasi-experimental study considered two factors. Firstly, the distance from the teams' table to the tutor's table. Secondly, students' orientation at each table, with either their front or back facing the tutor. Engagement was measured using two instruments, Situational Cognitive Engagement Measure and Classroom Engagement Survey at two TBL sessions - before and after swapping seating arrangements.

RESULTS

For experienced students, seating distance did not significantly affect engagement (p = 0.08-0.89). Novice student's engagement levels decreased significantly for those who moved further; M = 3.30 to 2.98 (p = 0.009-0.023). However, overall engagement also decreased post-swap regardless of direction moved; M = 3.26 to 3.00 (p = 0.004). For both cohorts, seating orientation did not significantly affect engagement (p = 0.07-0.62). Those unaffected by seating arrangement commended the classroom's design, such as screens all around and quality audio-visual system. Novice students exhibited a stronger preference to sit nearer to the tutor than experienced students. Both groups preferred sitting with their front-facing the tutor.

DISCUSSION

Within specially designed TBL classrooms, seating distance and orientation did not significantly affect engagement. Technologically enhanced team-centric spaces provide a favourable environment for TBL, though students' preferences for seats may change with more TBL experience.

SUPPLEMENTARY INFORMATION

The online version contains supplementary material available at 10.1007/s40670-021-01469-7.

Use of Team-Based Learning Pedagogy to Prepare for a Pharmacy School Accreditation Self-Study

Ensuring adequate engagement and preparation of all stakeholders in an accreditation self-study can be challenging for many reasons, including lack of motivation and inadequate understanding of expectations and procedures. The goal of this exploratory study was to determine whether using team-based learning (TBL) pedagogy to deliver an accreditation preparation workshop could effectively prepare and engage participants. A Likert-scale questionnaire was administered to workshop attendees (n = 52) to determine whether they found TBL-based training helpful and whether it promoted engagement. Twenty-four attendees completed the survey (46%). More than 80% of participants strongly agreed or agreed with 12 statements relating to perceptions of self and participant engagement within team activities and the usefulness of team activities. More than 65% of participants strongly agreed or agreed with statements relating to the helpfulness of the TBL approach in preparing for the self-study (five questions). Subgroup analysis showed no significant difference in responses based on whether on not participants had previously been involved in an accreditation self study. Our data indicate that a TBL approach can be an effective way to engage and prepare stakeholders for an accreditation self-study, and that TBL pedagogy has utility outside of the classroom setting.

Design of a foundational sciences curriculum: Applying the ICAP framework to pharmacology education in integrated medical curricula

Expectations for physicians are rapidly changing, as is the environment in which they will practice. In response, preclerkship medical education curricula are adapting to meet these demands, often by reducing the time for foundational sciences. This descriptive study compares preclerkship pharmacology education curricular practices from seven allopathic medical schools across the United States. We compare factors and practices that affect how pharmacology is integrated into the undergraduate medical education curriculum, including teaching techniques, resources, time allocated to pharmacology teaching, and assessment strategies. We use data from seven medical schools in the United States, along with results from a literature survey, to inform the strengths and weaknesses of various approaches and to raise important questions that can guide future research regarding integration of foundational sciences in medical school and health professions' curricula. In this comparative study, we found that there is significant heterogeneity in the number of hours dedicated to pharmacology in the preclerkship curriculum, whereas there was concordance in the use of active learning pedagogies for content delivery. Applying the ICAP (Interactive, Constructive, Active, Passive) Framework for cognitive engagement, our data showed that pharmacology was presented using more highly engaging pedagogies during sessions that are integrated with other foundational sciences. These findings can serve as a model that can be applied beyond pharmacology to other foundational sciences such as genetics, pathology, microbiology, biochemistry, etc.

Exploring the feasibility of using very short answer questions (VSAQs) in team-based learning (TBL)

BACKGROUND

Team-based learning (TBL) currently relies on single best answer questions (SBAQs) to provide immediate feedback. Very short answer questions (VSAQs) are a reliable and discriminatory alternative that encourage learners to use more authentic clinical reasoning strategies compared to SBAQs. However, the challenge of marking VSAQs has limited their integration into TBL; we therefore explored the feasibility of VSAQs within a TBL session.

METHODS

An online platform was developed to allow immediate marking of VSAQs during the TBL sessions. As part of the readiness assurance process, students completed VSAQs and SBAQs, which were marked in real time.

RESULTS

Instructors were able to mark all VSAQs during the individual readiness assurance test (iRAT), which facilitated the provision of immediate feedback during the team readiness assurance test (tRAT). The mean time to mark five VSAQs was 422 seconds (SD 73 seconds). For VSAQs, the number of attempts to reach the correct answer ranged from 1 to 38, compared to 1 to 4 for SBAQs. In total, 71.6% of students agreed that using VSAQs in TBL helped to emphasise group discussions.

DISCUSSION

The wide range of attempts at, and students' perspectives of VSAQs are suggestive of their positive impact on student discussion during TBL. We demonstrate how new technology allows VSAQs to be feasibly integrated into TBL with the potential to enrich group discussions.

Implementation of team-based learning on a large scale: Three factors to keep in mind

Team-based learning (TBL) is a structured form of small group learning that can be scaled up for delivery in large classes. The principles of successful TBL implementation are well established. TBL has become widely practiced in medical schools, but its use is typically limited to certain courses or parts of courses. Implementing TBL on a large scale, across different courses and disciplines, is the next logical step. The Lee Kong Chian School of Medicine (LKCMedicine), a partnership between Nanyang Technological University, Singapore and Imperial College London, admitted its first students in 2013. This new undergraduate medical program, developed collaboratively by faculty at both institutions, uses TBL as its main learning and teaching strategy, replacing all face-to-face lectures. TBL accounts for over 60% of the curriculum in the first two years, and there is continued learning through TBL during campus teaching in the remaining years. This paper describes our experience of rolling out TBL across all years of the medical curriculum, focusing on three success factors: (1) "team-centric" learning spaces, to foster active, collaborative learning; (2) an e-learning ecosystem, seamlessly integrated to support all phases of the TBL process and (3) teaching teams in which experts in pedagogical process (TBL Facilitators) co-teach with experts in subject matter (Content Experts).