Evaluation of self-directed clinical education: validation of an instrument

Evaluating real-patient learning in medical education - Hungarian validation of the Manchester Clinical Placement Index

Introduction

The Manchester Clinical Placement Index (MCPI) is an instrument to measure medical undergraduates' real-patient learning in communities of practice both in hospital and in GP placements. Its suitability to evaluate the quality of placement learning environments has been validated in an English-language context; however, there is a lack of evidence for its applicability in other languages. Our aim was to thoroughly explore the factor structure and the key psychometric properties of the Hungarian language version.

Methods

MCPI is an 8-item, mixed-method instrument which evaluates the quality of clinical placements as represented by the leadership, reception, supportiveness, facilities and organization of the placement (learning environment) as well as instruction, observation and feedback (training) on 7-point Likert scales with options for free-text comments on the strengths and weaknesses of the given placement on any of the items. We collected data online from medical students in their preclinical (1st, 2nd) as well as clinical years (4th, 5th) in a cross-sectional design in the academic years 2019-2020 and 2021-2022, by the end of their clinical placements. Our sample comprises data from 748 medical students. Exploratory and confirmatory factor analyses were performed, and higher-order factors were tested.

Results

Although a bifactor model gave the best model fit (RMSEA = 0.024, CFI = 0.999, and TLI = 0.998), a high explained common variance (ECV = 0.82) and reliability coefficients (ωH = 0.87) for the general factor suggested that the Hungarian version of the MCPI could be considered unidimensional. Individual application of either of the subscales was not supported statistically due to their low reliabilities.

Discussion

The Hungarian language version of MCPI proved to be a valid unidimensional instrument to measure the quality of undergraduate medical placements. The previously reported subscales were not robust enough, in the Hungarian context, to distinguish, statistically, the quality of learning environments from the training provided within those environments. This does not, however, preclude formative use of the subscales for quality improvement purposes.

Medical undergraduates' self-evaluation: before and after curriculum reform

BACKGROUND

In 2013, Taiwan launched a curriculum reform-the 7-year undergraduate medical education program was shortened to 6 years. This study explored the evaluation results from students regarding the curriculum reform and investigated graduates' perceptions regarding the curriculum organization of the two academic training programs affected by this curricular reform.

METHODS

A cross-sectional survey was conducted from May 14 to June 12, 2019. The 315 graduates from both the 7-year and 6-year curriculum programs in the same medical school in Taipei were invited to participate in this study. In total, 197 completed questionnaires were received, representing a response rate of 62.5%. The results of the principal component analysis confirmed the validity of the constructs employed in this self-administered questionnaire.

RESULTS

The t-test results yielded two main findings. First, the graduates from the 6-year program had significantly lower scores for preparedness for the upcoming postgraduate-year residency training than did their 7-year program counterparts. Additionally, the male graduates had significantly higher scores in terms of perceptions regarding curriculum organization and preparedness for postgraduate-year residency training than the female graduates. The results of stepwise regression also indicated that the sex difference was significantly correlated with graduates' readiness for their postgraduate-year residency training.

CONCLUSION

To avoid sex disparities in career development, a further investigation of female medical students' learning environment and conditions is necessary. In addition to the cross-sectional study of students' perceptions, further repeated measurements of the objective academic or clinical performance of graduates in clinical settings are desirable.

The evaluation of student-centredness of teaching and learning: a new mixed-methods approach

OBJECTIVE

The aim of the study was to develop and consider the usefulness of a new mixed-methods approach to evaluate the student-centredness of teaching and learning on undergraduate medical courses. An essential paradigm for the evaluation was the coherence between how teachers conceptualise their practice (espoused theories) and their actual practice (theories-in-use).

METHODS

The context was a module within an integrated basic sciences course in an undergraduate medical degree programme. The programme had an explicit intention of providing a student-centred curriculum. A content analysis framework based on Weimer's dimensions of student-centred teaching was used to analyze data collected from individual interviews with seven teachers to identify espoused theories and 34h of classroom observations and one student focus group to identify theories-in-use. The interviewees were identified by purposeful sampling. The findings from the three methods were triangulated to evaluate the student-centredness of teaching and learning on the course.

RESULTS

Different, but complementary, perspectives of the student-centredness of teaching and learning were identified by each method. The triangulation of the findings revealed coherence between the teachers' espoused theories and theories-in-use.

CONCLUSIONS

A mixed-methods approach that combined classroom observations with interviews from a purposeful sample of teachers and students offered a useful evaluation of the extent of student-centredness of teaching and learning of this basic science course. Our case study suggests that this new approach is applicable to other courses in medical education.

How clerkship students learn from real patients in practice settings

PURPOSE

To explore how undergraduate medical students learn from real patients in practice settings, the factors that affect their learning, and how clerkship learning might be enhanced.

METHOD

In 2009, 22 medical students in the three clerkship years of an undergraduate medical program in the United Kingdom made 119 near-contemporaneous audio diary entries reflecting how they learned from real patients. Nineteen attended focus groups; 18 were individually interviewed. The authors used a qualitative theory-building methodology with a conceptual orientation toward Communities of Practice theory. A learning theorist guided selective coding of a constant-comparative analysis.

RESULTS

Participants learned informally by participating in the communicative practices of workplaces. Two overlapping practices, patient care and education, were identified. Patient care created learning opportunities, which were enriched when practitioners intentionally supported participants' learning. Education, however, was not always coupled with patient care. So, talk positioned the boundaries of two practices in three configurations: education without patient care, education within patient care, and patient care without education. The nature and quality of participants' learning depended on how practitioners entered dialogue with them and linked the dialogue to authentic patient care.

CONCLUSIONS

Findings strongly suggest that medical students learn from real patients by participating in patient care within an educational practice. Their learning is affected by clinicians' willingness to engage in supportive dialogue. Promoting an informal, inclusive discourse of workplace learning might enhance clerkship education. This approach should take its place alongside-and perhaps ahead of-the currently dominant discourse of "clinical teaching."

Power in clinical teachers' discourses of a curriculum-in-action. Critical discourse analysis

"Curricula-in-action" generally differ from "official" curricula. That is particularly true of clerkship curricula because the practising doctors who supervise medical students' clinical activities are only secondarily educators. Clerkship education is evaluated, however, according to benchmarks set by official curricula. As a result, clerkship evaluations are important points of contact between clinical teachers and medical schools. We reasoned that an evaluation instrument is part of a medical school's official curriculum discourse and clinical teachers' reactions to it are a discourse of curriculum-in-action. We set out to answer the questions: What are clinical teachers' discourses of curriculum-in-action and how do they relate to an official curriculum discourse? Nineteen clerkship placement leads from two hospitals contributing to a single undergraduate medical programme participated. The evaluation instrument was the Manchester Clinical Placement Index, for which validity evidence has been published. Respondents were asked to say how they would react to junior students giving their placements low or high scores for each of 12 items from the Index. After transcription, we conducted a critical discourse analysis (CDA) of their audio-recorded answers. We purposefully selected the six items that elicited the widest spectrum of responses for analysis because quantity of material can compromise the quality of CDA. A dominant discourse of curriculum-in-action defined how teachers should "really" teach and junior students should learn. It deconstructed the need for teachers to be present when students performed clinical tasks because teachers' role was to give critical feedback on case presentations that were coincidental to clinical care. It positioned students at the bottom of a power hierarchy so they had to "struggle" to be taught. It placed respondents in a powerful position relative to "the hospital" and "the university", though there were tensions between respondents, patients, and nurses. Respondents dismissed criticism that was invalid according to their curriculum-in-action, which included most items in an evaluation instrument. There was a contrasting, non-dominant discourse of responding reflectively to feedback, which generated realistic ways of improving students' learning. The strength of respondents' emotions shows just how committed doctors are to students' learning. The strength of their expressions of power, however, explains why many of them teach in their own way rather than according to official curricula. Changes to clinical curricula, our findings suggest, will not be successful unless they are carefully negotiated with practising doctors.

Manchester Clinical Placement Index (MCPI). Conditions for medical students' learning in hospital and community placements

The drive to quality-manage medical education has created a need for valid measurement instruments. Validity evidence includes the theoretical and contextual origin of items, choice of response processes, internal structure, and interrelationship of a measure's variables. This research set out to explore the validity and potential utility of an 11-item measurement instrument, whose theoretical and empirical origins were in an Experience Based Learning model of how medical students learn in communities of practice (COPs), and whose contextual origins were in a community-oriented, horizontally integrated, undergraduate medical programme. The objectives were to examine the psychometric properties of the scale in both hospital and community COPs and provide validity evidence to support using it to measure the quality of placements. The instrument was administered twice to students learning in both hospital and community placements and analysed using exploratory factor analysis and a generalizability analysis. 754 of a possible 902 questionnaires were returned (84% response rate), representing 168 placements. Eight items loaded onto two factors, which accounted for 78% of variance in the hospital data and 82% of variance in the community data. One factor was the placement learning environment, whose five constituent items were how learners were received at the start of the placement, people's supportiveness, and the quality of organisation, leadership, and facilities. The other factor represented the quality of training-instruction in skills, observing students performing skills, and providing students with feedback. Alpha coefficients ranged between 0.89 and 0.93 and there were no redundant or ambiguous items. Generalisability analysis showed that between 7 and 11 raters would be needed to achieve acceptable reliability. There is validity evidence to support using the simple 8-item, mixed methods Manchester Clinical Placement Index to measure key conditions for undergraduate medical students' experience based learning: the quality of the learning environment and the training provided within it. Its conceptual orientation is towards Communities of Practice, which is a dominant contemporary theory in undergraduate medical education.

When only the real thing will do: junior medical students' learning from real patients

OBJECTIVES

This study aimed to explore how medical students experience contacts with real patients and what they learn from them.

METHODS

We carried out a post hoc, single-group study in one teaching sector of a 5-year, problem-based, horizontally integrated, outcome-based and community-oriented undergraduate programme, in which students lacked clinical exposure in the pre-clerkship phase. Subjects comprised five cohorts of students on their first clerkships. Data consisted of purposively selected, voluntary, self-report statements regarding real patient learning (RPL). Constant comparative analysis was performed by two independent researchers.

RESULTS

Respondents valued patients as an instructional resource that made learning more real. They reported learning through visual pattern recognition as well as through dialogue and physical examination. They more often used social than professional language to describe RPL. They reported affective outcomes including enhanced confidence, motivation, satisfaction and a sense of professional identity. They also reported cognitive outcomes including perspective, context, a temporal dimension, and an appreciation of complexity. Real patient learning helped respondents link theory learned earlier with reality as represented by verbal, visual and auditory experiences. It made learning easier, more meaningful and more focused. It helped respondents acquire complex skills and knowledge. Above all, RPL helped learners to remember subject matter. Most negative responses concerned the difficulty of acquiring appropriate experience, but RPL made a minority of respondents feel uncomfortable and incompetent.

CONCLUSIONS

Real patient learning led to a rich variety of learning outcomes, of which at least some medical students showed high metacognitive awareness. Sensitivity from clinical mentors towards the positive and negative outcomes of RPL reported here could support reflective clinical learning.

Student evaluation of the clinical 'curriculum in action'

AIM

To examine how students' evaluations of the environment, process and outcome of clinical learning interrelated and correlated with assessment results.

METHOD

A post hoc study in the 3rd of 5 years in a student-centred, horizontally integrated, objective-based medical curriculum. In the last week of each module, students evaluated what they had learned and how they had learned it using a previously validated, web-based scale. The interrelationships between scale variables and their relationships with summative assessment results were tested using factor analysis, correlation analysis and stepwise multiple regression analysis.

RESULTS

Student evaluation yielded 4 summary measures: 2 reflected learning outcomes ('real patient learning' and 'curriculum coverage'), 1 reflected process ('quality of instruction') and 1 reflected environment ('conditions for learning'). They fitted a causal model according to which instruction, conditions for learning and curriculum coverage favoured real patient learning. Real patient learning was rated higher in women than men, and the measures were associated more strongly in women. Performance in end-of-year summative assessments was predicted strongly by mid-year performance but by no other measure.

CONCLUSIONS

Students' evaluations of their learning environment and instructional processes correlated with their assessments of 2 outcomes of the curriculum in action: curriculum coverage and real patient learning. There was little shared variance between those measures and students' performance in summative assessments. Given its formative potential, students' evaluation of their curriculum in action could play a useful part in learner-centred clinical education. There is a possibility, which needs further research, that women's evaluations have greater predictive validity than men's. Assessment performance should be regarded not as a solitary gold standard but as just 1 measure of educational outcome.

How can medical students learn in a self-directed way in the clinical environment? Design-based research

AIM

This study aimed to establish whether and under what conditions medical students can learn in a self-directed manner in the clinical environment.

METHOD

A web-based learning management system brought 66 placement students, in a problem-based learning (PBL) medical curriculum, into closer touch with their clinical learning objectives and ways of achieving them. Free response comments from 16 of them during the 7 weeks they used it, transcripts of group discussions before and after the period of use, and responses from all 66 students to a questionnaire were analysed qualitatively.

RESULTS

Students were rarely fully autonomous or subservient. They valued affective and pedagogic support, and relied on teachers to manage their learning environment. With support, they were motivated and able to choose how and when to meet their learning needs. The new system was a useful adjunct.

CONCLUSIONS

Self-direction, interpreted literally, was a method of learning that students defaulted to when support and guidance were lacking. They found "supported participation" more valuable. Learning in the clinical environment was a social process with as many differences from, as similarities to, PBL.