1
|
Fisher J, Leahy D, Lim JJ, Astles E, Salvatore J, Thomson R. Question banks: credit? Or debit? A qualitative exploration of their use among medical students. BMC MEDICAL EDUCATION 2024; 24:569. [PMID: 38790034 PMCID: PMC11127331 DOI: 10.1186/s12909-024-05517-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/15/2024] [Accepted: 05/03/2024] [Indexed: 05/26/2024]
Abstract
BACKGROUND Online question banks are the most widely used education resource amongst medical students. Despite this there is an absence of literature outlining how and why they are used by students. Drawing on Deci and Ryan's self-determination theory, our study aimed to explore why and how early-stage medical students use question banks in their learning and revision strategies. METHODS The study was conducted at Newcastle University Medical School (United Kingdom and Malaysia). Purposive, convenience and snowball sampling of year two students were employed. Ten interviews were conducted. Thematic analysis was undertaken iteratively, enabling exploration of nascent themes. Data collection ceased when no new perspectives were identified. RESULTS Students' motivation to use question banks was predominantly driven by extrinsic motivators, with high-stakes exams and fear of failure being central. Their convenience and perceived efficiency promoted autonomy and thus motivation. Rapid feedback cycles and design features consistent with gamification were deterrents to intrinsic motivation. Potentially detrimental patterns of question bank use were evident: cueing, avoidance and memorising. Scepticism regarding veracity of question bank content was absent. CONCLUSIONS We call on educators to provide students with guidance about potential pitfalls associated with question banks and to reflect on potential inequity of access to these resources.
Collapse
Affiliation(s)
- James Fisher
- Newcastle University School of Medicine, Newcastle upon Tyne, UK.
- Northumbria Healthcare NHS Foundation Trust, North Shields, UK.
| | - Declan Leahy
- Newcastle University School of Medicine, Newcastle upon Tyne, UK
| | - Jun Jie Lim
- School of Medicine and Population Health, University of Sheffield, Sheffield, UK
- Newcastle University Medicine Malaysia, Gelang Patah, Johor, Malaysia
| | - Emily Astles
- Newcastle University School of Medicine, Newcastle upon Tyne, UK
| | - Jacobo Salvatore
- Newcastle University School of Medicine, Newcastle upon Tyne, UK
| | - Richard Thomson
- Newcastle University School of Medicine, Newcastle upon Tyne, UK
- Northumbria Healthcare NHS Foundation Trust, North Shields, UK
| |
Collapse
|
2
|
Menger NS, Tognetti A, Farruggia MC, Mucignat C, Bhutani S, Cooper KW, Rohlfs Domínguez P, Heinbockel T, Shields VDC, D'Errico A, Pereda-Loth V, Pierron D, Koyama S, Croijmans I. Giving a Voice to Patients With Smell Disorders Associated With COVID-19: Cross-Sectional Longitudinal Analysis Using Natural Language Processing of Self-Reports. JMIR Public Health Surveill 2024; 10:e47064. [PMID: 38728069 PMCID: PMC11127136 DOI: 10.2196/47064] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/09/2023] [Revised: 10/26/2023] [Accepted: 03/11/2024] [Indexed: 05/12/2024] Open
Abstract
BACKGROUND Smell disorders are commonly reported with COVID-19 infection. The smell-related issues associated with COVID-19 may be prolonged, even after the respiratory symptoms are resolved. These smell dysfunctions can range from anosmia (complete loss of smell) or hyposmia (reduced sense of smell) to parosmia (smells perceived differently) or phantosmia (smells perceived without an odor source being present). Similar to the difficulty that people experience when talking about their smell experiences, patients find it difficult to express or label the symptoms they experience, thereby complicating diagnosis. The complexity of these symptoms can be an additional burden for patients and health care providers and thus needs further investigation. OBJECTIVE This study aims to explore the smell disorder concerns of patients and to provide an overview for each specific smell disorder by using the longitudinal survey conducted in 2020 by the Global Consortium for Chemosensory Research, an international research group that has been created ad hoc for studying chemosensory dysfunctions. We aimed to extend the existing knowledge on smell disorders related to COVID-19 by analyzing a large data set of self-reported descriptive comments by using methods from natural language processing. METHODS We included self-reported data on the description of changes in smell provided by 1560 participants at 2 timepoints (second survey completed between 23 and 291 days). Text data from participants who still had smell disorders at the second timepoint (long-haulers) were compared with the text data of those who did not (non-long-haulers). Specifically, 3 aims were pursued in this study. The first aim was to classify smell disorders based on the participants' self-reports. The second aim was to classify the sentiment of each self-report by using a machine learning approach, and the third aim was to find particular food and nonfood keywords that were more salient among long-haulers than those among non-long-haulers. RESULTS We found that parosmia (odds ratio [OR] 1.78, 95% CI 1.35-2.37; P<.001) as well as hyposmia (OR 1.74, 95% CI 1.34-2.26; P<.001) were more frequently reported in long-haulers than in non-long-haulers. Furthermore, a significant relationship was found between long-hauler status and sentiment of self-report (P<.001). Finally, we found specific keywords that were more typical for long-haulers than those for non-long-haulers, for example, fire, gas, wine, and vinegar. CONCLUSIONS Our work shows consistent findings with those of previous studies, which indicate that self-reports, which can easily be extracted online, may offer valuable information to health care and understanding of smell disorders. At the same time, our study on self-reports provides new insights for future studies investigating smell disorders.
Collapse
Affiliation(s)
- Nick S Menger
- Institute of Medical Psychology and Behavioural Neurobiology, University of Tübingen, Tübingen, Germany
| | - Arnaud Tognetti
- Department of Clinical Neuroscience, Division of Psychology, Karolinska Institutet, Stockholm, Sweden
- Centre d'Economie de l'Environnement Montpellier, Centre National de la Recherche Scientifique, Institut National de Recherche pour l'Agriculture l'Alimentation et l'Environnement, Institut Agro, Université de Montpellier, Montpellier, France
| | - Michael C Farruggia
- Interdepartmental Neuroscience Program, Yale University, New Haven, CT, United States
| | - Carla Mucignat
- Department of Molecular Medicine, University of Padova, Padua, Italy
| | - Surabhi Bhutani
- School of Exercise and Nutritional Sciences, San Diego State University, San Diego, CA, United States
| | - Keiland W Cooper
- Department of Neurobiology and Behavior, University of California, Irvine, CA, United States
| | - Paloma Rohlfs Domínguez
- Department of Developmental and Educational Psychology, University of Basque Country, Leioa, Spain
| | - Thomas Heinbockel
- Department of Anatomy, Howard University College of Medicine, Washington, DC, United States
| | - Vonnie D C Shields
- Biological Sciences Department, Fisher College of Science and Mathematics, Towson University, Towson, MD, United States
| | - Anna D'Errico
- Goethe University of Frankfurt, Frankfurt am Main, Germany
| | | | - Denis Pierron
- Laboratoire Évolution et Santé Orale, Université Toulouse III, Toulouse, France
| | - Sachiko Koyama
- Department of Medicine, School of Medicine, Indiana University, Indianapolis, IN, United States
| | - Ilja Croijmans
- Language and Communication Department, Faculty of Arts, Radboud University, Nijmegen, Netherlands
- Centre for Language Studies, Radboud University, Nijmegen, Netherlands
| |
Collapse
|
3
|
Panchbudhe S, Shaikh S, Swami H, Kadam CY, Padalkar R, Shivkar RR, Gulavani G, Gulajkar S, Gawade S, Mujawar F. Efficacy of Google Form-based MCQ tests for formative assessment in medical biochemistry education. JOURNAL OF EDUCATION AND HEALTH PROMOTION 2024; 13:92. [PMID: 38726095 PMCID: PMC11081433 DOI: 10.4103/jehp.jehp_981_23] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/07/2023] [Accepted: 08/24/2023] [Indexed: 05/12/2024]
Abstract
BACKGROUND Biochemistry, being a vast and complex subject, can be challenging for Phase I MBBS students to comprehend and retain. Embracing rapidly evolving technology can facilitate a more accessible learning experience. In this study, we investigated the effectiveness of using Google Form-based multiple-choice question (MCQ) tests as a formative assessment tool after each biochemistry lecture series. The aim was to assess the improvement and gather feedback of Phase I MBBS students on the utility of this assessment tool. MATERIALS AND METHODS This educational prospective longitudinal study was conducted by the Department of Biochemistry at a university-affiliated medical college and tertiary care hospital. The study included 150 Phase I MBBS students as participants. Google Form-based MCQ tests were implemented as educational interventions after each lecture series during the study period. The study compared the internal assessment (IA) MCQ marks of students before and after the implementation of the intervention. In addition, feedback questionnaires were collected from the students. RESULTS There was a significant improvement in students' scores between the first IA (mean ± standard deviation [SD], 8.16 ± 3.08) and second IA (mean ± SD, 17.64 ± 2.02) (P < 0.0001). According to students' feedback, 149 out of 150 (99.3%) students found the use of Google Form-based MCQ tests as a formative assessment tool in the teaching-learning process to be highly beneficial and motivated them to engage in their biochemistry studies. CONCLUSION With the shift toward competency-based medical education (CBME) in India, it is crucial for educators to embrace novel teaching-learning and evaluation approaches. Our study highlighted the efficacy of employing Google Form-based MCQ tests in enhancing students' comprehension of the biochemistry subject, evaluating their scores and improving the overall quality of learning. Through this mode of assessment, teachers were able to provide targeted feedback on areas that required improvement, thereby enhancing the learning experience.
Collapse
Affiliation(s)
- Sanjyoti Panchbudhe
- Department of Biochemistry, Shrimati Kashibai Navale Medical College and General Hospital, Narhe, Pune, Maharashtra, India
| | - Simran Shaikh
- Department of Biochemistry, Shrimati Kashibai Navale Medical College and General Hospital, Narhe, Pune, Maharashtra, India
| | - Hanmant Swami
- Department of Biochemistry, Shrimati Kashibai Navale Medical College and General Hospital, Narhe, Pune, Maharashtra, India
| | - Charushila Y. Kadam
- Department of Biochemistry, RKDF Medical College Hospital and Research Centre, Bhopal, Madhya Pradesh, India
| | - Ramchandra Padalkar
- Department of Biochemistry, Shrimati Kashibai Navale Medical College and General Hospital, Narhe, Pune, Maharashtra, India
| | - Rajni R. Shivkar
- Department of Biochemistry, Shrimati Kashibai Navale Medical College and General Hospital, Narhe, Pune, Maharashtra, India
| | - Gouri Gulavani
- Department of Biochemistry, Shrimati Kashibai Navale Medical College and General Hospital, Narhe, Pune, Maharashtra, India
| | - Supriya Gulajkar
- Department of Biochemistry, Shrimati Kashibai Navale Medical College and General Hospital, Narhe, Pune, Maharashtra, India
| | - Shubhangi Gawade
- Department of Biochemistry, Shrimati Kashibai Navale Medical College and General Hospital, Narhe, Pune, Maharashtra, India
| | - Farheen Mujawar
- Department of Biochemistry, Shrimati Kashibai Navale Medical College and General Hospital, Narhe, Pune, Maharashtra, India
| |
Collapse
|
4
|
Brenner JM, Fulton TB, Kruidering M, Bird JB, Willey J, Qua K, Olvet DM. What have we learned about constructed response short-answer questions from students and faculty? A multi-institutional study. MEDICAL TEACHER 2024; 46:349-358. [PMID: 37688773 DOI: 10.1080/0142159x.2023.2249209] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/11/2023]
Abstract
PURPOSE The purpose of this study was to enrich understanding about the perceived benefits and drawbacks of constructed response short-answer questions (CR-SAQs) in preclerkship assessment using Norcini's criteria for good assessment as a framework. METHODS This multi-institutional study surveyed students and faculty at three institutions. A survey using Likert scale and open-ended questions was developed to evaluate faculty and student perceptions of CR-SAQs using the criteria of good assessment to determine the benefits and drawbacks. Descriptive statistics and Chi-square analyses are presented, and open responses were analyzed using directed content analysis to describe benefits and drawbacks of CR-SAQs. RESULTS A total of 260 students (19%) and 57 faculty (48%) completed the survey. Students and faculty report that the benefits of CR-SAQs are authenticity, deeper learning (educational effect), and receiving feedback (catalytic effect). Drawbacks included feasibility, construct validity, and scoring reproducibility. Students and faculty found CR-SAQs to be both acceptable (can show your reasoning, partial credit) and unacceptable (stressful, not USMLE format). CONCLUSIONS CR-SAQs are a method of aligning innovative curricula with assessment and could enrich the assessment toolkit for medical educators.
Collapse
Affiliation(s)
- Judith M Brenner
- Department of Science Education, Donald and Barbara Zucker School of Medicine, Hofstra/Northwell, Hempstead, New York, USA
| | - Tracy B Fulton
- Department of Biochemistry and Biophysics, University of California San Francisco, San Francisco, California, USA
| | - Marieke Kruidering
- Department of Cellular and Molecular Pharmacology, University of California San Francisco,San Francisco, California, USA
| | - Jeffrey B Bird
- Department of Science Education, Donald and Barbara Zucker School of Medicine, Hofstra/Northwell, Hempstead, New York, USA
| | - Joanne Willey
- Department of Science Education, Donald and Barbara Zucker School of Medicine, Hofstra/Northwell, Hempstead, New York, USA
| | - Kelli Qua
- Center for Medical Education, Case Western Reserve University School of Medicine, Cleveland, Ohio, USA
| | - Doreen M Olvet
- Department of Science Education, Donald and Barbara Zucker School of Medicine, Hofstra/Northwell, Hempstead, New York, USA
| |
Collapse
|
5
|
Shikino K, Nishizaki Y, Fukui S, Yokokawa D, Yamamoto Y, Kobayashi H, Shimizu T, Tokuda Y. Development of a Clinical Simulation Video to Evaluate Multiple Domains of Clinical Competence: Cross-Sectional Study. JMIR MEDICAL EDUCATION 2024; 10:e54401. [PMID: 38421691 PMCID: PMC10940988 DOI: 10.2196/54401] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/08/2023] [Revised: 12/14/2023] [Accepted: 01/28/2024] [Indexed: 03/02/2024]
Abstract
BACKGROUND Medical students in Japan undergo a 2-year postgraduate residency program to acquire clinical knowledge and general medical skills. The General Medicine In-Training Examination (GM-ITE) assesses postgraduate residents' clinical knowledge. A clinical simulation video (CSV) may assess learners' interpersonal abilities. OBJECTIVE This study aimed to evaluate the relationship between GM-ITE scores and resident physicians' diagnostic skills by having them watch a CSV and to explore resident physicians' perceptions of the CSV's realism, educational value, and impact on their motivation to learn. METHODS The participants included 56 postgraduate medical residents who took the GM-ITE between January 21 and January 28, 2021; watched the CSV; and then provided a diagnosis. The CSV and GM-ITE scores were compared, and the validity of the simulations was examined using discrimination indices, wherein ≥0.20 indicated high discriminatory power and >0.40 indicated a very good measure of the subject's qualifications. Additionally, we administered an anonymous questionnaire to ascertain participants' views on the realism and educational value of the CSV and its impact on their motivation to learn. RESULTS Of the 56 participants, 6 (11%) provided the correct diagnosis, and all were from the second postgraduate year. All domains indicated high discriminatory power. The (anonymous) follow-up responses indicated that the CSV format was more suitable than the conventional GM-ITE for assessing clinical competence. The anonymous survey revealed that 12 (52%) participants found the CSV format more suitable than the GM-ITE for assessing clinical competence, 18 (78%) affirmed the realism of the video simulation, and 17 (74%) indicated that the experience increased their motivation to learn. CONCLUSIONS The findings indicated that CSV modules simulating real-world clinical examinations were successful in assessing examinees' clinical competence across multiple domains. The study demonstrated that the CSV not only augmented the assessment of diagnostic skills but also positively impacted learners' motivation, suggesting a multifaceted role for simulation in medical education.
Collapse
Affiliation(s)
- Kiyoshi Shikino
- Department of Community-Oriented Medical Education, Chiba University Graduate School of Medicine, Chiba, Japan
- Department of General Medicine, Chiba University Hospital, Chiba, Japan
| | - Yuji Nishizaki
- Division of Medical Education, Juntendo University School of Medicine, Tokyo, Japan
| | - Sho Fukui
- Department of Emergency and General Medicine, Kyorin University, Tokyo, Japan
| | - Daiki Yokokawa
- Department of General Medicine, Chiba University Hospital, Chiba, Japan
| | - Yu Yamamoto
- Division of General Medicine, Center for Community Medicine, Jichi Medical University, Tochigi, Japan
| | - Hiroyuki Kobayashi
- Department of Internal Medicine, Mito Kyodo General Hospital, Tsukuba, Japan
| | - Taro Shimizu
- Department of Diagnostic and Generalist Medicine, Dokkyo Medical University Hospital, Tochigi, Japan
| | - Yasuharu Tokuda
- Muribushi Okinawa Center for Teaching Hospitals, Okinawa, Japan
- Tokyo Foundation for Policy Research, Tokyo, Japan
| |
Collapse
|
6
|
Rath A. Back to basics: reflective take on role of MCQs in undergraduate Malaysian dental professional qualifying exams. Front Med (Lausanne) 2023; 10:1287924. [PMID: 38098841 PMCID: PMC10719850 DOI: 10.3389/fmed.2023.1287924] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/03/2023] [Accepted: 11/06/2023] [Indexed: 12/17/2023] Open
Affiliation(s)
- Avita Rath
- Faculty of Dentistry, SEGi University, Petaling Jaya, Selangor, Malaysia
- Edinburgh Medical School- Clinical Education, University of Edinburgh, Edinburgh, United Kingdom
| |
Collapse
|
7
|
Jabbour Z. A Classroom Practice and Assessment Resource for Dental Clinical Treatment Planning. MEDEDPORTAL : THE JOURNAL OF TEACHING AND LEARNING RESOURCES 2023; 19:11353. [PMID: 37808364 PMCID: PMC10556203 DOI: 10.15766/mep_2374-8265.11353] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/20/2023] [Accepted: 07/18/2023] [Indexed: 10/10/2023]
Abstract
Introduction Dental students' opportunities to practice clinical treatment planning prior to the clinical experience are often limited. This resource permits students to practice treatment planning in the classroom and allows the instructor to efficiently provide feedback to a large number of students. Methods I developed a computer-based treatment plan worksheet (Microsoft Excel, Google Sheets, or Qualtrics) consisting of four sequence steps and a list of Common Dental Terminology (CDT) codes. I implemented this educational activity in the Case Management and Treatment Planning course (second year, spring quarter). The course included 10 weekly sessions of 1 hour each. I gave students a practice case at the beginning of the course. They submitted comprehensive sequenced treatment plans as assignments using the Qualtrics treatment plan worksheet 2 days after the interdisciplinary treatment planning didactic session. In the subsequent debrief session, I discussed the case with the entire class and focused on themes observed from the students' performance on the assignments. Results I graded all students' submissions (n = 87) using CDT codes both with and without consideration of sequence steps and observed variations in students' performance. Many students (n = 36, 41%) opted to give feedback through an anonymous survey, with 88%-96% of them agreeing or strongly agreeing that the educational activity allowed them to practice, demonstrate their knowledge, and improve their skills in treatment planning. Discussion This computer-based worksheet lets students practice clinical treatment planning in the classroom and allows the instructor to effectively assess a large number of students' assignments.
Collapse
Affiliation(s)
- Zaher Jabbour
- Assistant Clinical Professor, Interdisciplinary and Restorative Dentistry, University of California, Los Angeles, School of Dentistry
| |
Collapse
|
8
|
Olvet DM, Bird JB, Fulton TB, Kruidering M, Papp KK, Qua K, Willey JM, Brenner JM. A Multi-institutional Study of the Feasibility and Reliability of the Implementation of Constructed Response Exam Questions. TEACHING AND LEARNING IN MEDICINE 2023; 35:609-622. [PMID: 35989668 DOI: 10.1080/10401334.2022.2111571] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/14/2021] [Accepted: 07/27/2022] [Indexed: 06/15/2023]
Abstract
PROBLEM Some medical schools have incorporated constructed response short answer questions (CR-SAQs) into their assessment toolkits. Although CR-SAQs carry benefits for medical students and educators, the faculty perception that the amount of time required to create and score CR-SAQs is not feasible and concerns about reliable scoring may impede the use of this assessment type in medical education. INTERVENTION Three US medical schools collaborated to write and score CR-SAQs based on a single vignette. Study participants included faculty question writers (N = 5) and three groups of scorers: faculty content experts (N = 7), faculty non-content experts (N = 6), and fourth-year medical students (N = 7). Structured interviews were performed with question writers and an online survey was administered to scorers to gather information about their process for creating and scoring CR-SAQs. A content analysis was performed on the qualitative data using Bowen's model of feasibility as a framework. To examine inter-rater reliability between the content expert and other scorers, a random selection of fifty student responses from each site were scored by each site's faculty content experts, faculty non-content experts, and student scorers. A holistic rubric (6-point Likert scale) was used by two schools and an analytic rubric (3-4 point checklist) was used by one school. Cohen's weighted kappa (κw) was used to evaluate inter-rater reliability. CONTEXT This research study was implemented at three US medical schools that are nationally dispersed and have been administering CR-SAQ summative exams as part of their programs of assessment for at least five years. The study exam question was included in an end-of-course summative exam during the first year of medical school. IMPACT Five question writers (100%) participated in the interviews and twelve scorers (60% response rate) completed the survey. Qualitative comments revealed three aspects of feasibility: practicality (time, institutional culture, teamwork), implementation (steps in the question writing and scoring process), and adaptation (feedback, rubric adjustment, continuous quality improvement). The scorers' described their experience in terms of the need for outside resources, concern about lack of expertise, and value gained through scoring. Inter-rater reliability between the faculty content expert and student scorers was fair/moderate (κw=.34-.53, holistic rubrics) or substantial (κw=.67-.76, analytic rubric), but much lower between faculty content and non-content experts (κw=.18-.29, holistic rubrics; κw=.59-.66, analytic rubric). LESSONS LEARNED Our findings show that from the faculty perspective it is feasible to include CR-SAQs in summative exams and we provide practical information for medical educators creating and scoring CR-SAQs. We also learned that CR-SAQs can be reliably scored by faculty without content expertise or senior medical students using an analytic rubric, or by senior medical students using a holistic rubric, which provides options to alleviate the faculty burden associated with grading CR-SAQs.
Collapse
Affiliation(s)
- Doreen M Olvet
- Department of Science Education, Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, Hempstead, New York, USA
| | - Jeffrey B Bird
- Department of Science Education, Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, Hempstead, New York, USA
| | - Tracy B Fulton
- Department of Biochemistry and Biophysics, University of California San Francisco School of Medicine, San Francisco, California, USA
| | - Marieke Kruidering
- Department of Cellular & Molecular Pharmacology, University of California at San Francisco School of Medicine, San Francisco, California, USA
| | - Klara K Papp
- Center for Medical Education, Case Western Reserve University School of Medicine, Cleveland, Ohio, USA
| | - Kelli Qua
- Research and Evaluation, Case Western Reserve University School of Medicine, Cleveland, Ohio, USA
| | - Joanne M Willey
- Department of Science Education, Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, Hempstead, New York, USA
| | - Judith M Brenner
- Department of Science Education, Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, Hempstead, New York, USA
| |
Collapse
|
9
|
Stoehr F, Kämpgen B, Müller L, Zufiría LO, Junquero V, Merino C, Mildenberger P, Kloeckner R. Natural language processing for automatic evaluation of free-text answers - a feasibility study based on the European Diploma in Radiology examination. Insights Imaging 2023; 14:150. [PMID: 37726485 PMCID: PMC10509084 DOI: 10.1186/s13244-023-01507-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2023] [Accepted: 08/18/2023] [Indexed: 09/21/2023] Open
Abstract
BACKGROUND Written medical examinations consist of multiple-choice questions and/or free-text answers. The latter require manual evaluation and rating, which is time-consuming and potentially error-prone. We tested whether natural language processing (NLP) can be used to automatically analyze free-text answers to support the review process. METHODS The European Board of Radiology of the European Society of Radiology provided representative datasets comprising sample questions, answer keys, participant answers, and reviewer markings from European Diploma in Radiology examinations. Three free-text questions with the highest number of corresponding answers were selected: Questions 1 and 2 were "unstructured" and required a typical free-text answer whereas question 3 was "structured" and offered a selection of predefined wordings/phrases for participants to use in their free-text answer. The NLP engine was designed using word lists, rule-based synonyms, and decision tree learning based on the answer keys and its performance tested against the gold standard of reviewer markings. RESULTS After implementing the NLP approach in Python, F1 scores were calculated as a measure of NLP performance: 0.26 (unstructured question 1, n = 96), 0.33 (unstructured question 2, n = 327), and 0.5 (more structured question, n = 111). The respective precision/recall values were 0.26/0.27, 0.4/0.32, and 0.62/0.55. CONCLUSION This study showed the successful design of an NLP-based approach for automatic evaluation of free-text answers in the EDiR examination. Thus, as a future field of application, NLP could work as a decision-support system for reviewers and support the design of examinations being adjusted to the requirements of an automated, NLP-based review process. CLINICAL RELEVANCE STATEMENT Natural language processing can be successfully used to automatically evaluate free-text answers, performing better with more structured question-answer formats. Furthermore, this study provides a baseline for further work applying, e.g., more elaborated NLP approaches/large language models. KEY POINTS • Free-text answers require manual evaluation, which is time-consuming and potentially error-prone. • We developed a simple NLP-based approach - requiring only minimal effort/modeling - to automatically analyze and mark free-text answers. • Our NLP engine has the potential to support the manual evaluation process. • NLP performance is better on a more structured question-answer format.
Collapse
Affiliation(s)
- Fabian Stoehr
- Department of Diagnostic and Interventional Radiology, University Medical Center, Johannes Gutenberg-University Mainz, Langenbeckst, 1, 55131, Mainz, Germany
| | - Benedikt Kämpgen
- Empolis Information Management GmbH, Leightonstraße 2, 97074, Würzburg, Germany
| | - Lukas Müller
- Department of Diagnostic and Interventional Radiology, University Medical Center, Johannes Gutenberg-University Mainz, Langenbeckst, 1, 55131, Mainz, Germany
| | - Laura Oleaga Zufiría
- Department of Radiology, Hospital Clínic de Barcelona, C. de Villarroel, 170, 08036, Barcelona, Spain
| | | | | | - Peter Mildenberger
- Department of Diagnostic and Interventional Radiology, University Medical Center, Johannes Gutenberg-University Mainz, Langenbeckst, 1, 55131, Mainz, Germany
| | - Roman Kloeckner
- Institute of Interventional Radiology, University Hospital Schleswig-Holstein, Campus Luebeck, Ratzeburger Allee 160, 23583, Luebeck, Germany.
| |
Collapse
|
10
|
Bassey R, Hill RV, Nassrallah Z, Knutson S, Pinard B, Olvet DM, Rennie WP. Consolidation Carnival: A Case-Based Approach to Reviewing Musculoskeletal Anatomy in an Undergraduate Medical Curriculum. ADVANCES IN MEDICAL EDUCATION AND PRACTICE 2023; 14:889-897. [PMID: 37592958 PMCID: PMC10427757 DOI: 10.2147/amep.s409797] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/24/2023] [Accepted: 07/29/2023] [Indexed: 08/19/2023]
Abstract
Introduction Recent changes in undergraduate medical curricula have resulted in time constraints that are particularly challenging, especially when students must learn large blocks of musculoskeletal anatomy content. Consequently, students have resorted to rote memorization to cope, which counteracts our established collaborative self-directed learning model. Methods For a 6-week musculoskeletal anatomy course, two structured case-based review sessions are described, each following the completion of two five-hour lab sessions, two on the upper extremities and two on the lower extremities. These largely self-directed review sessions consisted of 6 students rotating through 7 to 8 stations every 10 minutes where clinical cases with follow-up questions were projected on large screens. The students were expected to work collaboratively to solve the cases utilizing the prosected specimens provided and discuss the accompanying answers at the end of each case. Results Ninety-four per cent of the students who participated in this study agreed that the case-based review sessions provided a helpful overview of musculoskeletal anatomy content. Student performance on the open-ended, case-based musculoskeletal examination questions showed no significant difference in performance on shoulder, hand, hip, thigh, and leg questions. There was, however, a statistically significant decrease in the students' scores on a forearm question in 2021 compared to 2019. Conclusion This paper describes our integrated, collaborative musculoskeletal course, including case-based review sessions, which was positively received by students as having value in reviewing the musculoskeletal content though it was not found to improve examination performance.
Collapse
Affiliation(s)
- Rosemary Bassey
- Department of Science Education, Donald and Barbara Zucker School of Medicine at Hofstra/ Northwell, Hempstead, NY, USA
| | - Robert V Hill
- Department of Science Education, Donald and Barbara Zucker School of Medicine at Hofstra/ Northwell, Hempstead, NY, USA
| | - Zeinab Nassrallah
- Department of Science Education, Donald and Barbara Zucker School of Medicine at Hofstra/ Northwell, Hempstead, NY, USA
| | - Shannon Knutson
- Department of Science Education, Donald and Barbara Zucker School of Medicine at Hofstra/ Northwell, Hempstead, NY, USA
| | - Brian Pinard
- Department of Science Education, Donald and Barbara Zucker School of Medicine at Hofstra/ Northwell, Hempstead, NY, USA
| | - Doreen M Olvet
- Department of Science Education, Donald and Barbara Zucker School of Medicine at Hofstra/ Northwell, Hempstead, NY, USA
| | - William P Rennie
- Department of Science Education, Donald and Barbara Zucker School of Medicine at Hofstra/ Northwell, Hempstead, NY, USA
| |
Collapse
|
11
|
Bassey RB, Hill RV, Rennie WP. Integration of physiology in a curriculum on human structure: a snapshot of the cardiovascular block. Front Physiol 2023; 14:1236409. [PMID: 37520828 PMCID: PMC10375018 DOI: 10.3389/fphys.2023.1236409] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/07/2023] [Accepted: 07/07/2023] [Indexed: 08/01/2023] Open
Abstract
With the gradual shift from discipline-based to competency-based medical education, the integrated curriculum has become a popular model for connecting basic science and clinical content in undergraduate medical education. Despite its popularity, there are concerns that important physiological concepts are not adequately addressed. We describe the spiral integration of physiology content in the 5-week Cardiovascular block of our Homeostasis course at the Zucker School of Medicine. We also describe our approach to incorporating physiology into an integrated, constructed response, short-answer assessment format. Our approach to spiral integration consists of rotating lab stations that highlight the distinction between normal and abnormal states, linked with appropriate clinical interventions. Physiology is at the core of integration in any curriculum and the basis of all applied fields of medicine, hence our approach is that teaching structural relationships would not be valuable without consideration of its functions, which can then be utilized in discussion of clinical presentations, imaging, and relevant pathologies. Likewise, our integrated assessments require the students to compose their answers to the questions from scratch, which creates a shift in mode of students' preparation from rote memorizations to more cognitive processing that enhances critical thinking.
Collapse
|
12
|
Woods C, Naroo S, Zeri F, Bakkar M, Barodawala F, Evans V, Fadel D, Kalikivayi L, Lira M, Maseedupally V, Huarte ST, Eperjesi F. Evidence for commonly used teaching, learning and assessment methods in contact lens clinical skills education. Cont Lens Anterior Eye 2023; 46:101821. [PMID: 36805277 DOI: 10.1016/j.clae.2023.101821] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2023] [Accepted: 02/08/2023] [Indexed: 02/18/2023]
Abstract
INTRODUCTION Evidence based practice is now an important part of healthcare education. The aim of this narrative literature review was to determine what evidence exists on the efficacy of commonly used teaching and learning and assessment methods in the realm of contact lens skills education (CLE) in order to provide insights into best practice. A summary of the global regulation and provision of postgraduate learning and continuing professional development in CLE is included. METHOD An expert panel of educators was recruited and completed a literature review of current evidence of teaching and learning and assessment methods in healthcare training, with an emphasis on health care, general optometry and CLE. RESULTS No direct evidence of benefit of teaching and learning and assessment methods in CLE were found. There was evidence for the benefit of some teaching and learning and assessment methods in other disciplines that could be transferable to CLE and could help students meet the intended learning outcomes. There was evidence that the following teaching and learning methods helped health-care and general optometry students meet the intended learning outcomes; clinical teaching and learning, flipped classrooms, clinical skills videos and clerkships. For assessment these methods were; essays, case presentations, objective structured clinical examinations, self-assessment and formative assessment. There was no evidence that the following teaching and learning methods helped health-care and general optometry students meet the intended learning outcomes; journal clubs and case discussions. Nor was any evidence found for the following assessment methods; multiple-choice questions, oral examinations, objective structured practical examinations, holistic assessment, and summative assessment. CONCLUSION Investigation into the efficacy of common teaching and learning and assessment methods in CLE are required and would be beneficial for the entire community of contact lens educators, and other disciplines that wish to adapt this approach of evidence-based teaching.
Collapse
Affiliation(s)
- Craig Woods
- School of Optometry and Vision Science, University of New South Wales, Australia; International Association of Contact Lens Educators, Canada
| | - Shehzad Naroo
- College of Health and Life Sciences, Aston University, UK; International Association of Contact Lens Educators, Canada
| | - Fabrizio Zeri
- College of Health and Life Sciences, Aston University, UK; University of Milano-Bicocca, Department of Materials Science, Milan, Italy; International Association of Contact Lens Educators, Canada
| | - May Bakkar
- Faculty of Applied Medical Sciences, Jordan University of Science and Technology, Jordan
| | - Fakhruddin Barodawala
- Faculty of Optometry and Vision Sciences, SEGi University, Malaysia; International Association of Contact Lens Educators, Canada
| | - Vicki Evans
- Faculty of Health, University of Canberra, Australia; International Association of Contact Lens Educators, Canada
| | - Daddi Fadel
- Center for Ocular Research & Education (CORE), School of Optometry & Vision Science, University of Waterloo, Waterloo, Canada
| | | | - Madalena Lira
- Physics Center of Minho and Porto Universities (CF-UM-UP), School of Sciences, University of Minho, Portugal; International Association of Contact Lens Educators, Canada
| | - Vinod Maseedupally
- School of Optometry and Vision Science, University of New South Wales, Australia
| | | | | |
Collapse
|
13
|
Lau KY, Ang JYH, Rajalingam P. Very Short Answer Questions in Team-Based Learning: Limited Effect on Peer Elaboration and Memory. MEDICAL SCIENCE EDUCATOR 2023; 33:139-145. [PMID: 36569367 PMCID: PMC9765373 DOI: 10.1007/s40670-022-01716-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Accepted: 12/06/2022] [Indexed: 06/17/2023]
Abstract
INTRODUCTION In team-based learning (TBL), single best answer questions (SBAQs) are traditionally used because immediate computer-assisted feedback facilitates team discussions. Recent improvements in digital marking systems and criticisms of non-analytical strategies in SBAQs have prompted the consideration of very short answer questions (VSAQs) as an alternative to SBAQs. We aim to compare the effect of VSAQs and SBAQs on peer elaboration and knowledge retention in TBL. MATERIALS AND METHODS Twenty-four second-year students from the Lee Kong Chian School of Medicine participated in a mixed-methods study that included a randomised controlled crossover trial with two intervention arms (TBL-VSAQs and TBL-SBAQs). Two TBL sessions were conducted, with one topic covered each. Students were randomly allocated into six teams of four members. Individual and team scores and completion times were measured, and students were surveyed on their TBL experience. A follow-up quiz on the same topics was administered two weeks later to assess knowledge retention. RESULTS Individuals scored lower for VSAQs than SBAQs in the second TBL topic (7.17 ± 1.52 versus 8.25 ± 1.48; p = 0.046), while findings in other metrics were non-significant. Follow-up quiz scores showed no significant difference in knowledge retention, although effect size and power were low. Students perceived VSAQs as more authentic and challenging, though most preferred the continued use of SBAQs for TBL. DISCUSSION VSAQs have a limited impact on peer elaboration and knowledge retention versus SBAQs in TBL. CONCLUSIONS We should be circumspect about implementing VSAQs in TBL especially for medical students at the pre-clinical level. SUPPLEMENTARY INFORMATION The online version contains supplementary material available at 10.1007/s40670-022-01716-5.
Collapse
Affiliation(s)
- Kai Yuan Lau
- Medical Education Research and Scholarship Unit (MERSU), Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore, Singapore
| | - Jessica Yang Huey Ang
- Medical Education Research and Scholarship Unit (MERSU), Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore, Singapore
| | - Preman Rajalingam
- Medical Education Research and Scholarship Unit (MERSU), Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore, Singapore
| |
Collapse
|
14
|
Lazarus MD, Gouda‐Vossos A, Ziebell A, Brand G. Fostering uncertainty tolerance in anatomy education: Lessons learned from how humanities, arts and social science (HASS) educators develop learners' uncertainty tolerance. ANATOMICAL SCIENCES EDUCATION 2023; 16:128-147. [PMID: 35114066 PMCID: PMC10078696 DOI: 10.1002/ase.2174] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 09/15/2021] [Revised: 01/24/2022] [Accepted: 01/28/2022] [Indexed: 06/14/2023]
Abstract
Uncertainty tolerance, individuals' perceptions/responses to uncertain stimuli, is increasingly recognized as critical to effective healthcare practice. While the Covid-19 pandemic generated collective uncertainty, healthcare-related uncertainty is omnipresent. Correspondingly, there is increasing focus on uncertainty tolerance as a health professional graduate "competency," and a concomitant interest in identifying pedagogy fostering learners' uncertainty tolerance. Despite these calls, practical guidelines for educators are lacking. There is some initial evidence that anatomy education can foster medical students' uncertainty tolerance (e.g., anatomical variation and dissection novelty), however, there remains a knowledge gap regarding robust curriculum-wide uncertainty tolerance teaching strategies. Drawing upon humanities, arts and social sciences (HASS) educators' established uncertainty tolerance pedagogies, this study sought to learn from HASS academics' experiences with, and teaching practices related to, uncertainty pedagogy using a qualitative, exploratory study design. Framework analysis was undertaken using an abductive approach, wherein researchers oscillate between inductive and deductive coding (comparing to the uncertainty tolerance conceptual model). During this analysis, the authors analyzed ~386 min of data from purposively sampled HASS academics' (n = 14) discussions to address the following research questions: (1) What teaching practices do HASS academics' perceive as impacting learners' uncertainty tolerance, and (2) How do HASS academics execute these teaching practices? The results extend current understanding of the moderating effects of education on uncertainty tolerance and supports prior findings that the anatomy learning environment is ripe for supporting learner uncertainty tolerance development. This study adds to growing literature on the powerful moderating effect education has on uncertainty tolerance and proposes translation of HASS uncertainty tolerance teaching practices to enhance anatomy education.
Collapse
Affiliation(s)
- Michelle D. Lazarus
- Centre for Human Anatomy EducationDepartment of Anatomy and Developmental BiologyFaculty of Medicine, Nursing and Health SciencesMonash UniversityClaytonVictoriaAustralia
- Monash Centre for Scholarship in Health Education, Faculty of Medicine, Nursing and Health SciencesMonash UniversityClaytonVictoriaAustralia
| | - Amany Gouda‐Vossos
- Centre for Human Anatomy EducationDepartment of Anatomy and Developmental BiologyFaculty of Medicine, Nursing and Health SciencesMonash UniversityClaytonVictoriaAustralia
| | - Angela Ziebell
- School of Life and Environmental SciencesDeakin University Burwood CampusBurwoodVictoriaAustralia
| | - Gabrielle Brand
- Monash Centre for Scholarship in Health Education, Faculty of Medicine, Nursing and Health SciencesMonash UniversityClaytonVictoriaAustralia
- School of Nursing and MidwiferyFaculty of Medicine, Nursing and Health SciencesMonash UniversityFrankstonVictoriaAustralia
| |
Collapse
|
15
|
Morjaria L, Burns L, Bracken K, Ngo QN, Lee M, Levinson AJ, Smith J, Thompson P, Sibbald M. Examining the Threat of ChatGPT to the Validity of Short Answer Assessments in an Undergraduate Medical Program. JOURNAL OF MEDICAL EDUCATION AND CURRICULAR DEVELOPMENT 2023; 10:23821205231204178. [PMID: 37780034 PMCID: PMC10540597 DOI: 10.1177/23821205231204178] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 05/17/2023] [Accepted: 09/01/2023] [Indexed: 10/03/2023]
Abstract
OBJECTIVES ChatGPT is an artificial intelligence model that can interpret free-text prompts and return detailed, human-like responses across a wide domain of subjects. This study evaluated the extent of the threat posed by ChatGPT to the validity of short-answer assessment problems used to examine pre-clerkship medical students in our undergraduate medical education program. METHODS Forty problems used in prior student assessments were retrieved and stratified by levels of Bloom's Taxonomy. Thirty of these problems were submitted to ChatGPT-3.5. For the remaining 10 problems, we retrieved past minimally passing student responses. Six tutors graded each of the 40 responses. Comparison of performance between student-generated and ChatGPT-generated answers aggregated as a whole and grouped by Bloom's levels of cognitive reasoning, was done using t-tests, ANOVA, Cronbach's alpha, and Cohen's d. Scores for ChatGPT-generated responses were also compared to historical class average performance. RESULTS ChatGPT-generated responses received a mean score of 3.29 out of 5 (n = 30, 95% CI 2.93-3.65) compared to 2.38 for a group of students meeting minimum passing marks (n = 10, 95% CI 1.94-2.82), representing higher performance (P = .008, η2 = 0.169), but was outperformed by historical class average scores on the same 30 problems (mean 3.67, P = .018) when including all past responses regardless of student performance level. There was no statistically significant trend in performance across domains of Bloom's Taxonomy. CONCLUSION While ChatGPT was able to pass short answer assessment problems spanning the pre-clerkship curriculum, it outperformed only underperforming students. We remark that tutors in several cases were convinced that ChatGPT-produced responses were produced by students. Risks to assessment validity include uncertainty in identifying struggling students and inability to intervene in a timely manner. The performance of ChatGPT on problems requiring increasing demands of cognitive reasoning warrants further research.
Collapse
Affiliation(s)
- Leo Morjaria
- Michael G. DeGroote School of Medicine, McMaster University, Hamilton, Ontario, Canada
| | - Levi Burns
- Michael G. DeGroote School of Medicine, McMaster University, Hamilton, Ontario, Canada
| | - Keyna Bracken
- Michael G. DeGroote School of Medicine, McMaster University, Hamilton, Ontario, Canada
- McMaster Education Research, Innovation and Theory (MERIT) Program, McMaster University, Hamilton, Ontario, Canada
| | - Quang N. Ngo
- Michael G. DeGroote School of Medicine, McMaster University, Hamilton, Ontario, Canada
- McMaster Education Research, Innovation and Theory (MERIT) Program, McMaster University, Hamilton, Ontario, Canada
| | - Mark Lee
- McMaster Education Research, Innovation and Theory (MERIT) Program, McMaster University, Hamilton, Ontario, Canada
| | - Anthony J. Levinson
- Michael G. DeGroote School of Medicine, McMaster University, Hamilton, Ontario, Canada
| | - John Smith
- Michael G. DeGroote School of Medicine, McMaster University, Hamilton, Ontario, Canada
| | - Penelope Thompson
- Michael G. DeGroote School of Medicine, McMaster University, Hamilton, Ontario, Canada
| | - Matthew Sibbald
- Michael G. DeGroote School of Medicine, McMaster University, Hamilton, Ontario, Canada
- McMaster Education Research, Innovation and Theory (MERIT) Program, McMaster University, Hamilton, Ontario, Canada
| |
Collapse
|
16
|
Ziehfreund S, Reifenrath J, Wijnen-Meijer M, Welzel J, Sauter F, Wecker H, Biedermann T, Zink A. Considering medical students' perception, concerns and needs for e-exam during COVID-19: a promising approach to improve subject specific e-exams. MEDICAL EDUCATION ONLINE 2022; 27:2114131. [PMID: 35993348 PMCID: PMC9397442 DOI: 10.1080/10872981.2022.2114131] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 03/13/2022] [Revised: 06/19/2022] [Accepted: 08/11/2022] [Indexed: 06/15/2023]
Abstract
The COVID-19 pandemic forced a rapid shift to digital strategies including e-exams in medical schools. However, there are significant concerns, predominately from student perspectives, and further data is required to successfully establish e-assessment in the medical curricula. The objective of the study was to examine medical students' perceptions, concerns, and needs regarding e-assessment to establish a comprehensive e-exam based on these and previous findings and to evaluate its effectiveness in terms of examinee perceptions and further needs. During the 2021 summer term, a cross-sectional study using qualitative and quantitative methods was conducted among all 1077 students at the School of Medicine, Technical University of Munich. They were asked to provide information regarding their characteristics, preferred exam format, e-assessment perception, concerns, and needs in an online questionnaire. Based on these findings, a pilot e-exam including an e-exam preparation for the students were established and subsequently evaluated among 125 pilot e-exam examinees under study consideration via an online-questionnaire. Of the 317 pre-exam participants (73.2% female), 70.3% preferred in-person exams and showed concerns about the technological framework, privacy, and examination requirements. Qualitative analysis showed that these concerns lead to additional exam stress and fear of failure. The 34 (79.4% female) participants who participated in the evaluation survey showed a significantly more positive e-exam perception. The fairness of the platform, the independence from an internet connection, the organization including the e-exam preparation, and the consideration of participant needs were discussed as particularly positive in the open-ended comments. In both surveys, participants requested uniform platforms and processes for all subjects. This study provides evidence for a positive, complementary role of student participation in a successful e-exam implementation. Furthermore, when establishing an e-exam format in the medical curricula, e-exam training, equal accessibility, availability offline, and all-round fairness should be considered.
Collapse
Affiliation(s)
- Stefanie Ziehfreund
- Department of Dermatology and Allergy, Technical University of Munich, School of Medicine, Munich, Germany
- Institute of General Practice and Health Services Research, School of Medicine, Technical University of Munich, Munich, Germany
| | - Johannes Reifenrath
- School of Medicine, TUM Medical Education Center (TUM MEC), Technical University of Munich, Munich, Germany
| | - Marjo Wijnen-Meijer
- School of Medicine, TUM Medical Education Center (TUM MEC), Technical University of Munich, Munich, Germany
| | - Julia Welzel
- Department of Dermatology, University Hospital Augsburg, Augsburg, Germany
| | - Fabian Sauter
- Department of Dermatology and Allergy, Technical University of Munich, School of Medicine, Munich, Germany
- Department of Informatics, Technical University of Munich, Munich, Germany
| | - Hannah Wecker
- Department of Dermatology and Allergy, Technical University of Munich, School of Medicine, Munich, Germany
| | - Tilo Biedermann
- Department of Dermatology and Allergy, Technical University of Munich, School of Medicine, Munich, Germany
| | - Alexander Zink
- Department of Dermatology and Allergy, Technical University of Munich, School of Medicine, Munich, Germany
- Division of Dermatology and Venereology, Department of Medicine Solna, Karolinska Institute, Stockholm, Sweden
| |
Collapse
|
17
|
El-Dwairi QA, Mustafeh I, Khatatbih M, Malki MI, Mustafa AG. Medical education across three colleges of medicine: perspectives of medical students. Heliyon 2022; 8:e11426. [PMID: 36387468 PMCID: PMC9647486 DOI: 10.1016/j.heliyon.2022.e11426] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/11/2022] [Revised: 09/26/2022] [Accepted: 10/31/2022] [Indexed: 11/08/2022] Open
Abstract
Aim This study aimed to explore and evaluate various components of the medical education process (lectures, labs, small-group discussions, clinical rotations, and undergraduate research) in three colleges of medicine in Jordan. Methods This cross-sectional questionnaire-based study included 849 undergraduate students from three main medical colleges in Jordan. Statically valid responses were considered for 684 students. The participants were from Jordan University of Science and Technology, Yarmouk University, and the University of Jordan. Results The distribution of students according to their admission status was 276 (40%) regular, 266 (38.9%) parallel, and 142 (20.8%) international programs. Personal interest and self-initiation were the major motives for studying medicine in 66.1%. Regarding the frequency of attending classes, University of Jordan students reported the highest rate of regular classes' attendance (93%). The study also reported that lecture notes and textbooks were the main sources of learning for medical students. The study also reported superior academic performance of students in the regular program compared to students in the parallel and international programs. Participants of the study criticized the medical curricula in the three colleges mentioned above because of the lack of active research programs. Most of the students (40%–56%) also complained that the lectures within the modules were not well-integrated, and they felt that the academic environment was moderate (48–59%). In addition, most students in the clinical phase complained of overcrowding in hospital wards during clinical rotation. Conclusions Based on students' feedback, multiple aspects of the medical education process require substantial reform to meet the expectations of medical students in Jordan.
Collapse
|
18
|
Nguyentan DC, Gruenberg K, Shin J. Should multiple-choice questions get the SAQ? Development of a short-answer question writing rubric. CURRENTS IN PHARMACY TEACHING & LEARNING 2022; 14:591-596. [PMID: 35715099 DOI: 10.1016/j.cptl.2022.04.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/25/2021] [Revised: 01/17/2022] [Accepted: 04/27/2022] [Indexed: 06/15/2023]
Abstract
INTRODUCTION Short-answer questions (SAQs) are often used to assess pharmacy student competency. However, the literature lacks guidance on SAQ development strategies, resulting in varying practices between SAQ writers. Understanding student and faculty perceptions of what constitutes a high-quality SAQ can identify best practices for SAQ development. METHODS We surveyed second-year pharmacy students at the University of California San Francisco (UCSF) to assess their perceptions of SAQs. Likert-type data were descriptively analyzed, and open-ended responses were analyzed using thematic analysis; we used these results to draft an initial SAQ checklist. We then conducted focus groups of UCSF pharmacy faculty to explore their experiences writing SAQs. Transcripts were analyzed using the survey codebook and de novo codes to generate themes. We used the focus group findings to finalize the checklist. RESULTS Seventy-five students (82%) completed the survey. Students identified "structure" (organizing into sections/lists) and "content" (clearly delineating student's task) as two ways to improve SAQ quality. Eight faculty participated in focus groups of two to three participants each. Faculty expanded on these previous themes and also identified a new theme, "process." This included peer review of SAQs as well as the iterative process involved in writing the SAQ, model answer, and grading rubric. CONCLUSIONS Content, structure, and process were the three areas identified for the improvement of SAQ quality at our institution. A checklist outlining best practices in these areas may be best implemented and adopted within the SAQ peer-review process.
Collapse
Affiliation(s)
- Ducanhhoa-Crystal Nguyentan
- Pharmacy Practice Resident, Department of Pharmacy, School of Pharmacy, University of Washington, 1959 NE Pacific Street, Seattle, WA 98195-7630, United States.
| | - Katherine Gruenberg
- Department of Clinical Pharmacy, School of Pharmacy, University of California, San Francisco, 521 Parnassus Avenue, Floor 3, San Francisco, CA 94143-0622, United States.
| | - Jaekyu Shin
- Department of Clinical Pharmacy, School of Pharmacy, University of California, San Francisco, 521 Parnassus Avenue, Floor 3, San Francisco, CA 94143-0622, United States.
| |
Collapse
|
19
|
Vegi VAK, Sudhakar PV, Bhimarasetty DM, Pamarthi K, Edara L, Kutikuppala LVS, Suvvari TK, Anand S. Multiple-choice questions in assessment: Perceptions of medical students from low-resource setting. JOURNAL OF EDUCATION AND HEALTH PROMOTION 2022; 11:103. [PMID: 35573621 PMCID: PMC9093664 DOI: 10.4103/jehp.jehp_621_21] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/03/2021] [Accepted: 08/11/2021] [Indexed: 06/15/2023]
Abstract
BACKGROUND In curriculum reforms under Competency-Based Medical Education (CBME) introduced by the National Medical Commission, medical students are recognized as one of the stake holders. The well-designed multiple-choice questions (MCQs) help students in assessment as well as promote deep learning. The objective of this study is to assess the perceptions of the Indian medical students towards the examination patterns in their undergraduate curriculum with particular reference to MCQs. MATERIALS AND METHODS An observational cross-sectional study was conducted among the medical colleges in India. A total of 7875 students took part in the study. The study instrument was a validated pretested self-administered questionnaire. The study was conducted as an online survey using the Google Forms platform. Inclusion criteria were students from all professional years as well as interns and those who were willing to participate in the study. RESULTS For the question of initiation of MCQs as a tool for the assessment, many of the students (46%) expressed that it should start from the 1st MBBS itself. The majority of the students (85%) felt that MCQ oriented preparation will be helpful for MBBS professional (theory and practical) examinations. Most of the students (90.9%) preferred that the MCQs should be concept and case scenario based. To the question of whether the current system of the university theory examinations of essays and short notes be replaced in total with MCQs so that they are more prepared for NEXT and USMLE examination, 65.5% responded with an answer of yes. CONCLUSION The findings of this study provide input to the regulatory authorities and other stakeholders responsible for designing and implementing the medical curriculum.
Collapse
Affiliation(s)
| | - P. V. Sudhakar
- Department of Plastic Surgery, Principal, Andhra Medical College, Visakhapatnam, Andhra Pradesh, India
| | | | - Kiran Pamarthi
- Department of Community Medicine, Andhra Medical College, Visakhapatnam, Andhra Pradesh, India
| | - Lokesh Edara
- Department of Medicine, Western Michigan School of Medicine, Kalamazoo, MI, USA
| | - L. V. Simhachalam Kutikuppala
- Department of General Medicine, Konaseema Institute of Medical Sciences and Research Foundation, Amalapuram, Andhra Pradesh, India
| | - Tarun Kumar Suvvari
- Department of General Medicine, Dr. NTR University of Health Sciences, Vijayawada, Andhra Pradesh, India
| | - Shubham Anand
- Department of General Medicine, GS Medical College, Pilkhuwa, Uttar Pradesh, India
| |
Collapse
|
20
|
Dalavaye N. A Response to "Assessing Medical Students' Knowledge of Genetics: Basis for Improving Genetics Curriculum for Future Clinical Practice" [Letter]. ADVANCES IN MEDICAL EDUCATION AND PRACTICE 2022; 13:121-122. [PMID: 35153526 PMCID: PMC8824752 DOI: 10.2147/amep.s356986] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Received: 01/04/2022] [Accepted: 01/27/2022] [Indexed: 06/14/2023]
|
21
|
Erturk S, van Tilburg WAP, Igou ER. Off the mark: Repetitive marking undermines essay evaluations due to boredom. MOTIVATION AND EMOTION 2022. [DOI: 10.1007/s11031-022-09929-2] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
Abstract
AbstractEssay-style assessment is widespread in education. Nonetheless, research shows that this tool can suffer from low reliability and validity. We attribute this problem partly to the boredom that marking multiple essays causes. Specifically, we propose that boredom in markers is associated with systematically lower marks on essays. To test this, we asked participants (N = 100) with an undergraduate degree to mark essays. The majority of these participants had at least some experience with marking. After marking each essay, participants indicated how bored they were. We found an increase in boredom over time and that higher boredom was associated with lower marks. Furthermore, offering a marking rubric did not prevent this problematic impact of boredom. These findings have implications for the validity of essays as an assessment tool and raise concerns about repetitive marking practices in general.
Collapse
|
22
|
Jabbour Z, Tran M. Creating an online tool for clinical treatment planning formative assessments. J Dent Educ 2021; 86 Suppl 1:792-794. [PMID: 34910299 DOI: 10.1002/jdd.12850] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/07/2021] [Revised: 10/31/2021] [Accepted: 11/20/2021] [Indexed: 11/12/2022]
Affiliation(s)
- Zaher Jabbour
- School of Dentistry, University of California, Los Angeles, California, USA
| | - Minh Tran
- Rossier School of Education, University of Southern California, Los Angeles, California, USA
| |
Collapse
|
23
|
Yaneva V, Clauser BE, Morales A, Paniagua M. Using Eye‐Tracking Data as Part of the Validity Argument for Multiple‐Choice Questions: A Demonstration. JOURNAL OF EDUCATIONAL MEASUREMENT 2021. [DOI: 10.1111/jedm.12304] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
|
24
|
Olvet DM, Bird JB, Fulton TB, Kruidering M, Papp KK, Qua K, Willey JM, Brenner JM. Can Content Experts Rely on Others to Reliably Score Open-Ended Questions on Summative Exams? ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2021; 96:S210. [PMID: 34705711 DOI: 10.1097/acm.0000000000004278] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Affiliation(s)
- Doreen M Olvet
- Author affiliations: D.M. Olvet, J.B. Bird, J.M. Willey, J.M. Brenner, Donald and Barbara Zucker School of Medicine at Hofstra/Northwell
| | - Jeffrey B Bird
- Author affiliations: D.M. Olvet, J.B. Bird, J.M. Willey, J.M. Brenner, Donald and Barbara Zucker School of Medicine at Hofstra/Northwell
| | - Tracy B Fulton
- T.B. Fulton, M. Kruidering, University of California, San Francisco School of Medicine
| | - Marieke Kruidering
- T.B. Fulton, M. Kruidering, University of California, San Francisco School of Medicine
| | - Klara K Papp
- K.K. Papp, K. Qua, Case Western Reserve University School of Medicine
| | - Kelli Qua
- K.K. Papp, K. Qua, Case Western Reserve University School of Medicine
| | - Joanne M Willey
- Author affiliations: D.M. Olvet, J.B. Bird, J.M. Willey, J.M. Brenner, Donald and Barbara Zucker School of Medicine at Hofstra/Northwell
| | - Judith M Brenner
- Author affiliations: D.M. Olvet, J.B. Bird, J.M. Willey, J.M. Brenner, Donald and Barbara Zucker School of Medicine at Hofstra/Northwell
| |
Collapse
|
25
|
Roxburgh M, Evans DJR. Assessing Anatomy Education: A Perspective from Design. ANATOMICAL SCIENCES EDUCATION 2021; 14:277-286. [PMID: 33544967 DOI: 10.1002/ase.2060] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/07/2020] [Revised: 01/26/2021] [Accepted: 02/01/2021] [Indexed: 06/12/2023]
Abstract
Medical and healthcare practice is likely to see fundamental changes in the future that will require a different approach to the way in which we educate, train, and assess the next generation of healthcare professionals. The anatomical sciences will need to be part of that challenge so they continue to play a full role in preparing students with the knowledge and ever increasingly the skills and competencies that will contribute to the fundamentals of their future capacity to practice effectively. Although there have been significant advances in anatomical science pedagogy, by reviewing learning and assessment in an apparently unrelated field, provides an opportunity to bring a different perspective and enable appropriate challenge of the current approaches in anatomy. Design learning has had to continually reimagine itself in response to the shifting landscape in design practice and the threats associated with technology and societal change. Design learning has also long used a student-centric active pedagogy and allied authentic assessment methods and, therefore, provides an ideal case study to help inform future changes required in anatomical learning and assessment.
Collapse
Affiliation(s)
- Mark Roxburgh
- School of Creative Industries, The University of Newcastle, Callaghan, New South Wales, Australia
| | - Darrell J R Evans
- School of Medicine and Public Health, The University of Newcastle, Callaghan, New South Wales, Australia
- Faculty of Medicine, Nursing and Health Sciences, Monash University, Clayton, Victoria, Australia
| |
Collapse
|
26
|
Goyal RK, Dawson CA, Epstein SB, Brach RJ, Finnie SM, Lounsbury KM, Lahey T, Eldakar-Hein ST. The design and implementation of a longitudinal social medicine curriculum at the University of Vermont's Larner College of Medicine. BMC MEDICAL EDUCATION 2021; 21:131. [PMID: 33627097 PMCID: PMC7903930 DOI: 10.1186/s12909-021-02533-x] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/11/2020] [Accepted: 02/01/2021] [Indexed: 06/12/2023]
Abstract
BACKGROUND Despite an abundant literature advocating that social determinants of health (SDH) be taught during undergraduate medical education, there are few detailed descriptions of how to design and implement longitudinal core curricula that is delivered to all students and accomplishes this goal. METHODS In this paper, we describe the design and implementation of a social medicine curriculum at the University of Vermont's Larner College of Medicine (UVM Larner). Using Kern's principles, we designed a longitudinal curriculum that extends through both preclinical and clinical training for all students and focused on integrating SDH material directly into basic science and clinical training. RESULTS We successfully developed and implemented two primary tools, a "Social Medicine Theme of the Week" (SMTW) in preclinical training, and SDH rounds in the clinical setting to deliver SDH content to all learners at UVM Larner. CONCLUSIONS Extensive student-faculty partnerships, robust needs assessment, and focusing on longitudinal and integrated SDH content delivery to all students were key features that contributed to successful design and implementation.
Collapse
Affiliation(s)
- Raghav K Goyal
- University of Vermont's Larner College of Medicine, UVMMC, 111 Colchester Ave, Smith 2, Burlington, VT, 05401, USA
| | - Christina A Dawson
- University of Vermont's Larner College of Medicine, UVMMC, 111 Colchester Ave, Smith 2, Burlington, VT, 05401, USA
| | - Samuel B Epstein
- University of Vermont's Larner College of Medicine, UVMMC, 111 Colchester Ave, Smith 2, Burlington, VT, 05401, USA
| | - Richard J Brach
- University of Vermont's Larner College of Medicine, UVMMC, 111 Colchester Ave, Smith 2, Burlington, VT, 05401, USA
| | - Sheridan M Finnie
- University of Vermont's Larner College of Medicine, UVMMC, 111 Colchester Ave, Smith 2, Burlington, VT, 05401, USA
| | - Karen M Lounsbury
- University of Vermont's Larner College of Medicine, UVMMC, 111 Colchester Ave, Smith 2, Burlington, VT, 05401, USA
| | - Timothy Lahey
- University of Vermont's Larner College of Medicine, UVMMC, 111 Colchester Ave, Smith 2, Burlington, VT, 05401, USA.
| | - Shaden T Eldakar-Hein
- University of Vermont's Larner College of Medicine, UVMMC, 111 Colchester Ave, Smith 2, Burlington, VT, 05401, USA
| |
Collapse
|
27
|
Ludley A. Context-rich short answer questions (CR-SAQs) in assessment for learning in undergraduate medical education. MEDICAL EDUCATION ONLINE 2019; 24:1674569. [PMID: 31570069 PMCID: PMC6781250 DOI: 10.1080/10872981.2019.1674569] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
|