Assessing the relationship between cardiac physical examination technique and accurate bedside diagnosis during an objective structured clinical examination (OSCE)

Designing a touchless physical examination for a virtual Objective Structured Clinical Examination

Purpose

Given the COVID-19 pandemic, many Objective Structured Clinical Examinations (OSCEs) have been adapted to virtual formats without addressing whether physical examination maneuvers can or should be assessed virtually. In response, we developed a novel touchless physical examination station for a virtual OSCE and gathered validity evidence for its use.

Methods

We used a touchless physical examination OSCE station pilot-tested in a virtual OSCE in which Internal Medicine residents had to verbalize their approach to the physical examination, interpret images and videos of findings provided upon request, and make a diagnosis. We explored differences in performance by training year using ANOVA. In addition, we analyzed data using elements of Bloom's taxonomy of learning, i.e. knowledge, understanding, and synthesis.

Results

Sixty-seven residents (PGY1-3) participated in the OSCE. Scores on the pilot station were significantly different between training levels (F=3.936, p = 0.024, ηp2 = 0.11). The pilot station-total correlation (STC) was r = 0.558, and the item-station correlations ranged from r = 0.115-0.571, with the most discriminating items being those that assessed application of knowledge (interpretation and synthesis) rather than recall.

Conclusion

This touchless physical examination station was feasible, had acceptable psychometric characteristics, and discriminated between residents at different levels of training.

Application of objective structured clinical examination (OSCE) for the evaluation of Kampo medicine training

BACKGROUND

The purpose of this study was to develop an objective, content-valid, and reliable assessment method for Kampo medicine using an objective structured clinical examination (OSCE) for the assessment of clinical competence in Kampo medicine.

METHODS

We developed a blueprint followed by a list of 47 assessment items and three task scenarios related to clinical competence in Kampo medicine. An eight-member test committee checked the relevance of the assessment items on a Likert scale. We calculated a content validity index and content validity ratio, and used the Angoff method to set the passing threshold. We trained a total of nine simulated patients with three assigned to each scenario. We conducted an OSCE for 11 candidates with varying medical abilities, and conducted three stations per person, which were evaluated by one evaluator in one room by direct observation. We used video recordings to test the inter-rater reliability of the three raters. We used the test results to verify the reliability of the assessment chart.

RESULTS

The inter-rater reliability (intraclass correlation coefficient [2,1]) was 0.973. The reliability of the assessment chart for each scenario (Cronbach's α) was 0.86, 0.89, and 0.85 for Scenarios 1, 2, and 3, respectively. The reliability of the assessment chart for the whole OSCE (Cronbach's α) was 0.90.

CONCLUSIONS

We developed a content-valid new OSCE assessment method for Kampo medicine and obtained high inter-rater and test reliabilities. Our findings suggest that this is one of the most reliable evaluation methods for assessing clinical competence in Kampo medicine.

Evaluating the effectiveness of a structured, simulator-assisted, peer-led training on cardiovascular physical examination in third-year medical students: a prospective, randomized, controlled trial

Background: Previous research suggests that cardiac examination skills in undergraduate medical students frequently need improvement. There are different ways to enhance physical examination (PE) skills such as simulator-based training or peer-assisted learning (PAL). Aim: The aim of this study was to evaluate the effectiveness of a structured, simulator-assisted, peer-led training on cardiovascular PE. Methods: Participants were third-year medical students at Leipzig University Faculty of Medicine. Students were randomly assigned to an intervention group (IG) and a control group (CG). In addition to standard curricular training, IG received a peer-led, simulator-based training in cardiac PE. Participant performance in cardiac PE was assessed using a standardized checklist with a maximum of 25 points. Primary outcome was assessed via checklist point distribution. Results: 89 students were randomised to either CG (n=43) or IG (n=46) with 70 completing the study. Overall, IG students performed significantly better than CG students did (max. points: 25, M±SD in IG was 17±3, in CG 12±4, p<.0001). Simple mistakes such as not using the stethoscope correctly were more frequent in CG students. Prior experience did not lead to a significant difference in performance. Conclusions: Structured, peer-led and simulator-assisted teaching sessions improve cardiac PE skills in this setting compared to control students that did not receive this training.

Physical examination skills training: Faculty staff vs. patient instructor feedback-A controlled trial

BACKGROUND

Standardized patients are widely used in training of medical students, both in teaching and assessment. They also frequently lead complete training sessions delivering physical examination skills without the aid of faculty teaching staff-acting as "patient instructors" (PIs). An important part of this training is their ability to provide detailed structured feedback to students which has a strong impact on their learning success. Yet, to date no study has assessed the quality of physical examination related feedback by PIs. Therefore, we conducted a randomized controlled study comparing feedback of PIs and faculty staff following a physical examination assessed by students and video assessors.

METHODS

14 PIs and 14 different faculty staff physicians both delivered feedback to 40 medical students that had performed a physical examination on the respective PI while the physicians observed the performance. The physical examination was rated by two independent video assessors to provide an objective performance standard (gold standard). Feedback of PI and physicians was content analyzed by two different independent video assessors based on a provided checklist and compared to the performance standard. Feedback of PIs and physicians was also rated by medical students and video assessors using a questionnaire consisting of 12 items.

RESULTS

There was no statistical significant difference concerning overall matching of physician or PI feedback with gold standard ratings by video assessment (p = .219). There was also no statistical difference when focusing only on items that were classified as major key steps (p = .802), mistakes or parts that were left out during physical examination (p = .219) or mistakes in communication items (p = .517). The feedback of physicians was significantly better rated than PI feedback both by students (p = .043) as well as by video assessors (p = .034).

CONCLUSIONS

In summary, our study demonstrates that trained PIs are able to provide feedback of equal quantitative value to that of faculty staff physicians with regard to a physical examination performed on them. However, both the students and the video raters judged the quality of the feedback given by the physicians to be significantly better than that of the PIs.

Digital rectal examination skills: first training experiences, the motives and attitudes of standardized patients

BACKGROUND

Physical clinical examination is a core clinical competence of medical doctors. In this regard, digital rectal examination (DRE) plays a central role in the detection of abnormalities of the anus and rectum. However, studies in undergraduate medical students as well as newly graduated doctors show that they are insufficiently prepared for performing DRE. Training units with Standardized Patients (SP) represent one method to deliver DRE skills. As yet, however, it is little known about SPs' attitudes.

METHODS

This is a qualitative study using a grounded theory approach. Interviews were conducted with 4 standardized patients about their experiences before, during and after structured SP training to deliver DRE competencies to medical students. The resulting data were subjected to thematic content analysis.

RESULTS

Results show that SPs do not have any predominant motives for DRE program participation. They participate in the SP training sessions with relatively little prejudice and do not anticipate feeling highly vulnerable within teaching sessions with undergraduate medical students.

CONCLUSIONS

The current study examined SPs' motives, views, expectations and experiences regarding a DRE program during their first SP training experiences. The results enabled us to derive distinct action guidelines for the recruitment, informing and briefing of SPs who are willing to participate in a DRE program.

Development and validation of a cardiac findings checklist for use with simulator-based assessments of cardiac physical examination competence

INTRODUCTION

Objective outcome measures for use with simulator-based assessments of cardiac physical examination competence are lacking. The current study describes the development and validation of an approach to scoring performance using a cardiac findings checklist.

METHODS

A cardiac findings checklist was developed and implemented for use with a simulator-based assessment of cardiac physical examination competence at a Canadian national specialty examination in internal medicine. Candidate performance as measured using the checklist was compared with global ratings of clinical performance on the cardiac patient simulator and with overall examination performance.

RESULTS

Interrater reliability for scoring the checklist ranged from 0.95 for scoring correct findings to 0.72 for scoring incorrect findings. A summary checklist score had a Pearson correlation of 0.60 with overall candidate performance on the simulator-based station.

CONCLUSION

Use of a cardiac findings checklist provides one objective measure of cardiac physical examination competence that may be used with simulator-based assessments.

Assessing cardiac physical examination skills using simulation technology and real patients: a comparison study

OBJECTIVE

High-stakes assessments of doctors' physical examination skills often employ standardised patients (SPs) who lack physical abnormalities. Simulation technology provides additional opportunities to assess these skills by mimicking physical abnormalities. The current study examined the relationship between internists' cardiac physical examination competence as assessed with simulation technology compared with that assessed with real patients (RPs).

METHODS

The cardiac physical examination skills and bedside diagnostic accuracy of 28 internists were assessed during an objective structured clinical examination (OSCE). The OSCE included 3 modalities of cardiac patients: RPs with cardiac abnormalities; SPs combined with computer-based, audio-video simulations of auscultatory abnormalities, and a cardiac patient simulator (CPS) manikin. Four cardiac diagnoses and their associated cardiac findings were matched across modalities. At each station, 2 examiners independently rated a participant's physical examination technique and global clinical competence. Two investigators separately scored diagnostic accuracy.

RESULTS

Inter-rater reliability between examiners for global ratings (GRs) ranged from 0.75-0.78 for the different modalities. Although there was no significant difference between participants' mean GRs for each modality, the correlations between participants' performances on each modality were low to modest: RP versus SP, r = 0.19; RP versus CPS, r = 0.22; SP versus CPS, r = 0.57 (P < 0.01).

CONCLUSIONS

Methodological limitations included variability between modalities in the components contributing to examiners' GRs, a paucity of objective outcome measures and restricted case sampling. No modality provided a clear 'gold standard' for the assessment of cardiac physical examination competence. These limitations need to be addressed before determining the optimal patient modality for high-stakes assessment purposes.

Twelve tips for excellent physical examination teaching

BACKGROUND

Physical examination (PEx) skills are declining among medical trainees, yet many institutions are not teaching these systematically and effectively. Many variables contribute to effective teaching: teachers' confidence in their clinical skills, ability to demonstrate and assess these skills; availability of suitable patients; trainee attitude and fatigue; belief that institutions do not value clinical teachers. Finally, the relevance and significance of a systematic exam must be demonstrated or the teaching degenerates into a 'show-and-tell' exercise.

AIMS

This paper describes twelve practical teaching tips that can be used to promote high quality PEx teaching in 5 minutes or 45 minutes. TEACHING TIPS: (1) Diagnostic hypotheses should guide reflective exam; (2) Teachers with the best clinical skills should be recruited; (3) A longitudinal and systematic curriculum can tailor teaching to multiple learner levels (4) Integration of simulation and bedside teaching can maximise learning; (5) Bedside detective work and games make learning fun; (6) The 6-step approach to teach procedures can be adopted to teach PEx; (7) Clinical teaching at the bedside should be increased; (8) Linking basic sciences to clinical findings will demonstrate relevance; (9) Since assessment drives learning, clinical skills should be systematically assessed; (10) Staff development can target improvement of teachers' clinical skills for effective teaching; (11) Technology should be used to study utility of clinical signs; (12) Institutions should elevate the importance of clinical skills teaching and recognize and reward teachers.

CONCLUSIONS

PEx is important in patient-physician interactions, a valuable contributor to accurate clinical diagnosis and can be taught effectively using practical tips. To reverse the trend of deficient clinical skills, precision of clinical findings should be studied and exam manoeuvres that do not contribute to diagnosis discarded; institutions should value clinical skills teaching, appoint and fund core faculty to teach and provide staff development to improve teaching skills.