Assessing Resident Diagnostic Skills Using a Modified Bronchiolitis Score

Background

Resident milestones are objective instruments that assess the resident's growth, progression in knowledge, and clinical diagnostic reasoning; but they rely on the subjective appraisal of the supervising attending. Little is known about the use of standardized instruments that may complement the evaluation of resident diagnostic skills in the academic setting.

Objectives

Evaluate a modified bronchiolitis severity assessment tool by appraising the inter-rater variability and reliability between pediatric attendings and pediatric residents.

Methods

Cross-sectional study of children under 24 months of age who presented to a Community Hospital's Emergency Department with bronchiolitis between January-June 2014. A paired pediatric attending and resident evaluated each patient. Evaluation included age-based respiratory rate (RR), retractions, peripheral saturation, and auscultation. Cohen's kappa (K) measured inter-rater agreement. Inter-rater reliability (IRR) was assessed using a one-way random, average measures intra-class correlation (ICC) to evaluate the degree of consistency and magnitude of disagreement between inter-raters. Value of >0.6 was considered substantial for kappa and good internal consistency for ICC.

Results

Twenty patients were evaluated. Analysis showed fair agreement for the presence of retractions (K=0.31), auscultation (K=0.33), and total score (K=0.3). The RR (ICC=0.97), SpO₂ (ICC=1.0), auscultation (ICC=0.77), and total score (ICC=0.84) were scored similarly across both raters, indicating excellent IRR. Identification of retractions had the least agreement across all statistical analysis.

Conclusion

The use of a standardized instrument, in conjunction with a trained resident-teaching staff, can help identify deficiencies in clinical competencies among residents and facilitate the learning process for the identification of pertinent clinical findings.

Evaluation of an adaptive virtual laboratory environment using Western Blotting for diagnosis of disease

BACKGROUND

Providing large numbers of undergraduate students in scientific disciplines with engaging, authentic laboratory experiences is important, but challenging. Virtual laboratories (vLABs) are a potential means to enable interactive learning experiences. A vLAB focusing on Western Blotting was developed and implemented in a 3rd year undergraduate Pathology course for science students to facilitate learning of technical molecular laboratory skills that are linked to development of diagnostic skills. Such skills are important for undergraduates in building a conceptual understanding of translation of laboratory techniques to changes in human biology due to disease.

METHODS

The Western Blotting vLAB was developed and deployed using the Adaptive eLearning Platform (AeLP) developed by Smart Sparrow (https://www.smartsparrow.com/). The vLAB was evaluated to assess students' perceptions of their laboratory skills relevant to the diagnosis of Muscular Dystrophy. A blended learning rotation model was applied in which wet laboratory and vLAB environments for Western Blotting were both delivered to three consecutive cohorts of 3rd year science undergraduates undertaking a Muscle Diseases practical class. Evaluation questionnaires were administered at the completion of the practical classes.

RESULTS

Students indicated in online questionnaires that the Western Blotting vLAB was at least equivalent to the real lab in their perceived development of concepts, laboratory skills and diagnosis of disease.

CONCLUSIONS

vLABs have great potential for improving students' development of diagnostic skills. Further studies are required to determine the impact of vLABs on student learning.

Dysfunctional doctors--will revalidation help?

Numbers of National Clinical Assessment Service referrals concerning practitioners' performance are much higher for the over-50s, especially in GPs. In 20% of cases there are concerns with diagnosis that are unlikely to be recognised by revalidation. Accurate and timely diagnosis is fundamental to patient safety, and the results of studies in which GPs were tested using mystery patients are not encouraging. We need to assess pratictioners' competencies in diagnosis throughout their careers. Plans to extend working life may require changes to the clinical responsibilities of older doctors.

Evaluation of Computer-aided Strategies for Teaching Medical Students Prenatal Ultrasound Diagnostic Skills

OBJECTIVE

To evaluate whether computer-based learning (CBL) improves newly acquired knowledge and is an effective strategy for teaching prenatal ultrasound diagnostic skills to third-year medical students when compared with instruction by traditional paper-based methods (PBM).

STUDY DESIGN

We conducted a randomized, prospective study involving volunteer junior (3(rd) year) medical students consecutively rotating through the Obstetrics and Gynecology clerkship during six months of the 2005-2006 academic year. The students were randomly assigned to permuted blocks and divided into two groups. Half of the participants received instruction in prenatal ultrasound diagnostics using an interactive CBL program; the other half received instruction using equivalent material by the traditional PBM. Outcomes were evaluated by comparing changes in pre-tutorial and post instruction examination scores.

RESULTS

All 36 potential participants (100%) completed the study curriculum. Students were divided equally between the CBL (n = 18) and PBM (n = 18) groups. Pre-tutorial exam scores (mean+/-s.d.) were 44%+/-11.1% for the CBL group and 44%+/-10.8% for the PBL cohort, indicating no statistically significant differences (p>0.05) between the two groups. After instruction, post-tutorial exam scores (mean+/-s.d.) were increased from the pre-tutorial scores, 74%+/-11% and 67%+/-12%, for students in the CBL and the PBM groups, respectively. The improvement in post-tutorial exam scores from the pre-test scores was considered significant (p<0.05). When post-test scores for the tutorial groups were compared, the CBL subjects achieved a score that was, on average, 7 percentage points higher than their PBM counterparts, a statistically significant difference (p < 0.05).

CONCLUSION

Instruction by either CBL or PBM strategies is associated with improvements in newly acquired knowledge as reflected by increased post-tutorial examination scores. Students that received CBL had significantlyhigher post-tutorial exam scores than those in the PBM group, indicating that CBL is an effective instruction strategy in this setting.