Enhancing diagnostic accuracy among nonexperts through use of video cases

Selection and ranking of patient video cases in paediatric neurology in relation to learner levels

BACKGROUND

Teaching and learning with patient video cases may add authenticity, enhance diagnostic accuracy and improve chances of early diagnosis. The aim of this study is firstly to identify selection criteria for key Patient video cases (PVCs), secondly to identify trends in relevance of PVCs for learner levels and thirdly, to rank PVCs for learner levels.

METHODS

Based on a literature review, we identified criteria for key PVCs for use in paediatric neurology. We then performed a multi-round Delphi analysis to obtain agreement between 28 expert clinician teachers concerning key PVCs for four learner levels.

RESULTS

We identified two major criteria: key PVCs should demonstrate key movements, and these movements should be subtle and/or difficult to note. The expert clinician teachers subsequently assessed a list of 14 topics for key PVCs. We found a clear, increasing trend in relevance scores, from medical students to young residents to experienced residents and specialists. For medical students and residents, epileptic spasms, Down syndrome, developmental delay, cerebral palsy and absence epilepsy were highly ranked. For specialists, conditions like chorea, focal seizures or eye movement disorders topped the ranking list, although ranking was less clear for this group of advanced learners.

DISCUSSION AND CONCLUSION

Key PVCs should demonstrate movements that are difficult to note for learners. Ranked lists of key PVCs for teaching and learning at different learner levels are now available and may help institutions build validated local libraries of PVCs.

Combining bimodal presentation schemes and buzz groups improves clinical reasoning and learning at morning report

Morning reports offer opportunities for intensive work-based learning. In this controlled study, we measured learning processes and outcomes with the report of paediatric emergency room patients. Twelve specialists and 12 residents were randomised into four groups and discussed the same two paediatric cases. The groups differed in their presentation modality (verbal only vs. verbal + text) and the use of buzz groups (with vs. without). The verbal interactions were analysed for clinical reasoning processes. Perceptions of learning and judgment of learning were reported in a questionnaire. Diagnostic accuracy was assessed by a 20-item multiple-choice test. Combined bimodal presentation and buzz groups increased the odds ratio of clinical reasoning to occur in the discussion of cases by a factor of 1.90 (p = 0.013), indicating superior reasoning for buzz groups working with bimodal materials. For specialists, a positive effect of bimodal presentation was found on perceptions of learning (p < 0.05), and for residents, a positive effect of buzz groups was found on judgment of learning (p < 0.005). A positive effect of bimodal presentation on diagnostic accuracy was noted in the specialists (p < 0.05). Combined bimodal presentation and buzz group discussion of emergency cases improves clinicians' clinical reasoning and learning.

Use of patient video cases in medical education

Patient video cases (PVCs) are brief video recordings of patients during spontaneous or instructed activity. PVCs are true to life and can be replayed, enabling the establishment of highly interactive, contextual and safe learning environments, with a minimum of facilitation. This article describes the use of workshops in which PVCs are used to assist with the development of observational skills and clinical reasoning in medical students and postgraduates. We describe why PVCs are a valuable addition to an educator's portfolio of resources, what evidence there is for their effectiveness, how to use videos for teaching, and some practical advice and tips on their collection and storage.

Medical students' assessment of pediatric patients - teaching and evaluation using video cases

BACKGROUND

We introduced video-based teaching in pediatrics. We evaluated the impact of a pediatric video program on student performance in assessing pediatric patients presented as video cases. The program consisted of a library of pediatric videos, and inclusion of these in the teaching and examination for pediatric medicine.

METHODS

Medical students on a pediatric clerkship at the University of Copenhagen assessed eight short pediatric video cases during autumn 2011 and spring 2012. Two independent observers evaluated a subset of records in a pilot study. A blind evaluation was made of the written records of 37 students before, and 58 students after, the introduction of the program using a Rubric score with four domains.

RESULTS

The intraobserver interclass correlation coefficient was 0.94 and the interobserver interclass correlation was 0.71(n=25). The students' mean total Rubric score in spring 2012 (7.0) was significantly higher (p<0.001, 95% CI 1.34-3.20) than autumn 2011 (4.7). Cohen's d was 1.1 (95% CI 0.6-1.7). Single domains scores increased significantly for general assessment (1.30 versus 0.57) (p<0.002, 95% CI 0.45-1.18), recognition of principal symptoms (1.38 versus 0.81) (p<0.008, 95% CI 0.22-0.91), appropriate diagnosis (2.28 versus 1.78) (p<0.002, 95% CI 0.16-0.82) and consistency between observed symptoms and diagnosis (1.94 versus 1.57) (p=0.0482, 95% CI 0.00-0.79).

CONCLUSIONS

Students improved in evaluating pediatric patients presented as video cases after the introduction of the program. The impact on real-life situations remains to be established.

International multispecialty consensus on how to evaluate ultrasound competence: a Delphi consensus survey

Objectives

To achieve international consensus across multiple specialties on a generic ultrasound rating scale using a Delphi technique.

Methods

Ultrasound experts from Obstetrics-Gynaecology, Surgery, Urology, Radiology, Rheumatology, Emergency Medicine, and Gastro-Enterology representing North America, Australia, and Europe were identified. A multi-round survey was conducted to obtain consensus between these experts. Of 60 invited experts, 44 experts agreed to participate in the first Delphi round, 41 remained in the second round, and 37 completed the third Delphi round. Seven key elements of the ultrasound examination were identified from existing literature and recommendations from international ultrasound societies. All experts rated the importance of these seven elements on a five-point Likert scale in the first round and suggested potential new elements for the assessment of ultrasound skills. In the second round, the experts re-rated all elements and a third round was conducted to allow final comments. Agreement on which elements to include in the final rating scale was pre-defined as more than 80% of the experts rating an element four or five, on importance to the ultrasound examination.

Results

Two additional elements were suggested by more than 10% of the experts in the first Delphi round. Consensus was obtained to include these two new elements along with five of the original elements in the final assessment instrument: 1) Indication for the examination 2) Applied knowledge of ultrasound equipment 3) Image optimization 4) Systematic examination 5) Interpretation of images 6) Documentation of examination and 7) Medical decision making.

Conclusion

International multispecialty consensus was achieved on the content of a generic ultrasound rating scale. This is the first step to ensure valid assessment of clinicians in different medical specialties using ultrasound.

Towards a conceptual framework demonstrating the effectiveness of audiovisual patient descriptions (patient video cases): a review of the current literature

BACKGROUND

Technological advances have enabled the widespread use of video cases via web-streaming and online download as an educational medium. The use of real subjects to demonstrate acute pathology should aid the education of health care professionals. However, the methodology by which this effect may be tested is not clear.

METHODS

We undertook a literature review of major databases, found relevant articles relevant to using patient video cases as educational interventions, extracted the methodologies used and assessed these methods for internal and construct validity.

RESULTS

A review of 2532 abstracts revealed 23 studies meeting the inclusion criteria and a final review of 18 of relevance. Medical students were the most commonly studied group (10 articles) with a spread of learner satisfaction, knowledge and behaviour tested. Only two of the studies fulfilled defined criteria on achieving internal and construct validity. The heterogeneity of articles meant it was not possible to perform any meta-analysis.

CONCLUSIONS

Previous studies have not well classified which facet of training or educational outcome the study is aiming to explore and had poor internal and construct validity. Future research should aim to validate a particular outcome measure, preferably by reproducing previous work rather than adopting new methods. In particular cognitive processing enhancement, demonstrated in a number of the medical student studies, should be tested at a postgraduate level.

Improving postgraduate clinical assessment tools: the introduction of video recordings to assess decision making

BACKGROUND

Competency in the management of acutely unwell patients has not previously been formally assessed as part of an objective structured clinical examination (OSCE).

AIM

The reliability of the paediatric postgraduate OSCE was calculated. An objective structured video examination was designed to assess candidates' clinical decision making ability when managing acutely unwell children.

METHODS

The performance of 3522 postgraduate paediatric trainees was assessed (2006-2008). OSCE reliability was determined using Cronbach's alpha and mean inter-item correlation. Performance in the video station was compared with overall performance (not including video station; Mann-Whitney U) and video station scores correlated with individual station scores (Spearman's Rho correlation coefficient).

RESULTS

Clinical examination pass rates for the 684 UK graduates, 1608 overseas candidates training in the UK and 1104 overseas candidates training overseas were 69.7%, 28% and 22.3%, respectively (graduation information not available for 126 candidates). Cronbach's alpha was 0.62. Mean inter-item correlation was 0.15. Candidates who passed the OSCE overall had significantly higher scores on the video station (t(3520) = 14.48); p < 0.001). There was significant positive correlation between scores on the video station, individual stations and overall total score (r's = 0.300; p = 0.001).

CONCLUSIONS

The postgraduate paediatric OSCE provides a sound and valid means of assessing clinical skills at the postgraduate level. The video station provides an important new method of assessment. Its use in other postgraduate clinical examinations should be explored.

Visual expertise in paediatric neurology

BACKGROUND

Visual expertise relies on perceptive as well as cognitive processes. At present, knowledge of these processes when diagnosing clinical cases mainly stems from studies with still pictures. In contrast, patient video cases constitute a dynamic diagnostic challenge that may simulate seeing and diagnosing a patient in person.

AIMS

This study investigates visual attention and the concomitant cognitive processes of clinicians diagnosing authentic paediatric video cases.

METHODS

A total of 43 clinicians with varying levels of expertise took part in this cross-sectional study. They diagnosed four brief video recordings of children: two with seizures and two with disorders imitating seizures. We used eye tracking to investigate time looking at relevant areas in the video cases and a concurrent think-aloud procedure to explore the associated clinical reasoning processes.

RESULTS

More experienced clinicians were more accurate in visual diagnosis and spent more of their time looking at relevant areas. At the same time, they explored data less, yet they built and evaluated more diagnostic hypotheses.

CONCLUSIONS

Clinicians of varying expertise analyse patient video cases differently. Clinical teachers should take these differences into account when optimising educational formats with patient video cases.