Objective structured clinical examination in basic thoracic ultrasound: a European study of validity evidence

Assessment of Basic Thoracic Ultrasound Skills in Immersive Virtual Reality: Gathering Validity Evidence

OBJECTIVE

Operator skills are essential for thoracic ultrasound (TUS) to ensure diagnostic accuracy. Immersive virtual reality (IVR) has shown potential within medical education but never for assessment of TUS skills. This study was aimed at developing an IVR test for assessing TUS skills, gathering validity evidence and establishing a pass/fail score.

METHODS

An expert panel developed a test based on the TUS protocol by the European Respiratory Society (ERS), including a tutorial and two clinical cases (pleural effusion and interstitial syndrome), using an IVR platform (VitaSim, Odense, Denmark). Four anterior, four lateral and six posterior zones were available for examination and decision of diagnosis. Each correct examination equaled one point. The contrasting groups' method was used to set a pass/fail score.

RESULTS

Data were collected during the 2022 ERS Congress. We included 13 novices (N, experience: 0 TUS), 22 intermediates (I, 1-50 TUS) and 11 experienced clinicians (E, >50 TUS). Cronbach's α was 0.86. The total mean point scores in case 1 (C1) were (N) 5.0 ± 2.7, (I) 7.3 ± 2.4 and (E) 8.7 ± 1.3, and the scores in case 2 (C2) were (N) 4.5 ± 1.8, (I) 6.7 ± 2.3 and (E) 8.5 ± 2.1. Significant differences were found between N and I for C1 (p = 0.007) and C2 (p = 0.02), I and E for C1 (p = 0.04) and C2 (p = 0.019) and N and E for C1 (p < 0.001) and C2 (p < 0.001). The pass/fail score was 7 points in each case.

CONCLUSION

We established an IVR test that can distinguish between operators with different TUS skills. This enables a standardized, objective and evidence-based approach to assessment of TUS skills.

The European Respiratory Society led training programme improves self-reported competency and increases the use of thoracic ultrasound

Thoracic ultrasound has become a well-implemented diagnostic tool for assessment and monitoring of patients with respiratory symptoms or disease. However, ultrasound examinations are user dependent and sufficient competencies are needed. The European Respiratory Society (ERS) hosts a structured and evidence-based training programme in thoracic ultrasound. This study aimed to explore and discuss the self-reported activity and self-reported competency of the participants during the ERS course. Online surveys were sent to the training programme participants before the second part of the course (practical part of the course), and before and 3 months after the third part of the course (final certification exam). A total of 77 participants completed the surveys. The self-reported frequency of thoracic ultrasound examinations increased during the course, and in the final survey more than 90% of the participants used thoracic ultrasound on weekly basis. The self-reported competency (on technical execution of the thoracic ultrasound examination and overall competency) also increased. The ERS thoracic ultrasound training programme forms the basis of broad theoretical knowledge and sufficient practical skills that seem to lead to behavioural changes, whereby a large proportion of the participants implemented ultrasound in their clinical practice.

Evidence-based training and certification: the ERS thoracic ultrasound training programme

Thoracic ultrasound has developed into an integral part of the respiratory physician's diagnostic and therapeutic toolbox, with high diagnostic accuracy for many diseases causing acute or chronic respiratory symptoms. However, it is vitally important that the operator has received the appropriate education and training to ensure a systematic and thorough examination, correct image interpretation, and that they then have the appropriate skills to integrate all the findings for patient benefit. In this review, we present the new European Respiratory Society thoracic ultrasound training programme, including a discussion of curriculum development, its implementation, and trainee evaluation. This programme enables participants to gain competence in thoracic ultrasound through structured, evidence-based training with robustly validated assessments and certification. The training programme consists of three components: an online, theoretical part (part 1), which is accessible all year; a practical course (part 2), with four courses held each year (two online courses and two on-site courses); and an examination (part 3) comprising an objective structured clinical examination (OSCE), which is hosted each year at the European Respiratory Society Congress.