Standardised assessment of competence in Focused Assessment with Sonography for Trauma

Education in Focused Assessment With Sonography for Trauma Using Immersive Virtual Reality: A Prospective, Interventional Cohort Study and Non-inferiority Analysis With a Historical Control

OBJECTIVE

Focused assessment with sonography for trauma (FAST) is a valuable ultrasound procedure in emergency settings, and there is a need for evidence-based education in FAST to ensure competencies. Immersive virtual reality (IVR) is a progressive training modality gaining traction in the field of ultrasound training. IVR holds several economic and practical advantages to the common instructor-led FAST courses using screen-based simulation (SBS).

METHODS

This prospective, interventional cohort study investigated whether training FAST using IVR unsupervised and out-of-hospital was non-inferior to a historical control group training at a 90 min SBS course in terms of developing FAST competencies in novices. Competencies were assessed in both groups using the same post-training simulation-based FAST test with validity evidence, and a non-inferiority margin of 2 points was chosen.

RESULTS

A total of 27 medical students attended the IVR course, and 27 junior doctors attended the SBS course. The IVR group trained for a median time of 117 min and scored a mean 14.2 ± 2.0 points, compared with a mean 13.7 ± 2.5 points in the SBS group. As the lower bound of the 95% confidence interval at 13.6 was within the range of the non-inferiority margin (11.7-13.7 points), training FAST in IVR for a median of 117 min was found non-inferior to training at a 90 min SBS course. No significant correlation was found between time spent in IVR and test scores.

CONCLUSION

Within the limitations of the use of a historical control group, the results suggest that IVR could be an alternative to SBS FAST training and suitable for unsupervised, out-of-hospital courses in basic FAST competencies.

Development and Usability of an Inexpensive and Reusable Phantom for Ultrasound-Guided Needle Cannulation

Introduction Ultrasound-guided peripheral venous catheter placement (UG-PVCP) is a key skill for establishing intravenous access, especially in patients with anatomical challenges. Ultrasound is highly operator-dependent, and it is essential to ensure a sufficient level of competence when educating healthcare professionals. Competence can be acquired through simulation-based training (SBT) using phantoms or simulators. We developed a phantom for SBT, and in this study, we explore the phantom's usability and technical fidelity. Methods Novices with no experience in UG-PVCP and experts who routinely performed the procedure were asked to perform three ultrasound-guided catheter placement attempts on the phantom. Afterward, they were asked to complete a usability questionnaire consisting of 14 questions exploring the usability and fidelity of the phantom. Results Fifty-seven participants were included in the study: 29 novices and 28 experts. When assessing positive questions about the frequency of use, ease of use, integration of functionality, quickness to learn, and confidence level, the study showed a median score of 4 to 5 out of 5 in the two groups. The median was 1 to 2 out of 5 for negative questions assessing cumbersomeness, unnecessary complexity, and model inconsistency. In an additional comment textbox, one participant mentioned that the cannulation did not feel realistic but that it was good for cannulation practice. Conclusions We believe the phantom is suitable for an educational curriculum since it shows a high level of usability, scoring high on positive questions while scoring low on negative questions, and having high functional fidelity.

Development of a multisystem point of care ultrasound skills assessment checklist

Background

Many institutions are training clinicians in point-of-care ultrasound (POCUS), but few POCUS skills checklists have been developed and validated. We developed a consensus-based multispecialty POCUS skills checklist with anchoring references for basic cardiac, lung, abdominal, and vascular ultrasound, and peripheral intravenous line (PIV) insertion.

Methods

A POCUS expert panel of 14 physicians specializing in emergency, critical care, and internal/hospital medicine participated in a modified-Delphi approach to develop a basic POCUS skills checklist by group consensus. Three rounds of voting were conducted, and consensus was defined by ≥ 80% agreement. Items achieving < 80% consensus were discussed and considered for up to two additional rounds of voting.

Results

Thirteen POCUS experts (93%) completed all three rounds of voting. Cardiac, lung, abdominal, and vascular ultrasound checklists included probe location and control, basic machine setup, image quality and optimization, and identification of anatomical structures. PIV insertion included additional items for needle tip tracking. During the first round of voting, 136 (82%) items achieved consensus, and after revision and revoting, an additional 21 items achieved consensus. A total of 153 (92%) items were included in the final checklist.

Conclusions

We have developed a consensus-based, multispecialty POCUS checklist to evaluate skills in image acquisition and anatomy identification for basic cardiac, lung, abdominal, and vascular ultrasound, and PIV insertion.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13089-022-00268-4.

Hands-On Time in Simulation-Based Ultrasound Training - A Dose-Related Response Study

Purpose Point of care ultrasound (POCUS) is widely used, but the sensitivity and specificity of the findings are highly user-dependent. There are many different approaches to ultrasound training. The aim of this study was to explore the effects of hands-on practice when learning POCUS. Methods Junior doctors with no or limited ultrasound experience were included in the study and divided into three groups. They all completed a Focused Assessment with Sonography for Trauma (FAST) course with different amounts of hands-on practice: 40 minutes (n=67), 60 minutes (n=12), and 90 minutes of hands-on time (n=27). By the end of the course, they all completed a previously validated test. Results More hands-on time improved the mean test scores and decreased the test time. The scores of the 40-, 60-, and 90-minute groups were 11.6 (SD 2.1), 12.8 (SD 2.5), and 13.7 (SD 2.5), respectively (p<0.001). The 90-minute group completed the test significantly faster than the other two groups (20 versus 26 minutes, p=0.003). A large inter-individual variation was seen. Conclusion The necessary amount of hands-on training is unknown. This study demonstrates that performance increases with prolonged hands-on time but the inter-individual variation among trainees is very large, thereby making it impossible to define the "optimal" time. This supports the use of the concept of mastery learning where each individual trainee can continue training until proficiency is reached.

Training and assessment of musculoskeletal ultrasound and injection skills-a systematic review

OBJECTIVES

To examine how residents are trained and assessed in musculoskeletal ultrasound (MSUS), MSUS-guided and landmark-guided joint aspiration and injection. Additionally, to present the available assessment tools and examine their supporting validity evidence.

METHODS

A systematic search of PubMed, Cochrane Library, and Embase was conducted in accordance with the PRISMA guidelines and studies published from January 1, 2000 to May 31, 2021 were included. Two independent reviewers performed the search and data extraction. The studies were evaluated using the Medical Education Research Quality Instrument (MERSQI).

RESULTS

9,884 articles were screened, and 43 were included; 3 were randomized studies, 21 pre- and post-test studies, 16 descriptive studies, and 3 studies developing assessment tools. The studies used various theoretical training modalities e.g. lectures, anatomical quizzes and e-learning. The practical training models varied from mannequins and cadavers to healthy volunteers and patients. The majority of studies used subjective "comfort level" as assessment, others used practical examination and/or theoretical examination. All training programs increased trainees' self-confidence, theoretical knowledge, and/or practical performance, however few used validated assessment tools to measure the effect. Only one study met the MERSQI high methodical quality cut-off score of 14.

CONCLUSION

The included studies were heterogeneous, and most were of poor methodological quality and not based on contemporary educational theories. This review highlights the need for educational studies using validated theoretical and practical assessment tools to ensure optimal MSUS training and assessment in rheumatology.

Are seminal vesicles a potential pitfall during pelvic exploration using point-of-care ultrasound (POCUS)?

Background

Trauma is a major cause of death among the working population. Many countries have now adopted a structured approach to trauma management in which ultrasound is used as a primary evaluation tool. While its use has direct therapeutic benefits, many artifacts and pitfalls are inherent to the technique. Knowledge of the most frequently encountered pitfalls in practice could thus help reduce the risk of error and lead to more accurate trauma assessments.

Objective

This study evaluates a potential pitfall caused by seminal vesicles during focused assessment with sonography for trauma examinations of the male pelvis performed by an emergency physician with experience in point-of-care ultrasound.

Methods

We took five static and five dynamic (3-s loops) transverse ultrasound images of the pelvis in five healthy males. The images and videos were then incorporated into an online survey and emailed through the World Interactive Network Focused On Critical UltraSound (WINFOCUS) in France and the Ultrasound and Emergency Medicine (UEM) Organization in Belgium. A questionnaire asked anonymous participants to assess the presence of free fluid in the static and dynamic images and to share information about their training and experience in point-of-care ultrasound. To validate the static and dynamic images, the survey was sent to three external radiologists for independent verification.

Results

A total of 191 individuals responded fully or partially to the survey, 114 of whom completed it. Among the 114 participants who completed the survey, the misinterpretation rate was 0.55 (95CI 0.51–0.60) for all static and dynamic ultrasound transverse pelvic views. The misinterpretation rate was 0.61 (95CI 0.55–0.66) and 0.50 (95CI 0.45–0.55) for static and dynamic ultrasound transverse pelvic views, respectively. The three external radiologists answered the questionnaire correctly without misinterpreting the survey ultrasound views.

Conclusions

Seminal vesicles are a potential pitfall when interpreting transverse ultrasound images of the male pelvis in the context of point-of-care ultrasound.

Consensus on technical procedures in radiology to include in simulation-based training for residents: a European-wide needs assessment

OBJECTIVES

To identify and prioritize technical procedures for simulation-based training that should be part of the education of residents in radiology.

METHODS

This European-wide needs assessment study used a modified Delphi technique to gather consensus from different key education stakeholders in the field. The first round was a brainstorming phase to identify all procedures that a newly specialized radiologist should potentially be able to do. In the second round, each procedure was explored for the need for simulation training; the participants determined frequency, number of radiologists performing the procedure, impact on patient comfort and safety, and feasibility of simulation. The result of this round was sent back to the participants for final evaluation and prioritization.

RESULTS

Seventy-one key education stakeholders from 27 European countries agreed to participate and were actively involved in the Delphi process: response rates were 72% and 82% in the second and third round, respectively. From 831 suggested procedures in the first round, these were grouped and categorized into 34 procedures that were pre-prioritized in the second round according to the need for simulation-based training. In the third round, 8 procedures were eliminated resulting in final inclusion of 26 procedures. Ultrasound procedures were highly ranked including basic skills such as probe handling; abdominal ultrasound; and ultrasound of kidneys, retroperitoneum, intestines, and scrotum.

CONCLUSION

The prioritized list of procedures represents a consensus document decided upon by educational stakeholders in radiology across Europe. These procedures are suitable for simulation and should be an integral part of the education of radiologists.

KEY POINTS

• The 26 identified procedures are listed according to priority and should be included as an integral part of simulation-based training curricula of radiologists across Europe. • This needs assessment is only the first step towards developing standardized simulation-based training programs that support the harmonization of education and training across Europe.

Simulation-Based Virtual-Reality Patient-Specific Rehearsal Prior to Endovascular Procedures: A Systematic Review

Patient-specific rehearsal (PsR) is a new concept whereby a procedure can be rehearsed virtually using the exact patient’s anatomical data prior to performing the real procedure. The aim of this study was to ascertain if endovascular virtual-reality PsR simulation enhanced performance in real life. This was done by performing a systematic review following the preferred reporting items for systematic reviews and meta-analysis (PRISMA) guidelines. A literature search was conducted in PubMed, Embase, The Cochrane Library and Web of Science concerning PsR in endovascular procedures. All publications were handled using Covidence. Reference lists were also screened. Data extracted from the studies were realism rating, procedure time, fluoroscopy time, contrast volume, number of angiograms and reduction of errors. Kirkpatrick’s four-level model for measuring the efficiency of training was used for guidance of the relevance of studies. The search yielded 1155 results after the exclusion of duplicates, and 11 studies were included. Four studies had a control group, including one randomized trial; the remaining seven were feasibility studies. The realism was rated high, and overall, the studies showed improvements in procedure time, fluoroscopy time and contrast volume after PsR. One study assessed and confirmed the reduction in errors after PsR. Only two studies included more than 15 patients in their cohort. Kirkpatrick’s model was applied to all studies, with one study reaching level 4. All studies found the concept of PsR to be feasible and realistic. The studies with a control group showed a reduction of overall procedure time, radiation exposure and potential errors in endovascular procedures following PsR.

Training and assessment of competence in resuscitative endovascular balloon occlusion of the aorta (REBOA) - a systematic review

BACKGROUND

Resuscitative endovascular balloon occlusion of the aorta (REBOA) is a potentially life-saving but high-risk emergency procedure in patients with haemorrhagic shock. Lack of physicians with competence in the procedure is a barrier to implementation of REBOA. It is currently unclear how training and assessment of competence should be done.

OBJECTIVES

To report and evaluate research in training and assessment of competence in REBOA and femoral arterial access with the aim to investigate the effect of simulation-based training in the procedure and to provide suggestions for the future design of training programs and assessment tools.

METHODS

Following PRISMA guidelines, PubMed, Embase, and Cochrane Library databases were searched for studies on training or assessment of competence in REBOA and femoral arterial access. Bias assessment was done using the Medical Education Research Study Quality Instrument. Evidence level was assessed using GRADE.

RESULTS

Sixteen studies were included, six of them published as abstracts. Full-text studies included 189 trainees ranging in experience level from military medics to surgical specialists. Outcome measures were heterogenous; the most used were rater checklists, knowledge testing, and procedure time. All studies confirmed an effect of training of REBOA on procedural competence in a simulation setting but had a high degree of bias. No study developed or used an assessment tool supported by validity evidence and no study investigated mid and long-term outcomes.

CONCLUSION

Simulation-based training of REBOA improves skills, however, the evidence level is very low and data cannot answer important questions on effect size, skill transfer and retention, and optimal course design. To advance research and training programmes, an assessment tool supported by validity evidence with broad applicability is needed.

Hip-fracture osteosynthesis training: exploring learning curves and setting proficiency standards

Background and purpose - Orthopedic surgeons must be able to perform internal fixation of proximal femoral fractures early in their career, but inexperienced trainees prolong surgery and cause increased reoperation rates. Simulation-based virtual reality (VR) training has been proposed to overcome the initial steep part of the learning curve but it is unknown how much simulation training is necessary before trainees can progress to supervised surgery on patients. We determined characteristics of learning curves for novices and experts and a pass/fail mastery-learning standard for junior trainees was established. Methods - 38 first-year residents and 8 consultants specialized in orthopedic trauma surgery performed cannulated screws, Hansson pins, and sliding hip screw on the Swemac TraumaVision VR simulator. A previously validated test was used. The participants repeated the procedures until they reached their learning plateau. Results - The novices and the experts reached their learning plateau after an average of 169 minutes (95% CI 152-87) and 143 minutes (CI 109-177), respectively. Highest achieved scores were 92% (CI 91-93) for novices and 96% (CI 94-97) for experts. Plateau score, defined as the average of the 4 last scores, was 85% (CI 82-87) and 92% (CI 89-96) for the novices and the experts, respectively. Interpretation - Training time to reach plateau varied widely and it is paramount that simulation-based training continues to a predefined standard instead of ending after a fixed number of attempts or amount of time. A score of 92% comparable to the experts' plateau score could be used as a mastery learning pass/fail standard.

Four Virtual-Reality Simulators for Diagnostic Abdominal Ultrasound Training in Radiology

Ultrasound exams need skilled examiners, and simulation-based training could provide standardized and safe skill training. This study aims to give an overview of different types of virtual-reality simulators for use in abdominal diagnostic ultrasound training in radiology. Fifteen specialized radiologists and radiological residents were presented with two similar cases on four different simulators for abdominal ultrasound training. A feedback sheet for each individual simulator and for an overall comparison was filled out by all participants. All means of scores were compared, and simulators were ranked from least to most favorable. One simulator was ranked most favorable in seven out of nine questions, but none of the mean scores had statistically significant differences. All simulators were recommended for training radiologists, and all simulators were perceived to benefit trainees more than experienced ultra-sonographers.