Learning curves for point-of-care ultrasound image acquisition for novice learners in a longitudinal curriculum

Artificial Intelligence-Guided Lung Ultrasound by Nonexperts

Importance

Lung ultrasound (LUS) aids in the diagnosis of patients with dyspnea, including those with cardiogenic pulmonary edema, but requires technical proficiency for image acquisition. Previous research has demonstrated the effectiveness of artificial intelligence (AI) in guiding novice users to acquire high-quality cardiac ultrasound images, suggesting its potential for broader use in LUS.

Objective

To evaluate the ability of AI to guide acquisition of diagnostic-quality LUS images by trained health care professionals (THCPs).

Design, Setting, and Participants

In this multicenter diagnostic validation study conducted between July 2023 and December 2023, participants aged 21 years or older with shortness of breath recruited from 4 clinical sites underwent 2 ultrasound examinations: 1 examination by a THCP operator using Lung Guidance AI and the other by a trained LUS expert without AI. The THCPs (including medical assistants, respiratory therapists, and nurses) underwent standardized AI training for LUS acquisition before participation.

Interventions

Lung Guidance AI software uses deep learning algorithms guiding LUS image acquisition and B-line annotation. Using an 8-zone LUS protocol, the AI software automatically captures images of diagnostic quality.

Main Outcomes and Measures

The primary end point was the proportion of THCP-acquired examinations of diagnostic quality according to a panel of 5 masked expert LUS readers, who provided remote review and ground truth validation.

Results

The intention-to-treat analysis included 176 participants (81 female participants [46.0%]; mean [SD] age, 63 [14] years; mean [SD] body mass index, 31 [8]). Overall, 98.3% (95% CI, 95.1%-99.4%) of THCP-acquired studies were of diagnostic quality, with no statistically significant difference in quality compared to LUS expert-acquired studies (difference, 1.7%; 95% CI, -1.6% to 5.0%).

Conclusions and Relevance

In this multicenter validation study, THCPs with AI assistance achieved LUS images meeting diagnostic standards compared with LUS experts without AI. This technology could extend access to LUS to underserved areas lacking expert personnel.

Trial Registration

ClinicalTrials.gov Identifier: NCT05992324.

Point-of-Care Ultrasound Competency, Credentialing and Policies

Point-of-care ultrasound (POCUS) enhances diagnostic accurate, reduces time to diagnosis, and improves patient satisfaction. Competenties encompass indications, image acquisition, interpretation, and medical decision-making. Education involves didactic methods and hands-on practice with emphasis on longitudinal training and feedback. Assessment tools target knowledge, psychomotor, attitudinal domains. POCUS portfolios and quality assurance ensure competence and continued overarching POCUS oversight and feedback. Scan numbers are often used as a surrogate for competency; however, it is important to note that learning curves vary by POCUS application. Proper POCUS use requires understanding its diagnostic power and potential limitations. POCUS indications vary by clinical scenarios and beneficial applications for training and credentialing vary by medical niche. This text underscores the importance of standardized training and evidence-based practices in POCUS utilization.

Lung imaging methods: indications, strengths and limitations

Imaging methods are fundamental tools to detect and diagnose lung diseases, monitor their treatment and detect possible complications. Each modality, starting from classical chest radiographs and computed tomography, as well as the ever more popular and easily available thoracic ultrasound, magnetic resonance imaging and nuclear medicine methods, and new techniques such as photon counting computed tomography, radiomics and application of artificial intelligence, has its strong and weak points, which we should be familiar with to properly choose between the methods and interpret their results. In this review, we present the indications, strengths and main limitations of methods for chest imaging.

Implementation of lung ultrasonography by general practitioners for lower respiratory tract infections: a feasibility study

OBJECTIVE

To evaluate the feasibility of lung ultrasonography (LUS) performed by novice users' general practitioners (GPs) in diagnosing lower respiratory tract infections (LRTIs) in primary health care settings.

DESIGN

A prospective interventional multicenter study (December 2019-March 2020).

SETTINGS AND SUBJECTS

Patients aged >3 months, suspected of having LRTI consulting in three different general practices (GPs) (rural, semirural and urban) in France.

MAIN OUTCOME MEASURES

Feasibility of LUS by GPs was assessed by (1) the proportion of patients where LUS was not performed, (2) technical breakdowns, (3) interpretability of images by GPs, (4) examination duration and (5) patient perception and acceptability.

RESULTS

A total of 151 patients were recruited, and GPs performed LUS for 111 (73.5%) patients (LUS group). In 99.1% (n = 110) of cases, GPs indicated that they were able to interpret images. The median [IQR] exam duration was 4 [3-5] minutes. LRTI was diagnosed in 70.3% and 60% of patients in the LUS and no-LUS groups, respectively (p = .43). After LUS, GPs changed their diagnosis from 'other' to 'LRTI' in six cases (+5.4%, p < .001), prescribed antibiotics for five patients (+4.5%, p = .164) and complementary chest imaging for 10 patients (+9%, p < .001). Patient stress was reported in 1.8% of cases, 81.7% of patients declared that they better understood the diagnosis, and 82% of patients thought that the GP diagnosis was more reliable after LUS.

CONCLUSIONS

LUS by GPs using handheld devices is a feasible diagnostic tool in primary health care for LRTI symptoms, demonstrating both effectiveness and positive patient reception.

TRIAL REGISTRATION NUMBER

Clinicaltrial.gov: NCT04602234, 20/10/2020.

Point of Care Ultrasounds Obtained by Novice Physician Assistant Residents (POCUS ON PAR)

INTRODUCTION

The integration of Point of Care Ultrasound (POCUS) into the care of trauma patients, specifically the E-FAST, has improved the accuracy of initial diagnoses and improved time to surgical intervention in critically ill patients. Physician assistants (PAs) are critically important members of any military trauma resuscitation team and are often team leaders in a pre-hospital setting. They may receive training in ultrasound but there are little data to support their use or evaluate their effectiveness in using POCUS. We designed a study to evaluate the image quality of an E-FAST Exam performed by Emergency Medicine Physician Assistant (EMPA) Fellows and Emergency Medicine (EM) Interns following identical training. Our hypothesis is that image quality obtained by EMPAs will be non-inferior to those images obtained by EM Interns.

MATERIALS AND METHODS

This is a prospective single-blinded study comparing the image quality of E-FAST exams performed by first year EM interns and first year EMPA fellows. All participants completed standard POCUS training prior to enrollment in the study. A total of 8 EMPAs and 8 EM first year residents completed 10 recorded E-FAST exams to be used as study images. Participants also viewed a 15-question slide show containing images of positive (6) and negative (9) E-FAST exams and recorded their interpretations. Images were reviewed by expert reviewers who were blinded to which images were collected by which group. An image quality score was recorded for each view as well as an overall image quality score. Image quality was rated on a 1 to 5 image quality scale.

RESULTS

For overall image quality, the mean score for EMPAs was 3.6 ± 0.5 and for EM residents was 3.2 ± 0.5 with statistical significance favoring better image quality from the EMPAs. The time to completion for the EFAST exam for EMPAs was 4.8 ± 1.3 minutes and for interns it was 3.4 ± 1.4 minutes (P value = 0.02). There was no difference in image interpretation quiz scores between the groups (mean score 92% among interns and 95% among PAs).

CONCLUSIONS

POCUS is an imaging modality which is very portable and relatively inexpensive which makes it ideal for military medicine. PAs are essential members of military trauma teams, and often run an initial trauma resuscitation. Being able to correctly identify patients who have free fluid early in the course of treatment allows for more correct evacuation criteria to ensure the sickest patients get to care the fastest. Although there are limited data to support POCUS use by non-physicians, our data support a growing body of evidence that it is not the profession or baseline medical education that determines an individual's ability to use and incorporate ultrasound into bedside and clinical practice. Our study shows that with training and experience PAs or other members of the military health care team can use the EFAST to better care for trauma patients.

Web-based objective and structured assessment of point-of-care lung ultrasound skills in resource-limited settings

BACKGROUND

Objective assessment of skills after training is essential for safe implementation of lung point-of-care ultrasound (POCUS). In low-and middle-income countries (LMIC) there is a need for assessment tools without onsite experts to scale up POCUS access. Our objective is to develop a web-based assessment tool and evaluate trainees across different countries and at different time points after initial lung POCUS training.

METHODS

We adapted the objective and validated lung ultrasound score (LUS-OSAUS) to a web-based tool with quiz and practical skills test. Trainees were evaluated after a short (4-day) standardized lung POCUS training and were classified in distinct groups according to (i) their geographical location (Benin vs. South-Africa) and (ii) time elapsed since training (Benin 0 months vs. Benin 6 months). The Benin 6 months group had minimal continuous education. Skills test images were read by two blinded experts. We report the overall success rates and then compare these rates based on location and timing since training, using the Fischer's exact test.

RESULTS

A total of 35 out of 43 participants completed the online LUS-OSAUS quiz and skills test. The overall success rate was 0.84 (95%CI 0.80-0.88), with lower success rates for "correct depth" 0.54 (0.37-0.71), "correct assessment of pleura" 0.63 (0.45-0.79) and "conclusion" 0.71 (0.54-0.85). There were no differences based on location, with respective rates of 0.86 (0.80-0.92) and 0.83 (0.75-0.91) (p-value = 0.125) for Benin and South Africa at 0 months, respectively. Similarly, there were no differences according to timing with success rates of 0.86 (0.80-0.92) and 0.82 (0.72-0.93) (p-value = 0.563) for Benin at 0 months and 6 months, respectively.

CONCLUSION

Web-based objective and structured assessment of lung POCUS skills in LMIC following a short-standardized training is feasible and has a good overall success rate with consistent results across regions and up to 6 months after training given minimal continuous education. Overall, technical and POCUS-based clinical conclusion skills are the most difficult to acquire.

Point-of-Care Ultrasound Psychomotor Learning Curves: A Systematic Review of the Literature

OBJECTIVES

Use of point-of-care ultrasound (POCUS) in clinical medicine and inclusion in medical training is increasing. Some professional societies recommend that 25-50 POCUS examinations be completed for each application learned; however, the amount of practice required is not well studied. As such, a better understanding of the learning curves of POCUS psychomotor skills is needed. This systematic review characterizes the learning curves for POCUS psychomotor skill acquisition.

METHODS

With the assistance of a research librarian, the available literature through August 28, 2023, was identified. The titles and abstracts, and then the full text were reviewed by two reviewers to screen for inclusion. All studies included after full-text review then underwent data extraction and analysis.

RESULTS

The search identified 893 unique studies. Forty-five studies underwent full-text review, with 17 meeting full inclusion criteria. Substantial heterogeneity was noted in study design, duration of education, number and type of learners, and methods for statistical analysis. Clear and validated definitions for learning endpoints, such as plateau points or competency, are lacking. Learning curves and endpoints differ for different applications of POCUS.

CONCLUSION

The results are overall supportive of the recommendations to complete 25-50 examinations per application of POCUS learned. However, specific applications require more practice than others. Certain applications, such as cardiac and the Focused Assessment with Sonography in Trauma (FAST) exams, are closer to 50; while others, such as soft tissue, airway, and eye require no more than 25.

Safety and efficacy of ultrasound-assisted bedside ureteric stent placement A prospective, single-institution study

INTRODUCTION

Previous studies have demonstrated the feasibility of bedside placement of ureteric stents; however, they have traditionally required two skilled operators and were associated with some stent malposition especially for proximal ureteric obstruction. We sought to investigate the efficacy and safety of a modified technique for ultrasound-assisted bedside ureteric stent insertion without the presence of a skilled assist.

METHODS

A single institution prospective study was performed from April-August 2023. Indications for stenting included infection, renal insufficiency, or intractable colic. Exclusion criteria included age <18 years, hemodynamic instability, and patients with history of chronic pain. Point-of-care ultrasound (POCUS) was used to confirm wire placement in the kidney and presence of a hydronephrotic drip from a 5 French ureteric catheter was used to confirm placement beyond the level of obstruction.

RESULTS

Of 28 patients, all patients underwent successful bedside ureteric stent placement. Mean age was 64.9 years and mean body mass index (BMI) was 33.2. Proximal ureter obstruction was present in 52% of patients and mid/distal obstruction in 48%. In cases with obstructing stones, the mean stone size was 8.1 mm with a range of 4-15 mm. Infection was the indication for stent placement in most patients (71%), followed by pain (4%) and acute kidney injury (AKI) (4%). All patients who underwent successful stent placement had presence of hydronephrotic drip from the ureteric catheter.

CONCLUSIONS

Ultrasound-assisted bedside ureteric stent insertion without a skilled assist is a safe and feasible option for management of acute ureteral obstruction. Presence of hydronephrotic drip can indicate successful access beyond the level of obstruction as an alternative to POCUS.

Teachability of multispectral optoacoustic tomography

To date, the appropriate training required for the reproducible operation of multispectral optoacoustic tomography (MSOT) is poorly discussed. Therefore, the aim of this study was to assess the teachability of MSOT imaging. Five operators (two experienced and three inexperienced) performed repositioning imaging experiments. The inexperienced received the following introductions: personal supervision, video meeting, or printed introduction. The task was to image the exact same position on the calf muscle for seven times on five volunteers in two rounds of investigations. In the first session, operators used ultrasound guidance during measurements while using only photoacoustic data in the second session. The performance comparison was carried out with full-reference image quality measures to quantitatively assess the difference between repeated scans. The study demonstrates that given a personal supervision and hybrid ultrasound real-time imaging in MSOT measurements, inexperienced operators are able to achieve the same level as experienced operators in terms of repositioning accuracy.

Implementation of a point-of-care ultrasound archiving system and governance framework in a UK physician-paramedic staffed helicopter emergency medical service

INTRODUCTION

There has been a rapid expansion in the use of point-of-care ultrasonography (POCUS) by emergency medical services (EMS). However, less than a third of UK EMS utilise imaging archiving for POCUS, and fewer review saved images as part of a clinical governance structure. This paper describes the implementation of a novel image archiving system and a robust clinical governance framework in our UK physician-paramedic staffed helicopter emergency medical service (HEMS).

METHODS

A retrospective database review was conducted of all patients attended by East Anglian Air Ambulance (EAAA) between the introduction of a new POCUS device and image archiving system on 1 December 2020 to 31 January 2024. All patients with recorded POCUS examinations were included. Images from POCUS examinations at EAAA are archived on a cloud-based server, and retrospectively reviewed within 24 h by an EAAA POCUS supervisor. Image quality is graded using a 5-point Likert-type scale, agreement between reviewer and clinician is recorded and feedback is provided on scanning technique. T-tests were used to assess the difference in image quality between physicians and paramedics. Inter-rater reliability between reviewers and clinicians was assessed using Cohen's kappa (κ).

RESULTS

During the study period, 5913 patients were attended by EAAA. Of these, 1097 patients had POCUS images recorded. The prevalence of POCUS during the study period was 18.6%. 1061 patient examinations underwent quality assurance (96.7%). The most common POCUS examination was echocardiography (60%), predominantly during cardiac arrest. The primary scanning clinician was a paramedic in 25.4% of POCUS examinations. Across all examination types; image quality was not significantly different between physicians and paramedics and agreement between reviewers and clinicians was strong (κ > 0.85).

CONCLUSIONS

In this service evaluation study, we have described outcomes following the introduction of a new POCUS device, image archiving system and governance framework in our HEMS. Paramedics were the primary scanning clinician in a quarter of scans, with image quality comparable to physicians. Almost all scans underwent quality assurance and inter-rater reliability was strong between clinicians and reviewers. Further research is required to investigate the diagnostic accuracy of POCUS and to demonstrate the effect of utilising prehospital POCUS to refine diagnosis on clinical outcomes.

Teaching Medical Students Rapid Ultrasound for shock and hypotension (RUSH): learning outcomes and clinical performance in a proof-of-concept study

BACKGROUND

Point-of-care ultrasound (POCUS) is a critical diagnostic tool in various medical settings, yet its instruction in medical education is inconsistent. The Rapid Ultrasound for Shock and Hypotension (RUSH) protocol is a comprehensive diagnostic tool, but its complexity poses challenges for teaching and learning. This study evaluates the effectiveness of a single-day training in RUSH for medical students by assessing their performance in clinical scenarios.

METHODS

In this prospective single-center observational proof-of-concept study, 16 medical students from Saarland University Medical Center underwent a single-day training in RUSH, followed by evaluations in clinical settings and on a high-fidelity simulator. Performance was assessed using a standardized scoring tool and time to complete the RUSH exam. Knowledge gain was measured with pre- and post-training written exams, and diagnostic performance was evaluated with an objective structured clinical examination (OSCE).

RESULTS

Students demonstrated high performance in RUSH exam views across patients (median performance: 85-87%) and improved scanning times, although not statistically significant. They performed better on simulators than on live patients. Written exam scores significantly improved post-training, suggesting a gain in theoretical knowledge. However, more than a third of students could not complete the RUSH exam within five minutes on live patients.

CONCLUSIONS

Single-day RUSH training improved medical students' theoretical knowledge and simulator performance but translating these skills to clinical settings proved challenging. The findings suggest that while short-term training can be beneficial, it may not suffice for clinical proficiency. This study underscores the need for structured and possibly longitudinal training programs to ensure skill retention and clinical applicability.