Impact of scan quality on AI assessment of hip dysplasia ultrasound

Overcoming barriers in the use of artificial intelligence in point of care ultrasound

Point-of-care ultrasound is a portable, low-cost imaging technology focused on answering specific clinical questions in real time. Artificial intelligence amplifies its capabilities by aiding clinicians in the acquisition and interpretation of the images; however, there are growing concerns on its effectiveness and trustworthiness. Here, we address key issues such as population bias, explainability and training of artificial intelligence in this field and propose approaches to ensure clinical effectiveness.

Deep learning-based automated guide for defining a standard imaging plane for developmental dysplasia of the hip screening using ultrasonography: a retrospective imaging analysis

BACKGROUND

We aimed to propose a deep-learning neural network model for automatically detecting five landmarks during a two-dimensional (2D) ultrasonography (US) scan to develop a standard plane for developmental dysplasia of the hip (DDH) screening.

METHOD

A model of global and local networks was developed to detect five landmarks for DDH screening during 2D US. Patients (N = 532) who underwent hip US for DDH screening from January 2016 to December 2021 at a tertiary medical center were enrolled. All datasets were randomly split into training, validation, and test sets in a 70:10:20 ratio for the final assessment of landmark detection. The performance of this model for detecting five landmarks for guiding DDH was analyzed using the root mean square error (RMSE) and dice similarity coefficient.

RESULTS

The RMSE value for the five landmarks for diagnosing and classifying DDH using global and local networks was 4.023 ± 3.723. The point results using EfficientNetB2 were 1.69 ± 1.26 (first point), 3.34 ± 2.37 (second point), 2.54 ± 1.61 (third point), 5.92 ± 4.25 (fourth point), and 6.61 ± 4.82 (fifth point).

CONCLUSIONS

Our deep-learning network model is feasible for detecting five landmarks for DDH using ultrasound images. The primary parameters to determine DDH will be significantly detected by applying the deep-learning model in clinical settings.

Artificial Intelligence (AI) Applications for Point of Care Ultrasound (POCUS) in Low-Resource Settings: A Scoping Review

Advancements in artificial intelligence (AI) for point-of-care ultrasound (POCUS) have ushered in new possibilities for medical diagnostics in low-resource settings. This review explores the current landscape of AI applications in POCUS across these environments, analyzing studies sourced from three databases-SCOPUS, PUBMED, and Google Scholars. Initially, 1196 records were identified, of which 1167 articles were excluded after a two-stage screening, leaving 29 unique studies for review. The majority of studies focused on deep learning algorithms to facilitate POCUS operations and interpretation in resource-constrained settings. Various types of low-resource settings were targeted, with a significant emphasis on low- and middle-income countries (LMICs), rural/remote areas, and emergency contexts. Notable limitations identified include challenges in generalizability, dataset availability, regional disparities in research, patient compliance, and ethical considerations. Additionally, the lack of standardization in POCUS devices, protocols, and algorithms emerged as a significant barrier to AI implementation. The diversity of POCUS AI applications in different domains (e.g., lung, hip, heart, etc.) illustrates the challenges of having to tailor to the specific needs of each application. By separating out the analysis by application area, researchers will better understand the distinct impacts and limitations of AI, aligning research and development efforts with the unique characteristics of each clinical condition. Despite these challenges, POCUS AI systems show promise in bridging gaps in healthcare delivery by aiding clinicians in low-resource settings. Future research endeavors should prioritize addressing the gaps identified in this review to enhance the feasibility and effectiveness of POCUS AI applications to improve healthcare outcomes in resource-constrained environments.

Artificial intelligence and point-of-care ultrasound: Benefits, limitations, and implications for the future

The utilization of artificial intelligence (AI) in medical imaging has become a rapidly growing field as a means to address contemporary demands and challenges of healthcare. Among the emerging applications of AI is point-of-care ultrasound (POCUS), in which the combination of these two technologies has garnered recent attention in research and clinical settings. In this Controversies paper, we will discuss the benefits, limitations, and future considerations of AI in POCUS for patients, clinicians, and healthcare systems.

A deep learning model adjusting for infant gender, age, height, and weight to determine whether the individual infant suit ultrasound examination of developmental dysplasia of the hip (DDH)

Objective

To examine the correlation between specific indicators and the quality of hip joint ultrasound images in infants and determine whether the individual infant suit ultrasound examination for developmental dysplasia of the hip (DDH).

Method

We retrospectively selected infants aged 0-6 months, undergone ultrasound imaging of the left hip joint between September 2021 and March 2022 at Shenzhen Children's Hospital. Using the entropy weighting method, weights were assigned to anatomical structures. Moreover, prospective data was collected from infants aged 5-11 months. The left hip joint was imaged, scored and weighted as before. The correlation between the weighted image quality scores and individual indicators were studied, with the last weighted image quality score used as the dependent variable and the individual indicators used as independent variables. A Long-short term memory (LSTM) model was used to fit the data and evaluate its effectiveness. Finally, The randomly selected images were manually measured and compared to measurements made using artificial intelligence (AI).

Results

According to the entropy weight method, the weights of each anatomical structure as follows: bony rim point 0.29, lower iliac limb point 0.41, and glenoid labrum 0.30. The final weighted score for ultrasound image quality is calculated by multiplying each score by its respective weight. Infant gender, age, height, and weight were found to be significantly correlated with the final weighted score of image quality (P < 0.05). The LSTM fitting model had a coefficient of determination (R2) of 0.95. The intra-class correlation coefficient (ICC) for the α and β angles between manual measurement and AI measurement was 0.98 and 0.93, respectively.

Conclusion

The quality of ultrasound images for infants can be influenced by the individual indicators (gender, age, height, and weight). The LSTM model showed good fitting efficiency and can help clinicians select whether the individual infant suit ultrasound examination of DDH.

Paediatric hip ultrasound

Aims

The aim of this study was to investigate the agreement in interpretation of the quality of the paediatric hip ultrasound examination, the reliability of geometric and morphological assessment, and the relationship between these measurements.

Methods

Four investigators evaluated 60 hip ultrasounds and assessed their quality based the standard plane of Graf et al. They measured geometric parameters, described the morphology of the hip, and assigned the Graf grade of dysplasia. They analyzed one self-selected image and one randomly selected image from the ultrasound series, and repeated the process four weeks later. The intra- and interobserver agreement, and correlations between various parameters were analyzed.

Results

In the assessment of quality, there a was moderate to substantial intraobserver agreement for each element investigated, but interobserver agreement was poor. Morphological features showed weak to moderate agreement across all parameters but improved to significant when responses were reduced. The geometric measurements showed nearly perfect agreement, and the relationship between them and the morphological features showed a dose response across all parameters with moderate to substantial correlations. There were strong correlations between geometric measurements. The Graf classification showed a fair to moderate interobserver agreement, and moderate to substantial intraobserver agreement.

Conclusion

This investigation into the reliability of the interpretation of hip ultrasound scans identified the difficulties in defining what is a high-quality ultrasound. We confirmed that geometric measurements are reliably interpreted and may be useful as a further measurement of quality. Morphological features are generally poorly interpreted, but a simpler binary classification considerably improves agreement. As there is a clear dose response relationship between geometric and morphological measurements, the importance of morphology in the diagnosis of hip dysplasia should be questioned.

Development of a Fully Automated Graf Standard Plane and Angle Evaluation Method for Infant Hip Ultrasound Scans

BACKGROUND

Graf's method is currently the most commonly used ultrasound-based technique for the diagnosis of developmental dysplasia of the hip (DDH). However, the efficiency and accuracy of diagnosis are highly affected by the sonographers' qualification and the time and effort expended, which has a significant intra- and inter-observer variability.

METHODS

Aiming to minimize the manual intervention in the diagnosis process, we developed a deep learning-based computer-aided framework for the DDH diagnosis, which can perform fully automated standard plane detection and angle measurement for Graf type I and type II hips. The proposed framework is composed of three modules: an anatomical structure detection module, a standard plane scoring module, and an angle measurement module. This framework can be applied to two common clinical scenarios. The first is the static mode, measurement and classification are performed directly based on the given standard plane. The second is the dynamic mode, where a standard plane from ultrasound video is first determined, and measurement and classification are then completed. To the best of our knowledge, our proposed framework is the first CAD method that can automatically perform the entire measurement process of Graf's method.

RESULTS

In our experiments, 1051 US images and 289 US videos of Graf type I and type II hips were used to evaluate the performance of the proposed framework. In static mode, the mean absolute error of α, β angles are 1.71° and 2.40°, and the classification accuracy is 94.71%. In dynamic mode, the mean absolute error of α, β angles are 1.97° and 2.53°, the classification accuracy is 89.51%, and the running speed is 31 fps.

CONCLUSIONS

Experimental results demonstrate that our fully automated framework can accurately perform standard plane detection and angle measurement of an infant's hip at a fast speed, showing great potential for clinical application.

Can AI Automatically Assess Scan Quality of Hip Ultrasound?

Ultrasound images can reliably detect Developmental Dysplasia of the Hip (DDH) during early infancy. Accuracy of diagnosis depends on the scan quality, which is subjectively assessed by the sonographer during ultrasound examination. Such assessment is prone to errors and often results in poor-quality scans not being reported, risking misdiagnosis. In this paper, we propose an Artificial Intelligence (AI) technique for automatically determining scan quality. We trained a Convolutional Neural Network (CNN) to categorize 3D Ultrasound (3DUS) hip scans as ‘adequate’ or ‘inadequate’ for diagnosis. We evaluated the performance of this AI technique on two datasets—Dataset 1 (DS1) consisting of 2187 3DUS images in which each image was assessed by one reader for scan quality on a scale of 1 (lowest quality) to 5 (optimal quality) and Dataset 2 (DS2) consisting of 107 3DUS images evaluated semi-quantitatively by four readers using a 10-point scoring system. As a binary classifier (adequate/inadequate), the AI technique gave highly accurate predictions on both datasets (DS1 accuracy = 96% and DS2 accuracy = 91%) and showed high agreement with expert readings in terms of Intraclass Correlation Coefficient (ICC) and Cohen’s kappa coefficient (K). Using our AI-based approach as a screening tool during ultrasound scanning or postprocessing would ensure high scan quality and lead to more reliable ultrasound hip examination in infants.

Interobserver Variability of Hip Dysplasia Indices on Sweep Ultrasound for Novices, Experts, and Artificial Intelligence

BACKGROUND

Ultrasound for developmental dysplasia of the hip (DDH) is challenging for nonexperts to perform and interpret. Recording "sweep" images allows more complete hip assessment, suitable for automation by artificial intelligence (AI), but reliability has not been established. We assessed agreement between readers of varying experience and a commercial AI algorithm, in DDH detection from infant hip ultrasound sweeps.

METHODS

We selected a full spectrum of poor-to-excellent quality images and normal to severe dysplasia, in 240 hips (120 single 2-dimensional images, 120 sweeps). For 12 readers (radiologists, sonographers, clinicians and researchers; 3 were DDH subspecialists), and a ultrasound-FDA-cleared AI software package (Medo Hip), we calculated interobserver reliability for alpha angle measurements by intraclass correlation coefficient (ICC2,1) and for DDH classification by Randolph Kappa.

RESULTS

Alpha angle reliability was high for AI versus subspecialists (ICC=0.87 for sweeps, 0.90 for single images). For DDH diagnosis from sweeps, agreement was high between subspecialists (kappa=0.72), and moderate for nonsubspecialists (0.54) and AI (0.47). Agreement was higher for single images (kappa=0.80, 0.66, 0.49). AI reliability deteriorated more than human readers for the poorest-quality images. The agreement of radiologists and clinicians with the accepted standard, while still high, was significantly poorer for sweeps than 2D images (P<0.05).

CONCLUSIONS

In a challenging exercise representing the wide spectrum of image quality and reader experience seen in real-world hip ultrasound, agreement on DDH diagnosis from easily obtained sweeps was only slightly lower than from single images, likely because of the additional step of selecting the best image. AI performed similarly to a nonsubspecialist human reader but was more affected by low-quality images.

Artificial Intelligence to Automatically Assess Scan Quality in Hip Ultrasound

PURPOSE

Since it is fast, inexpensive and increasingly portable, ultrasound can be used for early detection of Developmental Dysplasia of the Hip (DDH) in infants at point-of-care. However, accurate interpretation\is highly dependent on scan quality. Poor-quality images lead to misdiagnosis, but inexperienced users may not even recognize the deficiencies in the images. Currently, users assess scan quality subjectively, based on image landmarks which are prone to human errors. Instead, we propose using Artificial Intelligence (AI) to automatically assess scan quality.

METHODS

We trained separate Convolutional Neural Network (CNN) models to detect presence of each of four commonly used ultrasound landmarks in each hip image: straight horizontal iliac wing, labrum, os ischium and midportion of the femoral head. We used 100 3D ultrasound (3DUS) images for training and validated the technique on a set of 107 3DUS images also scored for landmarks by three non-expert readers and one expert radiologist.

RESULTS

We got AI ≥ 85% accuracy for all four landmarks (ilium = 0.89, labrum = 0.94, os ischium = 0.85, femoral head = 0.98) as a binary classifier between adequate and inadequate scan quality. Our technique also showed excellent agreement with manual assessment in terms of Intraclass Correlation Coefficient (ICC) and Cohen's kappa coefficient (K) for ilium (ICC = 0.81, K = 0.56), os ischium (ICC = 0.89, K = 0.63) and femoral head (ICC = 0.83, K = 0.66), and moderate to good agreement for labrum (ICC = 0.65, K = 0.33).

CONCLUSION

This new technique could ensure high scan quality and facilitate more widespread use of ultrasound in population screening of DDH.