Ultrasound volume projection image quality selection by ranking from convolutional RankNet

Automatic Assessment of Ultrasound Curvature Angle for Scoliosis Detection Using 3-D Ultrasound Volume Projection Imaging

OBJECTIVE

Scoliosis is a spinal deformation in which the spine takes a lateral curvature, generating an angle in the coronal plane. The conventional method for detecting scoliosis is measurement of the Cobb angle in spine images obtained by anterior X-ray scanning. Ultrasound imaging of the spine is found to be less ionising than traditional radiographic modalities. For posterior ultrasound scanning, alternate indices of the spinous process angle (SPA) and ultrasound curve angle (UCA) were developed and have proven comparable to those of the traditional Cobb angle. In SPA, the measurements are made using the spinous processes as an anatomical reference, leading to an underestimation of the traditionally used Cobb angles. Alternatively, in UCA, more lateral features of the spine are employed for measurement of the main thoracic and thoracolumbar angles; however, clear identification of bony features is required. The current practice of UCA angle measurement is manual. This research attempts to automate the process so that the errors related to human intervention can be avoided and the scalability of ultrasound scoliosis diagnosis can be improved. The key objective is to develop an automatic scoliosis diagnosis system using 3-D ultrasound imaging.

METHODS

The novel diagnosis system is a three-step process: (i) finding the ultrasound spine image with the most visible lateral features using the convolutional RankNet algorithm; (ii) segmenting the bony features from the noisy ultrasound images using joint spine segmentation and noise removal; and (iii) calculating the UCA automatically using a newly developed centroid pairing and inscribed rectangle slope method.

RESULTS

The proposed method was evaluated on 109 patients with scoliosis of different severity. The results obtained had a good correlation with manually measured UCAs (R2=0.9784 for the main thoracic angle andR2=0.9671 for the thoracolumbar angle) and a clinically acceptable mean absolute difference of the main thoracic angle (2.82 ± 2.67°) and thoracolumbar angle (3.34 ± 2.83°).

CONCLUSION

The proposed method establishes a very promising approach for enabling the applications of economic 3-D ultrasound volume projection imaging for mass screening of scoliosis.

Si-MSPDNet: A multiscale Siamese network with parallel partial decoders for the 3-D measurement of spines in 3D ultrasonic images

Early screening and frequent monitoring effectively decrease the risk of severe scoliosis, but radiation exposure is a consequence of traditional radiograph examinations. Additionally, traditional X-ray images on the coronal or sagittal plane have difficulty providing three-dimensional (3-D) information on spinal deformities. The Scolioscan system provides an innovative 3-D spine imaging approach via ultrasonic scanning, and its feasibility has been demonstrated in numerous studies. In this paper, to further examine the potential of spinal ultrasonic data for describing 3-D spinal deformities, we propose a novel deep-learning tracker named Si-MSPDNet for extracting widely employed landmarks (spinous process (SP)) in ultrasonic images of spines and establish a 3-D spinal profile to measure 3-D spinal deformities. Si-MSPDNet has a Siamese architecture. First, we employ two efficient two-stage encoders to extract features from the uncropped ultrasonic image and the patch centered on the SP cut from the image. Then, a fusion block is designed to strengthen the communication between encoded features and further refine them from channel and spatial perspectives. The SP is a very small target in ultrasonic images, so its representation is weak in the highest-level feature maps. To overcome this, we ignore the highest-level feature maps and introduce parallel partial decoders to localize the SP. The correlation evaluation in the traditional Siamese network is also expanded to multiple scales to enhance cooperation. Furthermore, we propose a binary guided mask based on vertebral anatomical prior knowledge, which can further improve the performance of our tracker by highlighting the potential region with SP. The binary-guided mask is also utilized for fully automatic initialization in tracking. We collected spinal ultrasonic data and corresponding radiographs on the coronal and sagittal planes from 150 patients to evaluate the tracking precision of Si-MSPDNet and the performance of the generated 3-D spinal profile. Experimental results revealed that our tracker achieved a tracking success rate of 100% and a mean IoU of 0.882, outperforming some commonly used tracking and real-time detection models. Furthermore, a high correlation existed on both the coronal and sagittal planes between our projected spinal curve and that extracted from the spinal annotation in X-ray images. The correlation between the tracking results of the SP and their ground truths on other projected planes was also satisfactory. More importantly, the difference in mean curvatures was slight on all projected planes between tracking results and ground truths. Thus, this study effectively demonstrates the promising potential of our 3-D spinal profile extraction method for the 3-D measurement of spinal deformities using 3-D ultrasound data.

Automatic 3-D spine curve measurement in freehand ultrasound via structure-aware reinforcement learning spinous process localization

Freehand 3-D ultrasound systems have been advanced in scoliosis assessment to avoid radiation hazards, especially for teenagers. This novel 3-D imaging method also makes it possible to evaluate the spine curvature automatically from the corresponding 3-D projection images. However, most approaches neglect the three-dimensional spine deformity by only using the rendering images, thus limiting their usage in clinical applications. In this study, we proposed a structure-aware localization model to directly identify the spinous processes for automatic 3-D spine curve measurement using the images acquired with freehand 3-D ultrasound imaging. The pivot is to leverage a novel reinforcement learning (RL) framework to localize the landmarks, which adopts a multi-scale agent to boost structure representation with positional information. We also introduced a structure similarity prediction mechanism to perceive the targets with apparent spinous process structures. Finally, a two-fold filtering strategy was proposed to screen the detected spinous processes landmarks iteratively, followed by a three-dimensional spine curve fitting for the spine curvature assessments. We evaluated the proposed model on 3-D ultrasound images among subjects with different scoliotic angles. The results showed that the mean localization accuracy of the proposed landmark localization algorithm was 5.95 pixels. Also, the curvature angles on the coronal plane obtained by the new method had a high linear correlation with those by manual measurement (R = 0.86, p < 0.001). These results demonstrated the potential of our proposed method for facilitating the 3-D assessment of scoliosis, especially for 3-D spine deformity assessment.

Semi-automatic method for pre-surgery scoliosis classification on X-ray images using Bending Asymmetry Index

PURPOSE

Bending Asymmetry Index (BAI) has been proposed to characterize the types of scoliotic curve in three-dimensional ultrasound imaging. Scolioscan has demonstrated its validity and reliability in scoliosis assessment with manual assessment-based X-ray imaging. The objective of this study is to investigate the ultrasound-derived BAI method to X-ray imaging of scoliosis, with supplementary information provided for the pre-surgery planning.

METHODS

About 30 pre-surgery scoliosis subjects (9 males and 21 females; Cobb: 50.9 ± 19.7°, range 18°-115°) were investigated retrospectively. Each subject underwent three-posture X-ray scanning supine on a plain mattress on the same day. BAI is an indicator to distinguish structural or non-structural curves through the spine flexibility information obtained from lateral bending spinal profiles. BAI was calculated semi-automatically with manual annotation of vertebral centroids and pelvis level inclination adjustment. BAI classification was validated with the scoliotic curve type and traditional Lenke classification using side-bending Cobb angle measurement (S-Cobb).

RESULTS

82 curves from 30 pre-surgery scoliosis patients were included. The correlation coefficient was R² = 0.730 (p < 0.05) between BAI and S-Cobb. In terms of scoliotic curve type classification, all curves were correctly classified; out of 30 subjects, 1 case was confirmed as misclassified when applying to Lenke classification earlier, thus has been adjusted.

CONCLUSION

BAI method has demonstrated its inter-modality versatility in X-ray imaging application. The curve type classification and the pre-surgery Lenke classification both indicated promising performances upon the exploratory dataset. A fully-automated of BAI measurement is surely an interesting direction to continue our endeavor. Deep learning on the vertebral-level segmentation should be involved in further study.

A Review of the Methods on Cobb Angle Measurements for Spinal Curvature

Scoliosis is a common disease of the spine and requires regular monitoring due to its progressive properties. A preferred indicator to assess scoliosis is by the Cobb angle, which is currently measured either manually by the relevant medical staff or semi-automatically, aided by a computer. These methods are not only labor-intensive but also vary in precision by the inter-observer and intra-observer. Therefore, a reliable and convenient method is urgently needed. With the development of computer vision and deep learning, it is possible to automatically calculate the Cobb angles by processing X-ray or CT/MR/US images. In this paper, the research progress of Cobb angle measurement in recent years is reviewed from the perspectives of computer vision and deep learning. By comparing the measurement effects of typical methods, their advantages and disadvantages are analyzed. Finally, the key issues and their development trends are also discussed.

3D ultrasound imaging provides reliable angle measurement with validity comparable to X-ray in patients with adolescent idiopathic scoliosis

BACKGROUND & OBJECTIVE

The application of ultrasound imaging for spine evaluation could minimize radiation exposure for patients with adolescence idiopathic scoliosis (AIS). A customized three-dimensional (3D) ultrasound imaging system has been demonstrated to provide reliable and valid coronal curvature measurements. However, these measurements were using the spinous processes as anatomical reference, leading to a predictable underestimation of the traditionally used Cobb angles. An alternative 3D ultrasound image reconstruction method was applied to create coronal images with more lateral features for angle measurement. The objective of this study was to test the reliability and the validity of this angle, the ultrasound curve angle (UCA), and compare the UCA with the Cobb angles on X-ray images of patients with AIS.

MATERIALS AND METHODS

This study was divided into: 1) Investigation of intra- and inter-reliability between two raters for measuring the UCA and two operators for acquiring ultrasound images; 2) Investigation of the validity between the radiographic Cobb angle and the UCA. Fifty patients and 164 patients with AIS, were included in the two stages, respectively. Patients underwent bi-planar X-ray and 3D ultrasound scanning on the same day. The proposed UCA was used to measure the coronal curvature from the ultrasound coronal images, which were formed using a newly customized volume projection imaging (VPI) method. The intra-rater/operator and inter-rater and operator reliability of the UCA were tested by intra-class correlation coefficient (ICC) (3,1) and (2,1), respectively. The validity of UCA measurements as compared to radiographic Cobb angles was tested by inter-method ICC (2,1), mean absolute difference (MAD), standard error of measurement (SEM), Pearson correlation coefficient and Bland-Altman statistics. The level of significance was set as 0.05.

RESULTS

Excellent intra-rater and intra-operator (ICC (3,1)≥0.973) and excellent inter-rater and inter-operator reliability (ICC (2,1)≥0.925) for UCA measurement, with overall MAD and SEM no more than 3.5° and 1.7° were demonstrated for both main thoracic and (thoraco)lumbar curvatures. Very good correlations were observed between UCA and Cobb angle for main thoracic (R 2 =0.893) and (thoraco)lumbar (R 2 =0.884) curves. The mean (SD) measurements in terms of radiographic Cobb and UCA were 27.2 ​± ​11.6° and 26.3 ​± ​11.4° for main thoracic curves; and 26.2 ​± ​11.4° and 24.8 ​± ​9.7° for (thoraco)lumbar curve respectively. One hundred sixty-four subjects (33 male and 131 female subjects; 11-18 years of age, mean of 15.1 ​± ​1.9 years) were included for the validity session. Excellent inter-method variations (ICC (2,K) ≥0.933) with overall MAD and SEM no more than 3.0° and 1.5° were demonstrated for both main thoracic and (thoraco)lumbar curvatures. In addition, Bland-Altman plots demonstrated an acceptable agreement between ultrasound and radiographic Cobb measurements.

CONCLUSION

In this study, very good correlations and agreement were demonstrated between the ultrasound and X-ray measurements of the scoliotic curvature. Judging from the promising results of this study, patients with AIS with different severity of curves can be evaluated and monitored by ultrasound imaging, reducing the usage of radiation during follow-ups. This method could also be used for scoliosis screening.The Translational potential of this article: Ultrasound curve angle (UCA) obtained from 3D ultrasound imaging system can provide reliable and valid evaluation on coronal curvature for patients with AIS, without the need of radiation.