Automatic Measurement of Spine Curvature on 3-D Ultrasound Volume Projection Image With Phase Features

ICPPNet: A semantic segmentation network model based on inter-class positional prior for scoliosis reconstruction in ultrasound images

OBJECTIVE

Considering the radiation hazard of X-ray, safer, more convenient and cost-effective ultrasound methods are gradually becoming new diagnostic approaches for scoliosis. For ultrasound images of spine regions, it is challenging to accurately identify spine regions in images due to relatively small target areas and the presence of a lot of interfering information. Therefore, we developed a novel neural network that incorporates prior knowledge to precisely segment spine regions in ultrasound images.

MATERIALS AND METHODS

We constructed a dataset of ultrasound images of spine regions for semantic segmentation. The dataset contains 3,136 images of 30 patients with scoliosis. And we propose a network model (ICPPNet), which fully utilizes inter-class positional prior knowledge by combining an inter-class positional probability heatmap, to achieve accurate segmentation of target areas.

RESULTS

ICPPNet achieved an average Dice similarity coefficient of 70.83% and an average 95% Hausdorff distance of 11.28 mm on the dataset, demonstrating its excellent performance. The average error between the Cobb angle measured by our method and the Cobb angle measured by X-ray images is 1.41 degrees, and the coefficient of determination is 0.9879 with a strong correlation.

DISCUSSION AND CONCLUSION

ICPPNet provides a new solution for the medical image segmentation task with positional prior knowledge between target classes. And ICPPNet strongly supports the subsequent reconstruction of spine models using ultrasound images.

Three-dimensional (3D) ultrasound imaging for quantitative assessment of frontal cobb angles in patients with idiopathic scoliosis - a systematic review and meta-analysis

BACKGROUND

Measurement of Cobb angle in the frontal plane from radiographs is the gold standard of quantifying spinal deformity in adolescent idiopathic scoliosis (AIS). As a radiation free alternative, ultrasonography (USG) for quantitative measurement of frontal cobb angles has been reported. However, a systematic review and meta-analysis on the reliability of ultrasound comparing with the gold standard have not yet been reported.

OBJECTIVES

This systematic review and meta-analysis aimed to evaluate (1) the reliability of ultrasound imaging compared with radiographs in measuring frontal cobb angle for screening or monitoring in AIS patients; (2) whether the performance of USG differ when using different anatomical landmarks for measurement of frontal cobb angles.

METHODS

Systematic search was performed on MEDLINE, EMBASE, CINAHL, and CENTRAL databases for relevant studies. QUADAS-2 was adopted for quality assessment. The intra- and inter-rater reliability of ultrasound measurement in terms of intra-class correlation coefficient (ICC) was recorded. Mean Absolute Difference (MAD) and Pearson correlation coefficients between frontal cobb angle measured from USG and radiographic measurements, were extracted with meta-analysis performed.

RESULTS AND DISCUSSION

Nineteen studies were included with a total of 2318 patients. The risk of bias of included studies were unclear or high. Pooled MAD of frontal cobb angle measured between USG and radiography was 4.02 degrees (95% CI: 3.28-4.76) with a pooled correlation coefficient of 0.91 (95% CI: 0.87-0.93). Subgroup analyses show that pooled correlation was > 0.87 across using various USG landmarks for measurement of frontal cobb angles. There was a high level of heterogeneity between results of the included studies with I2 > 90%. Potential sources of heterogeneity include curve severity, curve types, location of apex, scanning postures, patient demographics, equipment, and operator experience. Despite being the "gold standard", intrinsic errors in quantifying spinal deformities with radiographs may also be a source of inconsistency.

CONCLUSION

The current systematic review indicated that there is evidence in favor of using USG for quantitative evaluation of frontal cobb angle in AIS. However, the quality of evidence is low due to high risk of bias and heterogeneity between existing studies. Current literature is insufficient to support the use of USG as a screening and/or follow-up method for AIS. Further investigation addressing the limitations identified in this review is required before USG could be adapted for further clinical use.

A topological-aware automatic grading model corneal epithelial damage evaluation from full Corneal Fluorescence Staining images

Corneal Fluorescence Staining (CFS) imaging is commonly employed for assessing corneal epithelial damage. Automating the grading of CFS images can minimize subjectivity in clinical evaluations and enhance diagnostic efficiency. Existing methods typically depend on the texture and morphological information extracted from whole CFS images, often neglecting the spatial and distribution information between stained regions. This oversight hinders the accurate evaluation of corneal epithelial injury states. This study proposes a three-stage automatic corneal epithelial damage assessment model for full CFS images, optimizing grading by considering topological features among detected stained regions, which are crucial for accurately interpreting the spatial properties of objects within an image. Accurate corneal localization, robust to variations in contrast, is first achieved by integrating CFS images' intensity and phase information, subsequently by a multi-scale morphological top-hat operator concerning their prior shape to detect the stained regions. Finally, a multi-scale graph structure is constructed based on the detected stained areas, and distance-weighted topological features, along with textural and morphological features, are extracted into an automatic grading model based on an ensemble model. Experiments on an in-house dataset of CFS images annotated with six categories of Ocular Surface Staining (OSS) scores reveal that incorporating topological features achieves the highest Accuracy (0.7589), F1 score (0.7449), and AUC (0.9335). Moreover, topological features outperformed other individual features. These findings underscore the effectiveness of our proposed model in CFS grading, indicating its potential for assessing corneal epithelial damage. Additionally, the valuable insights provided by topological features into the spatial distribution patterns of staining suggest promising applications for enhancing disease classification and supporting more informed clinical decision-making in managing dry eye conditions.

Deep Learning Method for Precise Landmark Identification and Structural Assessment of Whole-Spine Radiographs

This study measured parameters automatically by marking the point for measuring each parameter on whole-spine radiographs. Between January 2020 and December 2021, 1017 sequential lateral whole-spine radiographs were retrospectively obtained. Of these, 819 and 198 were used for training and testing the performance of the landmark detection model, respectively. To objectively evaluate the program's performance, 690 whole-spine radiographs from four other institutions were used for external validation. The combined dataset comprised radiographs from 857 female and 850 male patients (average age 42.2 ± 27.3 years; range 20-85 years). The landmark localizer showed the highest accuracy in identifying cervical landmarks (median error 1.5-2.4 mm), followed by lumbosacral landmarks (median error 2.1-3.0 mm). However, thoracic landmarks displayed larger localization errors (median 2.4-4.3 mm), indicating slightly reduced precision compared with the cervical and lumbosacral regions. The agreement between the deep learning model and two experts was good to excellent, with intraclass correlation coefficient values >0.88. The deep learning model also performed well on the external validation set. There were no statistical differences between datasets in all parameters, suggesting that the performance of the artificial intelligence model created was excellent. The proposed automatic alignment analysis system identified anatomical landmarks and positions of the spine with high precision and generated various radiograph imaging parameters that had a good correlation with manual measurements.

Automatic Assessment of Ultrasound Curvature Angle for Scoliosis Detection Using 3-D Ultrasound Volume Projection Imaging

OBJECTIVE

Scoliosis is a spinal deformation in which the spine takes a lateral curvature, generating an angle in the coronal plane. The conventional method for detecting scoliosis is measurement of the Cobb angle in spine images obtained by anterior X-ray scanning. Ultrasound imaging of the spine is found to be less ionising than traditional radiographic modalities. For posterior ultrasound scanning, alternate indices of the spinous process angle (SPA) and ultrasound curve angle (UCA) were developed and have proven comparable to those of the traditional Cobb angle. In SPA, the measurements are made using the spinous processes as an anatomical reference, leading to an underestimation of the traditionally used Cobb angles. Alternatively, in UCA, more lateral features of the spine are employed for measurement of the main thoracic and thoracolumbar angles; however, clear identification of bony features is required. The current practice of UCA angle measurement is manual. This research attempts to automate the process so that the errors related to human intervention can be avoided and the scalability of ultrasound scoliosis diagnosis can be improved. The key objective is to develop an automatic scoliosis diagnosis system using 3-D ultrasound imaging.

METHODS

The novel diagnosis system is a three-step process: (i) finding the ultrasound spine image with the most visible lateral features using the convolutional RankNet algorithm; (ii) segmenting the bony features from the noisy ultrasound images using joint spine segmentation and noise removal; and (iii) calculating the UCA automatically using a newly developed centroid pairing and inscribed rectangle slope method.

RESULTS

The proposed method was evaluated on 109 patients with scoliosis of different severity. The results obtained had a good correlation with manually measured UCAs (R2=0.9784 for the main thoracic angle andR2=0.9671 for the thoracolumbar angle) and a clinically acceptable mean absolute difference of the main thoracic angle (2.82 ± 2.67°) and thoracolumbar angle (3.34 ± 2.83°).

CONCLUSION

The proposed method establishes a very promising approach for enabling the applications of economic 3-D ultrasound volume projection imaging for mass screening of scoliosis.

Anatomical Prior and Inter-Slice Consistency for Semi-Supervised Vertebral Structure Detection in 3D Ultrasound Volume

Three-dimensional (3D) ultrasound imaging technique has been applied for scoliosis assessment, but the current assessment method only uses coronal projection images and cannot illustrate the 3D deformity and vertebra rotation. The vertebra detection is essential to reveal 3D spine information, but the detection task is challenging due to complex data and limited annotations. We propose VertMatch to detect vertebral structures in 3D ultrasound volume containing a detector and classifier. The detector network finds the potential positions of structures on transverse slice globally, and then the local patches are cropped based on detected positions. The classifier is used to distinguish whether the patches contain real vertebral structures and screen the predicted positions from the detector. VertMatch utilizes unlabeled data in a semi-supervised manner, and we develop two novel techniques for semi-supervised learning: 1) anatomical prior is used to acquire high-quality pseudo labels; 2) inter-slice consistency is used to utilize more unlabeled data by inputting multiple adjacent slices. Experimental results demonstrate that VertMatch can detect vertebra accurately in ultrasound volume and outperforms state-of-the-art methods. Moreover, VertMatch is also validated in automatic spinous process angle measurement on forty subjects with scoliosis, and the results illustrate that it can be a promising approach for the 3D assessment of scoliosis.

Development of Automatic Assessment Framework for Spine Deformity Using Freehand 3-D Ultrasound Imaging System

A 3-D ultrasound (US) imaging technique has been studied to facilitate the diagnosis of spinal deformity without radiation. The objective of this article is to propose an assessment framework to automatically estimate spinal deformity in US spine images. The proposed framework comprises four major components, a US spine image generator, a novel transformer-based lightweight spine detector network, an angle evaluator, and a 3-D modeler. The principal component analysis (PCA) and discriminative scale space tracking (DSST) method are first adopted to generate the US spine images. The proposed detector is equipped with a redundancy queries removal (RQR) module and a regularization item to realize accurate and unique detection of spine images. Two clinical datasets, a total of 273 images from adolescents with idiopathic scoliosis, are used for the investigation of the proposed framework. The curvature is estimated by the angle evaluator, and the 3-D mesh model is established by the parametric modeling technique. The accuracy rate (AR) of the proposed detector can be achieved at 99.5%, with a minimal redundancy rate (RR) of 1.5%. The correlations between automatic curve measurements on US spine images from two datasets and manual measurements on radiographs are 0.91 and 0.88, respectively. The mean absolute difference (MAD) and standard deviation (SD) are 2.72° ± 2.14° and 2.91° ± 2.36° , respectively. The results demonstrate the effectiveness of the proposed framework to advance the application of the 3-D US imaging technique in clinical practice for scoliosis mass screening and monitoring.

A novel method for spine ultrasound and X-ray radiograph registration

Ultrasound is a promising imaging method for scoliosis evaluation because it is radiation free and provide real-time images. However, it cannot provide bony details because ultrasound cannot penetrate the bony structure. Therefore, registration of real-time ultrasound images with the previous X-ray radiograph can help physicians understand the spinal deformity of patients. In this study, an improved free-from deformation registration method based on mutual registration and hierarchical adaptive grid (MRHA-FFD) was developed. The method first performed registration grid preprocessing and then optimized control points and conducted mutual registration. Finally, a Blur-aware Attention Network was adopted for image deblurring. The performance of each step was verified by ablation experiments. Comparison experiment between the proposed method and traditional registration methods was also conducted. The qualitative and quantitative results suggested that MRHA-FFD is a promising approach for registering spine ultrasound image and X-ray radiograph.

Automatic Myotendinous Junction Identification in Ultrasound Images Based on Junction-Based Template Measurements

Tracking the myotendinous junction (MTJ) motion in consecutive ultrasound images is essential to assess muscle and tendon interaction and understand the mechanics' muscle-tendon unit and its pathological conditions during motion. However, the inherent speckle noises and ambiguous boundaries deter the reliable identification of MTJ, thus restricting their usage in human motion analysis. This study advances a fully automatic displacement measurement method for MTJ using prior shape knowledge on the Y-shape MTJ, precluding the influence of irregular and complicated hyperechoic structures in muscular ultrasound images. Our proposed method first adopts the junction candidate points using a combined measure of Hessian matrix and phase congruency, followed by a hierarchical clustering technique to refine the candidates approximating the position of the MTJ. Then, based on the prior knowledge of Y-shape MTJ, we finally identify the best matching junction points according to intensity distributions and directions of their branches using multiscale Gaussian templates and a Kalman filter. We evaluated our proposed method using the ultrasound scans of the gastrocnemius from 8 young, healthy volunteers. Our results present more consistent with the manual method in the MTJ tracking method than existing optical flow tracking methods, suggesting its potential in facilitating muscle and tendon function examinations with in vivo ultrasound imaging.

Ultrasound to X-ray synthesis generative attentional network (UXGAN) for adolescent idiopathic scoliosis

Standing X-ray radiograph with Cobb's method is the gold standard for scoliosis diagnosis. However, radiation hazard restricts its application, especially for close follow-up of adolescent patients. Compared with X-ray, ultrasound imaging has advantages of being radiation-free and real-time. To combine advantages of the above two imaging modalities, an ultrasound to X-ray synthesis generative attentional network (UXGAN) was proposed to synthesize ultrasound images into X-ray-like images. In this network, a cyclically consistent network was adopted and was trained end-to-end. An attention module was added and different residual blocks were designed. The quantitative comparison results demonstrated the superiority of our method to the state-of-the-art CycleGAN methods. We further compared the Cobb angle values measured on synthesized images and the real X-ray images, respectively. A good linear correlation (r = 0.95) was demonstrated between the two methods. The above results proved that the proposed method is of great significance for providing both X-ray images and ultrasound images based on the radiation-free ultrasound scanning.

Semi-automatic method for pre-surgery scoliosis classification on X-ray images using Bending Asymmetry Index

PURPOSE

Bending Asymmetry Index (BAI) has been proposed to characterize the types of scoliotic curve in three-dimensional ultrasound imaging. Scolioscan has demonstrated its validity and reliability in scoliosis assessment with manual assessment-based X-ray imaging. The objective of this study is to investigate the ultrasound-derived BAI method to X-ray imaging of scoliosis, with supplementary information provided for the pre-surgery planning.

METHODS

About 30 pre-surgery scoliosis subjects (9 males and 21 females; Cobb: 50.9 ± 19.7°, range 18°-115°) were investigated retrospectively. Each subject underwent three-posture X-ray scanning supine on a plain mattress on the same day. BAI is an indicator to distinguish structural or non-structural curves through the spine flexibility information obtained from lateral bending spinal profiles. BAI was calculated semi-automatically with manual annotation of vertebral centroids and pelvis level inclination adjustment. BAI classification was validated with the scoliotic curve type and traditional Lenke classification using side-bending Cobb angle measurement (S-Cobb).

RESULTS

82 curves from 30 pre-surgery scoliosis patients were included. The correlation coefficient was R² = 0.730 (p < 0.05) between BAI and S-Cobb. In terms of scoliotic curve type classification, all curves were correctly classified; out of 30 subjects, 1 case was confirmed as misclassified when applying to Lenke classification earlier, thus has been adjusted.

CONCLUSION

BAI method has demonstrated its inter-modality versatility in X-ray imaging application. The curve type classification and the pre-surgery Lenke classification both indicated promising performances upon the exploratory dataset. A fully-automated of BAI measurement is surely an interesting direction to continue our endeavor. Deep learning on the vertebral-level segmentation should be involved in further study.

Automatic recognition of whole-spine sagittal alignment and curvature analysis through a deep learning technique

PURPOSE

Artificial intelligence based on deep learning (DL) approaches enables the automatic recognition of anatomic landmarks and subsequent estimation of various spinopelvic parameters. The locations of inflection points (IPs) and apices (APs) in whole-spine lateral radiographs could be mathematically determined by a fully automatic spinal sagittal curvature analysis system.

METHODS

We developed a DL model for automatic spinal curvature analysis of whole-spine lateral plain radiographs by using 1800 annotated images of various spinal disease etiologies. The DL model comprised a landmark localizer to detect 25 vertebral landmarks and a numerical algorithm for the generation of an individualized spinal sagittal curvature. The characteristics of the spinal curvature, including the IPs, APs, and curvature angle, could thus be analyzed using mathematical definitions. The localization error of each landmark was calculated from the predictions of 300 test images to evaluate the performance of the landmark localizer. The interrater reliability among a senior orthopedic surgeon, a radiologist, and the DL model was assessed using the intraclass correlation coefficient (ICC).

RESULTS

The accuracy of the landmark localizer was within an acceptable range (median error: 1.7-4.1 mm), and the interrater reliabilities between the proposed DL model and each expert were good to excellent (all ICCs > 0.85) for the measurement of spinal curvature characteristics.

CONCLUSION

The interrater reliability between the proposed DL model and human experts was good to excellent in predicting the locations of IPs, APs, and curvature angles. Future applications should be explored to validate this system and improve its clinical efficiency.

Joint Spine Segmentation and Noise Removal From Ultrasound Volume Projection Images With Selective Feature Sharing

Volume Projection Imaging from ultrasound data is a promising technique to visualize spine features and diagnose Adolescent Idiopathic Scoliosis. In this paper, we present a novel multi-task framework to reduce the scan noise in volume projection images and to segment different spine features simultaneously, which provides an appealing alternative for intelligent scoliosis assessment in clinical applications. Our proposed framework consists of two streams: i) A noise removal stream based on generative adversarial networks, which aims to achieve effective scan noise removal in a weakly-supervised manner, i.e., without paired noisy-clean samples for learning; ii) A spine segmentation stream, which aims to predict accurate bone masks. To establish the interaction between these two tasks, we propose a selective feature-sharing strategy to transfer only the beneficial features, while filtering out the useless or harmful information. We evaluate our proposed framework on both scan noise removal and spine segmentation tasks. The experimental results demonstrate that our proposed method achieves promising performance on both tasks, which provides an appealing approach to facilitating clinical diagnosis.

Emerging Feature Extraction Techniques for Machine Learning-Based Classification of Carotid Artery Ultrasound Images

Plaque deposits in the carotid artery are the major cause of stroke and atherosclerosis. Ultrasound imaging is used as an early indicator of disease progression. Classification of the images to identify plaque presence and intima-media thickness (IMT) by machine learning algorithms requires features extracted from the images. A total of 361 images were used for feature extraction, which will assist in further classification of the carotid artery. This study presents the extraction of 65 features, which constitute of shape, texture, histogram, correlogram, and morphology features. Principal component analysis (PCA)-based feature selection is performed, and the 22 most significant features, which will improve the classification accuracy, are selected. Naive Bayes algorithm and dynamic learning vector quantization (DLVQ)-based machine learning classifications are performed with the extracted and selected features, and analysis is performed.

A Review of the Methods on Cobb Angle Measurements for Spinal Curvature

Scoliosis is a common disease of the spine and requires regular monitoring due to its progressive properties. A preferred indicator to assess scoliosis is by the Cobb angle, which is currently measured either manually by the relevant medical staff or semi-automatically, aided by a computer. These methods are not only labor-intensive but also vary in precision by the inter-observer and intra-observer. Therefore, a reliable and convenient method is urgently needed. With the development of computer vision and deep learning, it is possible to automatically calculate the Cobb angles by processing X-ray or CT/MR/US images. In this paper, the research progress of Cobb angle measurement in recent years is reviewed from the perspectives of computer vision and deep learning. By comparing the measurement effects of typical methods, their advantages and disadvantages are analyzed. Finally, the key issues and their development trends are also discussed.

Semi-automatic ultrasound curve angle measurement for adolescent idiopathic scoliosis

PURPOSE

Using X-ray to evaluate adolescent idiopathic scoliosis (AIS) conditions is the clinical gold standard, with potential radiation hazards. 3D ultrasound has demonstrated its validity and reliability of estimating X-ray Cobb angle (XCA) using spinous process angle (SPA), which can be automatically measured. While angle measurement with ultrasound using spine transverse process-related landmarks (UCA) shows better agreed with XCA, its automatic measurement is challenging and not available yet. This research aimed to analyze and measure scoliotic angles through a novel semi-automatic UCA method.

METHODS

100 AIS subjects (age: 15.0 ± 1.9 years, gender: 19 M and 81 F, Cobb: 25.5 ± 9.6°) underwent both 3D ultrasound and X-ray scanning on the same day. Scoliotic angles with XCA and UCA methods were measured manually; and transverse process-related features were identified/drawn for the semi-automatic UCA method. The semi-automatic method measured the spinal curvature with pairs of thoracic transverse processes and lumbar lumps in respective regions.

RESULTS

The new semi-automatic UCA method showed excellent correlations with manual XCA (R² = 0.815: thoracic angles R² = 0.857, lumbar angles R² = 0.787); and excellent correlations with manual UCA (R² = 0.866: thoracic angles R² = 0.921, lumbar angles R² = 0.780). The Bland-Altman plot also showed a good agreement against manual UCA/XCA. The MADs of semi-automatic UCA against XCA were less than 5°, which is clinically insignificant.

CONCLUSION

The semi-automatic UCA method had demonstrated the possibilities of estimating manual XCA and UCA. Further advancement in image processing to detect the vertebral landmarks in ultrasound images could help building a fully automated measurement method.

LEVEL OF EVIDENCE

Level III.

A Survey of Methods and Technologies Used for Diagnosis of Scoliosis

The purpose of this article is to present diagnostic methods used in the diagnosis of scoliosis in the form of a brief review. This article aims to point out the advantages of select methods. This article focuses on general issues without elaborating on problems strictly related to physiotherapy and treatment methods, which may be the subject of further discussions. By outlining and categorizing each method, we summarize relevant publications that may not only help introduce other researchers to the field but also be a valuable source for studying existing methods, developing new ones or choosing evaluation strategies.

Light-Convolution Dense Selection U-Net (LDS U-Net) for Ultrasound Lateral Bony Feature Segmentation

Scoliosis is a widespread medical condition where the spine becomes severely deformed and bends over time. It mostly affects young adults and may have a permanent impact on them. A periodic assessment, using a suitable modality, is necessary for its early detection. Conventionally, the usually employed modalities include X-ray and MRI, which employ ionising radiation and are expensive. Hence, a non-radiating 3D ultrasound imaging technique has been developed as a safe and economic alternative. However, ultrasound produces low-contrast images that are full of speckle noise, and skilled intervention is necessary for their processing. Given the prevalent occurrence of scoliosis and the limitations of scalability of human expert interventions, an automatic, fast, and low-computation assessment technique is being developed for mass scoliosis diagnosis. In this paper, a novel hybridized light-weight convolutional neural network architecture is presented for automatic lateral bony feature identification, which can help to develop a fully-fledged automatic scoliosis detection system. The proposed architecture, Light-convolution Dense Selection U-Net (LDS U-Net), can accurately segment ultrasound spine lateral bony features, from noisy images, thanks to its capabilities of smartly selecting only the useful information and extracting rich deep layer features from the input image. The proposed model is tested using a dataset of 109 spine ultrasound images. The segmentation result of the proposed network is compared with basic U-Net, Attention U-Net, and MultiResUNet using various popular segmentation indices. The results show that LDS U-Net provides a better segmentation performance compared to the other models. Additionally, LDS U-Net requires a smaller number of parameters and less memory, making it suitable for a large-batch screening process of scoliosis without a high computational requirement.

Automatic Detection and Measurement of Spinous Process Curve on Clinical Ultrasound Spine Images

The ultrasound (US) imaging technique has been applied to scoliosis assessment, and the proxy Cobb angle can be acquired on the US coronal images. The spinous process angle (SPA) is a valuable parameter to indicate 3-D deformity of spine. However, the SPA cannot be measured on US images since the spinous process (SP) is merged in the soft tissue layer and impossible to be identified on the coronal view directly. A new method based on the gradient vector flow (GVF) snake model was proposed to automatically locate SP position on the US transverse images, and the density-based spatial clustering of application with noise (DBSCAN) was used to remove the outliers out of the detected location results. With marking the SP points on the US coronal image, the SP curve was interpolated and the SPA was measured. The algorithm was evaluated on 50 subjects with various severity of scoliosis, and two raters measured the SPA on both US images and radiographs manually. The mean absolute differences (MADs) of SPAs obtained from the two modalities were 3.4° ± 2.4° and 3.6° ± 2.8° for the two raters, respectively, which were less than the clinical acceptance error (5°), and the results reported a good linear correlation ( ) between the US method and radiography. It indicates that the proposed method can be a promising approach for SPA measurement using the US imaging technique.

Dual-task ultrasound spine transverse vertebrae segmentation network with contour regularization

3D ultrasound imaging has become one of the common diagnosis ways to assess scoliosis since it is radiation-free, real-time, and low-cost. Spine curvature angle measurement is an important step to assess scoliosis precisely. One way to calculate the angle is using the vertebrae features of the 2-D coronal images to identify the most tilted vertebrae. To do the measurement, the segmentation of the transverse vertebrae is an important step. In this paper, we propose a dual-task ultrasound transverse vertebrae segmentation network (D-TVNet) based on U-Net. First, we arrange an auxiliary shape regularization network to learn the contour segmentation of the bones. It improves the boundary segmentation and anti-interference ability of the U-Net by fusing some of the features of the auxiliary task and the main task. Then, we introduce the atrous spatial pyramid pooling (ASPP) module to the end of the down-sampling stage of the main task stream to improve the relative feature extraction ability. To further improve the boundary segmentation, we extendedly fuse the down-sampling output features of the auxiliary network in the ASPP. The experiment results show that the proposed D-TVNet achieves the best dice score of 86.68% and the mean dice score of 86.17% based on cross-validation, which is an improvement of 5.17% over the baseline U-Net. An automatic ultrasound spine bone segmentation network with promising results has been achieved.

External Power-Driven Microrobotic Swarm: From Fundamental Understanding to Imaging-Guided Delivery

Untethered micro/nanorobots have been widely investigated owing to their potential in performing various tasks in different environments. The significant progress in this emerging interdisciplinary field has benefited from the distinctive features of those tiny active agents, such as wireless actuation, navigation under feedback control, and targeted delivery of small-scale objects. In recent studies, collective behaviors of these tiny machines have received tremendous attention because swarming agents can enhance the delivery capability and adaptability in complex environments and the contrast of medical imaging, thus benefiting the imaging-guided navigation and delivery. In this review, we summarize the recent research efforts on investigating collective behaviors of external power-driven micro/nanorobots, including the fundamental understanding of swarm formation, navigation, and pattern transformation. The fundamental understanding of swarming tiny machines provides the foundation for targeted delivery. We also summarize the swarm localization using different imaging techniques, including the imaging-guided delivery in biological environments. By highlighting the critical steps from understanding the fundamental interactions during swarm control to swarm localization and imaging-guided delivery applications, we envision that the microrobotic swarm provides a promising tool for delivering agents in an active, controlled manner.

Ultrasound volume projection image quality selection by ranking from convolutional RankNet

Periodic inspection and assessment are important for scoliosis patients. 3D ultrasound imaging has become an important means of scoliosis assessment as it is a real-time, cost-effective and radiation-free imaging technique. With the generation of a 3D ultrasound volume projection spine image using our Scolioscan system, a series of 2D coronal ultrasound images are produced at different depths with different qualities. Selecting a high quality image from these 2D images is the crucial task for further scoliosis measurement. However, adjacent images are similar and difficult to distinguish. To learn the nuances between these images, we propose selecting the best image automatically, based on their quality rankings. Here, the ranking algorithm we use is a pairwise learning-to-ranking network, RankNet. Then, to extract more efficient features of input images and to improve the discriminative ability of the model, we adopt the convolutional neural network as the backbone due to its high power of image exploration. Finally, by inputting the images in pairs into the proposed convolutional RankNet, we can select the best images from each case based on the output ranking orders. The experimental result shows that convolutional RankNet achieves better than 95.5% top-3 accuracy, and we prove that this performance is beyond the experience of a human expert.

Automatic Measurement of Spinous Process Angles on Ultrasound Spine Images

Ultrasound (US) imaging technique has been applied to measure the proxy Cobb angle and spinous process angle (SPA) for spinal curvatures of scoliosis. However manual measurement of ultrasound images is time consuming and greatly relying on the experience of raters. The objectives of this work are to develop an automatic measurement method to assess SPA of spine curves and to evaluate the accuracy and reliability of the method. The spinous process curves were identified and fitted on US images, and the automatically measured SPA were compared with the results from US manual and radiographic measurements. It illustrates that the US-auto measurement of SPA presents higher correlation and smaller difference with clinical standard radiographic results than the US-manual measurement.

3D Ultrasound Spine Image Selection Using Convolution Learning-to-Rank Algorithm

3D Ultrasound imaging has become an important means of scoliosis assessment as it is a real-time, cost-effective and radiation-free imaging technique. However, the coronal images from different depths of a 3D ultrasound image have different imaging definitions. So there is a need to select the coronal image that would give the best image definition. Also, manual selection of coronal images is time-consuming and limited to the discretion and capability of the assessor. Therefore, in this paper, we have developed a convolution learning-to-rank algorithm to select the best ultrasound images automatically using raw ultrasound images. The ranking is done based on the curve angle of the spinal cord. Firstly, we approached the image selection problem as a ranking problem; ranked based on probability of an image to be a good image. Here, we use the RankNet, a pairwise learning-to-rank method, to rank the images automatically. Secondly, we replaced the backbone of the RankNet, which is the traditional artificial neural network (ANN), with convolution neural network (CNN) to improve the feature extracting ability for the successive iterations. The experimental result shows that the proposed convolutional RankNet achieves the perfect accuracy of 100% while conventional DenseNet achieved 35% only. This proves that the convolutional RankNet is more suitable to highlight the best quality of ultrasound image from multiple mediocre ones.

Automating Spine Curvature Measurement in Volumetric Ultrasound via Adaptive Phase Features

Ultrasound volume projection imaging (VPI) has been recently suggested. This novel imaging method allows a non-radiation assessment of spine deformity with free-hand 3-D ultrasound imaging. This paper presents a fully automatic method to evaluate the spine curve in VPI images corresponding to different projection depth of the volumetric ultrasound, thus making it possible to analyze 3-D spine deformity. The new automatic method is based on prior knowledge about the geometric arrangement of the spinous processes. The frequency bandwidth of log-Gabor filters is adaptively adjusted to calculate the oriented phase congruency, facilitating the segmentation of the spinous column profile. And the spine curvature angle is finally calculated according to the inflection points of the curve over the segmented spinous column profile. The performance of the automatic method is evaluated on spine VPI images among patients with different scoliotic angles. The curvature angles obtained using the proposed method have a high linear correlation with those by the manual method (r = 0.90, p < 0.001) and X-ray Cobb's method (r = 0.87, p < 0.001). The feasibility of 3-D spine deformity assessment is also demonstrated using VPI images corresponding to various projection depth. The results suggest that this method can substantially improve the recognition of the spinous column profile, especially facilitating the applications of 3-D spine deformity assessment.

Spinal Cord Segmentation in Ultrasound Medical Imagery

In this paper, we study and evaluate the task of semantic segmentation of the spinal cord in ultrasound medical imagery. This task is useful for neurosurgeons to analyze the spinal cord movement during and after the laminectomy surgical operation. Laminectomy is performed on patients that suffer from an abnormal pressure made on the spinal cord. The surgeon operates by cutting the bones of the laminae and the intervening ligaments to relieve this pressure. During the surgery, ultrasound waves can pass through the laminectomy area to give real-time exploitable images of the spinal cord. The surgeon uses them to confirm spinal cord decompression or, occasionally, to assess a tumor adjacent to the spinal cord. The Freely pulsating spinal cord is a sign of adequate decompression. To evaluate the semantic segmentation approaches chosen in this study, we constructed two datasets using images collected from 10 different patients performing the laminectomy surgery. We found that the best solution for this task is Fully Convolutional DenseNets if the spinal cord is already in the train set. If the spinal cord does not exist in the train set, U-Net is the best. We also studied the effect of integrating inside both models some deep learning components like Atrous Spatial Pyramid Pooling (ASPP) and Depthwise Separable Convolution (DSC). We added a post-processing step and detailed the configurations to set for both models.

Phase asymmetry ultrasound despeckling with fractional anisotropic diffusion and total variation

We propose an ultrasound speckle filtering method for not only preserving various edge features but also filtering tissue-dependent complex speckle noises in ultrasound images. The key idea is to detect these various edges using a phase congruence-based edge significance measure called phase asymmetry (PAS), which is invariant to the intensity amplitude of edges and takes 0 in non-edge smooth regions and 1 at the idea step edge, while also taking intermediate values at slowly varying ramp edges. By leveraging the PAS metric in designing weighting coefficients to maintain a balance between fractional-order anisotropic diffusion and total variation (TV) filters in TV cost function, we propose a new fractional TV framework to not only achieve the best despeckling performance with ramp edge preservation but also reduce the staircase effect produced by integral-order filters. Then, we exploit the PAS metric in designing a new fractional-order diffusion coefficient to properly preserve low-contrast edges in diffusion filtering. Finally, different from fixed fractional-order diffusion filters, an adaptive fractional order is introduced based on the PAS metric to enhance various weak edges in the spatially transitional areas between objects. The proposed fractional TV model is minimized using the gradient descent method to obtain the final denoised image. The experimental results and real application of ultrasound breast image segmentation show that the proposed method outperforms other state-of-the-art ultrasound despeckling filters for both speckle reduction and feature preservation in terms of visual evaluation and quantitative indices. The best scores on feature similarity indices have achieved 0.867, 0.844 and 0.834 under three different levels of noise, while the best breast ultrasound segmentation accuracy in terms of the mean and median dice similarity coefficient are 96.25% and 96.15%, respectively.

Is Radiation-Free Ultrasound Accurate for Quantitative Assessment of Spinal Deformity in Idiopathic Scoliosis (IS): A Detailed Analysis With EOS Radiography on 952 Patients

Radiation exposure with repeated radiography required at follow-up poses serious health concerns for scoliosis patients. Although spinous process angle (SPA) measurement of spinal curvatures with ultrasound has been reported with promising results, an evidence-based account on its accuracy for translational application remains undefined. This prospective study involved 952 idiopathic scoliosis patients (75.7% female, mean age 16.7 ± 3.0 y, Cobb 28.7 ± 11.6°). Among 1432 curves (88.1%) detected by ultrasound, there was good correlation between radiologic Cobb angles measured manually on EOS (E_Cobb) whole-spine radiographs and automatic ultrasound SPA measurement for upper spinal curves (USCs) (r = 0.873, apices T7-T12/L1 intervertebral disc) and lower spinal curves (LSCs) (r = 0.740, apices L1 or below) (p < 0.001). Taller stature was associated with stronger correlation. For E_Cobb <30°, 66.6% USCs and 62.4% LSCs had absolute differences between E_Cobb and predicted Cobb angle calculated from SPA ≤5°. Ultrasound could be a viable option in lieu of radiography for measuring coronal curves with apices at T7 or lower and Cobb angle <30°.

Scoliotic Imaging With a Novel Double-Sweep 2.5-Dimensional Extended Field-of-View Ultrasound

Extended field-of-view ultrasound (US EFOV) imaging is a technique used extensively in the clinical field to attain interpretable panorama of anatomy; 2.5-D US EFOV has recently been proposed for spine imaging. In the original 2.5-D US EFOV, it makes use of a six degrees-of-freedom positional sensor attached to the US probe to record the corresponding position of each B-scan. By combining the positional information and the B-scan images, the 2.5-D EFOV can reconstruct a panorama on a curved image plane when the scanning trajectory of the US probe is curved. In this paper, an improved method based on the Bezier interpolation is proposed to better reconstruct 2.5-D US EFOV imaging, producing the panoramas with smoother texture and higher quality. To make it more applicable for scoliosis patients, we designed a novel method called double-sweep 2.5-D EFOV to better image the spinal tissues and easily compute the Cobb angle. In vitro and in vivo experiments demonstrated that the 2.5-D EFOV images obtained by the proposed method can present anatomical structures of the scanning region accurately.

A reliability and validity study for different coronal angles using ultrasound imaging in adolescent idiopathic scoliosis

BACKGROUND CONTEXT

Radiation exposure remains a big concern in adolescent idiopathic scoliosis (AIS). Ultrasound imaging of the spine could significantly reduce or possibly even eliminate this radiation hazard. The spinous processes (SPs) and transverse processes (TPs) were used to measure the coronal deformity. Both landmarks provided reliable information on the severity of the curve as related to the traditional Cobb angle. However, it remained unclear which coronal ultrasound angle is the most appropriate method to measure the curve severity.

PURPOSE

The objective of this study was to test the reliability and the validity of several ultrasound angle measurements in the coronal plane as compared with the radiographic coronal Cobb angle in patients with AIS.

STUDY DESIGN/SETTING

This is a cross-sectional study.

PATIENT SAMPLE

The study included 33 patients with AIS, both male and female (Cobb angle range: 3°-90°, primary and secondary curves), who underwent posterior-anterior radiography of the spine.

OUTCOME MEASURES

The outcome measures were the reliability (intraclass correlation coefficients [ICCs] for the intra- and interobserver variabilities) and the validity (linear regression analysis and Bland-Altman method, including the mean absolute difference [MAD]) of different ultrasound measurements.

MATERIALS AND METHODS

The patients were scanned using a dedicated ultrasound machine (Scolioscan, Telefield Medical Imaging Ltd, Hong Kong). The reliability and the validity were tested for three coronal ultrasound angles: an automatic and manual SP angle and a manual TP angle as compared with the radiographic coronal main thoracic or (thoraco)lumbar Cobb angles.

RESULTS

The ICC showed very reliable measurements of all ultrasound methods (ICC ≥0.84). The ultrasound angles were 15%-37% smaller as compared with the Cobb angles; however, excellent linear correlations were seen between all ultrasound angles and the Cobb angle (thoracic: R²≥0.987 and (thoraco)lumbar R²≥0.970), and the Bland-Altman plot showed a good agreement between all ultrasound angles and the Cobb angle. The MADs of the ultrasound angles, corrected using the linear regression equation, and the Cobb angles showed no significant difference between the different ultrasound angles (MAD: automatic SP angle 4.9°±3.2°, manual SP angle 4.5°±3.1°, and manual TP angle 4.7°±3.6°; p≥.388).

CONCLUSIONS

Coronal ultrasound angles are based on different landmarks than the traditional Cobb angle measurement and cannot represent the same angle values. In this study, we found excellent correlations between the ultrasound and Cobb measurements, without differences in the reliability and validity between the ultrasound angles based on the SPs and TPs. Therefore, the severity of the deformity in patients with AIS can be assessed by ultrasound imaging, avoiding hazardous ionizing radiation and enabling more individualized patient care. It also opens possibilities for screening.

2.5-D Extended Field-of-View Ultrasound

Recently, the growing emphasis on medical ultrasound (US) has led to a rapid development of US extended field-of-view (EFOV) techniques. US EFOV techniques can be classified into three categories: 2-D US EFOV, 3-D US, and 3-D US EFOV. In this paper, we propose a novel EFOV method called 2.5-D US EFOV that combines both the advantages of the 2-D US EFOV and the 3-D US by generating a panorama on a curved image plane guided by a curved scanning trajectory of the US probe. In 2.5-D US EFOV, the real-time position and orientation of the US image plane can be recorded via an electromagnetic spatial sensor attached to the probe. The scanning direction is not necessarily straight and can be curved according to the regions of interest (ROI). To form the curved panorama, an image cutting method is proposed. Finally, the curved panorama is rendered in a 3-D space using a surface rendering based on a texture mapping technique. This allows 3-D measurements of lines and angles. Phantom experiments demonstrated that 2.5-D US EFOV images could show anatomical structures of ROI accurately and rapidly. The overall average errors for the distance and angle measurements are -0.097 ± 0.128 cm (-1% ± 1.2%) and 1.50° ± 1.60° (1.9% ± 2%), respectively. A typical extended US image can be reconstructed from 321 B-scans images within 3 s. The satisfying quantitative result on the spinal tissues of a scoliosis subject demonstrates that our system has potential applications in the assessment of musculoskeletal issues.

Automatic Myotendinous Junction Tracking in Ultrasound Images with Phase-Based Segmentation

Displacement of the myotendinous junction (MTJ) obtained by ultrasound imaging is crucial to quantify the interactive length changes of muscles and tendons for understanding the mechanics and pathological conditions of the muscle-tendon unit during motion. However, the lack of a reliable automatic measurement method restricts its application in human motion analysis. This paper presents an automated measurement of MTJ displacement using prior knowledge on tendinous tissues and MTJ, precluding the influence of nontendinous components on the estimation of MTJ displacement. It is based on the perception of tendinous features from musculoskeletal ultrasound images using Radon transform and thresholding methods, with information about the symmetric measures obtained from phase congruency. The displacement of MTJ is achieved by tracking manually marked points on tendinous tissues with the Lucas-Kanade optical flow algorithm applied over the segmented MTJ region. The performance of this method was evaluated on ultrasound images of the gastrocnemius obtained from 10 healthy subjects (26.0 ± 2.9 years of age). Waveform similarity between the manual and automatic measurements was assessed by calculating the overall similarity with the coefficient of multiple correlation (CMC). In vivo experiments demonstrated that MTJ tracking with the proposed method (CMC = 0.97 ± 0.02) was more consistent with the manual measurements than existing optical flow tracking methods (CMC = 0.79 ± 0.11). This study demonstrated that the proposed method was robust to the interference of nontendinous components, resulting in a more reliable measurement of MTJ displacement, which may facilitate further research and applications related to the architectural change of muscles and tendons.

Machine Learning in Ultrasound Computer-Aided Diagnostic Systems: A Survey

The ultrasound imaging is one of the most common schemes to detect diseases in the clinical practice. There are many advantages of ultrasound imaging such as safety, convenience, and low cost. However, reading ultrasound imaging is not easy. To support the diagnosis of clinicians and reduce the load of doctors, many ultrasound computer-aided diagnosis (CAD) systems are proposed. In recent years, the success of deep learning in the image classification and segmentation led to more and more scholars realizing the potential of performance improvement brought by utilizing the deep learning in the ultrasound CAD system. This paper summarized the research which focuses on the ultrasound CAD system utilizing machine learning technology in recent years. This study divided the ultrasound CAD system into two categories. One is the traditional ultrasound CAD system which employed the manmade feature and the other is the deep learning ultrasound CAD system. The major feature and the classifier employed by the traditional ultrasound CAD system are introduced. As for the deep learning ultrasound CAD, newest applications are summarized. This paper will be useful for researchers who focus on the ultrasound CAD system.