Scoliotic Imaging With a Novel Double-Sweep 2.5-Dimensional Extended Field-of-View Ultrasound

ICPPNet: A semantic segmentation network model based on inter-class positional prior for scoliosis reconstruction in ultrasound images

OBJECTIVE

Considering the radiation hazard of X-ray, safer, more convenient and cost-effective ultrasound methods are gradually becoming new diagnostic approaches for scoliosis. For ultrasound images of spine regions, it is challenging to accurately identify spine regions in images due to relatively small target areas and the presence of a lot of interfering information. Therefore, we developed a novel neural network that incorporates prior knowledge to precisely segment spine regions in ultrasound images.

MATERIALS AND METHODS

We constructed a dataset of ultrasound images of spine regions for semantic segmentation. The dataset contains 3,136 images of 30 patients with scoliosis. And we propose a network model (ICPPNet), which fully utilizes inter-class positional prior knowledge by combining an inter-class positional probability heatmap, to achieve accurate segmentation of target areas.

RESULTS

ICPPNet achieved an average Dice similarity coefficient of 70.83% and an average 95% Hausdorff distance of 11.28 mm on the dataset, demonstrating its excellent performance. The average error between the Cobb angle measured by our method and the Cobb angle measured by X-ray images is 1.41 degrees, and the coefficient of determination is 0.9879 with a strong correlation.

DISCUSSION AND CONCLUSION

ICPPNet provides a new solution for the medical image segmentation task with positional prior knowledge between target classes. And ICPPNet strongly supports the subsequent reconstruction of spine models using ultrasound images.

Development of Automatic Assessment Framework for Spine Deformity Using Freehand 3-D Ultrasound Imaging System

A 3-D ultrasound (US) imaging technique has been studied to facilitate the diagnosis of spinal deformity without radiation. The objective of this article is to propose an assessment framework to automatically estimate spinal deformity in US spine images. The proposed framework comprises four major components, a US spine image generator, a novel transformer-based lightweight spine detector network, an angle evaluator, and a 3-D modeler. The principal component analysis (PCA) and discriminative scale space tracking (DSST) method are first adopted to generate the US spine images. The proposed detector is equipped with a redundancy queries removal (RQR) module and a regularization item to realize accurate and unique detection of spine images. Two clinical datasets, a total of 273 images from adolescents with idiopathic scoliosis, are used for the investigation of the proposed framework. The curvature is estimated by the angle evaluator, and the 3-D mesh model is established by the parametric modeling technique. The accuracy rate (AR) of the proposed detector can be achieved at 99.5%, with a minimal redundancy rate (RR) of 1.5%. The correlations between automatic curve measurements on US spine images from two datasets and manual measurements on radiographs are 0.91 and 0.88, respectively. The mean absolute difference (MAD) and standard deviation (SD) are 2.72° ± 2.14° and 2.91° ± 2.36° , respectively. The results demonstrate the effectiveness of the proposed framework to advance the application of the 3-D US imaging technique in clinical practice for scoliosis mass screening and monitoring.

Automatic Myotendinous Junction Identification in Ultrasound Images Based on Junction-Based Template Measurements

Tracking the myotendinous junction (MTJ) motion in consecutive ultrasound images is essential to assess muscle and tendon interaction and understand the mechanics' muscle-tendon unit and its pathological conditions during motion. However, the inherent speckle noises and ambiguous boundaries deter the reliable identification of MTJ, thus restricting their usage in human motion analysis. This study advances a fully automatic displacement measurement method for MTJ using prior shape knowledge on the Y-shape MTJ, precluding the influence of irregular and complicated hyperechoic structures in muscular ultrasound images. Our proposed method first adopts the junction candidate points using a combined measure of Hessian matrix and phase congruency, followed by a hierarchical clustering technique to refine the candidates approximating the position of the MTJ. Then, based on the prior knowledge of Y-shape MTJ, we finally identify the best matching junction points according to intensity distributions and directions of their branches using multiscale Gaussian templates and a Kalman filter. We evaluated our proposed method using the ultrasound scans of the gastrocnemius from 8 young, healthy volunteers. Our results present more consistent with the manual method in the MTJ tracking method than existing optical flow tracking methods, suggesting its potential in facilitating muscle and tendon function examinations with in vivo ultrasound imaging.

Ultrasound to X-ray synthesis generative attentional network (UXGAN) for adolescent idiopathic scoliosis

Standing X-ray radiograph with Cobb's method is the gold standard for scoliosis diagnosis. However, radiation hazard restricts its application, especially for close follow-up of adolescent patients. Compared with X-ray, ultrasound imaging has advantages of being radiation-free and real-time. To combine advantages of the above two imaging modalities, an ultrasound to X-ray synthesis generative attentional network (UXGAN) was proposed to synthesize ultrasound images into X-ray-like images. In this network, a cyclically consistent network was adopted and was trained end-to-end. An attention module was added and different residual blocks were designed. The quantitative comparison results demonstrated the superiority of our method to the state-of-the-art CycleGAN methods. We further compared the Cobb angle values measured on synthesized images and the real X-ray images, respectively. A good linear correlation (r = 0.95) was demonstrated between the two methods. The above results proved that the proposed method is of great significance for providing both X-ray images and ultrasound images based on the radiation-free ultrasound scanning.

Novel automated spinal ultrasound segmentation approach for scoliosis visualization

Scoliosis is a 3D deformity of the spine in which one or more segments of the spine curve laterally, usually with rotation of the vertebral body. Generally, having a Cobb angle (Cobb) greater than 10° can be considered scoliosis. In spine imaging, reliable and accurate identification and segmentation of bony features are crucial for scoliosis assessment, disease diagnosis, and treatment planning. Compared with commonly used X-ray detection methods, ultrasound has received extensive attention from researchers in the past years because of its lack of radiation, high real-time performance, and low price. On the basis of our previous research on spinal ultrasound imaging, this work combines artificial intelligence methods to create a new spine ultrasound image segmentation model called ultrasound global guidance block network (UGBNet), which provides a completely automatic and reliable spine segmentation and scoliosis visualization approach. Our network incorporates a global guidance block module that integrates spatial and channel attention, through which long-range feature dependencies and contextual scale information are learned. We evaluate the performance of the proposed model in semantic segmentation on spinal ultrasound datasets through extensive experiments with several classical learning segmentation methods, such as UNet. Results show that our method performs better than other approaches. Our UGBNet significantly improves segmentation precision, which can reach 74.2% on the evaluation metric of the Dice score.

A real-time freehand 3D ultrasound imaging method for scoliosis assessment

Real‐time 3D ultrasound has gained popularity in many fields because it can provide interactive feedback to help acquire high‐quality images or to conduct timely diagnosis. However, no comprehensive study has been reported on such an imaging method for scoliosis evaluation due to the complexity of this application. Meanwhile, the use of radiation‐free assessment of scoliosis is becoming increasingly popular. This study developed a real‐time 3D ultrasound imaging method for scoliosis assessment based on an incremental imaging method. In vivo experiments involving 36 patients with scoliosis were performed to test the performance of the proposed method. This new imaging method achieved a mean incremental frame rate of 82.7 ± 11.0 frames/s. The high repeatability of the intra‐operator test (intraclass correlation coefficient [ICC] = 0.92) and inter‐operator test (ICC = 0.91) demonstrated that the new method was very reliable. The result of spinous process angles obtained by the new method was linearly correlated (y = 0.97x, R² = 0.88) with that obtained by conventional 3D reconstruction. These results suggested that the newly developed imaging method can provide real‐time ultrasound imaging for scoliosis evaluation while preserving the comparative image quality of the conventional 3D reconstruction method.

Improvement of 3-D Ultrasound Spine Imaging Technique Using Fast Reconstruction Algorithm

Three-dimensional (3-D) freehand ultrasound (US) imaging has been applied to the investigation of spine deformity. However, it is a challenge for the current 3-D imaging reconstruction algorithms to achieve a balance between image quality and computation time. The objectives of this article are to implement a new fast reconstruction algorithm that can fulfill the request of immediate demonstration and processing for high-quality 3-D spine imaging, and to evaluate the reliability and accuracy of scoliotic curvature measurement when using the algorithm. The fast dot-projection (FDP) algorithm was applied for voxel-based nearest neighbor (VNN), multiple plane interpolation (MPI), and pixel nearest neighbor (PNN) protocols to reduce the reconstruction time. The 3-D image volume was reconstructed from the datasets acquired from scoliotic subjects. The computational cost, image characteristics, and statistical analyses of curve measurements were compared and evaluated among different reconstruction protocols. The results illustrated that the 3-D spine images using the FDP-MPI4 algorithm showed higher brightness (20%), contrast (14%), and signal-to-noise ratio (SNR) (26%) than FDP-VNN. The measurement performed by trainee rater exhibited significant improvement in measurement reliability and accuracy using FDP-MPI4 in comparison with FDP-VNN ( ), and the intraclass correlation coefficient (ICC) of interrater measurement increased from 0.88 to 0.96. The FDP-PNN method could acquire and reconstruct spine images simultaneously and present the results in 1-2 min, which showed the potential to provide the approximate real-time visualization for fast screening.

Automatic Detection and Measurement of Spinous Process Curve on Clinical Ultrasound Spine Images

The ultrasound (US) imaging technique has been applied to scoliosis assessment, and the proxy Cobb angle can be acquired on the US coronal images. The spinous process angle (SPA) is a valuable parameter to indicate 3-D deformity of spine. However, the SPA cannot be measured on US images since the spinous process (SP) is merged in the soft tissue layer and impossible to be identified on the coronal view directly. A new method based on the gradient vector flow (GVF) snake model was proposed to automatically locate SP position on the US transverse images, and the density-based spatial clustering of application with noise (DBSCAN) was used to remove the outliers out of the detected location results. With marking the SP points on the US coronal image, the SP curve was interpolated and the SPA was measured. The algorithm was evaluated on 50 subjects with various severity of scoliosis, and two raters measured the SPA on both US images and radiographs manually. The mean absolute differences (MADs) of SPAs obtained from the two modalities were 3.4° ± 2.4° and 3.6° ± 2.8° for the two raters, respectively, which were less than the clinical acceptance error (5°), and the results reported a good linear correlation ( ) between the US method and radiography. It indicates that the proposed method can be a promising approach for SPA measurement using the US imaging technique.

Validation of Scolioscan Air-Portable Radiation-Free Three-Dimensional Ultrasound Imaging Assessment System for Scoliosis

To diagnose scoliosis, the standing radiograph with Cobb's method is the gold standard for clinical practice. Recently, three-dimensional (3D) ultrasound imaging, which is radiation-free and inexpensive, has been demonstrated to be reliable for the assessment of scoliosis and validated by several groups. A portable 3D ultrasound system for scoliosis assessment is very much demanded, as it can further extend its potential applications for scoliosis screening, diagnosis, monitoring, treatment outcome measurement, and progress prediction. The aim of this study was to investigate the reliability of a newly developed portable 3D ultrasound imaging system, Scolioscan Air, for scoliosis assessment using coronal images it generated. The system was comprised of a handheld probe and tablet PC linking with a USB cable, and the probe further included a palm-sized ultrasound module together with a low-profile optical spatial sensor. A plastic phantom with three different angle structures built-in was used to evaluate the accuracy of measurement by positioning in 10 different orientations. Then, 19 volunteers with scoliosis (13F and 6M; Age: 13.6 ± 3.2 years) with different severity of scoliosis were assessed. Each subject underwent scanning by a commercially available 3D ultrasound imaging system, Scolioscan, and the portable 3D ultrasound imaging system, with the same posture on the same date. The spinal process angles (SPA) were measured in the coronal images formed by both systems and compared with each other. The angle phantom measurement showed the measured angles well agreed with the designed values, 59.7 ± 2.9 vs. 60 degrees, 40.8 ± 1.9 vs. 40 degrees, and 20.9 ± 2.1 vs. 20 degrees. For the subject tests, results demonstrated that there was a very good agreement between the angles obtained by the two systems, with a strong correlation (R2 = 0.78) for the 29 curves measured. The absolute difference between the two data sets was 2.9 ± 1.8 degrees. In addition, there was a small mean difference of 1.2 degrees, and the differences were symmetrically distributed around the mean difference according to the Bland-Altman test. Scolioscan Air was sufficiently comparable to Scolioscan in scoliosis assessment, overcoming the space limitation of Scolioscan and thus providing wider applications. Further studies involving a larger number of subjects are worthwhile to demonstrate its potential clinical values for the management of scoliosis.

A Single-Shot Region-Adaptive Network for Myotendinous Junction Segmentation in Muscular Ultrasound Images

Tracking the myotendinous junction (MTJ) in consecutive ultrasound images is crucial for understanding the mechanics and pathological conditions of the muscle-tendon unit. However, the lack of reliable and efficient identification of MTJ due to poor image quality and boundary ambiguity restricts its application in motion analysis. In recent years, with the rapid development of deep learning, the region-based convolution neural network (RCNN) has shown great potential in the field of simultaneous objection detection and instance segmentation in medical images. This article proposes a region-adaptive network (RAN) to localize MTJ region and to segment it in a single shot. Our model learns about the salient information of MTJ with the help of a composite architecture. Herein, a region-based multitask learning network explores the region containing MTJ, while a parallel end-to-end U-shaped path extracts the MTJ structure from the adaptively selected region for combating data imbalance and boundary ambiguity. By demonstrating the ultrasound images of the gastrocnemius, we showed that the RAN achieves superior segmentation performance when compared with the state-of-the-art Mask RCNN method with an average Dice score of 80.1%. Our proposed method is robust and reliable for advanced muscle and tendon function examinations obtained by ultrasound imaging.

Measurement of Quasi-Static 3-D Knee Joint Movement Based on the Registration From CT to US

The measurement of quasi-static 3-D knee joint movement is an important basis for studying the mechanism of knee joint injury. Most of the existing measurement methods make use of computed tomography (CT) and nuclear magnetic resonance (MR) imaging technology and hence have the disadvantages of invasiveness, ionizing radiation, low accuracy, and high cost. To overcome those drawbacks, this article innovatively proposes a 3-D motion measurement system for the knee joint based on the registration of CT images to ultrasound (US) images. More specifically, the lower limbs of a subject were first scanned once to acquire the CT images. A portable handheld device was designed to control a US probe for mechanically scanning the subject's lower limbs in a linear trajectory. During the movement of the subject's lower limbs, the US scanning was performed quasi-statically. The acquired US images were then registered to the CT images, and the 3-D motions of the lower limb bones could be recreated with the bones scanned in CT images. To guarantee the registration accuracy and efficiency, we used the H-shaped multiview slice assembly as the structural image content for the registration process. The experimental results show that our approach can accurately measure the 3-D motion of the knee joint and meet the needs of 3-D motion analysis of knee joint in practice.