Hole-filling based on content loss indexed 3D partial convolution network for freehand ultrasound reconstruction

Three-dimensional ultrasound image reconstruction based on 3D-ResNet in the musculoskeletal system using a 1D probe:ex vivoandin vivofeasibility studies

Objective. Three-dimensional (3D) ultrasound (US) is needed to provide sonographers with a more intuitive panoramic view of the complex anatomical structure, especially the musculoskeletal system. In actual scanning, sonographers may perform fast scanning using a one-dimensional (1D) array probe .at random angles to gain rapid feedback, which leads to a large US image interval and missing regions in the reconstructed volume.Approach.In this study, a 3D residual network (3D-ResNet) modified by a 3D global residual branch (3D-GRB) and two 3D local residual branches (3D-LRBs) was proposed to retain detail and reconstruct high-quality 3D US volumes with high efficiency using only sparse two-dimensional (2D) US images. The feasibility and performance of the proposed algorithm were evaluated onex vivoandin vivosets.Main results. High-quality 3D US volumes in the fingers, radial and ulnar bones, and metacarpophalangeal joints were obtained by the 3D-ResNet, respectively. Their axial, coronal, and sagittal slices exhibited rich texture and speckle details. Compared with kernel regression, voxel nearest-neighborhood, squared distance weighted methods, and a 3D convolution neural network in the ablation study, the mean peak-signal-to-noise ratio and mean structure similarity of the 3D-ResNet were up to 28.53 ± 1.29 dB and 0.98 ± 0.01, respectively, and the corresponding mean absolute error dropped to 0.023 ± 0.003 with a better resolution gain of 1.22 ± 0.19 and shorter reconstruction time.Significance.These results illustrate that the proposed algorithm can rapidly reconstruct high-quality 3D US volumes in the musculoskeletal system in cases of a large amount of data loss. This suggests that the proposed algorithm has the potential to provide rapid feedback and precise analysis of stereoscopic details in complex and meticulous musculoskeletal system scanning with a less limited scanning speed and pose variations for the 1D array probe.

Adaptive tetrahedral interpolation for reconstruction of uneven freehand 3D ultrasound

Objective.Freehand 3D ultrasound volume reconstruction has received considerable attention in medical research because it can freely perform spatial imaging at a low cost. However, the uneven distribution of the original ultrasound images in space reduces the reconstruction effect of the traditional method.Approach.An adaptive tetrahedral interpolation algorithm is proposed to reconstruct 3D ultrasound volume data. The algorithm adaptively divides the unevenly distributed images into numerous tetrahedrons and interpolates the voxel value in each tetrahedron to reconstruct 3D ultrasound volume data.Main results.Extensive experiments on simulated and clinical data confirm that the proposed method can achieve more accurate reconstruction than six benchmark methods. Specifically, the averaged interpolation error at the gray level can be reduced by 0.22-0.82, and the peak signal-to-noise ratio and the mean structure similarity can be improved by 0.32-1.83 dB and 0.01-0.05, respectively.Significance.With the parallel implementation of the algorithm, one 3D ultrasound volume data with size 279 × 279 × 276 can be reconstructed from 100 slices 2D ultrasound images with size 200 × 200 at 1.04 s. Such a quick and accurate approach has practical value in medical research.

A partial convolution generative adversarial network for lesion synthesis and enhanced liver tumor segmentation

Lesion segmentation is critical for clinicians to accurately stage the disease and determine treatment strategy. Deep learning based automatic segmentation can improve both the segmentation efficiency and accuracy. However, training a robust deep learning segmentation model requires sufficient training examples with sufficient diversity in lesion location and lesion size. This study is to develop a deep learning framework for generation of synthetic lesions with various locations and sizes that can be included in the training dataset to enhance the lesion segmentation performance. The lesion synthesis network is a modified generative adversarial network (GAN). Specifically, we innovated a partial convolution strategy to construct a U-Net-like generator. The discriminator is designed using Wasserstein GAN with gradient penalty and spectral normalization. A mask generation method based on principal component analysis (PCA) was developed to model various lesion shapes. The generated masks are then converted into liver lesions through a lesion synthesis network. The lesion synthesis framework was evaluated for lesion textures, and the synthetic lesions were used to train a lesion segmentation network to further validate the effectiveness of the lesion synthesis framework. All the networks are trained and tested on the LITS public dataset. Our experiments demonstrate that the synthetic lesions generated by our approach have very similar distributions for the two parameters, GLCM-energy and GLCM-correlation. Including the synthetic lesions in the segmentation network improved the segmentation dice performance from 67.3% to 71.4%. Meanwhile, the precision and sensitivity for lesion segmentation were improved from 74.6% to 76.0% and 66.1% to 70.9%, respectively. The proposed lesion synthesis approach outperforms the other two existing approaches. Including the synthetic lesion data into the training dataset significantly improves the segmentation performance.

Classification and Segmentation Algorithm in Benign and Malignant Pulmonary Nodules under Different CT Reconstruction

Methods

The imaging data of 55 patients with chest CT plain scan in the Xuancheng People's Hospital were collected retrospectively. The data of each patient included lung window reconstruction, mediastinum reconstruction, and bone window reconstruction. The depth neural network and 3D convolution neural network were used to construct the model and train the classification and segmentation algorithm. The pathological results were the gold standard for benign and malignant pulmonary nodules. The classification and segmentation algorithms under three CT reconstruction algorithms were compared and analyzed by analysis of variance.

Results

Under the three CT reconstruction algorithms, the classification accuracy of pulmonary nodule density types was 98.2%, 96.4%, and 94.5%, respectively. The Dice coefficients of all nodule segmentation were 80.32% ± 5.91%, 79.83% ± 6.12%, and 80.17% ± 5.89%, respectively. The diagnostic accuracy between benign and malignant pulmonary nodules under different reconstruction algorithms was 98.2%, 96.4%, and 94.5%, respectively. There was no significant difference in the classification accuracy, Dice coefficients, and diagnostic accuracy of pulmonary nodules under three different reconstruction algorithms (all P > 0.05).

Conclusion

The depth neural network algorithm combined with 3D convolution neural network has a good efficiency in identifying benign and malignant pulmonary nodules under different CT reconstruction classification and segmentation algorithms.