Deep generative models for automated muscle segmentation in computed tomography scanning

Deep Learning Technique for Automatic Segmentation of Proximal Hip Musculoskeletal Tissues From CT Scan Images: A MrOS Study

BACKGROUND

Age-related conditions, such as osteoporosis and sarcopenia, alongside chronic diseases, can result in significant musculoskeletal tissue loss. This impacts individuals' quality of life and increases risk of falls and fractures. Computed tomography (CT) has been widely used for assessing musculoskeletal tissues. Although automatic techniques have been investigated for segmenting tissues in the abdomen and mid-thigh regions, studies in proximal hip remain limited. This study aims to develop a deep learning technique for segmentation and quantification of musculoskeletal tissues in CT scans of proximal hip.

METHODS

We examined 300 participants (men, 73 ± 6 years) from two cohorts of the Osteoporotic Fractures in Men Study (MrOS). We manually segmented cortical bone, trabecular bone, marrow adipose tissue (MAT), haematopoietic bone marrow (HBM), muscle, intermuscular adipose tissue (IMAT) and subcutaneous adipose tissue (SAT) from CT scan images at the proximal hip level. Using these data, we trained a U-Net-like deep learning model for automatic segmentation. The association between model-generated quantitative results and outcome variables such as grip strength, chair sit-to-stand time, walking speed, femoral neck and spine bone mineral density (BMD), and total lean mass was calculated.

RESULTS

An average Dice similarity coefficient (DSC) above 90% was observed across all tissue types in the test dataset. Grip strength showed positive correlations with cortical bone area (coefficient: 0.95, 95% confidence interval: [0.10, 1.80]), muscle area (0.41, [0.19, 0.64]) and average Hounsfield unit for muscle adjusted for height squared (AHU/h2) (1.1, [0.53, 1.67]), while it was negatively correlated with IMAT (-1.45, [-2.21, -0.70]) and SAT (-0.32, [-0.50, -0.13]). Gait speed was directly related to muscle area (0.01, [0.00, 0.02]) and inversely to IMAT (-0.04, [-0.07, -0.01]), while chair sit-to-stand time was associated with muscle area (0.98, [0.98, 0.99]), IMAT area (1.04, [1.01, 1.07]), SAT area (1.01, [1.01, 1.02]) and AHU/h2 for muscle (0.97, [0.95, 0.99]). MAT area showed a potential link to non-trauma fractures post-50 years (1.67, [0.98, 2.83]). Femoral neck BMD was associated with cortical bone (0.09, [0.08, 0.10]), MAT (-0.11, [-0.13, -0.10]), MAT adjusted for total bone marrow area (-0.06, [-0.07, -0.05]) and AHU/h2 for muscle (0.01, [0.00, 0.02]). Total spine BMD showed similar associations and with AHU for muscle (0.02, [0.00, 0.05]). Total lean mass was correlated with cortical bone (517.3, [148.26, 886.34]), trabecular bone (924, [262.55, 1585.45]), muscle (381.71, [291.47, 471.96]), IMAT (-1096.62, [-1410.34, -782.89]), SAT (-413.28, [-480.26, -346.29]), AHU (527.39, [159.12, 895.66]) and AHU/h2 (300.03, [49.23, 550.83]).

CONCLUSION

Our deep learning-based technique offers a fast and accurate method for segmentation and quantification of musculoskeletal tissues in proximal hip, with potential clinical value.

Validation of musculoskeletal segmentation model with uncertainty estimation for bone and muscle assessment in hip-to-knee clinical CT images

Deep learning-based image segmentation has allowed for the fully automated, accurate, and rapid analysis of musculoskeletal (MSK) structures from medical images. However, current approaches were either applied only to 2D cross-sectional images, addressed few structures, or were validated on small datasets, which limit the application in large-scale databases. This study aimed to validate an improved deep learning model for volumetric MSK segmentation of the hip and thigh with uncertainty estimation from clinical computed tomography (CT) images. Databases of CT images from multiple manufacturers/scanners, disease status, and patient positioning were used. The segmentation accuracy, and accuracy in estimating the structures volume and density, i.e., mean HU, were evaluated. An approach for segmentation failure detection based on predictive uncertainty was also investigated. The model has improved all segmentation accuracy and structure volume/density evaluation metrics compared to a shallower baseline model with a smaller training database (N = 20). The predictive uncertainty yielded large areas under the receiver operating characteristic (AUROC) curves (AUROCs ≥ .95) in detecting inaccurate and failed segmentations. Furthermore, the study has shown an impact of the disease severity status on the model's predictive uncertainties when applied to a large-scale database. The high segmentation and muscle volume/density estimation accuracy and the high accuracy in failure detection based on the predictive uncertainty exhibited the model's reliability for analyzing individual MSK structures in large-scale CT databases.

Accelerated muscle mass estimation from CT images through transfer learning

BACKGROUND

The cost of labeling to collect training data sets using deep learning is especially high in medical applications compared to other fields. Furthermore, due to variances in images depending on the computed tomography (CT) devices, a deep learning based segmentation model trained with a certain device often does not work with images from a different device.

METHODS

In this study, we propose an efficient learning strategy for deep learning models in medical image segmentation. We aim to overcome the difficulties of segmentation in CT images by training a VNet segmentation model which enables rapid labeling of organs in CT images with the model obtained by transfer learning using a small number of manually labeled images, called SEED images. We established a process for generating SEED images and conducting transfer learning a model. We evaluate the performance of various segmentation models such as vanilla UNet, UNETR, Swin-UNETR and VNet. Furthermore, assuming a scenario that a model is repeatedly trained with CT images collected from multiple devices, in which is catastrophic forgetting often occurs, we examine if the performance of our model degrades.

RESULTS

We show that transfer learning can train a model that does a good job of segmenting muscles with a small number of images. In addition, it was confirmed that VNet shows better performance when comparing the performance of existing semi-automated segmentation tools and other deep learning networks to muscle and liver segmentation tasks. Additionally, we confirmed that VNet is the most robust model to deal with catastrophic forgetting problems.

CONCLUSION

In the 2D CT image segmentation task, we confirmed that the CNN-based network shows better performance than the existing semi-automatic segmentation tool or latest transformer-based networks.

UDBRNet: A novel uncertainty driven boundary refined network for organ at risk segmentation

Organ segmentation has become a preliminary task for computer-aided intervention, diagnosis, radiation therapy, and critical robotic surgery. Automatic organ segmentation from medical images is a challenging task due to the inconsistent shape and size of different organs. Besides this, low contrast at the edges of organs due to similar types of tissue confuses the network's ability to segment the contour of organs properly. In this paper, we propose a novel convolution neural network based uncertainty-driven boundary-refined segmentation network (UDBRNet) that segments the organs from CT images. The CT images are segmented first and produce multiple segmentation masks from multi-line segmentation decoder. Uncertain regions are identified from multiple masks and the boundaries of the organs are refined based on uncertainty data. Our method achieves remarkable performance, boasting dice accuracies of 0.80, 0.95, 0.92, and 0.94 for Esophagus, Heart, Trachea, and Aorta respectively on the SegThor dataset, and 0.71, 0.89, 0.85, 0.97, and 0.97 for Esophagus, Spinal Cord, Heart, Left-Lung, and Right-Lung respectively on the LCTSC dataset. These results demonstrate the superiority of our uncertainty-driven boundary refinement technique over state-of-the-art segmentation networks such as UNet, Attention UNet, FC-denseNet, BASNet, UNet++, R2UNet, TransUNet, and DS-TransUNet. UDBRNet presents a promising network for more precise organ segmentation, particularly in challenging, uncertain conditions. The source code of our proposed method will be available at https://github.com/riadhassan/UDBRNet.

K. SK, Jayaraman S. Automatic foot ulcer segmentation using conditional generative adversarial network (AFSegGAN): A wound management system

Effective wound care is essential to prevent further complications, promote healing, and reduce the risk of infection and other health issues. Chronic wounds, particularly in older adults, patients with disabilities, and those with pressure, venous, or diabetic foot ulcers, cause significant morbidity and mortality. Due to the positive trend in the number of individuals with chronic wounds, particularly among the growing elderly and diabetes populations, it is imperative to develop novel technologies and practices for the best practice clinical management of chronic wounds to minimize the potential health and economic burdens on society. As wound care is managed in hospitals and community care, it is crucial to have quantitative metrics like wound boundary and morphological features. The traditional visual inspection technique is purely subjective and error-prone, and digitization provides an appealing alternative. Various deep-learning models have earned confidence; however, their accuracy primarily relies on the image quality, the dataset size to learn the features, and experts' annotation. This work aims to develop a wound management system that automates wound segmentation using a conditional generative adversarial network (cGAN) and estimate the wound morphological parameters. AFSegGAN was developed and validated on the MICCAI 2021-foot ulcer segmentation dataset. In addition, we use adversarial loss and patch-level comparison at the discriminator network to improve the segmentation performance and balance the GAN network training. Our model outperformed state-of-the-art methods with a Dice score of 93.11% and IoU of 99.07%. The proposed wound management system demonstrates its abilities in wound segmentation and parameter estimation, thereby reducing healthcare workers' efforts to diagnose or manage wounds and facilitating remote healthcare.

Deep Learning-Based Medical Images Segmentation of Musculoskeletal Anatomical Structures: A Survey of Bottlenecks and Strategies

By leveraging the recent development of artificial intelligence algorithms, several medical sectors have benefited from using automatic segmentation tools from bioimaging to segment anatomical structures. Segmentation of the musculoskeletal system is key for studying alterations in anatomical tissue and supporting medical interventions. The clinical use of such tools requires an understanding of the proper method for interpreting data and evaluating their performance. The current systematic review aims to present the common bottlenecks for musculoskeletal structures analysis (e.g., small sample size, data inhomogeneity) and the related strategies utilized by different authors. A search was performed using the PUBMED database with the following keywords: deep learning, musculoskeletal system, segmentation. A total of 140 articles published up until February 2022 were obtained and analyzed according to the PRISMA framework in terms of anatomical structures, bioimaging techniques, pre/post-processing operations, training/validation/testing subset creation, network architecture, loss functions, performance indicators and so on. Several common trends emerged from this survey; however, the different methods need to be compared and discussed based on each specific case study (anatomical region, medical imaging acquisition setting, study population, etc.). These findings can be used to guide clinicians (as end users) to better understand the potential benefits and limitations of these tools.

AUE-Net: Automated Generation of Ultrasound Elastography Using Generative Adversarial Network

Problem: Ultrasonography is recommended as the first choice for evaluation of thyroid nodules, however, conventional ultrasound features may not be able to adequately predict malignancy. Ultrasound elastography, adjunct to conventional B-mode ultrasound, can effectively improve the diagnostic accuracy of thyroid nodules. However, this technology requires professional elastography equipment and experienced physicians. Aim: in the field of computational medicine, Generative Adversarial Networks (GANs) were proven to be a powerful tool for generating high-quality images. This work therefore utilizes GANs to generate ultrasound elastography images. Methods: this paper proposes a new automated generation method of ultrasound elastography (AUE-net) to generate elastography images from conventional ultrasound images. The AUE-net was based on the U-Net architecture and optimized by attention modules and feature residual blocks, which could improve the adaptability of feature extraction for nodules of different sizes. The additional color loss function was used to balance color distribution. In this network, we first attempted to extract the tissue features of the ultrasound image in the latent space, then converted the attributes by modeling the strain, and finally reconstructed them into the corresponding elastography image. Results: a total of 726 thyroid ultrasound elastography images with corresponding conventional images from 397 patients were obtained between 2019 and 2021 as the dataset (646 in training set and 80 in testing set). The mean rating accuracy of the AUE-net generated elastography images by ultrasound specialists was 84.38%. Compared with that of the existing models in the visual aspect, the presented model generated relatively higher quality elastography images. Conclusion: the AUE-net generated ultrasound elastography images showed natural appearance and retained tissue information. Accordingly, it seems that B-mode ultrasound harbors information that can link to tissue elasticity. This study may pave the way to generate ultrasound elastography images readily without the need for professional equipment.

Generative adversarial networks in medical image segmentation: A review

PURPOSE

Since Generative Adversarial Network (GAN) was introduced into the field of deep learning in 2014, it has received extensive attention from academia and industry, and a lot of high-quality papers have been published. GAN effectively improves the accuracy of medical image segmentation because of its good generating ability and capability to capture data distribution. This paper introduces the origin, working principle, and extended variant of GAN, and it reviews the latest development of GAN-based medical image segmentation methods.

METHOD

To find the papers, we searched on Google Scholar and PubMed with the keywords like "segmentation", "medical image", and "GAN (or generative adversarial network)". Also, additional searches were performed on Semantic Scholar, Springer, arXiv, and the top conferences in computer science with the above keywords related to GAN.

RESULTS

We reviewed more than 120 GAN-based architectures for medical image segmentation that were published before September 2021. We categorized and summarized these papers according to the segmentation regions, imaging modality, and classification methods. Besides, we discussed the advantages, challenges, and future research directions of GAN in medical image segmentation.

CONCLUSIONS

We discussed in detail the recent papers on medical image segmentation using GAN. The application of GAN and its extended variants has effectively improved the accuracy of medical image segmentation. Obtaining the recognition of clinicians and patients and overcoming the instability, low repeatability, and uninterpretability of GAN will be an important research direction in the future.