Deep learning-based thigh muscle segmentation for reproducible fat fraction quantification using fat-water decomposition MRI

Importance of neural network complexity for the automatic segmentation of individual thigh muscles in MRI images from patients with neuromuscular diseases

OBJECTIVE

Segmentation of individual thigh muscles in MRI images is essential for monitoring neuromuscular diseases and quantifying relevant biomarkers such as fat fraction (FF). Deep learning approaches such as U-Net have demonstrated effectiveness in this field. However, the impact of reducing neural network complexity remains unexplored in the FF quantification in individual muscles.

MATERIAL AND METHODS

U-Net architectures with different complexities have been compared for the quantification of the fat fraction in each muscle group selected in the central part of the thigh region. The corresponding performance has been assessed in terms of Dice score (DSC) and FF quantification error. The database contained 1450 thigh images from 59 patients and 14 healthy subjects (age: 47 ± 17 years, sex: 36F, 37M). Ten individual muscles were segmented in each image. The performance of each model was compared to nnU-Net, a complex architecture with 4.35 × 107 parameters, 12.8 Gigabytes of peak memory usage and 167 h of training time.

RESULTS

As expected, nnU-Net achieved the highest DSC (94.77 ± 0.13%). A simpler U-Net (5.81 × 105 parameters, 2.37 Gigabytes, 14 h of training time) achieved a lower DSC but still above 90%. Surprisingly, both models achieved a comparable FF estimation.

DISCUSSION

The poor correlation between observed DSC and FF indicates that less complex architectures, reducing GPU memory utilization and training time, can still accurately quantify FF.

Automated analysis of quantitative muscle MRI and its reliability in patients with Duchenne muscular dystrophy

BACKGROUND

Quantitative muscle MRI is one of the most promising biomarkers to detect subclinical disease progression in patients with neuromuscular disorders, including Duchenne muscular dystrophy (DMD). However, its clinical application has been limited partly due to the time-intensive process of manual segmentation.

OBJECTIVE

We present a simple and fast automated approach to obtain quantitative measurement of thigh muscle fat fraction and investigate its reliability in patients with DMD.

METHODS

Clinical and radiological baseline and 6-month follow-up data of 41 ambulant patients with DMD were analysed retrospectively. Axial 2-point Dixon MR images of all thigh muscles were used to quantify mean fat fraction, while clinical outcomes were measured by the Motor Function Measure (MFM) and its D1 domain. Data obtained by automated segmentation were compared to manual segmentation and correlated with clinical outcomes. Results were also used to compare the statistical power when using automated or manual segmentation.

RESULTS

A mean increase of 3.55% in thigh muscle fat fraction at 6-month follow-up could be detected by both methods without any significant difference between them (p=0.437). The automated muscle segmentation method demonstrated a strong correlation with manually segmented data (Pearson's ρ = 0.97). Additionally, there was no statistically significant difference between the automated and manual segmentation methods in their association with clinical progression, as measured by the total MFM score and its D1 domain (p = 0.235 and p = 0.425, respectively).

CONCLUSIONS

The presented automated segmentation technique is a fast and reliable tool for assessing disease progression, particularly in the early stages of DMD. It is one of the few studies validated using manual segmentation, and with further refinement, it has the potential to become a good surrogate marker for disease progression in various neuromuscular disorders.

Semantic segmentation for individual thigh skeletal muscles of athletes on magnetic resonance images

The skeletal muscles that athletes should train vary depending on their discipline and position. Therefore, individual skeletal muscle cross-sectional area assessment is important in the development of training strategies. To measure the cross-sectional area of skeletal muscle, manual segmentation of each muscle is performed using magnetic resonance (MR) imaging. This task is time-consuming and requires significant effort. Additionally, interobserver variability can sometimes be problematic. The purpose of this study was to develop an automated computerized method for semantic segmentation of individual thigh skeletal muscles from MR images of athletes. Our database consisted of 697 images from the thighs of 697 elite athletes. The images were randomly divided into a training dataset (70%), a validation dataset (10%), and a test dataset (20%). A label image was generated for each image by manually annotating 15 object classes: 12 different skeletal muscles, fat, bones, and vessels and nerves. Using the validation dataset, DeepLab v3+ was chosen from three different semantic segmentation models as a base model for segmenting individual thigh skeletal muscles. The feature extractor in DeepLab v3+ was also optimized to ResNet50. The mean Jaccard index and Dice index for the proposed method were 0.853 and 0.916, respectively, which were significantly higher than those from conventional DeepLab v3+ (Jaccard index: 0.810, p < .001; Dice index: 0.887, p < .001). The proposed method achieved a mean area error for 15 objective classes of 3.12%, useful in the assessment of skeletal muscle cross-sectional area from MR images.

Muscle imaging in facioscapulohumeral muscular dystrophy research: A scoping review and expert recommendations

Clinical trial readiness is an important topic in the field of facioscapulohumeral muscular dystrophy (FSHD). As FSHD is a slowly progressive and clinically heterogeneous disease, imaging biomarkers have been proposed to complement clinical outcome measures. Muscle magnetic resonance imaging (MRI), ultrasound and dual energy X-ray absorptiometry (DEXA) have been used to measure disease severity, activity and progression. We conducted a scoping review of the literature on these imaging modalities to assess gaps in knowledge and subsequently collaborated with a panel of neuromuscular imaging experts to generate recommendations on the road ahead. We systematically searched PubMed, EMBASE and Cochrane Library databases. Three-hundred and twenty-eight studies were screened and one hundred and five studies were included. MRI indices related to intramuscular fat content, STIR positivity and T2water are used as diagnostic as well as prognostic and monitoring biomarkers. Ultrasound echogenicity can be used as a diagnostic and potentially as a prognostic and monitoring biomarker. DEXA lean muscle mass may be used as an additional monitoring biomarker. Each imaging modality has its own benefits but also challenges. Based on our expert opinions, we propose a roadmap to address these challenges, ensuring the optimal use of each modality in multi-center clinical trials in FSHD.

Comparison of Manual vs Artificial Intelligence-Based Muscle MRI Segmentation for Evaluating Disease Progression in Patients With CMT1A

BACKGROUND AND OBJECTIVES

Intramuscular fat fraction (FF), assessed with quantitative MRI (qMRI), has emerged as one of the few responsive outcome measures in CMT1A patients. The main limitation for its use in future therapeutic trials is the time required for the manual segmentation of individual muscles. This study aimed to evaluate the accuracy and responsiveness of a fully automatic artificial intelligence (AI)-based segmentation pipeline to assess disease progression in a cohort of CMT1A patients over 1 year.

METHODS

Twenty CMT1A patients were included in this observational, prospective, longitudinal study. FF was measured twice a year apart using qMRI in the lower limbs. Individual muscle segmentation was performed fully automatically using a trained convolutional neural network with or without human quality check (QC). The corresponding results were compared with those obtained by fully manual (FM) segmentation using the Dice similarity coefficient (DSC). FF progression and its standardized response mean (SRM) were also computed in individual muscles over the single central slice and a 3D volume to define the most sensitive region of interest.

RESULTS

AI-based segmentation showed excellent DSC values (>0.90). Significant global FF progression was observed at thigh (+0.71% ± 1.28%; p = 0.016) and leg (+1.73% ± 2.88%, p = 0.007) levels, similarly to that calculated using the FM technique (p = 0.363 and p = 0.634). FF progression of each individual muscle was comparable when computed from either the central slice or the 3D volume. The best SRM value (0.70) was obtained for the FF progression computed using the AI-based technique with human QC in the 3D volume at the leg level. The time required for fully automatic segmentation using AI with a QC was 10 hours for the entire data set compared with 90 hours for the FM.

DISCUSSION

qMRI combined with AI-based segmentation can be considered as a process ready for assessing longitudinal FF changes in CMT1A patients. Given the slow FF progression at a thigh level and the large heterogeneity between muscles and individuals, FF should be quantified from a 3D volume at the leg level for longitudinal analyses. A QC performed after the AI-based segmentation is still advised given the increased SRM value.

Efficient data labeling strategies for automated muscle segmentation in lower leg MRIs of Charcot-Marie-Tooth disease patients

We aimed to develop efficient data labeling strategies for ground truth segmentation in lower-leg magnetic resonance imaging (MRI) of patients with Charcot-Marie-Tooth disease (CMT) and to develop an automated muscle segmentation model using different labeling approaches. The impact of using unlabeled data on model performance was further examined. Using axial T1-weighted MRIs of 120 patients with CMT (60 each with mild and severe intramuscular fat infiltration), we compared the performance of segmentation models obtained using several different labeling strategies. The effect of leveraging unlabeled data on segmentation performance was evaluated by comparing the performances of few-supervised, semi-supervised (mean teacher model), and fully-supervised learning models. We employed a 2D U-Net architecture and assessed its performance by comparing the average Dice coefficients (ADC) using paired t-tests with Bonferroni correction. Among few-supervised models utilizing 10% labeled data, labeling three slices (the uppermost, central, and lowermost slices) per subject exhibited a significantly higher ADC (90.84±3.46%) compared with other strategies using a single image slice per subject (uppermost, 87.79±4.41%; central, 89.42±4.07%; lowermost, 89.29±4.71%, p < 0.0001) or all slices per subject (85.97±9.82%, p < 0.0001). Moreover, semi-supervised learning significantly enhanced the segmentation performance. The semi-supervised model using the three-slices strategy showed the highest segmentation performance (91.03±3.67%) among 10% labeled set models. Fully-supervised model showed an ADC of 91.39±3.76. A three-slice-based labeling strategy for ground truth segmentation is the most efficient method for developing automated muscle segmentation models of CMT lower leg MRI. Additionally, semi-supervised learning with unlabeled data significantly enhances segmentation performance.

Deep network and multi-atlas segmentation fusion for delineation of thigh muscle groups in three-dimensional water-fat separated MRI

Purpose

Segmentation is essential for tissue quantification and characterization in studies of aging and age-related and metabolic diseases and the development of imaging biomarkers. We propose a multi-method and multi-atlas methodology for automated segmentation of functional muscle groups in three-dimensional (3D) thigh magnetic resonance images. These groups lie anatomically adjacent to each other, rendering their manual delineation a challenging and time-consuming task.

Approach

We introduce a framework for automated segmentation of the four main functional muscle groups of the thigh, gracilis, hamstring, quadriceps femoris, and sartorius, using chemical shift encoded water-fat magnetic resonance imaging (CSE-MRI). We propose fusing anatomical mappings from multiple deformable models with 3D deep learning model-based segmentation. This approach leverages the generalizability of multi-atlas segmentation (MAS) and accuracy of deep networks, hence enabling accurate assessment of volume and fat content of muscle groups.

Results

For segmentation performance evaluation, we calculated the Dice similarity coefficient (DSC) and Hausdorff distance 95th percentile (HD-95). We evaluated the proposed framework, its variants, and baseline methods on 15 healthy subjects by threefold cross-validation and tested on four patients. Fusion of multiple atlases, deformable registration models, and deep learning segmentation produced the top performance with an average DSC of 0.859 and HD-95 of 8.34 over all muscles.

Conclusions

Fusion of multiple anatomical mappings from multiple MAS techniques enriches the template set and improves the segmentation accuracy. Additional fusion with deep network decisions applied to the subject space offers complementary information. The proposed approach can produce accurate segmentation of individual muscle groups in 3D thigh MRI scans.

Convolutional Neural Network-Based Automated Segmentation of Skeletal Muscle and Subcutaneous Adipose Tissue on Thigh MRI in Muscular Dystrophy Patients

We aim to develop a deep learning-based algorithm for automated segmentation of thigh muscles and subcutaneous adipose tissue (SAT) from T1-weighted muscle MRIs from patients affected by muscular dystrophies (MDs). From March 2019 to February 2022, adult and pediatric patients affected by MDs were enrolled from Azienda Ospedaliera Universitaria Pisana, Pisa, Italy (Institution 1) and the IRCCS Stella Maris Foundation, Calambrone-Pisa, Italy (Institution 2), respectively. All patients underwent a bilateral thighs MRI including an axial T1 weighted in- and out-of-phase (dual-echo). Both muscles and SAT were manually and separately segmented on out-of-phase image sets by a radiologist with 6 years of experience in musculoskeletal imaging. A U-Net1 and U-Net3 were built to automatically segment the SAT, all the thigh muscles together and the three muscular compartments separately. The dataset was randomly split into the on train, validation, and test set. The segmentation performance was assessed through the Dice similarity coefficient (DSC). The final cohort included 23 patients. The estimated DSC for U-Net1 was 96.8%, 95.3%, and 95.6% on train, validation, and test set, respectively, while the estimated accuracy for U-Net3 was 94.1%, 92.9%, and 93.9%. Both of the U-Nets achieved a median DSC of 0.95 for SAT segmentation. The U-Net1 and the U-Net3 achieved an optimal agreement with manual segmentation for the automatic segmentation. The so-developed neural networks have the potential to automatically segment thigh muscles and SAT in patients affected by MDs.

One-Year Longitudinal Assessment of Patients With CMT1A Using Quantitative MRI

BACKGROUND AND OBJECTIVES

Intramuscular fat fraction (FF) assessed using quantitative MRI (qMRI) has emerged as one of the few responsive outcome measures in CMT1A suitable for future clinical trials. This study aimed to identify the relevance of multiple qMRI biomarkers for tracking longitudinal changes in CMT1A and to assess correlations between MRI metrics and clinical parameters.

METHODS

qMRI was performed in CMT1A patients at 2 time points, a year apart, and various metrics were extracted from 3-dimensional volumes of interest at thigh and leg levels. A semiautomated segmentation technique was used, enabling the analysis of central slices and a larger 3D muscle volume. Metrics included proton density (PD), magnetization transfer ratio (MTR), and intramuscular FF. The sciatic and tibial nerves were also assessed. Disease severity was gauged using Charcot Marie Tooth Neurologic Score (CMTNSv2), Charcot Marie Tooth Examination Score, Overall Neuropathy Limitation Scale scores, and Medical Research Council (MRC) muscle strength.

RESULTS

Twenty-four patients were included. FF significantly rose in the 3D volume at both thigh (+1.04% ± 2.19%, p = 0.041) and leg (+1.36% ± 1.87%, p = 0.045) levels. The 3D analyses unveiled a length-dependent gradient in FF, ranging from 22.61% ± 10.17% to 26.17% ± 10.79% at the leg level. There was noticeable variance in longitudinal changes between muscles: +3.17% ± 6.86% (p = 0.028) in the tibialis anterior compared with 0.37% ± 4.97% (p = 0.893) in the gastrocnemius medialis. MTR across the entire thigh volume showed a significant decline between the 2 time points -2.75 ± 6.58 (p = 0.049), whereas no significant differences were noted for the 3D muscle volume and PD. No longitudinal changes were observed in any nerve metric. Potent correlations were identified between FF and primary clinical measures: CMTNSv2 (ρ = 0.656; p = 0.001) and MRC in the lower limbs (ρ = -0.877; p < 0.001).

DISCUSSION

Our results further support that qMRI is a promising tool for following up longitudinal changes in CMT1A patients, FF being the paramount MRI metric for both thigh and leg regions. It is crucial to scrutinize the postimaging data extraction methods considering that annual changes are minimal (around +1.5%). Given the varied FF distribution, the existence of a length-dependent gradient, and the differential fatty involution across muscles, 3D volume analysis appeared more suitable than single slice analysis.

Automatic segmentation of lower limb muscles from MR images of post-menopausal women based on deep learning and data augmentation

Individual muscle segmentation is the process of partitioning medical images into regions representing each muscle. It can be used to isolate spatially structured quantitative muscle characteristics, such as volume, geometry, and the level of fat infiltration. These features are pivotal to measuring the state of muscle functional health and in tracking the response of the body to musculoskeletal and neuromusculoskeletal disorders. The gold standard approach to perform muscle segmentation requires manual processing of large numbers of images and is associated with significant operator repeatability issues and high time requirements. Deep learning-based techniques have been recently suggested to be capable of automating the process, which would catalyse research into the effects of musculoskeletal disorders on the muscular system. In this study, three convolutional neural networks were explored in their capacity to automatically segment twenty-three lower limb muscles from the hips, thigh, and calves from magnetic resonance images. The three neural networks (UNet, Attention UNet, and a novel Spatial Channel UNet) were trained independently with augmented images to segment 6 subjects and were able to segment the muscles with an average Relative Volume Error (RVE) between -8.6% and 2.9%, average Dice Similarity Coefficient (DSC) between 0.70 and 0.84, and average Hausdorff Distance (HD) between 12.2 and 46.5 mm, with performance dependent on both the subject and the network used. The trained convolutional neural networks designed, and data used in this study are openly available for use, either through re-training for other medical images, or application to automatically segment new T1-weighted lower limb magnetic resonance images captured with similar acquisition parameters.

The Future of Artificial Intelligence in Sports Medicine and Return to Play

Artificial intelligence (AI) has shown tremendous growth over the last decade, with the more recent development of clinical applications in health care. The ability of AI to synthesize large amounts of complex data automatically allows health care providers to access previously unavailable metrics and thus enhance and personalize patient care. These innovations include AI-assisted diagnostic tools, prediction models for each treatment pathway, and various tools for workflow optimization. The extension of AI into sports medicine is still early, but numerous AI-driven algorithms, devices, and research initiatives have delved into predicting and preventing athlete injury, aiding in injury assessment, optimizing recovery plans, monitoring rehabilitation progress, and predicting return to play.

Human lower leg muscles grow asynchronously

Muscle volume must increase substantially during childhood growth to generate the power required to propel the growing body. One unresolved but fundamental question about childhood muscle growth is whether muscles grow at equal rates; that is, if muscles grow in synchrony with each other. In this study, we used magnetic resonance imaging (MRI) and advances in artificial intelligence methods (deep learning) for medical image segmentation to investigate whether human lower leg muscles grow in synchrony. Muscle volumes were measured in 10 lower leg muscles in 208 typically developing children (eight infants aged less than 3 months and 200 children aged 5 to 15 years). We tested the hypothesis that human lower leg muscles grow synchronously by investigating whether the volume of individual lower leg muscles, expressed as a proportion of total lower leg muscle volume, remains constant with age. There were substantial age-related changes in the relative volume of most muscles in both boys and girls (p < 0.001). This was most evident between birth and five years of age but was still evident after five years. The medial gastrocnemius and soleus muscles, the largest muscles in infancy, grew faster than other muscles in the first five years. The findings demonstrate that muscles in the human lower leg grow asynchronously. This finding may assist early detection of atypical growth and allow targeted muscle-specific interventions to improve the quality of life, particularly for children with neuromotor conditions such as cerebral palsy.

Automatic MRI-based rotator cuff muscle segmentation using U-Nets

BACKGROUND

The rotator cuff (RC) is a crucial anatomical element within the shoulder joint, facilitating an extensive array of motions while maintaining joint stability. Comprised of the subscapularis, infraspinatus, supraspinatus, and teres minor muscles, the RC plays an integral role in shoulder functionality. RC injuries represent prevalent, incapacitating conditions that impose a substantial impact on approximately 8% of the adult population in the USA. Segmentation of these muscles provides valuable anatomical information for evaluating muscle quality and allows for better treatment planning.

MATERIALS AND METHODS

We developed a model based on residual deep convolutional encoder-decoder U-net to segment RC muscles on oblique sagittal T1-weighted images MRI. Our data consisted of shoulder MRIs from a cohort of 157 individuals, consisting of individuals without RC tendon tear (N=79) and patients with partial RC tendon tear (N=78). We evaluated different modeling approaches. The performance of the models was evaluated by calculating the Dice coefficient on the hold out test set.

RESULTS

The best-performing model's median Dice coefficient was measured to be 89% (Q1:85%, Q3:96%) for the supraspinatus, 86% (Q1:82%, Q3:88%) for the subscapularis, 86% (Q1:82%, Q3:90%) for the infraspinatus, and 78% (Q1:70%, Q3:81%) for the teres minor muscle, indicating a satisfactory level of accuracy in the model's predictions.

CONCLUSION

Our computational models demonstrated the capability to delineate RC muscles with a level of precision akin to that of experienced radiologists. As hypothesized, the proposed algorithm exhibited superior performance when segmenting muscles with well-defined boundaries, including the supraspinatus, subscapularis, and infraspinatus muscles.

The Impact of Fatty Infiltration on MRI Segmentation of Lower Limb Muscles in Neuromuscular Diseases: A Comparative Study of Deep Learning Approaches

BACKGROUND

Deep learning methods have been shown to be useful for segmentation of lower limb muscle MRIs of healthy subjects but, have not been sufficiently evaluated on neuromuscular disease (NDM) patients.

PURPOSE

Evaluate the influence of fat infiltration on convolutional neural network (CNN) segmentation of MRIs from NMD patients.

STUDY TYPE

Retrospective study.

SUBJECTS

Data were collected from a hospital database of 67 patients with NMDs and 14 controls (age: 53 ± 17 years, sex: 48 M, 33 F). Ten individual muscles were segmented from the thigh and six from the calf (20 slices, 200 cm section).

FIELD STRENGTH/SEQUENCE

A 1.5 T. Sequences: 2D T1 -weighted fast spin echo. Fat fraction (FF): three-point Dixon 3D GRE, magnetization transfer ratio (MTR): 3D MT-prepared GRE, T2: 2D multispin-echo sequence.

ASSESSMENT

U-Net 2D, U-Net 3D, TransUNet, and HRNet were trained to segment thigh and leg muscles (101/11 and 95/11 training/validation images, 10-fold cross-validation). Automatic and manual segmentations were compared based on geometric criteria (Dice coefficient [DSC], outlier rate, absence rate) and reliability of measured MRI quantities (FF, MTR, T2, volume).

STATISTICAL TESTS

Bland-Altman plots were chosen to describe agreement between manual vs. automatic estimated FF, MTR, T2 and volume. Comparisons were made between muscle populations with an FF greater than 20% (G20+) and lower than 20% (G20-).

RESULTS

The CNNs achieved equivalent results, yet only HRNet recognized every muscle in the database, with a DSC of 0.91 ± 0.08, and measurement biases reaching -0.32% ± 0.92% for FF, 0.19 ± 0.77 for MTR, -0.55 ± 1.95 msec for T2, and - 0.38 ± 3.67 cm3 for volume. The performances of HRNet, between G20- and G20+ decreased significantly.

DATA CONCLUSION

HRNet was the most appropriate network, as it did not omit any muscle. The accuracy obtained shows that CNNs could provide fully automated methods for studying NMDs. However, the accuracy of the methods may be degraded on the most infiltrated muscles (>20%).

EVIDENCE LEVEL

4.

TECHNICAL EFFICACY

Stage 1.

Artificial Intelligence-powered automatic volume calculation in medical images - available tools, performance and challenges for nuclear medicine

Volumetry is crucial in oncology and endocrinology, for diagnosis, treatment planning, and evaluating response to therapy for several diseases. The integration of Artificial Intelligence (AI) and Deep Learning (DL) has significantly accelerated the automatization of volumetric calculations, enhancing accuracy and reducing variability and labor. In this review, we show that a high correlation has been observed between Machine Learning (ML) methods and expert assessments in tumor volumetry; Yet, it is recognized as more challenging than organ volumetry. Liver volumetry has shown progression in accuracy with a decrease in error. If a relative error below 10 % is acceptable, ML-based liver volumetry can be considered reliable for standardized imaging protocols if used in patients without major anomalies. Similarly, ML-supported automatic kidney volumetry has also shown consistency and reliability in volumetric calculations. In contrast, AI-supported thyroid volumetry has not been extensively developed, despite initial works in 3D ultrasound showing promising results in terms of accuracy and reproducibility. Despite the advancements presented in the reviewed literature, the lack of standardization limits the generalizability of ML methods across diverse scenarios. The domain gap, i. e., the difference in probability distribution of training and inference data, is of paramount importance before clinical deployment of AI, to maintain accuracy and reliability in patient care. The increasing availability of improved segmentation tools is expected to further incorporate AI methods into routine workflows where volumetry will play a more prominent role in radionuclide therapy planning and quantitative follow-up of disease evolution.

Semi-automatic muscle segmentation in MR images using deep registration-based label propagation

Fully automated approaches based on convolutional neural networks have shown promising performances on muscle segmentation from magnetic resonance (MR) images, but still rely on an extensive amount of training data to achieve valuable results. Muscle segmentation for pediatric and rare diseases cohorts is therefore still often done manually. Producing dense delineations over 3D volumes remains a time-consuming and tedious task, with significant redundancy between successive slices. In this work, we propose a segmentation method relying on registration-based label propagation, which provides 3D muscle delineations from a limited number of annotated 2D slices. Based on an unsupervised deep registration scheme, our approach ensures the preservation of anatomical structures by penalizing deformation compositions that do not produce consistent segmentation from one annotated slice to another. Evaluation is performed on MR data from lower leg and shoulder joints. Results demonstrate that the proposed few-shot multi-label segmentation model outperforms state-of-the-art techniques.

Deep learning-based automatic pipeline for quantitative assessment of thigh muscle morphology and fatty infiltration

PURPOSE

Fast and accurate thigh muscle segmentation from MRI is important for quantitative assessment of thigh muscle morphology and composition. A novel deep learning (DL) based thigh muscle and surrounding tissues segmentation model was developed for fully automatic and reproducible cross-sectional area (CSA) and fat fraction (FF) quantification and tested in patients at 10 years after anterior cruciate ligament reconstructions.

METHODS

A DL model combining UNet and DenseNet was trained and tested using manually segmented thighs from 16 patients (32 legs). Segmentation accuracy was evaluated using Dice similarity coefficients (DSC) and average symmetric surface distance (ASSD). A UNet model was trained for comparison. These segmentations were used to obtain CSA and FF quantification. Reproducibility of CSA and FF quantification was tested with scan and rescan of six healthy subjects.

RESULTS

The proposed UNet and DenseNet had high agreement with manual segmentation (DSC >0.97, ASSD < 0.24) and improved performance compared with UNet. For hamstrings of the operated knee, the automated pipeline had largest absolute difference of 6.01% for CSA and 0.47% for FF as compared to manual segmentation. In reproducibility analysis, the average difference (absolute) in CSA quantification between scan and rescan was better for the automatic method as compared with manual segmentation (2.27% vs. 3.34%), whereas the average difference (absolute) in FF quantification were similar.

CONCLUSIONS

The proposed method exhibits excellent accuracy and reproducibility in CSA and FF quantification compared with manual segmentation and can be used in large-scale patient studies.

Automated MRI quantification of volumetric per-muscle fat fractions in the proximal leg of patients with muscular dystrophies

Muscular dystrophies (MD) are a class of rare genetic diseases resulting in progressive muscle weakness affecting specific muscle groups, depending on the type of disease. Disease progression is characterized by the gradual replacement of muscle tissue by fat, which can be assessed with fat-sensitive magnetic resonance imaging (MRI) and objectively evaluated by quantifying the fat fraction percentage (FF%) per muscle. Volumetric quantification of fat replacement over the full 3D extent of each muscle is more precise and potentially more sensitive than 2D quantification in few selected slices only, but it requires an accurate 3D segmentation of each muscle individually, which is time consuming when this has to be performed manually for a large number of muscles. A reliable, largely automated approach for 3D muscle segmentation is thus needed to facilitate the adoption of fat fraction quantification as a measure of MD disease progression in clinical routine practice, but this is challenging due to the variable appearance of the images and the ambiguity in the discrimination of the contours of adjacent muscles, especially when the normal image contrast is affected and diminished by the fat replacement. To deal with these challenges, we used deep learning to train AI-models to segment the muscles in the proximal leg from knee to hip in Dixon MRI images of healthy subjects as well as patients with MD. We demonstrate state-of-the-art segmentation results of all 18 muscles individually in terms of overlap (Dice score, DSC) with the manual ground truth delineation for images of cases with low fat infiltration (mean overall FF%: 11.3%; mean DSC: 95.3% per image, 84.4-97.3% per muscle) as well as with medium and high fat infiltration (mean overall FF%: 44.3%; mean DSC: 89.0% per image, 70.8-94.5% per muscle). In addition, we demonstrate that the segmentation performance is largely invariant to the field of view of the MRI scan, is generalizable to patients with different types of MD and that the manual delineation effort to create the training set can be drastically reduced without significant loss of segmentation quality by delineating only a subset of the slices.

Deformable image registration based on single or multi-atlas methods for automatic muscle segmentation and the generation of augmented imaging datasets

Muscle segmentation is a process relied upon to gather medical image-based muscle characterisation, useful in directly assessing muscle volume and geometry, that can be used as inputs to musculoskeletal modelling pipelines. Manual or semi-automatic techniques are typically employed to segment the muscles and quantify their properties, but they require significant manual labour and incur operator repeatability issues. In this study an automatic process is presented, aiming to segment all lower limb muscles from Magnetic Resonance (MR) imaging data simultaneously using three-dimensional (3D) deformable image registration (single inputs or multi-atlas). Twenty-three of the major lower limb skeletal muscles were segmented from five subjects, with an average Dice similarity coefficient of 0.72, and average absolute relative volume error (RVE) of 12.7% (average relative volume error of -2.2%) considering the optimal subject combinations. The multi-atlas approach showed slightly better accuracy (average DSC: 0.73; average RVE: 1.67%). Segmented MR imaging datasets of the lower limb are not widely available in the literature, limiting the potential of new, probabilistic methods such as deep learning to be used in the context of muscle segmentation. In this work, Non-linear deformable image registration is used to generate 69 manually checked, segmented, 3D, artificial datasets, allowing access for future studies to use these new methods, with a large amount of reliable reference data.

Deep learning-based fully automated body composition analysis of thigh CT: comparison with DXA measurement

OBJECTIVES

To compare volumetric CT with DL-based fully automated segmentation and dual-energy X-ray absorptiometry (DXA) in the measurement of thigh tissue composition.

METHODS

This prospective study was performed from January 2019 to December 2020. The participants underwent DXA to determine the body composition of the whole body and thigh. CT was performed in the thigh region; the images were automatically segmented into three muscle groups and adipose tissue by custom-developed DL-based automated segmentation software. Subsequently, the program reported the tissue composition of the thigh. The correlation and agreement between variables measured by DXA and CT were assessed. Then, CT thigh tissue volume prediction equations based on DXA-derived thigh tissue mass were developed using a general linear model.

RESULTS

In total, 100 patients (mean age, 44.9 years; 60 women) were evaluated. There was a strong correlation between the CT and DXA measurements (R = 0.813~0.98, p < 0.001). There was no significant difference in total soft tissue mass between DXA and CT measurement (p = 0.183). However, DXA overestimated thigh lean (muscle) mass and underestimated thigh total fat mass (p < 0.001). The DXA-derived lean mass was an average of 10% higher than the CT-derived lean mass and 47% higher than the CT-derived lean muscle mass. The DXA-derived total fat mass was approximately 20% lower than the CT-derived total fat mass. The predicted CT tissue volume using DXA-derived data was highly correlated with actual CT-measured tissue volume in the validation group (R² = 0.96~0.97, p < 0.001).

CONCLUSIONS

Volumetric CT measurements with DL-based fully automated segmentation are a rapid and more accurate method for measuring thigh tissue composition.

KEY POINTS

• There was a positive correlation between CT and DXA measurements in both the whole body and thigh. • DXA overestimated thigh lean mass by 10%, lean muscle mass by 47%, but underestimated total fat mass by 20% compared to the CT method. • The equations for predicting CT volume (cm³) were developed using DXA data (g), age, height (cm), and body weight (kg) and good model performance was proven in the validation study.

MRI biomarker of muscle composition is associated with severity of pelvic organ prolapse

BACKGROUND

The pathophysiology of pelvic organ prolapse is largely unknown. We hypothesized that reduced muscle mass on magnetic resonance defecography (MRD) is associated with increased pelvic floor laxity. The aim of this study was to compare the psoas and puborectalis muscle mass composition and cross-sectional area among patients with or without pelvic laxity.

METHODS

An observational retrospective study was conducted on women > age 18 years old who had undergone MRD for pelvic floor complaints from January 2020 to December 2020 at Stanford Pelvic Health Center. Pelvic floor laxity, pelvic organ descent, and rectal prolapse were characterized by standard measurements on MRD and compared to the psoas (L4 level) and puborectalis muscle index (cross-sectional area adjusted by height) and relative fat fraction, quantified by utilizing a 2-point Dixon technique. Regression analysis was used to quantify the association between muscle characteristics and pelvic organ measurements.

RESULTS

The psoas fat fraction was significantly elevated in patients with abnormally increased resting and strain H and M lines (p < 0.05) and increased with rising grades of Oxford rectal prolapse (p = 0.0001), uterovaginal descent (p = 0.001) and bladder descent (p = 0.0005). In multivariate regression analysis, adjusted for age and body mass index, the psoas fat fraction (not muscle index) was an independent risk factor for abnormal strain H and M line; odds ratio (95% confidence interval) of 17.8 (2-155.4) and 18.5 (1.3-258.3) respectively, and rising Oxford grade of rectal prolapse 153.9 (4.4-5383) and bladder descent 12.4 (1.5-106). Puborectalis fat fraction was increased by rising grades of Oxford rectal prolapse (p = 0.0002).

CONCLUSIONS

Severity of pelvic organ prolapse appears to be associated with increasing psoas muscle fat fraction, a biomarker for reduced skeletal muscle mass. Future prospective research is needed to determine if sarcopenia may predict postsurgical outcomes after pelvic organ prolapse repair.

Artificial intelligence for MRI diagnosis of joints: a scoping review of the current state-of-the-art of deep learning-based approaches

Deep learning-based MRI diagnosis of internal joint derangement is an emerging field of artificial intelligence, which offers many exciting possibilities for musculoskeletal radiology. A variety of investigational deep learning algorithms have been developed to detect anterior cruciate ligament tears, meniscus tears, and rotator cuff disorders. Additional deep learning-based MRI algorithms have been investigated to detect Achilles tendon tears, recurrence prediction of musculoskeletal neoplasms, and complex segmentation of nerves, bones, and muscles. Proof-of-concept studies suggest that deep learning algorithms may achieve similar diagnostic performances when compared to human readers in meta-analyses; however, musculoskeletal radiologists outperformed most deep learning algorithms in studies including a direct comparison. Earlier investigations and developments of deep learning algorithms focused on the binary classification of the presence or absence of an abnormality, whereas more advanced deep learning algorithms start to include features for characterization and severity grading. While many studies have focused on comparing deep learning algorithms against human readers, there is a paucity of data on the performance differences of radiologists interpreting musculoskeletal MRI studies without and with artificial intelligence support. Similarly, studies demonstrating the generalizability and clinical applicability of deep learning algorithms using realistic clinical settings with workflow-integrated deep learning algorithms are sparse. Contingent upon future studies showing the clinical utility of deep learning algorithms, artificial intelligence may eventually translate into clinical practice to assist detection and characterization of various conditions on musculoskeletal MRI exams.

Automatic quadriceps and patellae segmentation of MRI with cascaded U2 -Net and SASSNet deep learning model

PURPOSE

Automatic muscle segmentation is critical for advancing our understanding of human physiology, biomechanics, and musculoskeletal pathologies, as it allows for timely exploration of large multi-dimensional image sets. Segmentation models are rarely developed/validated for the pediatric model. As such, autosegmentation is not available to explore how muscle architectural changes during development and how disease/pathology affects the developing musculoskeletal system. Thus, we aimed to develop and validate an end-to-end, fully automated, deep learning model for accurate segmentation of the rectus femoris and vastus lateral, medialis, and intermedialis using a pediatric database.

METHODS

We developed a two-stage cascaded deep learning model in a coarse-to-fine manner. In the first stage, the U² -Net roughly detects the muscle subcompartment region. Then, in the second stage, the shape-aware 3D semantic segmentation method SASSNet refines the cropped target regions to generate the more finer and accurate segmentation masks. We utilized multifeature image maps in both stages to stabilize performance and validated their use with an ablation study. The second-stage SASSNet was independently run and evaluated with three different cropped region resolutions: the original image resolution, and images downsampled 2× and 4× (high, mid, and low). The relationship between image resolution and segmentation accuracy was explored. In addition, the patella was included as a comparator to past work. We evaluated segmentation accuracy using leave-one-out testing on a database of 3D MR images (0.43 × 0.43 × 2 mm) from 40 pediatric participants (age 15.3 ± 1.9 years, 55.8 ± 11.8 kg, 164.2 ± 7.9 cm, 38F/2 M).

RESULTS

The mid-resolution second stage produced the best results for the vastus medialis, rectus femoris, and patella (Dice similarity coefficient = 95.0%, 95.1%, 93.7%), whereas the low-resolution second stage produced the best results for the vastus lateralis and vastus intermedialis (DSC = 94.5% and 93.7%). In comparing the low- to mid-resolution cases, the vasti intermedialis, vastus medialis, rectus femoris, and patella produced significant differences (p = 0.0015, p = 0.0101, p < 0.0001, p = 0.0003) and the vasti lateralis did not (p = 0.2177). The high-resolution stage 2 had significantly lower accuracy (1.0 to 4.4 dice percentage points) compared to both the mid- and low-resolution routines (p value ranged from < 0.001 to 0.04). The one exception was the rectus femoris, where there was no difference between the low- and high-resolution cases. The ablation study demonstrated that the multifeature is more reliable than the single feature.

CONCLUSIONS

Our successful implementation of this two-stage segmentation pipeline provides a critical tool for expanding pediatric muscle physiology and clinical research. With a relatively small and variable dataset, our fully automatic segmentation technique produces accuracies that matched or exceeded the current state of the art. The two-stage segmentation avoids memory issues and excessive run times by using a first stage focused on cropping out unnecessary data. The excellent Dice similarity coefficients improve upon previous template-based automatic and semiautomatic methodologies targeting the leg musculature. More importantly, with a naturally variable dataset (size, shape, etc.), the proposed model demonstrates slightly improved accuracies, compared to previous neural networks methods.

Local MRI-based measures of thigh adipose tissue derived from fully automated deep convolutional neural network-based segmentation show a comparable responsiveness to bidirectional change in body weight as from quality controlled manual segmentation

BACKGROUND

Thigh intermuscular (IMF) and subcutaneous (SCF) fat are associated with joint function, inflammation and knee osteoarthritis. Fully automated segmentation from MRI is important to study the above relationship in larger cohorts. However, such algorithms are not clinically evaluated for longitudinal studies. Our aim was to evaluate a fully automated U-Net segmentation approach and its ability to detect longitudinal changes in thigh IMF and SCF during weight changes compared to manual segmentation.

METHODS

103 Osteoarthritis Initiative subjects, were studied, 52 with> 10% weight loss, and 51 with> 10% weight gain over 2-years. Longitudinal change in IMF and SCF were determined from baseline and year-2 axial thigh MRIs using U-Net segmentation. The standardised response mean (SRM) was used as measure of sensitivity to change.

RESULTS

The U-Net took substantially less time (single-slice MRI:< 1 s) and IMF and SCF showed very similar sensitivity to change as manual segmentation: With an average weight gain of + 14%, we observed an + 12% /+ 26% increase in IMF / SCF (SRM=0.99 /1.03) using the U-Net, compared with + 21% /+ 27% (SRM=0.60 /1.07) for manual segmentation. During an average weight loss of - 18%, we observed an - 14% /- 22% reduction in IMF /SCF (SRM = - 1.04 /-1.20) using the U-Net, compared with - 16% /- 22% (SRM = - 0.70 /-1.23) for manual segmentation.

CONCLUSION

U-Net segmentation replicates longitudinal changes of IMF and SCF associated with weight changes with a similar sensitivity to change as manual segmentation. This method is applicable to large databases for studying relationships between IMF and SCF and various disease conditions.

Two decades of advances in muscle imaging in children: from pattern recognition of muscle diseases to quantification and machine learning approaches

Muscle imaging has progressively gained popularity in the neuromuscular field. Together with detailed clinical examination and muscle biopsy, it has become one of the main tools for deep phenotyping and orientation of etiological diagnosis. Even in the current era of powerful new generation sequencing, muscle MRI has arisen as a tool for prioritization of certain genetic entities, supporting the pathogenicity of variants of unknown significance and facilitating diagnosis in cases with an initially inconclusive genetic study. Although the utility of muscle imaging is increasingly clear, it has not reached its full potential in clinical practice. Pattern recognition is known for a number of diseases and will certainly be enhanced by the use of machine learning approaches. For instance, MRI heatmap representations might be confronted with molecular results by obtaining a probabilistic diagnosis based in each disease "MRI fingerprints". Muscle ultrasound as a screening tool and quantified techniques such as Dixon MRI seem still underdeveloped. In this paper, we aim to appraise the advances in recent years in pediatric muscle imaging and try to define areas of uncertainty and potential advances that might become standardized to be widely used in the future.

Deep learning for automatic segmentation of thigh and leg muscles

Objective

In this study we address the automatic segmentation of selected muscles of the thigh and leg through a supervised deep learning approach.

Material and methods

The application of quantitative imaging in neuromuscular diseases requires the availability of regions of interest (ROI) drawn on muscles to extract quantitative parameters. Up to now, manual drawing of ROIs has been considered the gold standard in clinical studies, with no clear and universally accepted standardized procedure for segmentation. Several automatic methods, based mainly on machine learning and deep learning algorithms, have recently been proposed to discriminate between skeletal muscle, bone, subcutaneous and intermuscular adipose tissue. We develop a supervised deep learning approach based on a unified framework for ROI segmentation.

Results

The proposed network generates segmentation maps with high accuracy, consisting in Dice Scores ranging from 0.89 to 0.95, with respect to “ground truth” manually segmented labelled images, also showing high average performance in both mild and severe cases of disease involvement (i.e. entity of fatty replacement).

Discussion

The presented results are promising and potentially translatable to different skeletal muscle groups and other MRI sequences with different contrast and resolution.

Fully automated pelvic bone segmentation in multiparameteric MRI using a 3D convolutional neural network

BACKGROUND

Accurate segmentation of pelvic bones is an initial step to achieve accurate detection and localisation of pelvic bone metastases. This study presents a deep learning-based approach for automated segmentation of normal pelvic bony structures in multiparametric magnetic resonance imaging (mpMRI) using a 3D convolutional neural network (CNN).

METHODS

This retrospective study included 264 pelvic mpMRI data obtained between 2018 and 2019. The manual annotations of pelvic bony structures (which included lumbar vertebra, sacrococcyx, ilium, acetabulum, femoral head, femoral neck, ischium, and pubis) on diffusion-weighted imaging (DWI) and apparent diffusion coefficient (ADC) images were used to create reference standards. A 3D U-Net CNN was employed for automatic pelvic bone segmentation. Additionally, 60 mpMRI data from 2020 were included and used to evaluate the model externally.

RESULTS

The CNN achieved a high Dice similarity coefficient (DSC) average in both testing (0.80 [DWI images] and 0.85 [ADC images]) and external (0.79 [DWI images] and 0.84 [ADC images]) validation sets. Pelvic bone volumes measured with manual and CNN-predicted segmentations were highly correlated (R² value of 0.84-0.97) and in close agreement (mean bias of 2.6-4.5 cm³). A SCORE system was designed to qualitatively evaluate the model for which both testing and external validation sets achieved high scores in terms of both qualitative evaluation and concordance between two readers (ICC = 0.904; 95% confidence interval: 0.871-0.929).

CONCLUSIONS

A deep learning-based method can achieve automated pelvic bone segmentation on DWI and ADC images with suitable quantitative and qualitative performance.

Machine Learning and Radiomics Applications in Esophageal Cancers Using Non-Invasive Imaging Methods-A Critical Review of Literature

Esophageal cancer (EC) is of public health significance as one of the leading causes of cancer death worldwide. Accurate staging, treatment planning and prognostication in EC patients are of vital importance. Recent advances in machine learning (ML) techniques demonstrate their potential to provide novel quantitative imaging markers in medical imaging. Radiomics approaches that could quantify medical images into high-dimensional data have been shown to improve the imaging-based classification system in characterizing the heterogeneity of primary tumors and lymph nodes in EC patients. In this review, we aim to provide a comprehensive summary of the evidence of the most recent developments in ML application in imaging pertinent to EC patient care. According to the published results, ML models evaluating treatment response and lymph node metastasis achieve reliable predictions, ranging from acceptable to outstanding in their validation groups. Patients stratified by ML models in different risk groups have a significant or borderline significant difference in survival outcomes. Prospective large multi-center studies are suggested to improve the generalizability of ML techniques with standardized imaging protocols and harmonization between different centers.

Overview of MR Image Segmentation Strategies in Neuromuscular Disorders

Neuromuscular disorders are rare diseases for which few therapeutic strategies currently exist. Assessment of therapeutic strategies efficiency is limited by the lack of biomarkers sensitive to the slow progression of neuromuscular diseases (NMD). Magnetic resonance imaging (MRI) has emerged as a tool of choice for the development of qualitative scores for the study of NMD. The recent emergence of quantitative MRI has enabled to provide quantitative biomarkers more sensitive to the evaluation of pathological changes in muscle tissue. However, in order to extract these biomarkers from specific regions of interest, muscle segmentation is mandatory. The time-consuming aspect of manual segmentation has limited the evaluation of these biomarkers on large cohorts. In recent years, several methods have been proposed to make the segmentation step automatic or semi-automatic. The purpose of this study was to review these methods and discuss their reliability, reproducibility, and limitations in the context of NMD. A particular attention has been paid to recent deep learning methods, as they have emerged as an effective method of image segmentation in many other clinical contexts.