Unsupervised classification of multi-contrast magnetic resonance histology of peripheral arterial disease lesions using a convolutional variational autoencoder with a Gaussian mixture model in latent space: A technical feasibility study

PURPOSE

To investigate the feasibility of a deep learning algorithm combining variational autoencoder (VAE) and two-dimensional (2D) convolutional neural networks (CNN) for automatically quantifying hard tissue presence and morphology in multi-contrast magnetic resonance (MR) images of peripheral arterial disease (PAD) occlusive lesions.

METHODS

Multi-contrast MR images (T2-weighted and ultrashort echo time) were acquired from lesions harvested from six amputated legs with high isotropic spatial resolution (0.078 mm and 0.156 mm, respectively) at 9.4 T. A total of 4014 pseudo-color combined images were generated, with 75% used to train a VAE employing custom 2D CNN layers. A Gaussian mixture model (GMM) was employed to classify the latent space data into four tissue classes: I) concentric calcified (c), II) eccentric calcified (e), III) occluded with hard tissue (h) and IV) occluded with soft tissue (s). Test image probabilities, encoded by the trained VAE were used to evaluate model performance.

RESULTS

GMM component classification probabilities ranged from 0.92 to 0.97 for class (c), 1.00 for class (e), 0.82-0.95 for class (h) and 0.56-0.93 for the remaining class (s). Due to the complexity of soft-tissue lesions reflected in the heterogeneity of the pseudo-color images, more GMM components (n=17) were attributed to class (s), compared to the other three (c, e and h) (n=6).

CONCLUSION

Combination of 2D CNN VAE and GMM achieves high classification probabilities for hard tissue-containing lesions. Automatic recognition of these classes may aid therapeutic decision-making and identifying uncrossable lesions prior to endovascular intervention.

A comparative study of two automated solutions for cross-sectional skeletal muscle measurement from abdominal computed tomography images

BACKGROUND

Measurement of cross-sectional muscle area (CSMA) at the mid third lumbar vertebra (L3) level from computed tomography (CT) images is becoming one of the reference methods for sarcopenia diagnosis. However, manual skeletal muscle segmentation is tedious and is thus restricted to research. Automated solutions are required for use in clinical practice.

PURPOSE

The aim of this study was to compare the reliability of two automated solutions for the measurement of CSMA.

METHODS

We conducted a retrospective analysis of CT images in our hospital database. We included consecutive individuals hospitalized at the Grenoble University Hospital in France between January and May 2018 with abdominal CT images and sagittal reconstruction. We used two types of software to automatically segment skeletal muscle: ABACS, a module of the SliceOmatic software solution "ABACS-SliceOmatic," and a deep learning-based solution called "AutoMATiCA." Manual segmentation was performed by a medical expert to generate reference data using "SliceOmatic." The Dice similarity coefficient (DSC) was used to measure overlap between the results of the manual and the automated segmentations. The DSC value for each method was compared with the Mann-Whitney U test.

RESULTS

A total of 676 hospitalized individuals was retrospectively included (365 males [53.8%] and 312 females [46.2%]). The median DSC for SliceOmatic vs AutoMATiCA (0.969 [5th percentile: 0.909]) was greater than the median DSC for SliceOmatic vs. ABACS-SliceOmatic (0.949 [5th percentile: 0.836]) (p < 0.001).

CONCLUSIONS

AutoMATiCA, which used artificial intelligence, was more reliable than ABACS-SliceOmatic for skeletal muscle segmentation at the L3 level in a cohort of hospitalized individuals. The next step is to develop and validate a neural network that can identify L3 slices, which is currently a fastidious process.

Automated Opportunistic Trabecular Volumetric Bone Mineral Density Extraction Outperforms Manual Measurements for the Prediction of Vertebral Fractures in Routine CT

Opportunistic osteoporosis screening using multidetector CT-scans (MDCT) and convolutional neural network (CNN)-derived segmentations of the spine to generate volumetric bone mineral density (vBMD) bears the potential to improve incidental osteoporotic vertebral fracture (VF) prediction. However, the performance compared to the established manual opportunistic vBMD measures remains unclear. Hence, we investigated patients with a routine MDCT of the spine who had developed a new osteoporotic incidental VF and frequency matched to patients without incidental VFs as assessed on follow-up MDCT images after 1.5 years. Automated vBMD was generated using CNN-generated segmentation masks and asynchronous calibration. Additionally, manual vBMD was sampled by two radiologists. Automated vBMD measurements in patients with incidental VFs at 1.5-years follow-up (n = 53) were significantly lower compared to patients without incidental VFs (n = 104) (83.6 ± 29.4 mg/cm³ vs. 102.1 ± 27.7 mg/cm³, p < 0.001). This comparison was not significant for manually assessed vBMD (99.2 ± 37.6 mg/cm³ vs. 107.9 ± 33.9 mg/cm³, p = 0.30). When adjusting for age and sex, both automated and manual vBMD measurements were significantly associated with incidental VFs at 1.5-year follow-up, however, the associations were stronger for automated measurements (β = -0.32; 95% confidence interval (CI): -20.10, 4.35; p < 0.001) compared to manual measurements (β = -0.15; 95% CI: -11.16, 5.16; p < 0.03). In conclusion, automated opportunistic measurements are feasible and can be useful for bone mineral density assessment in clinical routine.

Transfer-Learning Deep Radiomics and Hand-Crafted Radiomics for Classifying Lymph Nodes from Contrast-Enhanced Computed Tomography in Lung Cancer

OBJECTIVES

Positron emission tomography (PET) is currently considered the non-invasive reference standard for lymph node (N-)staging in lung cancer. However, not all patients can undergo this diagnostic procedure due to high costs, limited availability, and additional radiation exposure. The purpose of this study was to predict the PET result from traditional contrast-enhanced computed tomography (CT) and to test different feature extraction strategies.

METHODS

In this study, 100 lung cancer patients underwent a contrast-enhanced ¹⁸F-fluorodeoxyglucose (FDG) PET/CT scan between August 2012 and December 2019. We trained machine learning models to predict FDG uptake in the subsequent PET scan. Model inputs were composed of (i) traditional "hand-crafted" radiomics features from the segmented lymph nodes, (ii) deep features derived from a pretrained EfficientNet-CNN, and (iii) a hybrid approach combining (i) and (ii).

RESULTS

In total, 2734 lymph nodes [555 (20.3%) PET-positive] from 100 patients [49% female; mean age 65, SD: 14] with lung cancer (60% adenocarcinoma, 21% plate epithelial carcinoma, 8% small-cell lung cancer) were included in this study. The area under the receiver operating characteristic curve (AUC) ranged from 0.79 to 0.87, and the scaled Brier score (SBS) ranged from 16 to 36%. The random forest model (iii) yielded the best results [AUC 0.871 (0.865-0.878), SBS 35.8 (34.2-37.2)] and had significantly higher model performance than both approaches alone (AUC: p < 0.001, z = 8.8 and z = 22.4; SBS: p < 0.001, z = 11.4 and z = 26.6, against (i) and (ii), respectively).

CONCLUSION

Both traditional radiomics features and transfer-learning deep radiomics features provide relevant and complementary information for non-invasive N-staging in lung cancer.

Deep Learning Neural Networks to Predict Serious Complications After Bariatric Surgery: Analysis of Scandinavian Obesity Surgery Registry Data

Background

Obesity is one of today’s most visible public health problems worldwide. Although modern bariatric surgery is ostensibly considered safe, serious complications and mortality still occur in some patients.

Objective

This study aimed to explore whether serious postoperative complications of bariatric surgery recorded in a national quality registry can be predicted preoperatively using deep learning methods.

Methods

Patients who were registered in the Scandinavian Obesity Surgery Registry (SOReg) between 2010 and 2015 were included in this study. The patients who underwent a bariatric procedure between 2010 and 2014 were used as training data, and those who underwent a bariatric procedure in 2015 were used as test data. Postoperative complications were graded according to the Clavien-Dindo classification, and complications requiring intervention under general anesthesia or resulting in organ failure or death were considered serious. Three supervised deep learning neural networks were applied and compared in our study: multilayer perceptron (MLP), convolutional neural network (CNN), and recurrent neural network (RNN). The synthetic minority oversampling technique (SMOTE) was used to artificially augment the patients with serious complications. The performances of the neural networks were evaluated using accuracy, sensitivity, specificity, Matthews correlation coefficient, and area under the receiver operating characteristic curve.

Results

In total, 37,811 and 6250 patients were used as the training data and test data, with incidence rates of serious complication of 3.2% (1220/37,811) and 3.0% (188/6250), respectively. When trained using the SMOTE data, the MLP appeared to have a desirable performance, with an area under curve (AUC) of 0.84 (95% CI 0.83-0.85). However, its performance was low for the test data, with an AUC of 0.54 (95% CI 0.53-0.55). The performance of CNN was similar to that of MLP. It generated AUCs of 0.79 (95% CI 0.78-0.80) and 0.57 (95% CI 0.59-0.61) for the SMOTE data and test data, respectively. Compared with the MLP and CNN, the RNN showed worse performance, with AUCs of 0.65 (95% CI 0.64-0.66) and 0.55 (95% CI 0.53-0.57) for the SMOTE data and test data, respectively.

Conclusions

MLP and CNN showed improved, but limited, ability for predicting the postoperative serious complications after bariatric surgery in the Scandinavian Obesity Surgery Registry data. However, the overfitting issue is still apparent and needs to be overcome by incorporating intra- and perioperative information.