A systematic literature review: exploring the challenges of ensemble model for medical imaging

BACKGROUND

Medical imaging has been essential and has provided clinicians with useful information about the human body to diagnose various health issues. Early diagnosis of diseases based on medical imaging can mitigate the risk of severe consequences and enhance long-term health outcomes. Nevertheless, the task of diagnosing diseases based on medical imaging can be challenging due to the exclusive ability of clinicians to interpret the outcomes of medical imaging, which is time-consuming and susceptible to human fallibility. The ensemble model has the potential to enhance the accuracy of diagnoses of diseases based on medical imaging by analyzing vast volumes of data and identifying trends that may not be immediately apparent to doctors. However, it takes a lot of memory and processing resources to train and maintain several ensemble models. These challenges highlight the necessity of effective and scalable ensemble models that can manage the intricacies of medical imaging assignments.

METHODS

This study employed an SLR technique to explore the latest advancements and approaches. By conducting a thorough and systematic search of Scopus and Web of Science databases in accordance with the principles outlined in the PRISMA, employing keywords namely ensemble model and medical imaging.

RESULTS

This study included a total of 75 papers that were published between 2019 and 2024. The categorization, methodologies, and use of medical imaging were key factors examined in the analysis of the 30 cited papers included in this study, with a focus on diagnosing diseases.

CONCLUSIONS

Researchers have observed the emergence of an ensemble model for disease diagnosis using medical imaging since it has demonstrated improved accuracy and may guide future studies by highlighting the limitations of the ensemble model.

Automated prognosis of renal function decline in ADPKD patients using deep learning

An accurate prognosis of renal function decline in Autosomal Dominant Polycystic Kidney Disease (ADPKD) is crucial for early intervention. Current biomarkers used are height-adjusted total kidney volume (HtTKV), estimated glomerular filtration rate (eGFR), and patient age. However, manually measuring kidney volume is time-consuming and subject to observer variability. Additionally, incorporating automatically generated features from kidney MRI images, along with conventional biomarkers, can enhance prognostic improvement. To address these issues, we developed two deep-learning algorithms. Firstly, an automated kidney volume segmentation model accurately calculates HtTKV. Secondly, we utilize segmented kidney volumes, predicted HtTKV, age, and baseline eGFR to predict chronic kidney disease (CKD) stages >=3A, >=3B, and a 30% decline in eGFR after 8 years from the baseline visit. Our approach combines a convolutional neural network (CNN) and a multi-layer perceptron (MLP). Our study included 135 subjects and the AUC scores obtained were 0.96, 0.96, and 0.95 for CKD stages >=3A, >=3B, and a 30% decline in eGFR, respectively. Furthermore, our algorithm achieved a Pearson correlation coefficient of 0.81 between predicted and measured eGFR decline. We extended our approach to predict distinct CKD stages after eight years with an AUC of 0.97. The proposed approach has the potential to enhance monitoring and facilitate prognosis in ADPKD patients, even in the early disease stages.

Automated peripancreatic vessel segmentation and labeling based on iterative trunk growth and weakly supervised mechanism

Peripancreatic vessel segmentation and anatomical labeling are pivotal aspects in aiding surgical planning and prognosis for patients with pancreatic tumors. Nevertheless, prevailing techniques often fall short in achieving satisfactory segmentation performance for the peripancreatic vein (PPV), leading to predictions characterized by poor integrity and connectivity. Besides, unsupervised labeling algorithms usually cannot deal with complex anatomical variation while fully supervised methods require a large number of voxel-wise annotations for training, which is very labor-intensive and time-consuming. To address these two problems, we propose an Automated Peripancreatic vEssel Segmentation and lAbeling (APESA) framework, to not only highly improve the segmentation performance for PPV, but also efficiently identify the peripancreatic artery (PPA) branches. There are two core modules in our proposed APESA framework: iterative trunk growth module (ITGM) for vein segmentation and weakly supervised labeling mechanism (WSLM) for artery labeling. The ITGM is composed of a series of iterative submodules, each of which chooses the largest connected component of the previous PPV segmentation as the trunk of a tree structure, seeks for the potential missing branches around the trunk by our designed branch proposal network, and facilitates trunk growth under the connectivity constraint. The WSLM incorporates the rule-based pseudo label generation with less expert participation, an anatomical labeling network to learn the branch distribution voxel by voxel, and adaptive radius-based postprocessing to refine the branch structures of the labeling predictions. Our achieved Dice of 94.01% for PPV segmentation on our collected dataset represents an approximately 10% accuracy improvement compared to state-of-the-art methods. Additionally, we attained a Dice of 97.01% for PPA segmentation and competitive labeling performance for PPA labeling compared to prior works. Our source codes will be publicly available at https://github.com/ZouLiwen-1999/APESA.

Fine-Grained and Multiple Classification for Alzheimer's Disease With Wavelet Convolution Unit Network

In this article, we propose a novel wavelet convolution unit for the image-oriented neural network to integrate wavelet analysis with a vanilla convolution operator to extract deep abstract features more efficiently. On one hand, in order to acquire non-local receptive fields and avoid information loss, we define a new convolution operation by composing a traditional convolution function and approximate and detailed representations after single-scale wavelet decomposition of source images. On the other hand, multi-scale wavelet decomposition is introduced to obtain more comprehensive multi-scale feature information. Then, we fuse all these cross-scale features to improve the problem of inaccurate localization of singular points. Given the novel wavelet convolution unit, we further design a network based on it for fine-grained Alzheimer's disease classifications (i.e., Alzheimer's disease, Normal controls, early mild cognitive impairment, late mild cognitive impairment). Up to now, only a few methods have studied one or several fine-grained classifications, and even fewer methods can achieve both fine-grained and multi-class classifications. We adopt the novel network and diffuse tensor images to achieve fine-grained classifications, which achieved state-of-the-art accuracy for all eight kinds of fine-grained classifications, up to 97.30%, 95.78%, 95.00%, 94.00%, 97.89%, 95.71%, 95.07%, 93.79%. In order to build a reference standard for Alzheimer's disease classifications, we actually implemented all twelve coarse-grained and fine-grained classifications. The results show that the proposed method achieves solidly high accuracy for them. Its classification ability greatly exceeds any kind of existing Alzheimer's disease classification method.

Deep Learning-Based Automated Magnetic Resonance Image Segmentation of the Lumbar Structure and Its Adjacent Structures at the L4/5 Level

(1) Background: This study aims to develop a deep learning model based on a 3D Deeplab V3+ network to automatically segment multiple structures from magnetic resonance (MR) images at the L4/5 level. (2) Methods: After data preprocessing, the modified 3D Deeplab V3+ network of the deep learning model was used for the automatic segmentation of multiple structures from MR images at the L4/5 level. We performed five-fold cross-validation to evaluate the performance of the deep learning model. Subsequently, the Dice Similarity Coefficient (DSC), precision, and recall were also used to assess the deep learning model's performance. Pearson's correlation coefficient analysis and the Wilcoxon signed-rank test were employed to compare the morphometric measurements of 3D reconstruction models generated by manual and automatic segmentation. (3) Results: The deep learning model obtained an overall average DSC of 0.886, an average precision of 0.899, and an average recall of 0.881 on the test sets. Furthermore, all morphometry-related measurements of 3D reconstruction models revealed no significant difference between ground truth and automatic segmentation. Strong linear relationships and correlations were also obtained in the morphometry-related measurements of 3D reconstruction models between ground truth and automated segmentation. (4) Conclusions: We found it feasible to perform automated segmentation of multiple structures from MR images, which would facilitate lumbar surgical evaluation by establishing 3D reconstruction models at the L4/5 level.

EmergeNet: A novel deep-learning based ensemble segmentation model for emergence timing detection of coleoptile

The emergence timing of a plant, i.e., the time at which the plant is first visible from the surface of the soil, is an important phenotypic event and is an indicator of the successful establishment and growth of a plant. The paper introduces a novel deep-learning based model called EmergeNet with a customized loss function that adapts to plant growth for coleoptile (a rigid plant tissue that encloses the first leaves of a seedling) emergence timing detection. It can also track its growth from a time-lapse sequence of images with cluttered backgrounds and extreme variations in illumination. EmergeNet is a novel ensemble segmentation model that integrates three different but promising networks, namely, SEResNet, InceptionV3, and VGG19, in the encoder part of its base model, which is the UNet model. EmergeNet can correctly detect the coleoptile at its first emergence when it is tiny and therefore barely visible on the soil surface. The performance of EmergeNet is evaluated using a benchmark dataset called the University of Nebraska-Lincoln Maize Emergence Dataset (UNL-MED). It contains top-view time-lapse images of maize coleoptiles starting before the occurrence of their emergence and continuing until they are about one inch tall. EmergeNet detects the emergence timing with 100% accuracy compared with human-annotated ground-truth. Furthermore, it significantly outperforms UNet by generating very high-quality segmented masks of the coleoptiles in both natural light and dark environmental conditions.

Investigation of biases in convolutional neural networks for semantic segmentation using performance sensitivity analysis

The application of deep neural networks for segmentation in medical imaging has gained substantial interest in recent years. In many cases, this variant of machine learning has been shown to outperform other conventional segmentation approaches. However, little is known about its general applicability. Especially the robustness against image modifications (e.g., intensity variations, contrast variations, spatial alignment) has hardly been investigated. Data augmentation is often used to compensate for sensitivity to such changes, although its effectiveness has not yet been studied. Therefore, the goal of this study was to systematically investigate the sensitivity to variations in input data with respect to segmentation of medical images using deep learning. This approach was tested with two publicly available segmentation frameworks (DeepMedic and TractSeg). In the case of DeepMedic, the performance was tested using ground truth data, while in the case of TractSeg, the STAPLE technique was employed. In both cases, sensitivity analysis revealed significant dependence of the segmentation performance on input variations. The effects of different data augmentation strategies were also shown, making this type of analysis a useful tool for selecting the right parameters for augmentation. The proposed analysis should be applied to any deep learning image segmentation approach, unless the assessment of sensitivity to input variations can be directly derived from the network.

Can Generative Adversarial Networks help to overcome the limited data problem in segmentation?

PURPOSE

For image translational tasks, the application of deep learning methods showed that Generative Adversarial Network (GAN) architectures outperform the traditional U-Net networks, when using the same training data size. This study investigates whether this performance boost can also be expected for segmentation tasks with small training dataset size.

MATERIALS/METHODS

Two models were trained on varying training dataset sizes ranging from 1-100 patients: a) U-Net and b) U-Net with patch discriminator (conditional GAN). The performance of both models to segment the male pelvis on CT-data was evaluated (Dice similarity coefficient, Hausdorff) with respect to training data size.

RESULTS

No significant differences were observed between the U-Net and cGAN when the models were trained with the same training sizes up to 100 patients. The training dataset size had a significant impact on the models' performances, with vast improvements when increasing dataset sizes from 1 to 20 patients.

CONCLUSION

When introducing GANs for the segmentation task no significant performance boost was observed in our experiments, even in segmentation models developed on small datasets.

Deep Learning-Based Total Kidney Volume Segmentation in Autosomal Dominant Polycystic Kidney Disease Using Attention, Cosine Loss, and Sharpness Aware Minimization

Early detection of the autosomal dominant polycystic kidney disease (ADPKD) is crucial as it is one of the most common causes of end-stage renal disease (ESRD) and kidney failure. The total kidney volume (TKV) can be used as a biomarker to quantify disease progression. The TKV calculation requires accurate delineation of kidney volumes, which is usually performed manually by an expert physician. However, this is time-consuming and automated segmentation is warranted. Furthermore, the scarcity of large annotated datasets hinders the development of deep learning solutions. In this work, we address this problem by implementing three attention mechanisms into the U-Net to improve TKV estimation. Additionally, we implement a cosine loss function that works well on image classification tasks with small datasets. Lastly, we apply a technique called sharpness aware minimization (SAM) that helps improve the generalizability of networks. Our results show significant improvements (p-value < 0.05) over the reference kidney segmentation U-Net. We show that the attention mechanisms and/or the cosine loss with SAM can achieve a dice score (DSC) of 0.918, a mean symmetric surface distance (MSSD) of 1.20 mm with the mean TKV difference of −1.72%, and R2 of 0.96 while using only 100 MRI datasets for training and testing. Furthermore, we tested four ensembles and obtained improvements over the best individual network, achieving a DSC and MSSD of 0.922 and 1.09 mm, respectively.

Automatic Segmentation of Novel Coronavirus Pneumonia Lesions in CT Images Utilizing Deep-Supervised Ensemble Learning Network

Background: The novel coronavirus disease 2019 (COVID-19) has been spread widely in the world, causing a huge threat to the living environment of people. Objective: Under CT imaging, the structure features of COVID-19 lesions are complicated and varied greatly in different cases. To accurately locate COVID-19 lesions and assist doctors to make the best diagnosis and treatment plan, a deep-supervised ensemble learning network is presented for COVID-19 lesion segmentation in CT images. Methods: Since a large number of COVID-19 CT images and the corresponding lesion annotations are difficult to obtain, a transfer learning strategy is employed to make up for the shortcoming and alleviate the overfitting problem. Based on the reality that traditional single deep learning framework is difficult to extract complicated and varied COVID-19 lesion features effectively that may cause some lesions to be undetected. To overcome the problem, a deep-supervised ensemble learning network is presented to combine with local and global features for COVID-19 lesion segmentation. Results: The performance of the proposed method was validated in experiments with a publicly available dataset. Compared with manual annotations, the proposed method acquired a high intersection over union (IoU) of 0.7279 and a low Hausdorff distance (H) of 92.4604. Conclusion: A deep-supervised ensemble learning network was presented for coronavirus pneumonia lesion segmentation in CT images. The effectiveness of the proposed method was verified by visual inspection and quantitative evaluation. Experimental results indicated that the proposed method has a good performance in COVID-19 lesion segmentation.

Automated Screening for Abdominal Aortic Aneurysm in CT Scans under Clinical Conditions Using Deep Learning

Abdominal aortic aneurysms (AAA) may remain clinically silent until they enlarge and patients present with a potentially lethal rupture. This necessitates early detection and elective treatment. The goal of this study was to develop an easy-to-train algorithm which is capable of automated AAA screening in CT scans and can be applied to an intra-hospital environment. Three deep convolutional neural networks (ResNet, VGG-16 and AlexNet) were adapted for 3D classification and applied to a dataset consisting of 187 heterogenous CT scans. The 3D ResNet outperformed both other networks. Across the five folds of the first training dataset it achieved an accuracy of 0.856 and an area under the curve (AUC) of 0.926. Subsequently, the algorithms performance was verified on a second data set containing 106 scans, where it ran fully automated and resulted in an accuracy of 0.953 and an AUC of 0.971. A layer-wise relevance propagation (LRP) made the decision process interpretable and showed that the network correctly focused on the aortic lumen. In conclusion, the deep learning-based screening proved to be robust and showed high performance even on a heterogeneous multi-center data set. Integration into hospital workflow and its effect on aneurysm management would be an exciting topic of future research.

Local-global active contour model based on tensor-based representation for 3D ultrasound vessel segmentation

Three-dimensional (3D) vessel segmentation can provide full spatial information about an anatomic structure to help physicians gain increased understanding of vascular structures, which plays an utmost role in many medical image-processing and analysis applications. The purpose of this paper aims to develop a 3D vessel-segmentation method that can improve segmentation accuracy in 3D ultrasound (US) images. We propose a 3D tensor-based active contour model method for accurate 3D vessel segmentation. With our method, the contrast-independent multiscale bottom-hat tensor representation and local-global information are captured. This strategy ensures the effective extraction of the boundaries of vessels from inhomogeneous and homogeneous regions without being affected by the noise and low-contrast of the 3D US images. Experimental results in clinical 3D US and public 3D Multiphoton Microscopy datasets are used for quantitative and qualitative comparison with several state-of-the-art vessel segmentation methods. Clinical experiments demonstrate that our method can achieve a smoother and more accurate boundary of the vessel object than competing methods. The mean SE, SP and ACC of the proposed method are: 0.7768 ± 0.0597, 0.9978 ± 0.0013 and 0.9971 ± 0.0015 respectively. Experiments on the public dataset show that our method can segment complex vessels in different medical images with noise and low- contrast.