Automatic segmentation of abdominal aortic aneurysms from CT angiography using a context-aware cascaded U-Net

Slimmable transformer with hybrid axial-attention for medical image segmentation

The transformer architecture has achieved remarkable success in medical image analysis owing to its powerful capability for capturing long-range dependencies. However, due to the lack of intrinsic inductive bias in modeling visual structural information, the transformer generally requires a large-scale pre-training schedule, limiting the clinical applications over expensive small-scale medical data. To this end, we propose a slimmable transformer to explore intrinsic inductive bias via position information for medical image segmentation. Specifically, we empirically investigate how different position encoding strategies affect the prediction quality of the region of interest (ROI) and observe that ROIs are sensitive to different position encoding strategies. Motivated by this, we present a novel Hybrid Axial-Attention (HAA) that can be equipped with pixel-level spatial structure and relative position information as inductive bias. Moreover, we introduce a gating mechanism to achieve efficient feature selection and further improve the representation quality over small-scale datasets. Experiments on LGG and COVID-19 datasets prove the superiority of our method over the baseline and previous works. Internal workflow visualization with interpretability is conducted to validate our success better; the proposed slimmable transformer has the potential to be further developed into a visual software tool for improving computer-aided lesion diagnosis and treatment planning.

Exploring a frequency-domain attention-guided cascade U-Net: Towards spatially tunable segmentation of vasculature

Developing fully automatic and highly accurate medical image segmentation methods is critically important for vascular disease diagnosis and treatment planning. Although advances in convolutional neural networks (CNNs) have spawned an array of automatic segmentation models converging to saturated high performance, none have explored whether CNNs can achieve (spatially) tunable segmentation. As a result, we propose multiple attention modules from a frequency-domain perspective to construct a unified CNN architecture for segmenting vasculature with desired (spatial) scales. The proposed CNN architecture is named frequency-domain attention-guided cascaded U-Net (FACU-Net). Specifically, FACU-Net contains two innovative components: (1) a frequency-domain-based channel attention module that adaptively tunes channel-wise feature responses and (2) a frequency-domain-based spatial attention module that enables the deep network to concentrate on foreground regions of interest (ROIs) effectively. Furthermore, we devised a novel frequency-domain-based content attention module to enhance or weaken the high (spatial) frequency information, allowing us to strengthen or eliminate vessels of interest. Extensive experiments using clinical data from patients with intracranial aneurysms (IA) and abdominal aortic aneurysms (AAA) demonstrated that the proposed FACU-Net met its design goal. In addition, we further investigated the association between varying (spatial) frequency components and the desirable vessel size/scale attributes. In summary, our preliminary findings are encouraging, and further developments may lead to deployable image segmentation models that are spatially tunable for clinical applications.

Automated polyp segmentation based on a multi-distance feature dissimilarity-guided fully convolutional network

Colorectal malignancies often arise from adenomatous polyps, which typically begin as solitary, asymptomatic growths before progressing to malignancy. Colonoscopy is widely recognized as a highly efficacious clinical polyp detection method, offering valuable visual data that facilitates precise identification and subsequent removal of these tumors. Nevertheless, accurately segmenting individual polyps poses a considerable difficulty because polyps exhibit intricate and changeable characteristics, including shape, size, color, quantity and growth context during different stages. The presence of similar contextual structures around polyps significantly hampers the performance of commonly used convolutional neural network (CNN)-based automatic detection models to accurately capture valid polyp features, and these large receptive field CNN models often overlook the details of small polyps, which leads to the occurrence of false detections and missed detections. To tackle these challenges, we introduce a novel approach for automatic polyp segmentation, known as the multi-distance feature dissimilarity-guided fully convolutional network. This approach comprises three essential components, i.e., an encoder-decoder, a multi-distance difference (MDD) module and a hybrid loss (HL) module. Specifically, the MDD module primarily employs a multi-layer feature subtraction (MLFS) strategy to propagate features from the encoder to the decoder, which focuses on extracting information differences between neighboring layers' features at short distances, and both short and long-distance feature differences across layers. Drawing inspiration from pyramids, the MDD module effectively acquires discriminative features from neighboring layers or across layers in a continuous manner, which helps to strengthen feature complementary across different layers. The HL module is responsible for supervising the feature maps extracted at each layer of the network to improve prediction accuracy. Our experimental results on four challenge datasets demonstrate that the proposed approach exhibits superior automatic polyp performance in terms of the six evaluation criteria compared to five current state-of-the-art approaches.

S-Net: a multiple cross aggregation convolutional architecture for automatic segmentation of small/thin structures for cardiovascular applications

With the success of U-Net or its variants in automatic medical image segmentation, building a fully convolutional network (FCN) based on an encoder-decoder structure has become an effective end-to-end learning approach. However, the intrinsic property of FCNs is that as the encoder deepens, higher-level features are learned, and the receptive field size of the network increases, which results in unsatisfactory performance for detecting low-level small/thin structures such as atrial walls and small arteries. To address this issue, we propose to keep the different encoding layer features at their original sizes to constrain the receptive field from increasing as the network goes deeper. Accordingly, we develop a novel S-shaped multiple cross-aggregation segmentation architecture named S-Net, which has two branches in the encoding stage, i.e., a resampling branch to capture low-level fine-grained details and thin/small structures and a downsampling branch to learn high-level discriminative knowledge. In particular, these two branches learn complementary features by residual cross-aggregation; the fusion of the complementary features from different decoding layers can be effectively accomplished through lateral connections. Meanwhile, we perform supervised prediction at all decoding layers to incorporate coarse-level features with high semantic meaning and fine-level features with high localization capability to detect multi-scale structures, especially for small/thin volumes fully. To validate the effectiveness of our S-Net, we conducted extensive experiments on the segmentation of cardiac wall and intracranial aneurysm (IA) vasculature, and quantitative and qualitative evaluations demonstrated the superior performance of our method for predicting small/thin structures in medical images.

Radiomic-based Textural Analysis of Intraluminal Thrombus in Aortic Abdominal Aneurysms: A Demonstration of Automated Workflow

Our main objective is to investigate how the structural information of intraluminal thrombus (ILT) can be used to predict abdominal aortic aneurysms (AAA) growth status through an automated workflow. Fifty-four human subjects with ILT in their AAAs were identified from our database; those AAAs were categorized as slowly- (< 5 mm/year) or fast-growing (≥ 5 mm/year) AAAs. In-house deep-learning image segmentation models were used to generate 3D geometrical AAA models, followed by automated analysis. All features were fed into a support vector machine classifier to predict AAA's growth status.The most accurate prediction model was achieved through four geometrical parameters measuring the extent of ILT, two parameters quantifying the constitution of ILT, antihypertensive medication, and the presence of co-existing coronary artery disease. The predictive model achieved an AUROC of 0.89 and a total accuracy of 83%. When ILT was not considered, our prediction's AUROC decreased to 0.75 (P-value < 0.001).

Deep-learning-based image segmentation for image-based computational hemodynamic analysis of abdominal aortic aneurysms: a comparison study

Computational hemodynamics is increasingly being used to quantify hemodynamic characteristics in and around abdominal aortic aneurysms (AAA) in a patient-specific fashion. However, the time-consuming manual annotation hinders the clinical translation of computational hemodynamic analysis. Thus, we investigate the feasibility of using deep-learning-based image segmentation methods to reduce the time required for manual segmentation. Two of the latest deep-learning-based image segmentation methods, ARU-Net and CACU-Net, were used to test the feasibility of automated computer model creation for computational hemodynamic analysis. Morphological features and hemodynamic metrics of 30 computed tomography angiography (CTA) scans were compared between pre-dictions and manual models. The DICE score for both networks was 0.916, and the correlation value was above 0.95, indicating their ability to generate models comparable to human segmentation. The Bland-Altman analysis shows a good agreement between deep learning and manual segmentation results. Compared with manual (computational hemodynamics) model recreation, the time for automated computer model generation was significantly reduced (from ∼2 h to ∼10 min). Automated image segmentation can significantly reduce time expenses on the recreation of patient-specific AAA models. Moreover, our study showed that both CACU-Net and ARU-Net could accomplish AAA segmentation, and CACU-Net outperformed ARU-Net in terms of accuracy and time-saving.

USING CONVOLUTIONAL NEURAL NETWORK-BASED SEGMENTATION FOR IMAGE-BASED COMPUTATIONAL FLUID DYNAMICS SIMULATIONS OF BRAIN ANEURYSMS: INITIAL EXPERIENCE IN AUTOMATED MODEL CREATION

"Image-based" computational fluid dynamics (CFD) simulations provide insights into each patient's hemodynamic environment. However, current standard procedures for creating CFD models start with manual segmentation and are time-consuming, hindering the clinical translation of image-based CFD simulations. This feasibility study adopts deep-learning-based image segmentation (hereafter referred to as Artificial Intelligence (AI) segmentation) to replace manual segmentation to accelerate CFD model creation. Two published convolutional neural network-based AI methods (MIScnn and DeepMedic) were selected to perform CFD model extraction from three-dimensional (3D) rotational angiography data containing intracranial aneurysms. In this study, aneurysm morphological and hemodynamic results using models generated by AI segmentation methods were compared with those obtained by two human users for the same data. Interclass coefficients (ICC), Bland-Altman plots, and Pearson's correlation coefficients (PCC) were combined to assess how well AI-generated CFD models were performed. We found that almost perfect agreement was obtained between the human and AI results for all eleven morphological and five out of eight hemodynamic parameters, while a moderate agreement was obtained from the remaining three hemodynamic parameters. Given this level of agreement, using AI segmentation to create CFD models is feasible, given more developments.