Direct Quantification of Coronary Artery Stenosis Through Hierarchical Attentive Multi-View Learning

Intraoperative stenosis detection in X-ray coronary angiography via temporal fusion and attention-based CNN

BACKGROUND AND OBJECTIVE

Coronary artery disease (CAD), the leading cause of mortality, is caused by atherosclerotic plaque buildup in the arteries. The gold standard for the diagnosis of CAD is via X-ray coronary angiography (XCA) during percutaneous coronary intervention, where locating coronary artery stenosis is fundamental and essential. However, due to complex vascular features and motion artifacts caused by heartbeat and respiratory movement, manually recognizing stenosis is challenging for physicians, which may prolong the surgery decision-making time and lead to irreversible myocardial damage. Therefore, we aim to provide an automatic method for accurate stenosis localization.

METHODS

In this work, we present a convolutional neural network (CNN) with feature-level temporal fusion and attention modules to detect coronary artery stenosis in XCA images. The temporal fusion module, composed of the deformable convolution and the correlation-based module, is proposed to integrate time-varifying vessel features from consecutive frames. The attention module adopts channel-wise recalibration to capture global context as well as spatial-wise recalibration to enhance stenosis features with local width and morphology information.

RESULTS

We compare our method to the commonly used attention methods, state-of-the-art object detection methods, and stenosis detection methods. Experimental results show that our fusion and attention strategy significantly improves performance in discerning stenosis (P<0.05), achieving the best average recall score on two different datasets.

CONCLUSIONS

This is the first study to integrate both temporal fusion and attention mechanism into a novel feature-level hybrid CNN framework for stenosis detection in XCA images, which is proved effective in improving detection performance and therefore is potentially helpful in intraoperative stenosis localization.

Non-invasive physiological assessment of intermediate coronary stenoses from plain angiography through artificial intelligence: the STARFLOW system

BACKGROUND

Despite evidence supporting use of fractional flow reserve (FFR) and instantaneous waves-free ratio (iFR) to improve outcome of patients undergoing coronary angiography (CA) and percutaneous coronary intervention, such techniques are still underused in clinical practice due to economic and logistic issues.

OBJECTIVES

We aimed to develop an artificial intelligence (AI)-based application to compute FFR and iFR from plain CA.

METHODS AND RESULTS

Consecutive patients performing FFR or iFR or both were enrolled. A specific multi-task deep network exploiting 2 projections of the coronary of interest from standard CA was appraised. Accuracy of prediction of FFR/iFR of the AI model was the primary endpoint, along with sensitivity and specificity. Prediction was tested both for continuous values and for dichotomous classification (positive/negative) for FFR or iFR. Subgroup analyses were performed for FFR and iFR.A total of 389 patients from 5 centers were enrolled. Mean age was 67.9 ± 9.6 and 39.2% of patients were admitted for acute coronary syndrome. Overall, the accuracy was 87.3% (81.2-93.4%), with a sensitivity of 82.4% (71.9-96.4%) and a specificity of 92.2% (90.4-93.9%). For FFR, accuracy was 84.8% (77.8-91.8%), with a sensitivity of 81.9% (69.4-94.4%) and a specificity of 87.7% (85.5-89.9%), while for iFR accuracy was 90.2% (86.0-94.6%), with a sensitivity of 87.2% (76.6-97.8%) and a specificity of 93.2% (91.7-94.7%, all confidence intervals 95%).

CONCLUSION

The presented machine-learning based tool showed high accuracy in prediction of wire-based FFR and iFR.

DDUM: Deformable Dilated U-structure Module for coronary stenosis detection

Deep learning methods are increasingly popular in assisting physicians with diagnosing coronary artery disease and reducing errors caused by subjective judgment. However, accessing and labeling medical imaging data, especially coronary angiography data, is challenging. Consequently, models trained on such datasets often exhibit low accuracy, high false-positive rates, and limited generalization capabilities. We propose a Deformable Dilatable U-structure Module that can specialize a common network for coronary stenosis detection and enhance its generalization ability. Experiments demonstrate that our proposed module significantly improves the performance of various models. When applying DDUM to a model with ResNet50 as the backbone and faster R-CNN as the detector, the mean average precision increases from 33.76 to 42.39, a 25.56% improvement. Additionally, we show that DDUM enhances the network's generalization ability through transfer learning experiments. This module can improve the network's accuracy for stenosis detection and enhance the generalization ability of the original model. Fine-tuning reduces training costs and ensures that the model can be easily adapted and deployed across different devices.

CDKD-w+: A Keyframe Recognition Method for Coronary Digital Subtraction Angiography Video Sequence Based on w+ Space Encoding

Currently, various deep learning methods can assist in medical diagnosis. Coronary Digital Subtraction Angiography (DSA) is a medical imaging technology used in cardiac interventional procedures. By employing X-ray sensors to visualize the coronary arteries, it generates two-dimensional images from any angle. However, due to the complexity of the coronary structures, the 2D images may sometimes lack sufficient information, necessitating the construction of a 3D model. Camera-level 3D modeling can be realized based on deep learning. Nevertheless, the beating of the heart results in varying degrees of arterial vasoconstriction and vasodilation, leading to substantial discrepancies between DSA sequences, which introduce errors in 3D modeling of the coronary arteries, resulting in the inability of the 3D model to reflect the coronary arteries. We propose a coronary DSA video sequence keyframe recognition method, CDKD-w+, based on w+ space encoding. The method utilizes a pSp encoder to encode the coronary DSA images, converting them into latent codes in the w+ space. Differential analysis of inter-frame latent codes is employed for heartbeat keyframe localization, aiding in coronary 3D modeling. Experimental results on a self-constructed coronary DSA heartbeat keyframe recognition dataset demonstrate an accuracy of 97%, outperforming traditional metrics such as L1, SSIM, and PSNR.

A Convolutional-Transformer Model for FFR and iFR Assessment From Coronary Angiography

The quantification of stenosis severity from X-ray catheter angiography is a challenging task. Indeed, this requires to fully understand the lesion's geometry by analyzing dynamics of the contrast material, only relying on visual observation by clinicians. To support decision making for cardiac intervention, we propose a hybrid CNN-Transformer model for the assessment of angiography-based non-invasive fractional flow-reserve (FFR) and instantaneous wave-free ratio (iFR) of intermediate coronary stenosis. Our approach predicts whether a coronary artery stenosis is hemodynamically significant and provides direct FFR and iFR estimates. This is achieved through a combination of regression and classification branches that forces the model to focus on the cut-off region of FFR (around 0.8 FFR value), which is highly critical for decision-making. We also propose a spatio-temporal factorization mechanisms that redesigns the transformer's self-attention mechanism to capture both local spatial and temporal interactions between vessel geometry, blood flow dynamics, and lesion morphology. The proposed method achieves state-of-the-art performance on a dataset of 778 exams from 389 patients. Unlike existing methods, our approach employs a single angiography view and does not require knowledge of the key frame; supervision at training time is provided by a classification loss (based on a threshold of the FFR/iFR values) and a regression loss for direct estimation. Finally, the analysis of model interpretability and calibration shows that, in spite of the complexity of angiographic imaging data, our method can robustly identify the location of the stenosis and correlate prediction uncertainty to the provided output scores.

PEA-Net: A progressive edge information aggregation network for vessel segmentation

Automatic vessel segmentation is a critical area of research in medical image analysis, as it can greatly assist doctors in accurately and efficiently diagnosing vascular diseases. However, accurately extracting the complete vessel structure from images remains a challenge due to issues such as uneven contrast and background noise. Existing methods primarily focus on segmenting individual pixels and often fail to consider vessel features and morphology. As a result, these methods often produce fragmented results and misidentify vessel-like background noise, leading to missing and outlier points in the overall segmentation. To address these issues, this paper proposes a novel approach called the progressive edge information aggregation network for vessel segmentation (PEA-Net). The proposed method consists of several key components. First, a dual-stream receptive field encoder (DRE) is introduced to preserve fine structural features and mitigate false positive predictions caused by background noise. This is achieved by combining vessel morphological features obtained from different receptive field sizes. Second, a progressive complementary fusion (PCF) module is designed to enhance fine vessel detection and improve connectivity. This module complements the decoding path by combining features from previous iterations and the DRE, incorporating nonsalient information. Additionally, segmentation-edge decoupling enhancement (SDE) modules are employed as decoders to integrate upsampling features with nonsalient information provided by the PCF. This integration enhances both edge and segmentation information. The features in the skip connection and decoding path are iteratively updated to progressively aggregate fine structure information, thereby optimizing segmentation results and reducing topological disconnections. Experimental results on multiple datasets demonstrate that the proposed PEA-Net model and strategy achieve optimal performance in both pixel-level and topology-level metrics.

CASMatching strategy for automated detection and quantification of carotid artery stenosis based on digital subtraction angiography

BACKGROUND AND OBJECTIVE

Automated detection and quantification of carotid artery stenosis is a crucial task in establishing a computer-aided diagnostic system for brain diseases. Digital subtraction angiography (DSA) is known as the "gold standard" for carotid stenosis diagnosis. It is commonly used to identify carotid artery stenosis and measure morphological indices of the stenosis. However, using deep learning to detect stenosis based on DSA images and further quantitatively predicting the morphological indices remain a challenge due the absence of prior work. In this paper, we propose a quantitative method for predicting morphological indices of carotid stenosis.

METHODS

Our method adopts a two-stage pipeline, first locating regions suitable for predicting morphological indices by object detection model, and then using a regression model to predict indices. A novel Carotid Artery Stenosis Matching (CASMatching) strategy is introduced into the object detection to model the matching relationship between a stenosis and multiple normal vessel segments. The proposed Match-ness branch predicts a Match-ness score for each normal vessel segment to indicate the degree of matching to the stenosis. A novel Direction Distance-IoU (2DIoU) loss based on the Distance-IoU loss is proposed to make the model focused more on the bounding box regression in the direction of vessel extension. After detection, the normal vessel segment with the highest Match-ness score and the stenosis are intercepted from the original image, then fed into a regression model to predict morphological indices and calculate the degree of stenosis.

RESULTS

Our method is trained and evaluated on a dataset collected from three different manufacturers' monoplane X-ray systems. The results show that the proposed components in the object detector substantially improve the detection performance of normal vascular segments. For the prediction of morphological indices, our model achieves Mean Absolute Error of 0.378, 0.221, 4.9 on reference vessel diameter (RVD), minimum lumen diameter (MLD) and stenosis degree.

CONCLUSIONS

Our method can precisely localize the carotid stenosis and the normal vessel segment suitable for predicting RVD of the stenosis, and further achieve accurate quantification, providing a novel solution for the quantification of carotid artery stenosis.

Selective ensemble methods for deep learning segmentation of major vessels in invasive coronary angiography

BACKGROUND

Invasive coronary angiography (ICA) is a primary imaging modality that visualizes the lumen area of coronary arteries for diagnosis and interventional guidance. In the current practice of quantitative coronary analysis (QCA), semi-automatic segmentation tools require labor-intensive and time-consuming manual correction, limiting their application in the catheterization room.

PURPOSE

This study aims to propose rank-based selective ensemble methods that improve the segmentation performance and reduce morphological errors that limit fully automated quantification of coronary artery using deep-learning segmentation of ICA.

METHODS

Two selective ensemble methods proposed in this work integrated the weighted ensemble approach with per-image quality estimation. The segmentation outcomes from five base models with different loss functions were ranked either by mask morphology or estimated dice similarity coefficient (DSC). The final output was determined by imposing different weights according to the ranks. The ranking criteria based on mask morphology were formulated from empirical insight to avoid frequent types of segmentation errors (MSEN), while the estimation of DSCs was performed by comparing the pseudo-ground truth generated from a meta-learner (ESEN). Five-fold cross-validation was performed with the internal dataset of 7426 coronary angiograms from 2924 patients, and prediction model was externally validated with 556 images of 226 patients.

RESULTS

The selective ensemble methods improved the segmentation performance with DSCs up to 93.07% and provided a better delineation of coronary lesion with local DSCs of up to 93.93%, outperforming all individual models. Proposed methods also minimized the chances of mask disconnection in the most narrowed regions to 2.10%. The robustness of the proposed methods was also evident in the external validation. Inference time for major vessel segmentation was approximately one-sixth of a second.

CONCLUSION

Proposed methods successfully reduced morphological errors in the predicted masks and were able to enhance the robustness of the automatic segmentation. The results suggest better applicability of real-time QCA-based diagnostic methods in routine clinical settings.

A framework of myocardial bridge detection with x-ray angiography sequence

BACKGROUND

Myocardial bridges are congenital anatomical abnormalities in which myocardium covers a segment of coronary arteries, leading to stenocardia, myocardial ischemia, and sudden cardiac death in severe cases. However, automatic diagnosis of myocardial bridge presents significant challenges.

METHOD

A novel framework of myocardial bridge detection with x-ray angiography sequence is proposed, which can realize automatic detection of vessel stenosis and myocardial bridge. Firstly, we employ a novel neural network model for coronary vessel segmentation, which consists of both CNNs and transformer structures to effectively extract both local and global information of the vessels. Secondly, we describe the vessel segment information, establish the vessel tree in the image, and fuse the vessel tree information between sequences. Finally, based on vessel stenosis detection, we realize automatic detection of the myocardial bridge by querying the blood vessels between the image sequence information.

RESULTS

In experiment, we evaluate the segmentation results using two metrics, Dice and ASD, and achieve scores of 0.917 and 1.39, respectively. In the stenosis detection, we achieve an average accuracy rate of 92.7% in stenosis detection among 262 stenoses. In multi-frame image processing, vessels in different frames can be well-matched, and the accuracy of myocardial bridge detection achieves 75%.

CONCLUSIONS

Our experimental results demonstrate that the algorithm can automatically detect stenosis and myocardial bridge, providing a new idea for subsequent automatic diagnosis of coronary vessels.

Affinity Feature Strengthening for Accurate, Complete and Robust Vessel Segmentation

Vessel segmentation is crucial in many medical image applications, such as detecting coronary stenoses, retinal vessel diseases and brain aneurysms. However, achieving high pixel-wise accuracy, complete topology structure and robustness to various contrast variations are critical and challenging, and most existing methods focus only on achieving one or two of these aspects. In this paper, we present a novel approach, the affinity feature strengthening network (AFN), which jointly models geometry and refines pixel-wise segmentation features using a contrast-insensitive, multiscale affinity approach. Specifically, we compute a multiscale affinity field for each pixel, capturing its semantic relationships with neighboring pixels in the predicted mask image. This field represents the local geometry of vessel segments of different sizes, allowing us to learn spatial- and scale-aware adaptive weights to strengthen vessel features. We evaluate our AFN on four different types of vascular datasets: X-ray angiography coronary vessel dataset (XCAD), portal vein dataset (PV), digital subtraction angiography cerebrovascular vessel dataset (DSA) and retinal vessel dataset (DRIVE). Extensive experimental results demonstrate that our AFN outperforms the state-of-the-art methods in terms of both higher accuracy and topological metrics, while also being more robust to various contrast changes.

Diagnostic performance of IVUS-FFR analysis based on generative adversarial network and bifurcation fractal law for assessing myocardial ischemia

Background

IVUS-based virtual FFR (IVUS-FFR) can provide additional functional assessment information to IVUS imaging for the diagnosis of coronary stenosis. IVUS image segmentation and side branch blood flow can affect the accuracy of virtual FFR. The purpose of this study was to evaluate the diagnostic performance of an IVUS-FFR analysis based on generative adversarial networks and bifurcation fractal law, using invasive FFR as a reference.

Method

In this study, a total of 108 vessels were retrospectively collected from 87 patients who underwent IVUS and invasive FFR. IVUS-FFR was performed by analysts who were blinded to invasive FFR. We evaluated the diagnostic performance and computation time of IVUS-FFR, and compared it with that of the FFR-branch (considering side branch blood flow by manually extending the side branch from the bifurcation ostia). We also compared the effects of three bifurcation fractal laws on the accuracy of IVUS-FFR.

Result

The diagnostic accuracy, sensitivity, and specificity for IVUS-FFR to identify invasive FFR≤0.80 were 90.7% (95% CI, 83.6–95.5), 89.7% (95% CI, 78.8–96.1), 92.0% (95% CI, 80.8–97.8), respectively. A good correlation and agreement between IVUS-FFR and invasive FFR were observed. And the average computation time of IVUS-FFR was shorter than that of FFR-branch. In addition to this, we also observe that the HK model is the most accurate among the three bifurcation fractal laws.

Conclusion

Our proposed IVUS-FFR analysis correlates and agrees well with invasive FFR and shows good diagnostic performance. Compared with FFR-branch, IVUS-FFR has the same level of diagnostic performance with significantly lower computation time.

Progressive Perception Learning for Main Coronary Segmentation in X-Ray Angiography

Main coronary segmentation from the X-ray angiography images is important for the computer-aided diagnosis and treatment of coronary disease. However, it confronts the challenge at three different image granularities (the semantic, surrounding, and local levels). The challenge includes the semantic confusion between the main and collateral vessels, low contrast between the foreground vessel and background surroundings, and local ambiguity near the vessel boundaries. The traditional hand-crafted feature-based methods may be insufficient because they may lack the semantic relationship information and may not distinguish the main and collateral vessels. The existing deep learning-based methods seem to have issues due to the deficiency in the long-distance semantic relationship capture, the foreground and background interference adaptability, and the boundary detail information preservation. To solve the main coronary segmentation challenge, we propose the progressive perception learning (PPL) framework to inspect these three different image granularities. Specifically, the PPL contains the context, interference, and boundary perception modules. The context perception is designed to focus on the main coronary vessel based on the semantic dependence capture among different coronary segments. The interference perception is designed to purify the feature maps based on the foreground vessel enhancement and background artifact suppression. The boundary perception is designed to highlight the boundary details based on boundary feature extraction through the intersection between the foreground and background predictions. Extensive experiments on 1085 subjects show that the PPL is effective (e.g., the overall Dice is greater than 95%), and superior to thirteen state-of-the-art coronary segmentation methods.

Coronary artery stenosis detection via proposal-shifted spatial-temporal transformer in X-ray angiography

Accurate detection of coronary artery stenosis in X-ray angiography (XRA) images is crucial for the diagnosis and treatment of coronary artery disease. However, stenosis detection remains a challenging task due to complicated vascular structures, poor imaging quality, and fickle lesions. While devoted to accurate stenosis detection, most methods are inefficient in the exploitation of spatio-temporal information of XRA sequences, leading to a limited performance on the task. To overcome the problem, we propose a new stenosis detection framework based on a Transformer-based module to aggregate proposal-level spatio-temporal features. In the module, proposal-shifted spatio-temporal tokenization (PSSTT) scheme is devised to gather spatio-temporal region-of-interest (RoI) features for obtaining visual tokens within a local window. Then, the Transformer-based feature aggregation (TFA) network takes the tokens as the inputs to enhance the RoI features by learning the long-range spatio-temporal context for final stenosis prediction. The effectiveness of our method was validated by conducting qualitative and quantitative experiments on 233 XRA sequences of coronary artery. Our method achieves a high F1 score of 90.88%, outperforming other 15 state-of-the-art detection methods. It demonstrates that our method can perform accurate stenosis detection from XRA images due to the strong ability to aggregate spatio-temporal features.

Segmentation-Based vs. Regression-Based Biomarker Estimation: A Case Study of Fetus Head Circumference Assessment from Ultrasound Images

The fetus head circumference (HC) is a key biometric to monitor fetus growth during pregnancy, which is estimated from ultrasound (US) images. The standard approach to automatically measure the HC is to use a segmentation network to segment the skull, and then estimate the head contour length from the segmentation map via ellipse fitting, usually after post-processing. In this application, segmentation is just an intermediate step to the estimation of a parameter of interest. Another possibility is to estimate directly the HC with a regression network. Even if this type of segmentation-free approaches have been boosted with deep learning, it is not yet clear how well direct approach can compare to segmentation approaches, which are expected to be still more accurate. This observation motivates the present study, where we propose a fair, quantitative comparison of segmentation-based and segmentation-free (i.e., regression) approaches to estimate how far regression-based approaches stand from segmentation approaches. We experiment various convolutional neural networks (CNN) architectures and backbones for both segmentation and regression models and provide estimation results on the HC18 dataset, as well agreement analysis, to support our findings. We also investigate memory usage and computational efficiency to compare both types of approaches. The experimental results demonstrate that even if segmentation-based approaches deliver the most accurate results, regression CNN approaches are actually learning to find prominent features, leading to promising yet improvable HC estimation results.

Direct left-ventricular global longitudinal strain (GLS) computation with a fully convolutional network

This study's purpose was to develop a direct MRI-based, deep-learning semantic segmentation approach for computing global longitudinal strain (GLS), a known metric for detecting left-ventricular (LV) cardiotoxicity in breast cancer. Displacement Encoding with Stimulated Echoes cardiac image phases acquired from 30 breast cancer patients and 30 healthy females were unwrapped via a DeepLabV3 + fully convolutional network (FCN). Myocardial strains were directly computed from the unwrapped phases with the Radial Point Interpolation Method. FCN-unwrapped phases of a phantom's rotating gel were validated against quality-guided phase-unwrapping (QGPU) and robust transport of intensity equation (RTIE) phase-unwrapping. FCN performance on unwrapping human LV data was measured with F1 and Dice scores versus QGPU ground-truth. The reliability of FCN-based strains was assessed against RTIE-based strains with Cronbach's alpha (C-α) intraclass correlation coefficient. Mean squared error (MSE) of unwrapping the phantom experiment data at 0 dB signal-to-noise ratio were 1.6, 2.7 and 6.1 with FCN, QGPU and RTIE techniques. Human data classification accuracies were F1 = 0.95 (Dice = 0.96) with FCN and F1 = 0.94 (Dice = 0.95) with RTIE. GLS results from FCN and RTIE were -16 ± 3% vs. -16 ± 3% (C-α = 0.9) for patients and -20 ± 3% vs. -20 ± 3% (C-α = 0.9) for healthy subjects. The low MSE from the phantom validation demonstrates accuracy of phase-unwrapping with the FCN and comparable human subject results versus RTIE demonstrate GLS analysis accuracy. A deep-learning methodology for phase-unwrapping in medical images and GLS computation was developed and validated in a heterogeneous cohort.

Segmentation of Coronary Arteries Images Using Spatio-temporal Feature Fusion Network with Combo Loss

PURPOSE

Coronary heart disease is a serious disease that endangers human health and life. In recent years, the incidence and mortality of coronary heart disease have increased rapidly. The quantification of the coronary artery is critical in diagnosing coronary heart disease.

METHODS

In this paper, we improve the coronary arteries segmentation performance from two aspects of network model and algorithm. We proposed a U-shaped network based on spatio-temporal feature fusion structure to segment coronary arteries from 2D slices of computed tomography angiography (CTA) heart images. The spatio-temporal feature combines features of multiple levels and different receptive fields separately to get more precise boundaries. It is easy to cause over-segmented for the small proportion of coronary arteries in CTA images. For this reason, a combo loss function was designed to deal with the notorious imbalance between inputs and outputs that plague learning models. Input imbalance refers to the class imbalance in the input training samples. The output imbalance refers to the imbalance between the false positive and false negative of the inference model. The two imbalances in training and inference were divided and conquered with our combo loss function. Specifically, a gradient harmonizing mechanism (GHM) loss was employed to balance the gradient contribution of the input samples and at the same time punish false positives/negatives using another sensitivity-precision loss term to learn better model parameters gradually.

RESULTS

Compared with some existing methods, our proposed method improves the segmentation accuracy significantly, achieving the mean Dice coefficient of 0.87. In addition, accurate results can be obtained with little data using our method. Code is available at: https://github.com/Ariel97-star/FFNet-CoronaryArtery-Segmentation .

CONCLUSIONS

Our method can intelligently capture coronary artery structure and achieve accurate flow reserve fraction (FFR) analysis. Through a series of steps such as CPR curved reconstruction, the detection of coronary heart disease can be achieved without affecting the patient's body. In addition, our work can be used as an effective means to assist in the detection of stenosis. In the screening of coronary heart disease among high-risk cardiovascular factors, automatic detection of luminal stenosis can be performed based on the application of later algorithm transformation.

A 3D multiscale view convolutional neural network with attention for mental disease diagnosis on MRI images

Computer Assisted Diagnosis (CAD) based on brain Magnetic Resonance Imaging (MRI) is a popular research field for the computer science and medical engineering. Traditional machine learning and deep learning methods were employed in the classification of brain MRI images in the previous studies. However, the current algorithms rarely take into consideration the influence of multi-scale brain connectivity disorders on some mental diseases. To improve this defect, a deep learning structure was proposed based on MRI images, which was designed to consider the brain's connections at different sizes and the attention of connections. In this work, a Multiscale View (MV) module was proposed, which was designed to detect multi-scale brain network disorders. On the basis of the MV module, the path attention module was also proposed to simulate the attention selection of the parallel paths in the MV module. Based on the two modules, we proposed a 3D Multiscale View Convolutional Neural Network with Attention (3D MVA-CNN) for classification of MRI images for mental disease. The proposed method outperformed the previous 3D CNN structures in the structural MRI data of ADHD-200 and the functional MRI data of schizophrenia. Finally, we also proposed a preliminary framework for clinical application using 3D CNN, and discussed its limitations on data accessing and reliability. This work promoted the assisted diagnosis of mental diseases based on deep learning and provided a novel 3D CNN method based on MRI data.

Multi-level semantic adaptation for few-shot segmentation on cardiac image sequences

Obtaining manual labels is time-consuming and labor-intensive on cardiac image sequences. Few-shot segmentation can utilize limited labels to learn new tasks. However, it suffers from two challenges: spatial-temporal distribution bias and long-term information bias. These challenges derive from the impact of the time dimension on cardiac image sequences, resulting in serious over-adaptation. In this paper, we propose the multi-level semantic adaptation (MSA) for few-shot segmentation on cardiac image sequences. The MSA addresses the two biases by exploring the domain adaptation and the weight adaptation on the semantic features in multiple levels, including sequence-level, frame-level, and pixel-level. First, the MSA proposes the dual-level feature adjustment for domain adaptation in spatial and temporal directions. This adjustment explicitly aligns the frame-level feature and the sequence-level feature to improve the model adaptation on diverse modalities. Second, the MSA explores the hierarchical attention metric for weight adaptation in the frame-level feature and the pixel-level feature. This metric focuses on the similar frame and the target region to promote the model discrimination on the border features. The extensive experiments demonstrate that our MSA is effective in few-shot segmentation on cardiac image sequences with three modalities, i.e. MR, CT, and Echo (e.g. the average Dice is 0.9243), as well as superior to the ten state-of-the-art methods.

Mass segmentation for whole mammograms via attentive multi-task learning framework

Mass segmentation in the mammogram is a necessary and challenging task in the computer-aided diagnosis of breast cancer. Most of the existing methods tended to segment the mass by manually or automatically extracting mass-centered image patches. However, manual patch extraction is time-consuming, and automatic patch extraction can introduce errors that will affect the performance of subsequent segmentation.In order to improve the efficiency of mass segmentation and reduce segmentation errors, we proposed a novel mass segmentation method based on an attentive multi-task learning network (MTLNet), which is an end-to-end model to accurately segment mass in the whole mammogram directly, without the need for extraction in advance with the center of mass image patch. In MTLNet, we applied group convolution to the feature extraction network, which not only reduced the redundancy of the network but also improved the capacity of feature learning. Secondly, an attention mechanism is added to the backbone to highlight the feature channels that contain rich information. Eventually, the multi-task learning framework is employed in the model, which reduces the risk of model overfitting and enables the model not only to segment the mass but also to classify and locate the mass. We \hl{used five-fold cross validation to evaluate the performance of the proposed method under detection and segmentation tasks respectively on the two public mammographic datasets INbreast and CBIS-DDSM, and our method achieved the Dice index of 0.826 on INbreast and 0.863 on CBIS-DDSM.