Multitask Learning for Estimating Multitype Cardiac Indices in MRI and CT Based on Adversarial Reverse Mapping

EFNet: A multitask deep learning network for simultaneous quantification of left ventricle structure and function

PURPOSE

The purpose of this study is to develop an automated method using deep learning for the reliable and precise quantification of left ventricle structure and function from echocardiogram videos, eliminating the need to identify end-systolic and end-diastolic frames. This addresses the variability and potential inaccuracies associated with manual quantification, aiming to improve the diagnosis and management of cardiovascular conditions.

METHODS

A single, fully automated multitask network, the EchoFused Network (EFNet) is introduced that simultaneously addresses both left ventricle segmentation and ejection fraction estimation tasks through cross-module fusion. Our proposed approach utilizes semi-supervised learning to estimate the ejection fraction from the entire cardiac cycle, yielding more dependable estimations and obviating the need to identify specific frames. To facilitate joint optimization, the losses from task-specific modules are combined using a normalization technique, ensuring commensurability on a comparable scale.

RESULTS

The assessment of the proposed model on a publicly available dataset, EchoNet-Dynamic, shows significant performance improvement, achieving an MAE of 4.35% for ejection fraction estimation and DSC values of 0.9309 (end-diastolic) and 0.9135 (end-systolic) for left ventricle segmentation.

CONCLUSIONS

The study demonstrates the efficacy of EFNet, a multitask deep learning network, in simultaneously quantifying left ventricle structure and function through cross-module fusion on echocardiogram data.

HAMMF: Hierarchical attention-based multi-task and multi-modal fusion model for computer-aided diagnosis of Alzheimer's disease

Alzheimer's disease (AD) is a progressive neurodegenerative condition, and early intervention can help slow its progression. However, integrating multi-dimensional information and deep convolutional networks increases the model parameters, affecting diagnosis accuracy and efficiency and hindering clinical diagnostic model deployment. Multi-modal neuroimaging can offer more precise diagnostic results, while multi-task modeling of classification and regression tasks can enhance the performance and stability of AD diagnosis. This study proposes a Hierarchical Attention-based Multi-task Multi-modal Fusion model (HAMMF) that leverages multi-modal neuroimaging data to concurrently learn AD classification tasks, cognitive score regression, and age regression tasks using attention-based techniques. Firstly, we preprocess MRI and PET image data to obtain two modal data, each containing distinct information. Next, we incorporate a novel Contextual Hierarchical Attention Module (CHAM) to aggregate multi-modal features. This module employs channel and spatial attention to extract fine-grained pathological features from unimodal image data across various dimensions. Using these attention mechanisms, the Transformer can effectively capture correlated features of multi-modal inputs. Lastly, we adopt multi-task learning in our model to investigate the influence of different variables on diagnosis, with a primary classification task and a secondary regression task for optimal multi-task prediction performance. Our experiments utilized MRI and PET images from 720 subjects in the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset. The results show that our proposed model achieves an overall accuracy of 93.15% for AD/NC recognition, and the visualization results demonstrate its strong pathological feature recognition performance.

Empowering precise advertising with Fed-GANCC: A novel federated learning approach leveraging Generative Adversarial Networks and group clustering

In the realm of targeted advertising, the demand for precision is paramount, and the traditional centralized machine learning paradigm fails to address this necessity effectively. Two critical challenges persist in the current advertising ecosystem: the data privacy concerns leading to isolated data islands and the complexity in handling non-Independent and Identically Distributed (non-IID) data and concept drift due to the specificity and diversity in user behavior data. Current federated learning frameworks struggle to overcome these hurdles satisfactorily. This paper introduces Fed-GANCC, an innovative federated learning framework that synergizes Generative Adversarial Networks (GANs) and Group Clustering. The framework incorporates a user data augmentation algorithm predicated on adversarial generative networks to enrich user behavior data, curtail the impact of non-uniform data distribution, and enhance the applicability of the global machine learning model. Unlike traditional approaches, our framework offers user data augmentation algorithms based on adversarial generative networks, which not only enriches user behavior data but also reduces the challenges posed by non-uniform data distribution, thereby enhancing the applicability of the global machine learning (ML) model. The effectiveness of Fed-GANCC is distinctly showcased through experimental results, outperforming contemporary methods like FED-AVG and FED-SGD in terms of accuracy, loss value, and receiver operating characteristic (ROC) indicators within the same computing time. Experimental results vindicate the effectiveness of Fed-GANCC, revealing substantial enhancements in accuracy, loss value, and receiver operating characteristic (ROC) metrics compared to FED-AVG and FED-SGD given the same computational time. These outcomes underline Fed-GANCC's exceptional prowess in mitigating issues such as isolated data islands, non-IID data, and concept drift. With its novel approach to addressing the prevailing challenges in targeted advertising such as isolated data islands, non-IID data, and concept drift, the Fed-GANCC framework stands as a benchmark, paving the way for future advancements in federated learning solutions tailored for the advertising domain. The Fed-GANCC framework promises to offer pivotal insights for the future development of efficient and advanced federated learning solutions for targeted advertising.

Multi-modality cardiac image computing: A survey

Multi-modality cardiac imaging plays a key role in the management of patients with cardiovascular diseases. It allows a combination of complementary anatomical, morphological and functional information, increases diagnosis accuracy, and improves the efficacy of cardiovascular interventions and clinical outcomes. Fully-automated processing and quantitative analysis of multi-modality cardiac images could have a direct impact on clinical research and evidence-based patient management. However, these require overcoming significant challenges including inter-modality misalignment and finding optimal methods to integrate information from different modalities. This paper aims to provide a comprehensive review of multi-modality imaging in cardiology, the computing methods, the validation strategies, the related clinical workflows and future perspectives. For the computing methodologies, we have a favored focus on the three tasks, i.e., registration, fusion and segmentation, which generally involve multi-modality imaging data, either combining information from different modalities or transferring information across modalities. The review highlights that multi-modality cardiac imaging data has the potential of wide applicability in the clinic, such as trans-aortic valve implantation guidance, myocardial viability assessment, and catheter ablation therapy and its patient selection. Nevertheless, many challenges remain unsolved, such as missing modality, modality selection, combination of imaging and non-imaging data, and uniform analysis and representation of different modalities. There is also work to do in defining how the well-developed techniques fit in clinical workflows and how much additional and relevant information they introduce. These problems are likely to continue to be an active field of research and the questions to be answered in the future.

Distilling sub-space structure across views for cardiac indices estimation

Cardiac indices estimation in multi-view images attracts great attention due to its capability for cardiac function assessment. However, the variation of the cardiac indices across views causes that most cardiac indices estimation methods can only be trained separately in each view, resulting in low data utilization. To solve this problem, we have proposed distilling the sub-space structure across views to explore the multi-view data fully for cardiac indices estimation. In particular, the sub-space structure is obtained via building a n×n covariance matrix to describe the correlation between the output dimensions of all views. Then, an alternate convex search algorithm is proposed to optimize the cross-view learning framework by which: (i) we train the model with regularization of sub-space structure in each view; (ii) we update the sub-space structure based on the learned parameters from all views. In the end, we have conducted a series of experiments to verify the effectiveness of our proposed framework. The model is trained on three views (short axis, 2-chamber view and 4-chamber view) with two modalities (magnetic resonance imaging and computed tomography). Compared to the state-of-the-art methods, our method has demonstrated superior performance on cardiac indices estimation tasks.

Multi-task deep learning for medical image computing and analysis: A review

The renaissance of deep learning has provided promising solutions to various tasks. While conventional deep learning models are constructed for a single specific task, multi-task deep learning (MTDL) that is capable to simultaneously accomplish at least two tasks has attracted research attention. MTDL is a joint learning paradigm that harnesses the inherent correlation of multiple related tasks to achieve reciprocal benefits in improving performance, enhancing generalizability, and reducing the overall computational cost. This review focuses on the advanced applications of MTDL for medical image computing and analysis. We first summarize four popular MTDL network architectures (i.e., cascaded, parallel, interacted, and hybrid). Then, we review the representative MTDL-based networks for eight application areas, including the brain, eye, chest, cardiac, abdomen, musculoskeletal, pathology, and other human body regions. While MTDL-based medical image processing has been flourishing and demonstrating outstanding performance in many tasks, in the meanwhile, there are performance gaps in some tasks, and accordingly we perceive the open challenges and the perspective trends. For instance, in the 2018 Ischemic Stroke Lesion Segmentation challenge, the reported top dice score of 0.51 and top recall of 0.55 achieved by the cascaded MTDL model indicate further research efforts in high demand to escalate the performance of current models.

MNC-Net: Multi-task graph structure learning based on node clustering for early Parkinson's disease diagnosis

PURPOSE

The identification of early-stage Parkinson's disease (PD) is important for the effective management of patients, affecting their treatment and prognosis. Recently, structural brain networks (SBNs) have been used to diagnose PD. However, how to mine abnormal patterns from high-dimensional SBNs has been a challenge due to the complex topology of the brain. Meanwhile, the existing prediction mechanisms of deep learning models are often complicated, and it is difficult to extract effective interpretations. In addition, most works only focus on the classification of imaging and ignore clinical scores in practical applications, which limits the ability of the model. Inspired by the regional modularity of SBNs, we adopted graph learning from the perspective of node clustering to construct an interpretable framework for PD classification.

METHODS

In this study, a multi-task graph structure learning framework based on node clustering (MNC-Net) is proposed for the early diagnosis of PD. Specifically, we modeled complex SBNs into modular graphs that facilitated the representation learning of abnormal patterns. Traditional graph neural networks are optimized through graph structure learning based on node clustering, which identifies potentially abnormal brain regions and reduces the impact of irrelevant noise. Furthermore, we employed a regression task to link clinical scores to disease classification, and incorporated latent domain information into model training through multi-task learning.

RESULTS

We validated the proposed approach on the Parkinsons Progression Markers Initiative dataset. Experimental results showed that our MNC-Net effectively separated the early-stage PD from healthy controls(HC) with an accuracy of 95.5%. The t-SNE figures have showed that our graph structure learning method can capture more efficient and discriminatory features. Furthermore, node clustering parameters were used as important weights to extract salient task-related brain regions(ROIs). These ROIs are involved in the development of mood disorders, tremors, imbalances and other symptoms, highlighting the importance of memory, language and mild motor function in early PD. In addition, statistical results from clinical scores confirmed that our model could capture abnormal connectivity that was significantly different between PD and HC. These results are consistent with previous studies, demonstrating the interpretability of our methods.

Multiple Adversarial Learning based Angiography Reconstruction for Ultra-low-dose Contrast Medium CT

Iodinated contrast medium (ICM) dose reduction is beneficial for decreasing potential health risk to renal-insufficiency patients in CT scanning. Due to the lowintensity vessel in ultra-low-dose-ICM CT angiography, it cannot provide clinical diagnosis of vascular diseases. Angiography reconstruction for ultra-low-dose-ICM CT can enhance vascular intensity for directly vascular diseases diagnosis. However, the angiography reconstruction is challenging since patient individual differences and vascular disease diversity. In this paper, we propose a Multiple Adversarial Learning based Angiography Reconstruction (i.e., MALAR) framework to enhance vascular intensity. Specifically, a bilateral learning mechanism is developed for mapping a relationship between source and target domains rather than the image-to-image mapping. Then, a dual correlation constraint is introduced to characterize both distribution uniformity from across-domain features and sample inconsistency with domain simultaneously. Finally, an adaptive fusion module by combining multiscale information and long-range interactive dependency is explored to alleviate the interference of high-noise metal. Experiments are performed on CT sequences with different ICM doses. Quantitative results based on multiple metrics demonstrate the effectiveness of our MALAR on angiography reconstruction. Qualitative assessments by radiographers confirm the potential of our MALAR for the clinical diagnosis of vascular diseases. The code and model are available at https://github.com/HIC-SYSU/MALAR.

TRSA-Net: Task Relation Spatial co-Attention for Joint Segmentation, Quantification and Uncertainty Estimation on Paired 2D Echocardiography

Clinical workflow of cardiac assessment on 2D echocardiography requires both accurate segmentation and quantification of the Left Ventricle (LV) from paired apical 4-chamber and 2-chamber. Moreover, uncertainty estimation is significant in clinically understanding the performance of a model. However, current research on 2D echocardiography ignores this vital task while joint segmentation with quantification, hence motivating the need for a unified optimization method. In this paper, we propose a multitask model with Task Relation Spatial co-Attention (referred as TRSA-Net) for joint segmentation, quantification, and uncertainty estimation on paired 2D echo. TRSA-Net achieves multitask joint learning by novelly exploring the spatial correlation between tasks. The task relation spatial co-attention learns the spatial mapping among task-specific features by non-local and co-excitation, which forcibly joints embedded spatial information in the segmentation and quantification. The Boundary-aware Structure Consistency (BSC) and Joint Indices Constraint (JIC) are integrated into the multitask learning optimization objective to guide the learning of segmentation and quantification paths. The BSC creatively promotes structural similarity of predictions, and JIC explores the internal relationship between three quantitative indices. We validate the efficacy of our TRSA-Net on the public CAMUS dataset. Extensive comparison and ablation experiments show that our approach can achieve competitive segmentation performance and highly accurate results on quantification.

Direct left-ventricular global longitudinal strain (GLS) computation with a fully convolutional network

This study's purpose was to develop a direct MRI-based, deep-learning semantic segmentation approach for computing global longitudinal strain (GLS), a known metric for detecting left-ventricular (LV) cardiotoxicity in breast cancer. Displacement Encoding with Stimulated Echoes cardiac image phases acquired from 30 breast cancer patients and 30 healthy females were unwrapped via a DeepLabV3 + fully convolutional network (FCN). Myocardial strains were directly computed from the unwrapped phases with the Radial Point Interpolation Method. FCN-unwrapped phases of a phantom's rotating gel were validated against quality-guided phase-unwrapping (QGPU) and robust transport of intensity equation (RTIE) phase-unwrapping. FCN performance on unwrapping human LV data was measured with F1 and Dice scores versus QGPU ground-truth. The reliability of FCN-based strains was assessed against RTIE-based strains with Cronbach's alpha (C-α) intraclass correlation coefficient. Mean squared error (MSE) of unwrapping the phantom experiment data at 0 dB signal-to-noise ratio were 1.6, 2.7 and 6.1 with FCN, QGPU and RTIE techniques. Human data classification accuracies were F1 = 0.95 (Dice = 0.96) with FCN and F1 = 0.94 (Dice = 0.95) with RTIE. GLS results from FCN and RTIE were -16 ± 3% vs. -16 ± 3% (C-α = 0.9) for patients and -20 ± 3% vs. -20 ± 3% (C-α = 0.9) for healthy subjects. The low MSE from the phantom validation demonstrates accuracy of phase-unwrapping with the FCN and comparable human subject results versus RTIE demonstrate GLS analysis accuracy. A deep-learning methodology for phase-unwrapping in medical images and GLS computation was developed and validated in a heterogeneous cohort.

One-pot synthesis of multifunctional radiolabeled upconversion nanorods for enhanced multimodal imaging performance

We report on the synthesis of radiolabeled NaYF4: Yb/Er/Gd(18/2/60mol%) upconversion nanorods (UCNRs) by one-pot solvothermal process. The synthesized UCNRs were transferred to water by using poly(maleic anhydride-alt-1-octadecene)-mPEG 5000 (C18PMH-PEG). Subsequently...

DSI-Net: Deep Synergistic Interaction Network for Joint Classification and Segmentation With Endoscope Images

Automatic classification and segmentation of wireless capsule endoscope (WCE) images are two clinically significant and relevant tasks in a computer-aided diagnosis system for gastrointestinal diseases. Most of existing approaches, however, considered these two tasks individually and ignored their complementary information, leading to limited performance. To overcome this bottleneck, we propose a deep synergistic interaction network (DSI-Net) for joint classification and segmentation with WCE images, which mainly consists of the classification branch (C-Branch), the coarse segmentation (CS-Branch) and the fine segmentation branches (FS-Branch). In order to facilitate the classification task with the segmentation knowledge, a lesion location mining (LLM) module is devised in C-Branch to accurately highlight lesion regions through mining neglected lesion areas and erasing misclassified background areas. To assist the segmentation task with the classification prior, we propose a category-guided feature generation (CFG) module in FS-Branch to improve pixel representation by leveraging the category prototypes of C-Branch to obtain the category-aware features. In such way, these modules enable the deep synergistic interaction between these two tasks. In addition, we introduce a task interaction loss to enhance the mutual supervision between the classification and segmentation tasks and guarantee the consistency of their predictions. Relying on the proposed deep synergistic interaction mechanism, DSI-Net achieves superior classification and segmentation performance on public dataset in comparison with state-of-the-art methods. The source code is available at https://github.com/CityU-AIM-Group/DSI-Net.

Constraint-based Unsupervised Domain Adaptation network for Multi-Modality Cardiac Image Segmentation

The cardiac CT and MRI images depict the various structures of the heart, which are very valuable for analyzing heart function. However, due to the difference in the shape of the cardiac images and imaging techniques, automatic segmentation is challenging. To solve this challenge, in this paper, we propose a new constraint-based unsupervised domain adaptation network. This network first performs mutual translation of images between different domains, it can provide training data for the segmentation model, and ensure domain invariance at the image level. Then, we input the target domain into the source domain segmentation model to obtain pseudo-labels and introduce cross-domain self-supervised learning between the two segmentation models. Here, a new loss function is designed to ensure the accuracy of the pseudo-labels. In addition, a cross-domain consistency loss is also introduced. Finally, we construct a multi-level aggregation segmentation network to obtain more refined target domain information. We validate our method on the public whole heart image segmentation challenge dataset and obtain experimental results of 82.9% and 5.5 on dice and average symmetric surface distance (ASSD), respectively. These experimental results prove that our method can provide important assistance in the clinical evaluation of unannotated cardiac datasets.

TSU-net: Two-stage multi-scale cascade and multi-field fusion U-net for right ventricular segmentation

Accurate segmentation of the right ventricle from cardiac magnetic resonance images (MRI) is a critical step in cardiac function analysis and disease diagnosis. It is still an open problem due to some difficulties, such as a large variety of object sizes and ill-defined borders. In this paper, we present a TSU-net network that grips deeper features and captures targets of different sizes with multi-scale cascade and multi-field fusion in the right ventricle. TSU-net mainly contains two major components: Dilated-Convolution Block (DB) and Multi-Layer-Pool Block (MB). DB extracts and aggregates multi-scale features for the right ventricle. MB mainly relies on multiple effective field-of-views to detect objects at different sizes and fill boundary features. Different from previous networks, we used DB and MB to replace the convolution layer in the encoding layer, thus, we can gather multi-scale information of right ventricle, detect different size targets and fill boundary information in each encoding layer. In addition, in the decoding layer, we used DB to replace the convolution layer, so that we can aggregate the multi-scale features of the right ventricle in each decoding layer. Furthermore, the two-stage U-net structure is used to further improve the utilization of DB and MB through a two-layer encoding/decoding layer. Our method is validated on the RVSC, a public right ventricular data set. The results demonstrated that TSU-net achieved an average Dice coefficient of 0.86 on endocardium and 0.90 on the epicardium, thereby outperforming other models. It effectively assists doctors to diagnose the disease and promotes the development of medical images. In addition, we also provide an intuitive explanation of our network, which fully explain MB and TSU-net's ability to detect targets of different sizes and fill in boundary features.

Multi-level semantic adaptation for few-shot segmentation on cardiac image sequences

Obtaining manual labels is time-consuming and labor-intensive on cardiac image sequences. Few-shot segmentation can utilize limited labels to learn new tasks. However, it suffers from two challenges: spatial-temporal distribution bias and long-term information bias. These challenges derive from the impact of the time dimension on cardiac image sequences, resulting in serious over-adaptation. In this paper, we propose the multi-level semantic adaptation (MSA) for few-shot segmentation on cardiac image sequences. The MSA addresses the two biases by exploring the domain adaptation and the weight adaptation on the semantic features in multiple levels, including sequence-level, frame-level, and pixel-level. First, the MSA proposes the dual-level feature adjustment for domain adaptation in spatial and temporal directions. This adjustment explicitly aligns the frame-level feature and the sequence-level feature to improve the model adaptation on diverse modalities. Second, the MSA explores the hierarchical attention metric for weight adaptation in the frame-level feature and the pixel-level feature. This metric focuses on the similar frame and the target region to promote the model discrimination on the border features. The extensive experiments demonstrate that our MSA is effective in few-shot segmentation on cardiac image sequences with three modalities, i.e. MR, CT, and Echo (e.g. the average Dice is 0.9243), as well as superior to the ten state-of-the-art methods.

DarkASDNet: Classification of ASD on Functional MRI Using Deep Neural Network

Non-invasive whole-brain scans aid the diagnosis of neuropsychiatric disorder diseases such as autism, dementia, and brain cancer. The assessable analysis for autism spectrum disorders (ASD) is rationally challenging due to the limitations of publicly available datasets. For diagnostic or prognostic tools, functional Magnetic Resonance Imaging (fMRI) exposed affirmation to the biomarkers in neuroimaging research because of fMRI pickup inherent connectivity between the brain and regions. There are profound studies in ASD with introducing machine learning or deep learning methods that have manifested advanced steps for ASD predictions based on fMRI data. However, utmost antecedent models have an inadequacy in their capacity to manipulate performance metrics such as accuracy, precision, recall, and F1-score. To overcome these problems, we proposed an avant-garde DarkASDNet, which has the competence to extract features from a lower level to a higher level and bring out promising results. In this work, we considered 3D fMRI data to predict binary classification between ASD and typical control (TC). Firstly, we pre-processed the 3D fMRI data by adopting proper slice time correction and normalization. Then, we introduced a novel DarkASDNet which surpassed the benchmark accuracy for the classification of ASD. Our model's outcomes unveil that our proposed method established state-of-the-art accuracy of 94.70% to classify ASD vs. TC in ABIDE-I, NYU dataset. Finally, we contemplated our model by performing evaluation metrics including precision, recall, F1-score, ROC curve, and AUC score, and legitimize by distinguishing with recent literature descriptions to vindicate our outcomes. The proposed DarkASDNet architecture provides a novel benchmark approach for ASD classification using fMRI processed data.

Using Machine Learning to Predict Five-Year Reintervention Risk in Type B Aortic Dissection Patients After Thoracic Endovascular Aortic Repair

Purpose: Type B aortic dissection (TBAD) is a high-risk disease, commonly treated with thoracic endovascular aortic repair (TEVAR). However, for the long-term follow-up, it is associated with a high 5-year reintervention rate for patients after TEVAR. There is no accurate definition of prognostic risk factors for TBAD in medical guidelines, and there is no scientific judgment standard for patients’ quality of life or survival outcome in the next five years in clinical practice. A large amount of medical data features makes prognostic analysis difficult. However, machine learning (ML) permits lots of objective data features to be considered for clinical risk stratification and patient management. We aimed to predict the 5-year prognosis in TBAD after TEVAR by Ml, based on baseline, stent characteristics and computed tomography angiography (CTA) imaging data, and provided a certain degree of scientific basis for prognostic risk score and stratification in medical guidelines. Materials and Methods: Dataset we recorded was obtained from 172 TBAD patients undergoing TEVAR. Totally 40 features were recorded, including 14 baseline, 5 stent characteristics and 21 CTA imaging data. Information gain (IG) was used to select features highly associated with adverse outcome. Then, the Gradient Boost classifier was trained using grid search and stratified 5-fold cross-validation, and Its predictive performance was evaluated by the area under the curve (AUC) in the receiver operating characteristic (ROC). Results: Totally 60 patients underwent reintervention during follow-up. Combing 24 features selected by IG, ML model predicted prognosis well in TBAD after TEVAR, with an AUC of 0.816 and a 95% confidence interval of 0.797 to 0.837. Reintervention rate of prediction was slightly higher than the actual (48.2% vs. 34.8%). Conclusion: Machine learning, which combined with baseline, stent characteristics and CTA imaging data for personalized risk computations, effectively predicted reintervention risk in TBAD patients after TEVAR in 5-year follow-up. The model could be used to efficiently assist the clinical management of TBAD patients and prompt high-risk factors.

Mass segmentation for whole mammograms via attentive multi-task learning framework

Mass segmentation in the mammogram is a necessary and challenging task in the computer-aided diagnosis of breast cancer. Most of the existing methods tended to segment the mass by manually or automatically extracting mass-centered image patches. However, manual patch extraction is time-consuming, and automatic patch extraction can introduce errors that will affect the performance of subsequent segmentation.In order to improve the efficiency of mass segmentation and reduce segmentation errors, we proposed a novel mass segmentation method based on an attentive multi-task learning network (MTLNet), which is an end-to-end model to accurately segment mass in the whole mammogram directly, without the need for extraction in advance with the center of mass image patch. In MTLNet, we applied group convolution to the feature extraction network, which not only reduced the redundancy of the network but also improved the capacity of feature learning. Secondly, an attention mechanism is added to the backbone to highlight the feature channels that contain rich information. Eventually, the multi-task learning framework is employed in the model, which reduces the risk of model overfitting and enables the model not only to segment the mass but also to classify and locate the mass. We \hl{used five-fold cross validation to evaluate the performance of the proposed method under detection and segmentation tasks respectively on the two public mammographic datasets INbreast and CBIS-DDSM, and our method achieved the Dice index of 0.826 on INbreast and 0.863 on CBIS-DDSM.

Direct Quantification of Coronary Artery Stenosis Through Hierarchical Attentive Multi-View Learning

Quantification of coronary artery stenosis on X-ray angiography (XRA) images is of great importance during the intraoperative treatment of coronary artery disease. It serves to quantify the coronary artery stenosis by estimating the clinical morphological indices, which are essential in clinical decision making. However, stenosis quantification is still a challenging task due to the overlapping, diversity and small-size region of the stenosis in the XRA images. While efforts have been devoted to stenosis quantification through low-level features, these methods have difficulty in learning the real mapping from these features to the stenosis indices. These methods are still cumbersome and unreliable for the intraoperative procedures due to their two-phase quantification, which depends on the results of segmentation or reconstruction of the coronary artery. In this work, we are proposing a hierarchical attentive multi-view learning model (HEAL) to achieve a direct quantification of coronary artery stenosis, without the intermediate segmentation or reconstruction. We have designed a multi-view learning model to learn more complementary information of the stenosis from different views. For this purpose, an intra-view hierarchical attentive block is proposed to learn the discriminative information of stenosis. Additionally, a stenosis representation learning module is developed to extract the multi-scale features from the keyframe perspective for considering the clinical workflow. Finally, the morphological indices are directly estimated based on the multi-view feature embedding. Extensive experiment studies on clinical multi-manufacturer dataset consisting of 228 subjects show the superiority of our HEAL against nine comparing methods, including direct quantification methods and multi-view learning methods. The experimental results demonstrate the better clinical agreement between the ground truth and the prediction, which endows our proposed method with a great potential for the efficient intraoperative treatment of coronary artery disease.