Cascade knowledge diffusion network for skin lesion diagnosis and segmentation

PMFSNet: Polarized multi-scale feature self-attention network for lightweight medical image segmentation

BACKGROUND AND OBJECTIVES

Current state-of-the-art medical image segmentation methods prioritize precision but often at the expense of increased computational demands and larger model sizes. Applying these large-scale models to the relatively limited scale of medical image datasets tends to induce redundant computation, complicating the process without the necessary benefits. These approaches increase complexity and pose challenges for integrating and deploying lightweight models on edge devices. For instance, recent transformer-based models have excelled in 2D and 3D medical image segmentation due to their extensive receptive fields and high parameter count. However, their effectiveness comes with the risk of overfitting when applied to small datasets. It often neglects the vital inductive biases of Convolutional Neural Networks (CNNs), essential for local feature representation.

METHODS

In this work, we propose PMFSNet, a novel medical imaging segmentation model that effectively balances global and local feature processing while avoiding the computational redundancy typical of larger models. PMFSNet streamlines the UNet-based hierarchical structure and simplifies the self-attention mechanism's computational complexity, making it suitable for lightweight applications. It incorporates a plug-and-play PMFS block, a multi-scale feature enhancement module based on attention mechanisms, to capture long-term dependencies.

RESULTS

The extensive comprehensive results demonstrate that our method achieves superior performance in various segmentation tasks on different data scales even with fewer than a million parameters. Results reveal that our PMFSNet achieves IoU of 84.68%, 82.02%, 78.82%, and 76.48% on public datasets of 3D CBCT Tooth, ovarian tumors ultrasound (MMOTU), skin lesions dermoscopy (ISIC 2018), and gastrointestinal polyp (Kvasir SEG), and yields DSC of 78.29%, 77.45%, and 78.04% on three retinal vessel segmentation datasets, DRIVE, STARE, and CHASE-DB1, respectively.

CONCLUSION

Our proposed model exhibits competitive performance across various datasets, accomplishing this with significantly fewer model parameters and inference time, demonstrating its value in model integration and deployment. It strikes an optimal compromise between efficiency and performance and can be a highly efficient solution for medical image analysis in resource-constrained clinical environments. The source code is available at https://github.com/yykzjh/PMFSNet.

An attention enhanced dilated bottleneck network for kidney disease classification

Computer-Aided Design (CAD) techniques have been developed to assist nephrologists by optimising clinical workflows, ensuring accurate results and effectively handling extensive datasets. The proposed work introduces a Dilated Bottleneck Attention-based Renal Network (DBAR-Net) to automate the diagnosis and classification of kidney diseases like cysts, stones, and tumour. To overcome the challenges caused by complex and overlapping features, the DBAR_Net model implements a multi-feature fusion technique. Two fold convolved layer normalization blocks [Formula: see text]& [Formula: see text] capture fine-grained detail and abstract patterns to achieve faster convergence and improved robustness. Spatially focused features and channel-wise refined features are generated through dual bottleneck attention modules [Formula: see text] to improve the representation of convolved features by highlighting channel and spatial regions resulting enhanced interpretability and feature generalisation. Additionally, adaptive contextual features are obtained from a dilated convolved layer normalisation block [Formula: see text], which effectively captures contextual insights from semantic feature interpretation. The resulting features are fused additively and processed through a linear layer with global average pooling and layer normalization. This combination effectively reduces spatial dimensions, internal covariate shifts and improved generalization along with essential features. The proposed approach was evaluated using the CT KIDNEY DATASET that includes 8750 CT images classified into four categories: Normal, Cyst, Tumour, and Stone. Experimental results showed that [Formula: see text] improved feature detection ability enhanced the performance of DBAR_Net model attaining a F1 score as 0.98 with minimal computational complexity and optimum classification accuracy of 98.86%. The integration of these blocks resulted in precise multi-class kidney disease detection, thereby leading to the superior performance of DBAR_Net compared to other transfer learning models like VGG16, VGG19, ResNet50, EfficientNetB0, Inception V3, MobileNetV2, and Xception.

Semi-supervised tissue segmentation from histopathological images with consistency regularization and uncertainty estimation

Pathologists have depended on their visual experience to assess tissue structures in smear images, which was time-consuming, error-prone, and inconsistent. Deep learning, particularly Convolutional Neural Networks (CNNs), offers the ability to automate this procedure by recognizing patterns in tissue images. However, training these models necessitates huge amounts of labeled data, which can be difficult to come by due to the skill required for annotation and the unavailability of data, particularly for rare diseases. This work introduces a new semi-supervised method for tissue structure semantic segmentation in histopathological images. The study presents a CNN based teacher model that generates pseudo-labels to train a student model, aiming to overcome the drawbacks of conventional supervised learning approaches. Self-supervised training is used to improve the teacher model's performance on smaller datasets. Consistency regularization is integrated to efficiently train the student model on labeled data. Further, the study uses Monte Carlo dropout to estimate the uncertainty of proposed model. The proposed model demonstrated promising results by achieving an mIoU score of 0.64 on a public dataset, highlighting its potential to improve segmentation accuracy in histopathological image analysis.

A Unified Multi-Task Learning Model with Joint Reverse Optimization for Simultaneous Skin Lesion Segmentation and Diagnosis

Classifying and segmenting skin cancer represent pivotal objectives for automated diagnostic systems that utilize dermoscopy images. However, these tasks present significant challenges due to the diverse shape variations of skin lesions and the inherently fuzzy nature of dermoscopy images, including low contrast and the presence of artifacts. Given the robust correlation between the classification of skin lesions and their segmentation, we propose that employing a combined learning method holds the promise of considerably enhancing the performance of both tasks. In this paper, we present a unified multi-task learning strategy that concurrently classifies abnormalities of skin lesions and allows for the joint segmentation of lesion boundaries. This approach integrates an optimization technique known as joint reverse learning, which fosters mutual enhancement through extracting shared features and limiting task dominance across the two tasks. The effectiveness of the proposed method was assessed using two publicly available datasets, ISIC 2016 and PH2, which included melanoma and benign skin cancers. In contrast to the single-task learning strategy, which solely focuses on either classification or segmentation, the experimental findings demonstrated that the proposed network improves the diagnostic capability of skin tumor screening and analysis. The proposed method achieves a significant segmentation performance on skin lesion boundaries, with Dice Similarity Coefficients (DSC) of 89.48% and 88.81% on the ISIC 2016 and PH2 datasets, respectively. Additionally, our multi-task learning approach enhances classification, increasing the F1 score from 78.26% (baseline ResNet50) to 82.07% on ISIC 2016 and from 82.38% to 85.50% on PH2. This work showcases its potential applicability across varied clinical scenarios.

A multi-task learning model for clinically interpretable sesamoiditis grading

Sesamoiditis is a common equine disease with varying severity, leading to increased injury risks and performance degradation in horses. Accurate grading of sesamoiditis is crucial for effective treatment. Although deep learning-based approaches for grading sesamoiditis show promise, they remain underexplored and often lack clinical interpretability. To address this issue, we propose a novel, clinically interpretable multi-task learning model that integrates clinical knowledge with machine learning. The proposed model employs a dual-branch decoder to simultaneously perform sesamoiditis grading and vascular channel segmentation. Feature fusion is utilized to transfer knowledge between these tasks, enabling the identification of subtle radiographic variations. Additionally, our model generates a diagnostic report that, along with the vascular channel mask, serves as an explanation of the model's grading decisions, thereby increasing the transparency of the decision-making process. We validate our model on two datasets, demonstrating its superior performance compared to state-of-the-art models in terms of accuracy and generalization. This study provides a foundational framework for the interpretable grading of similar diseases.

Hyb_SEnc: An Antituberculosis Peptide Predictor Based on a Hybrid Feature Vector and Stacked Ensemble Learning

Tuberculosis has plagued mankind since ancient times, and the struggle between humans and tuberculosis continues. Mycobacterium tuberculosis is the leading cause of tuberculosis, infecting nearly one-third of the world's population. The rise of peptide drugs has created a new direction in the treatment of tuberculosis. Therefore, for the treatment of tuberculosis, the prediction of anti-tuberculosis peptides is crucial. This paper proposes an anti-tuberculosis peptide prediction method based on hybrid features and stacked ensemble learning. First, a random forest (RF) and extremely randomized tree (ERT) are selected as first-level learning of stacked ensembles. Then, the five best-performing feature encoding methods are selected to obtain the hybrid feature vector, and then the decision tree and recursive feature elimination (DT-RFE) are used to refine the hybrid feature vector. After selection, the optimal feature subset is used as the input of the stacked ensemble model. At the same time, logistic regression (LR) is used as a stacked ensemble secondary learner to build the final stacked ensemble model Hyb_SEnc. The prediction accuracy of Hyb_SEnc achieved 94.68% and 95.74% on the independent test sets of AntiTb_MD and AntiTb_RD, respectively.

Intelligent skin lesion segmentation using deformable attention Transformer U-Net with bidirectional attention mechanism in skin cancer images

BACKGROUND

In recent years, the increasing prevalence of skin cancers, particularly malignant melanoma, has become a major concern for public health. The development of accurate automated segmentation techniques for skin lesions holds immense potential in alleviating the burden on medical professionals. It is of substantial clinical importance for the early identification and intervention of skin cancer. Nevertheless, the irregular shape, uneven color, and noise interference of the skin lesions have presented significant challenges to the precise segmentation. Therefore, it is crucial to develop a high-precision and intelligent skin lesion segmentation framework for clinical treatment.

METHODS

A precision-driven segmentation model for skin cancer images is proposed based on the Transformer U-Net, called BiADATU-Net, which integrates the deformable attention Transformer and bidirectional attention blocks into the U-Net. The encoder part utilizes deformable attention Transformer with dual attention block, allowing adaptive learning of global and local features. The decoder part incorporates specifically tailored scSE attention modules within skip connection layers to capture image-specific context information for strong feature fusion. Additionally, deformable convolution is aggregated into two different attention blocks to learn irregular lesion features for high-precision prediction.

RESULTS

A series of experiments are conducted on four skin cancer image datasets (i.e., ISIC2016, ISIC2017, ISIC2018, and PH2). The findings show that our model exhibits satisfactory segmentation performance, all achieving an accuracy rate of over 96%.

CONCLUSION

Our experiment results validate the proposed BiADATU-Net achieves competitive performance supremacy compared to some state-of-the-art methods. It is potential and valuable in the field of skin lesion segmentation.

Multi-view attribute learning and context relationship encoding enhanced segmentation of lung tumors from CT images

Graph convolutional neural networks (GCN) have shown the promise in medical image segmentation due to the flexibility of representing diverse range of image regions using graph nodes and propagating knowledge via graph edges. However, existing methods did not fully exploit the various attributes of image nodes and the context relationship among their attributes. We propose a new segmentation method with multi-similarity view enhancement and node attribute context learning (MNSeg). First, multiple views were formed by measuring the similarities among the image nodes, and MNSeg has a GCN based multi-view image node attribute learning (MAL) module to integrate various node attributes learnt from multiple similarity views. Each similarity view contains the specific similarities among all the image nodes, and it was integrated with the node attributes from all the channels to form the enhanced attributes of image nodes. Second, the context relationships among the attributes of image nodes are formulated by a transformer-based context relationship encoding (CRE) strategy to propagate these relationships across all the image nodes. During the transformer-based learning, the relationships were estimated based on the self-attention on all the image nodes, and then they were encoded into the learned node features. Finally, we design an attention at attribute category level (ACA) to discriminate and fuse the learnt diverse information from MAL, CRE, and the original node attributes. ACA identifies the more informative attribute categories by adaptively learn their importance. We validate the performance of MNSeg on a public lung tumor CT dataset and an in-house non-small cell lung cancer (NSCLC) dataset collected from the hospital. The segmentation results show that MNSeg outperformed the compared segmentation methods in terms of spatial overlap and the shape similarities. The ablation studies demonstrated the effectiveness of MAL, CRE, and ACA. The generalization ability of MNSeg was proved by the consistent improved segmentation performances using different 3D segmentation backbones.

LeaNet: Lightweight U-shaped architecture for high-performance skin cancer image segmentation

Skin cancer diagnosis often relies on image segmentation as a crucial aid, and a high-performance segmentation can lower misdiagnosis risks. Part of the medical devices often have limited computing power for deploying image segmentation algorithms. However, existing high-performance algorithms for image segmentation primarily rely on computationally intensive large models, making it challenging to meet the lightweight deployment requirement of medical devices. State-of-the-art lightweight models are not able to capture both local and global feature information of lesion edges due to their model structures, result in pixel loss of lesion edge. To tackle this problem, we propose LeaNet, a novel U-shaped network for high-performance yet lightweight skin cancer image segmentation. Specifically, LeaNet employs multiple attention blocks in a lightweight symmetric U-shaped design. Each blocks contains a dilated efficient channel attention (DECA) module for global and local contour information and an inverted external attention (IEA) module to improve information correlation between data samples. Additionally, LeaNet uses an attention bridge (AB) module to connect the left and right sides of the U-shaped architecture, thereby enhancing the model's multi-level feature extraction capability. We tested our model on ISIC2017 and ISIC2018 datasets. Compared with large models like ResUNet, LeaNet improved the ACC, SEN, and SPEC metrics by 1.09 %, 2.58 %, and 1.6 %, respectively, while reducing the model's parameter number and computational complexity by 570x and 1182x. Compared with lightweight models like MALUNet, LeaNet achieved improvements of 2.07 %, 4.26 %, and 3.11 % in ACC, SEN, and SPEC, respectively, reducing the parameter number and computational complexity by 1.54x and 1.04x.

Federated and Transfer Learning Methods for the Classification of Melanoma and Nonmelanoma Skin Cancers: A Prospective Study

Skin cancer is considered a dangerous type of cancer with a high global mortality rate. Manual skin cancer diagnosis is a challenging and time-consuming method due to the complexity of the disease. Recently, deep learning and transfer learning have been the most effective methods for diagnosing this deadly cancer. To aid dermatologists and other healthcare professionals in classifying images into melanoma and nonmelanoma cancer and enabling the treatment of patients at an early stage, this systematic literature review (SLR) presents various federated learning (FL) and transfer learning (TL) techniques that have been widely applied. This study explores the FL and TL classifiers by evaluating them in terms of the performance metrics reported in research studies, which include true positive rate (TPR), true negative rate (TNR), area under the curve (AUC), and accuracy (ACC). This study was assembled and systemized by reviewing well-reputed studies published in eminent fora between January 2018 and July 2023. The existing literature was compiled through a systematic search of seven well-reputed databases. A total of 86 articles were included in this SLR. This SLR contains the most recent research on FL and TL algorithms for classifying malignant skin cancer. In addition, a taxonomy is presented that summarizes the many malignant and non-malignant cancer classes. The results of this SLR highlight the limitations and challenges of recent research. Consequently, the future direction of work and opportunities for interested researchers are established that help them in the automated classification of melanoma and nonmelanoma skin cancers.

Precision in Dermatology: Developing an Optimal Feature Selection Framework for Skin Lesion Classification

Melanoma is widely recognized as one of the most lethal forms of skin cancer, with its incidence showing an upward trend in recent years. Nonetheless, the timely detection of this malignancy substantially enhances the likelihood of patients' long-term survival. Several computer-based methods have recently been proposed, in the pursuit of diagnosing skin lesions at their early stages. Despite achieving some level of success, there still remains a margin of error that the machine learning community considers to be an unresolved research challenge. The primary objective of this study was to maximize the input feature information by combining multiple deep models in the first phase, and then to avoid noisy and redundant information by downsampling the feature set, using a novel evolutionary feature selection technique, in the second phase. By maintaining the integrity of the original feature space, the proposed idea generated highly discriminant feature information. Recent deep models, including Darknet53, DenseNet201, InceptionV3, and InceptionResNetV2, were employed in our study, for the purpose of feature extraction. Additionally, transfer learning was leveraged, to enhance the performance of our approach. In the subsequent phase, the extracted feature information from the chosen pre-existing models was combined, with the aim of preserving maximum information, prior to undergoing the process of feature selection, using a novel entropy-controlled gray wolf optimization (ECGWO) algorithm. The integration of fusion and selection techniques was employed, initially to incorporate the feature vector with a high level of information and, subsequently, to eliminate redundant and irrelevant feature information. The effectiveness of our concept is supported by an assessment conducted on three benchmark dermoscopic datasets: PH2, ISIC-MSK, and ISIC-UDA. In order to validate the proposed methodology, a comprehensive evaluation was conducted, including a rigorous comparison to established techniques in the field.

A survey on deep learning for skin lesion segmentation

Skin cancer is a major public health problem that could benefit from computer-aided diagnosis to reduce the burden of this common disease. Skin lesion segmentation from images is an important step toward achieving this goal. However, the presence of natural and artificial artifacts (e.g., hair and air bubbles), intrinsic factors (e.g., lesion shape and contrast), and variations in image acquisition conditions make skin lesion segmentation a challenging task. Recently, various researchers have explored the applicability of deep learning models to skin lesion segmentation. In this survey, we cross-examine 177 research papers that deal with deep learning-based segmentation of skin lesions. We analyze these works along several dimensions, including input data (datasets, preprocessing, and synthetic data generation), model design (architecture, modules, and losses), and evaluation aspects (data annotation requirements and segmentation performance). We discuss these dimensions both from the viewpoint of select seminal works, and from a systematic viewpoint, examining how those choices have influenced current trends, and how their limitations should be addressed. To facilitate comparisons, we summarize all examined works in a comprehensive table as well as an interactive table available online³.

Skin lesion segmentation using two-phase cross-domain transfer learning framework

BACKGROUND AND OBJECTIVE

Deep learning (DL) models have been used for medical imaging for a long time but they did not achieve their full potential in the past because of insufficient computing power and scarcity of training data. In recent years, we have seen substantial growth in DL networks because of improved technology and an abundance of data. However, previous studies indicate that even a well-trained DL algorithm may struggle to generalize data from multiple sources because of domain shifts. Additionally, ineffectiveness of basic data fusion methods, complexity of segmentation target and low interpretability of current DL models limit their use in clinical decisions. To meet these challenges, we present a new two-phase cross-domain transfer learning system for effective skin lesion segmentation from dermoscopic images.

METHODS

Our system is based on two significant technical inventions. We examine a two- phase cross-domain transfer learning approach, including model-level and data-level transfer learning, by fine-tuning the system on two datasets, MoleMap and ImageNet. We then present nSknRSUNet, a high-performing DL network, for skin lesion segmentation using broad receptive fields and spatial edge attention feature fusion. We examine the trained model's generalization capabilities on skin lesion segmentation to quantify these two inventions. We cross-examine the model using two skin lesion image datasets, MoleMap and HAM10000, obtained from varied clinical contexts.

RESULTS

At data-level transfer learning for the HAM10000 dataset, the proposed model obtained 94.63% of DSC and 99.12% accuracy. In cross-examination at data-level transfer learning for the Molemap dataset, the proposed model obtained 93.63% of DSC and 97.01% of accuracy.

CONCLUSION

Numerous experiments reveal that our system produces excellent performance and improves upon state-of-the-art methods on both qualitative and quantitative measures.

Intra-class consistency and inter-class discrimination feature learning for automatic skin lesion classification

Automated skin lesion classification has been proved to be capable of improving the diagnostic performance for dermoscopic images. Although many successes have been achieved, accurate classification remains challenging due to the significant intra-class variation and inter-class similarity. In this article, a deep learning method is proposed to increase the intra-class consistency as well as the inter-class discrimination of learned features in the automatic skin lesion classification. To enhance the inter-class discriminative feature learning, a CAM-based (class activation mapping) global-lesion localization module is proposed by optimizing the distance of CAMs for the same dermoscopic image generated by different skin lesion tasks. Then, a global features guided intra-class similarity learning module is proposed to generate the class center according to the deep features of all samples in one class and the history feature of one sample during the learning process. In this way, the performance can be improved with the collaboration of CAM-based inter-class feature discriminating and global features guided intra-class feature concentrating. To evaluate the effectiveness of the proposed method, extensive experiments are conducted on the ISIC-2017 and ISIC-2018 datasets. Experimental results with different backbones have demonstrated that the proposed method has good generalizability and can adaptively focus on more discriminative regions of the skin lesion.

Robust fusion for skin lesion segmentation of dermoscopic images

Robust skin lesion segmentation of dermoscopic images is still very difficult. Recent methods often take the combinations of CNN and Transformer for feature abstraction and multi-scale features for further classification. Both types of combination in general rely on some forms of feature fusion. This paper considers these fusions from two novel points of view. For abstraction, Transformer is viewed as the affinity exploration of different patch tokens and can be applied to attend CNN features in multiple scales. Consequently, a new fusion module, the Attention-based Transformer-And-CNN fusion module (ATAC), is proposed. ATAC augments the CNN features with more global contexts. For further classification, adaptively combining the information from multiple scales according to their contributions to object recognition is expected. Accordingly, a new fusion module, the GAting-based Multi-Scale fusion module (GAMS), is also introduced, which adaptively weights the information from multiple scales by the light-weighted gating mechanism. Combining ATAC and GAMS leads to a new encoder-decoder-based framework. In this method, ATAC acts as an encoder block to progressively abstract strong CNN features with rich global contexts attended by long-range relations, while GAMS works as an enhancement of the decoder to generate the discriminative features through adaptive fusion of multi-scale ones. This framework is especially good at lesions of varying sizes and shapes and of low contrasts and its performances are demonstrated with extensive experiments on public skin lesion segmentation datasets.

Saliency-CCE: Exploiting colour contextual extractor and saliency-based biomedical image segmentation

Biomedical image segmentation is one critical component in computer-aided system diagnosis. However, various non-automatic segmentation methods are usually designed to segment target objects with single-task driven, ignoring the potential contribution of multi-task, such as the salient object detection (SOD) task and the image segmentation task. In this paper, we propose a novel dual-task framework for white blood cell (WBC) and skin lesion (SL) saliency detection and segmentation in biomedical images, called Saliency-CCE. Saliency-CCE consists of a preprocessing of hair removal for skin lesions images, a novel colour contextual extractor (CCE) module for the SOD task and an improved adaptive threshold (AT) paradigm for the image segmentation task. In the SOD task, we perform the CCE module to extract hand-crafted features through a novel colour channel volume (CCV) block and a novel colour activation mapping (CAM) block. We first exploit the CCV block to generate a target object's region of interest (ROI). After that, we employ the CAM block to yield a refined salient map as the final salient map from the extracted ROI. We propose a novel adaptive threshold (AT) strategy in the segmentation task to automatically segment the WBC and SL from the final salient map. We evaluate our proposed Saliency-CCE on the ISIC-2016, the ISIC-2017, and the SCISC datasets, which outperform representative state-of-the-art SOD and biomedical image segmentation approaches. Our code is available at https://github.com/zxg3017/Saliency-CCE.

A survey, review, and future trends of skin lesion segmentation and classification

The Computer-aided Diagnosis or Detection (CAD) approach for skin lesion analysis is an emerging field of research that has the potential to alleviate the burden and cost of skin cancer screening. Researchers have recently indicated increasing interest in developing such CAD systems, with the intention of providing a user-friendly tool to dermatologists to reduce the challenges encountered or associated with manual inspection. This article aims to provide a comprehensive literature survey and review of a total of 594 publications (356 for skin lesion segmentation and 238 for skin lesion classification) published between 2011 and 2022. These articles are analyzed and summarized in a number of different ways to contribute vital information regarding the methods for the development of CAD systems. These ways include: relevant and essential definitions and theories, input data (dataset utilization, preprocessing, augmentations, and fixing imbalance problems), method configuration (techniques, architectures, module frameworks, and losses), training tactics (hyperparameter settings), and evaluation criteria. We intend to investigate a variety of performance-enhancing approaches, including ensemble and post-processing. We also discuss these dimensions to reveal their current trends based on utilization frequencies. In addition, we highlight the primary difficulties associated with evaluating skin lesion segmentation and classification systems using minimal datasets, as well as the potential solutions to these difficulties. Findings, recommendations, and trends are disclosed to inform future research on developing an automated and robust CAD system for skin lesion analysis.

ACCPG-Net: A skin lesion segmentation network with Adaptive Channel-Context-Aware Pyramid Attention and Global Feature Fusion

The computer-aided diagnosis system based on dermoscopic images has played an important role in the clinical treatment of skin lesion. An accurate, efficient, and automatic skin lesion segmentation method is an important auxiliary tool for clinical diagnosis. At present, skin lesion segmentation still suffers from great challenges. Existing deep-learning-based automatic segmentation methods frequently use convolutional neural networks (CNN). However, the globally-sharing feature re-weighting vector may not be optimal for the prediction of lesion areas in dermoscopic images. The presence of hairs and spots in some samples aggravates the interference of similar categories, and reduces the segmentation accuracy. To solve this problem, this paper proposes a new deep network for precise skin lesion segmentation based on a U-shape structure. To be specific, two lightweight attention modules: adaptive channel-context-aware pyramid attention (ACCAPA) module and global feature fusion (GFF) module, are embedded in the network. The ACCAPA module can model the characteristics of the lesion areas by dynamically learning the channel information, contextual information and global structure information. GFF is used for different levels of semantic information interaction between encoder and decoder layers. To validate the effectiveness of the proposed method, we test the performance of ACCPG-Net on several public skin lesion datasets. The results show that our method achieves better segmentation performance compared to other state-of-the-art methods.

A Collaborative Learning Model for Skin Lesion Segmentation and Classification

The automatic segmentation and classification of skin lesions are two essential tasks in computer-aided skin cancer diagnosis. Segmentation aims to detect the location and boundary of the skin lesion area, while classification is used to evaluate the type of skin lesion. The location and contour information of lesions provided by segmentation is essential for the classification of skin lesions, while the skin disease classification helps generate target localization maps to assist the segmentation task. Although the segmentation and classification are studied independently in most cases, we find meaningful information can be explored using the correlation of dermatological segmentation and classification tasks, especially when the sample data are insufficient. In this paper, we propose a collaborative learning deep convolutional neural networks (CL-DCNN) model based on the teacher-student learning method for dermatological segmentation and classification. To generate high-quality pseudo-labels, we provide a self-training method. The segmentation network is selectively retrained through classification network screening pseudo-labels. Specially, we obtain high-quality pseudo-labels for the segmentation network by providing a reliability measure method. We also employ class activation maps to improve the location ability of the segmentation network. Furthermore, we provide the lesion contour information by using the lesion segmentation masks to improve the recognition ability of the classification network. Experiments are carried on the ISIC 2017 and ISIC Archive datasets. The CL-DCNN model achieved a Jaccard of 79.1% on the skin lesion segmentation task and an average AUC of 93.7% on the skin disease classification task, which is superior to the advanced skin lesion segmentation methods and classification methods.

A Novel Multi-Task Learning Network Based on Melanoma Segmentation and Classification with Skin Lesion Images

Melanoma is known worldwide as a malignant tumor and the fastest-growing skin cancer type. It is a very life-threatening disease with a high mortality rate. Automatic melanoma detection improves the early detection of the disease and the survival rate. In accordance with this purpose, we presented a multi-task learning approach based on melanoma recognition with dermoscopy images. Firstly, an effective pre-processing approach based on max pooling, contrast, and shape filters is used to eliminate hair details and to perform image enhancement operations. Next, the lesion region was segmented with a VGGNet model-based FCN Layer architecture using enhanced images. Later, a cropping process was performed for the detected lesions. Then, the cropped images were converted to the input size of the classifier model using the very deep super-resolution neural network approach, and the decrease in image resolution was minimized. Finally, a deep learning network approach based on pre-trained convolutional neural networks was developed for melanoma classification. We used the International Skin Imaging Collaboration, a publicly available dermoscopic skin lesion dataset in experimental studies. While the performance measures of accuracy, specificity, precision, and sensitivity, obtained for segmentation of the lesion region, were produced at rates of 96.99%, 92.53%, 97.65%, and 98.41%, respectively, the performance measures achieved rates for classification of 97.73%, 99.83%, 99.83%, and 95.67%, respectively.

Topological structure and global features enhanced graph reasoning model for non-small cell lung cancer segmentation from CT

Objective.Accurate and automated segmentation of lung tumors from computed tomography (CT) images is critical yet challenging. Lung tumors are of various sizes and locations and have indistinct boundaries adjacent to other normal tissues.Approach.We propose a new segmentation model that can integrate the topological structure and global features of image region nodes to address the challenges. Firstly, we construct a weighted graph with image region nodes. The graph topology reflects the complex spatial relationships among these nodes, and each node has its specific attributes. Secondly, we propose a node-wise topological feature learning module based on a new graph convolutional autoencoder (GCA). Meanwhile, a node information supplementation (GNIS) module is established by integrating specific features of each node extracted by a convolutional neural network (CNN) into each encoding layer of GCA. Afterwards, we construct a global feature extraction model based on multi-layer perceptron (MLP) to encode the features learnt from all the image region nodes which are crucial complementary information for tumor segmentation.Main results.Ablation study results over the public lung tumor segmentation dataset demonstrate the contributions of our major technical innovations. Compared with other segmentation methods, our new model improves the segmentation performance and has generalization ability on different 3D image segmentation backbones. Our model achieved Dice of 0.7827, IoU of 0.6981, and HD of 32.1743 mm on the public dataset 2018 Medical Segmentation Decathlon challenge, and Dice of 0.7004, IoU of 0.5704 and HD of 64.4661 mm on lung tumor dataset from Shandong Cancer Hospital.Significance. The novel model improves automated lung tumor segmentation performance especially the challenging and complex cases using topological structure and global features of image region nodes. It is of great potential to apply the model to other CT segmentation tasks.

SL-HarDNet: Skin lesion segmentation with HarDNet

Automatic segmentation of skin lesions from dermoscopy is of great significance for the early diagnosis of skin cancer. However, due to the complexity and fuzzy boundary of skin lesions, automatic segmentation of skin lesions is a challenging task. In this paper, we present a novel skin lesion segmentation network based on HarDNet (SL-HarDNet). We adopt HarDNet as the backbone, which can learn more robust feature representation. Furthermore, we introduce three powerful modules, including: cascaded fusion module (CFM), spatial channel attention module (SCAM) and feature aggregation module (FAM). Among them, CFM combines the features of different levels and effectively aggregates the semantic and location information of skin lesions. SCAM realizes the capture of key spatial information. The cross-level features are effectively fused through FAM, and the obtained high-level semantic position information features are reintegrated with the features from CFM to improve the segmentation performance of the model. We apply the challenge dataset ISIC-2016&PH2 and ISIC-2018, and extensively evaluate and compare the state-of-the-art skin lesion segmentation methods. Experiments show that our SL-HarDNet performance is always superior to other segmentation methods and achieves the latest performance.

Color-invariant skin lesion semantic segmentation based on modified U-Net deep convolutional neural network

Melanoma is a type of skin lesion that is less common than other types of skin lesions, but it is fast growing and spreading. Therefore, it is classified as a serious disease that directly threatens human health and life. Recently, the number of deaths due to this disease has increased significantly. Thus, researchers are interested in creating computer-aided diagnostic systems that aid in the proper diagnosis and detection of these lesions from dermoscopy images. Relying on manual diagnosis is time consuming in addition to requiring enough experience from dermatologists. Current skin lesion segmentation systems use deep convolutional neural networks to detect skin lesions from RGB dermoscopy images. However, relying on RGB color model is not always the optimal choice to train such networks because some fine details of lesion parts in the dermoscopy images can not clearly appear using RGB color model. Other color models exhibit invariant features of the dermoscopy images so that they can improve the performance of deep neural networks. In the proposed Color Invariant U-Net (CIU-Net) model, a color mixture block is added at the beginning of the contracting path of U-Net. The color mixture block acts as a mixer to learn the fusion of various input color models and create a new one with three channels. Furthermore, a new channel-attention module is included in the connection path between encoder and decoder paths. This channel attention module is developed to enrich the extracted color features. From the experimental result, we found that the proposed CIU-Net works in harmony with the new proposed hybrid loss function to enhance skin segmentation results. The performance of the proposed CIU-Net architecture is evaluated using ISIC 2018 dataset and the results are compared with other recent approaches. Our proposed method outperformed other recent approaches and achieved the best Dice and Jaccard coefficient with values 92.56% and 91.40%, respectively.

Graph based multi-scale neighboring topology deep learning for kidney and tumor segmentation

Objective.Effective learning and modelling of spatial and semantic relations between image regions in various ranges are critical yet challenging in image segmentation tasks.Approach.We propose a novel deep graph reasoning model to learn from multi-order neighborhood topologies for volumetric image segmentation. A graph is first constructed with nodes representing image regions and graph topology to derive spatial dependencies and semantic connections across image regions. We propose a new node attribute embedding mechanism to formulate topological attributes for each image region node by performing multi-order random walks (RW) on the graph and updating neighboring topologies at different neighborhood ranges. Afterwards, multi-scale graph convolutional autoencoders are developed to extract deep multi-scale topological representations of nodes and propagate learnt knowledge along graph edges during the convolutional and optimization process. We also propose a scale-level attention module to learn the adaptive weights of topological representations at multiple scales for enhanced fusion. Finally, the enhanced topological representation and knowledge from graph reasoning are integrated with content features before feeding into the segmentation decoder.Main results.The evaluation results over public kidney and tumor CT segmentation dataset show that our model outperforms other state-of-the-art segmentation methods. Ablation studies and experiments using different convolutional neural networks backbones show the contributions of major technical innovations and generalization ability.Significance.We propose for the first time an RW-driven MCG with scale-level attention to extract semantic connections and spatial dependencies between a diverse range of regions for accurate kidney and tumor segmentation in CT volumes.

MulTNet: A Multi-Scale Transformer Network for Marine Image Segmentation toward Fishing

Image segmentation plays an important role in the sensing systems of autonomous underwater vehicles for fishing. Via accurately perceiving the marine organisms and surrounding environment, the automatic catch of marine products can be implemented. However, existing segmentation methods cannot precisely segment marine animals due to the low quality and complex shapes of collected marine images in the underwater situation. A novel multi-scale transformer network (MulTNet) is proposed for improving the segmentation accuracy of marine animals, and it simultaneously possesses the merits of a convolutional neural network (CNN) and a transformer. To alleviate the computational burden of the proposed network, a dimensionality reduction CNN module (DRCM) based on progressive downsampling is first designed to fully extract the low-level features, and then they are fed into a proposed multi-scale transformer module (MTM). For capturing the rich contextural information from different subregions and scales, four parallel small-scale encoder layers with different heads are constructed, and then they are combined with a large-scale transformer layer to form a multi-scale transformer module. The comparative results demonstrate MulTNet outperforms the existing advanced image segmentation networks, with MIOU improvements of 0.76% in the marine animal dataset and 0.29% in the ISIC 2018 dataset. Consequently, the proposed method has important application value for segmenting underwater images.

DTP-Net: A convolutional neural network model to predict threshold for localizing the lesions on dermatological macro-images

Highly focused images of skin captured with ordinary cameras, called macro-images, are extensively used in dermatology. Being highly focused views, the macro-images contain only lesions and background regions. Hence, the localization of lesions on the macro-images is a simple thresholding problem. However, algorithms that offer an accurate estimate of threshold and retain consistent performance on different dermatological macro-images are rare. A deep learning model, termed 'Deep Threshold Prediction Network (DTP-Net)', is proposed in this paper to address this issue. For training the model, grayscale versions of the macro-images are fed as input to the model, and the corresponding gray-level threshold values at which the Dice similarity index (DSI) between the segmented and the ground-truth images are maximized are defined as the targets. The DTP-Net exhibited the least value of root mean square error for the predicted threshold, compared with 11 state-of-the-art threshold estimation algorithms (such as Otsu's thresholding, Valley emphasized otsu's thresholding, Isodata thresholding, Histogram slope difference distribution-based thresholding, Minimum error thresholding, Poisson's distribution-based minimum error thresholding, Kapur's maximum entropy thresholding, Entropy-weighted otsu's thresholding, Minimum cross-entropy thresholding, Type-2 fuzzy-based thresholding, and Fuzzy entropy thresholding). The DTP-Net could learn the difference between the lesion and background in the intensity space and accurately predict the threshold that separates the lesion from the background. The proposed DTP-Net can be integrated into the segmentation module in automated tools that detect skin cancer from dermatological macro-images.

ATGPred-FL: sequence-based prediction of autophagy proteins with feature representation learning

Autophagy plays an important role in biological evolution and is regulated by many autophagy proteins. Accurate identification of autophagy proteins is crucially important to reveal their biological functions. Due to the expense and labor cost of experimental methods, it is urgent to develop automated, accurate and reliable sequence-based computational tools to enable the identification of novel autophagy proteins among numerous proteins and peptides. For this purpose, a new predictor named ATGPred-FL was proposed for the efficient identification of autophagy proteins. We investigated various sequence-based feature descriptors and adopted the feature learning method to generate corresponding, more informative probability features. Then, a two-step feature selection strategy based on accuracy was utilized to remove irrelevant and redundant features, leading to the most discriminative 14-dimensional feature set. The final predictor was built using a support vector machine classifier, which performed favorably on both the training and testing sets with accuracy values of 94.40% and 90.50%, respectively. ATGPred-FL is the first ATG machine learning predictor based on protein primary sequences. We envision that ATGPred-FL will be an effective and useful tool for autophagy protein identification, and it is available for free at http://lab.malab.cn/~acy/ATGPred-FL , the source code and datasets are accessible at https://github.com/jiaoshihu/ATGPred .

Multiclass Skin Lesion Classification Using Hybrid Deep Features Selection and Extreme Learning Machine

The variation in skin textures and injuries, as well as the detection and classification of skin cancer, is a difficult task. Manually detecting skin lesions from dermoscopy images is a difficult and time-consuming process. Recent advancements in the domains of the internet of things (IoT) and artificial intelligence for medical applications demonstrated improvements in both accuracy and computational time. In this paper, a new method for multiclass skin lesion classification using best deep learning feature fusion and an extreme learning machine is proposed. The proposed method includes five primary steps: image acquisition and contrast enhancement; deep learning feature extraction using transfer learning; best feature selection using hybrid whale optimization and entropy-mutual information (EMI) approach; fusion of selected features using a modified canonical correlation based approach; and, finally, extreme learning machine based classification. The feature selection step improves the system's computational efficiency and accuracy. The experiment is carried out on two publicly available datasets, HAM10000 and ISIC2018. The achieved accuracy on both datasets is 93.40 and 94.36 percent. When compared to state-of-the-art (SOTA) techniques, the proposed method's accuracy is improved. Furthermore, the proposed method is computationally efficient.

FAT-Net: Feature adaptive transformers for automated skin lesion segmentation

Skin lesion segmentation from dermoscopic image is essential for improving the quantitative analysis of melanoma. However, it is still a challenging task due to the large scale variations and irregular shapes of the skin lesions. In addition, the blurred lesion boundaries between the skin lesions and the surrounding tissues may also increase the probability of incorrect segmentation. Due to the inherent limitations of traditional convolutional neural networks (CNNs) in capturing global context information, traditional CNN-based methods usually cannot achieve a satisfactory segmentation performance. In this paper, we propose a novel feature adaptive transformer network based on the classical encoder-decoder architecture, named FAT-Net, which integrates an extra transformer branch to effectively capture long-range dependencies and global context information. Furthermore, we also employ a memory-efficient decoder and a feature adaptation module to enhance the feature fusion between the adjacent-level features by activating the effective channels and restraining the irrelevant background noise. We have performed extensive experiments to verify the effectiveness of our proposed method on four public skin lesion segmentation datasets, including the ISIC 2016, ISIC 2017, ISIC 2018, and PH2 datasets. Ablation studies demonstrate the effectiveness of our feature adaptive transformers and memory-efficient strategies. Comparisons with state-of-the-art methods also verify the superiority of our proposed FAT-Net in terms of both accuracy and inference speed. The code is available at https://github.com/SZUcsh/FAT-Net.

Skin Lesion Extraction Using Multiscale Morphological Local Variance Reconstruction Based Watershed Transform and Fast Fuzzy C-Means Clustering

Early identification of melanocytic skin lesions increases the survival rate for skin cancer patients. Automated melanocytic skin lesion extraction from dermoscopic images using the computer vision approach is a challenging task as the lesions present in the image can be of different colors, there may be a variation of contrast near the lesion boundaries, lesions may have different sizes and shapes, etc. Therefore, lesion extraction from dermoscopic images is a fundamental step for automated melanoma identification. In this article, a watershed transform based on the fast fuzzy c-means (FCM) clustering algorithm is proposed for the extraction of melanocytic skin lesion from dermoscopic images. Initially, the proposed method removes the artifacts from the dermoscopic images and enhances the texture regions. Further, it is filtered using a Gaussian filter and a local variance filter to enhance the lesion boundary regions. Later, the watershed transform based on MMLVR (multiscale morphological local variance reconstruction) is introduced to acquire the superpixels of the image with accurate boundary regions. Finally, the fast FCM clustering technique is implemented in the superpixels of the image to attain the final lesion extraction result. The proposed method is tested in the three publicly available skin lesion image datasets, i.e., ISIC 2016, ISIC 2017 and ISIC 2018. Experimental evaluation shows that the proposed method achieves a good result.