High-Resolution Encoder-Decoder Networks for Low-Contrast Medical Image Segmentation

A bibliometric of publication trends in medical image segmentation: Quantitative and qualitative analysis

PURPOSE

Medical images are important in diagnosing disease and treatment planning. Computer algorithms that describe anatomical structures that highlight regions of interest and remove unnecessary information are collectively known as medical image segmentation algorithms. The quality of these algorithms will directly affect the performance of the following processing steps. There are many studies about the algorithms of medical image segmentation and their applications, but none involved a bibliometric of medical image segmentation.

METHODS

This bibliometric work investigated the academic publication trends in medical image segmentation technology. These data were collected from the Web of Science (WoS) Core Collection and the Scopus. In the quantitative analysis stage, important visual maps were produced to show publication trends from five different perspectives including annual publications, countries, top authors, publication sources, and keywords. In the qualitative analysis stage, the frequently used methods and research trends in the medical image segmentation field were analyzed from 49 publications with the top annual citation rates.

RESULTS

The analysis results showed that the number of publications had increased rapidly by year. The top related countries include the Chinese mainland, the United States, and India. Most of these publications were conference papers, besides there are also some top journals. The research hotspot in this field was deep learning-based medical image segmentation algorithms based on keyword analysis. These publications were divided into three categories: reviews, segmentation algorithm publications, and other relevant publications. Among these three categories, segmentation algorithm publications occupied the vast majority, and deep learning neural network-based algorithm was the research hotspots and frontiers.

CONCLUSIONS

Through this bibliometric research work, the research hotspot in the medical image segmentation field is uncovered and can point to future research in the field. It can be expected that more researchers will focus their work on deep learning neural network-based medical image segmentation.

Automatic segmentation of hippocampus in hippocampal sparing whole brain radiotherapy: A multitask edge-aware learning

PURPOSE

This study aimed to improve the accuracy of the hippocampus segmentation through multitask edge-aware learning.

METHOD

We developed a multitask framework for computerized hippocampus segmentation. We used three-dimensional (3D) U-net as our backbone model with two training objectives: (a) to minimize the difference between the targeted binary mask and the model prediction; and (b) to optimize an auxiliary edge-prediction task which is designed to guide the model detection of the weak boundary of the hippocampus in model optimization. To balance the multiple task objectives, we proposed an improved gradient normalization by adaptively adjusting the weight of losses from different tasks. A total of 247 T1-weighted MRIs including 131 without contrast and 116 with contrast were collected from 247 patients to train and validate the proposed method. Segmentation was quantitatively evaluated with the dice coefficient (Dice), Hausdorff distance (HD), and average Hausdorff distance (AVD). The 3D U-net was used for baseline comparison. We used a Wilcoxon signed-rank test to compare repeated measurements (Dice, HD, and AVD) by different segmentations.

RESULTS

Through fivefold cross-validation, our multitask edge-aware learning achieved Dice of 0.8483 ± 0.0036, HD of 7.5706 ± 1.2330 mm, and AVD of 0.1522 ± 0.0165 mm, respectively. Conversely, the baseline results were 0.8340 ± 0.0072, 10.4631 ± 2.3736 mm, and 0.1884 ± 0.0286 mm, respectively. With a Wilcoxon signed-rank test, we found that the differences between our method and the baseline were statistically significant (P < 0.05).

CONCLUSION

Our results demonstrated the efficiency of multitask edge-aware learning in hippocampus segmentation for hippocampal sparing whole-brain radiotherapy. The proposed framework may also be useful for other low-contrast small organ segmentations on medical imaging modalities.

A shape-constraint adversarial framework with instance-normalized spatio-temporal features for inter-fetal membrane segmentation

BACKGROUND AND OBJECTIVES

During Twin-to-Twin Transfusion Syndrome (TTTS), abnormal vascular anastomoses in the monochorionic placenta can produce uneven blood flow between the fetuses. In the current practice, this syndrome is surgically treated by closing the abnormal connections using laser ablation. Surgeons commonly use the inter-fetal membrane as a reference. Limited field of view, low fetoscopic image quality and high inter-subject variability make the membrane identification a challenging task. However, currently available tools are not optimal for automatic membrane segmentation in fetoscopic videos, due to membrane texture homogeneity and high illumination variability.

METHODS

To tackle these challenges, we present a new deep-learning framework for inter-fetal membrane segmentation on in-vivo fetoscopic videos. The framework enhances existing architectures by (i) encoding a novel (instance-normalized) dense block, invariant to illumination changes, that extracts spatio-temporal features to enforce pixel connectivity in time, and (ii) relying on an adversarial training, which constrains macro appearance.

RESULTS

We performed a comprehensive validation using 20 different videos (2000 frames) from 20 different surgeries, achieving a mean Dice Similarity Coefficient of 0.8780±0.1383.

CONCLUSIONS

The proposed framework has great potential to positively impact the actual surgical practice for TTTS treatment, allowing the implementation of surgical guidance systems that can enhance context awareness and potentially lower the duration of the surgeries.

Discriminative Feature Learning for Thorax Disease Classification in Chest X-ray Images

This paper focuses on the thorax disease classification problem in chest X-ray (CXR) images. Different from the generic image classification task, a robust and stable CXR image analysis system should consider the unique characteristics of CXR images. Particularly, it should be able to: 1) automatically focus on the disease-critical regions, which usually are of small sizes; 2) adaptively capture the intrinsic relationships among different disease features and utilize them to boost the multi-label disease recognition rates jointly. In this paper, we propose to learn discriminative features with a two-branch architecture, named ConsultNet, to achieve those two purposes simultaneously. ConsultNet consists of two components. First, an information bottleneck constrained feature selector extracts critical disease-specific features according to the feature importance. Second, a spatial-and-channel encoding based feature integrator enhances the latent semantic dependencies in the feature space. ConsultNet fuses these discriminative features to improve the performance of thorax disease classification in CXRs. Experiments conducted on the ChestX-ray14 and CheXpert dataset demonstrate the effectiveness of the proposed method.

Multi-task edge-recalibrated network for male pelvic multi-organ segmentation on CT images

Automated male pelvic multi-organ segmentation on CT images is highly desired for applications, including radiotherapy planning. To further improve the performance and efficiency of existing automated segmentation methods, in this study, we propose a multi-task edge-recalibrated network (MTER-Net), which aims to overcome the challenges, including blurry boundaries, large inter-patient appearance variations, and low soft-tissue contrast. The proposed MTER-Net is equipped with the following novel components. (a) To exploit the saliency and stability of femoral heads, we employed a light-weight localization module to locate the target region and efficiently remove the complex background. (b) We add an edge stream to the regular segmentation stream to focus on processing the edge-related information, distinguish the organs with blurry boundaries, and then boost the overall segmentation performance. Between the regular segmentation stream and edge stream, we introduce an edge recalibration module at each resolution level to connect the intermediate layers and deliver the higher-level activations from the regular stream to the edge stream to denoise the irrelevant activations. (c) Finally, using a 3D Atrous Spatial Pyramid Pooling (ASPP) feature fusion module, we fuse the features at different scales in the regular stream and the predictions from the edge stream to form the final segmentation result. The proposed segmentation network was evaluated on 200 prostate cancer patient CT images with manually delineated contours of bladder, rectum, seminal vesicle, and prostate. The segmentation performance of the proposed method was quantitatively evaluated using three metrics including Dice similarity coefficient (DSC), average surface distance (ASD), and 95% surface distance (95SD). The proposed MTER-Net achieves average DSC of 86.35%, ASD of 1.09 mm, and 95SD of 3.53 mm on the four organs, which outperforms the state-of-the-art segmentation networks by a large margin. Specifically, the quantitative DSC evaluation results of the four organs are 96.49% (bladder), 86.39% (rectum), 76.38% (seminal vesicle), and 86.14% (prostate), respectively. In conclusion, we demonstrate that the proposed MTER-Net efficiently attains superior performance to state-of-the-art pelvic organ segmentation methods.

SESV: Accurate Medical Image Segmentation by Predicting and Correcting Errors

Medical image segmentation is an essential task in computer-aided diagnosis. Despite their prevalence and success, deep convolutional neural networks (DCNNs) still need to be improved to produce accurate and robust enough segmentation results for clinical use. In this paper, we propose a novel and generic framework called Segmentation-Emendation-reSegmentation-Verification (SESV) to improve the accuracy of existing DCNNs in medical image segmentation, instead of designing a more accurate segmentation model. Our idea is to predict the segmentation errors produced by an existing model and then correct them. Since predicting segmentation errors is challenging, we design two ways to tolerate the mistakes in the error prediction. First, rather than using a predicted segmentation error map to correct the segmentation mask directly, we only treat the error map as the prior that indicates the locations where segmentation errors are prone to occur, and then concatenate the error map with the image and segmentation mask as the input of a re-segmentation network. Second, we introduce a verification network to determine whether to accept or reject the refined mask produced by the re-segmentation network on a region-by-region basis. The experimental results on the CRAG, ISIC, and IDRiD datasets suggest that using our SESV framework can improve the accuracy of DeepLabv3+ substantially and achieve advanced performance in the segmentation of gland cells, skin lesions, and retinal microaneurysms. Consistent conclusions can also be drawn when using PSPNet, U-Net, and FPN as the segmentation network, respectively. Therefore, our SESV framework is capable of improving the accuracy of different DCNNs on different medical image segmentation tasks.

COVID-19–affected medical image analysis using DenserNet

The COrona VIrus Disease (COVID-19) outbreak has been announced as a pandemic by the World Health Organization (WHO) in mid-February 2020. With the current pandemic situation, the testing and detection of this disease are becoming a challenge in many regions across the globe because of the insufficiency of the suitable testing infrastructure. The shortage of kits to test COVID-19 has led to another crisis owing to worldwide supply-demand mismatch, and thereby, widen up a new research area that deals with the detection of COVID-19 without the test kit. In this paper, we investigate medical images, mostly chest X-ray images and thorax computed tomography (CT) scans to identify the attack of COVID-19. In countries, where the number of medical experts is lesser than the expected as recommended by WHO, this computer-aided system can be useful as it requires minimal human intervention. Consequently, this technology reduces the chances of contagious infection. This study may further help in the early detection of people with some similar symptoms of coronavirus. Early detection and intervention can play a pivotal role in coronavirus treatment. The primary goal of our work is to detect COVID-19–affected cases. However, this work can be extended to detect pneumonia because of Severe Acute Respiratory Syndrome, Acute Respiratory Distress Syndrome, Middle East Respiratory Syndrome, and bacteria-like Streptococcus. In this paper, we employ publicly available medical images obtained from various demographics, and propose a rapid cost-effective test leveraging a deep learning-based framework. Here, we propose a new architecture based on a densely connected convolutional neural network to analyze the COVID-19–affected medical images. We name our proposed architecture as DenserNet, which is an improvisation of DenseNet. Our proposed Denser Net architecture achieved 96.18% and 87.19% accuracies on two publicly available databases containing chest X-ray images and thorax CT scans, respectively, for the task of separating COVID-19 and non-COVID-19 images, which is quite encouraging.

Identification of Melanoma From Hyperspectral Pathology Image Using 3D Convolutional Networks

Skin biopsy histopathological analysis is one of the primary methods used for pathologists to assess the presence and deterioration of melanoma in clinical. A comprehensive and reliable pathological analysis is the result of correctly segmented melanoma and its interaction with benign tissues, and therefore providing accurate therapy. In this study, we applied the deep convolution network on the hyperspectral pathology images to perform the segmentation of melanoma. To make the best use of spectral properties of three dimensional hyperspectral data, we proposed a 3D fully convolutional network named Hyper-net to segment melanoma from hyperspectral pathology images. In order to enhance the sensitivity of the model, we made a specific modification to the loss function with caution of false negative in diagnosis. The performance of Hyper-net surpassed the 2D model with the accuracy over 92%. The false negative rate decreased by nearly 66% using Hyper-net with the modified loss function. These findings demonstrated the ability of the Hyper-net for assisting pathologists in diagnosis of melanoma based on hyperspectral pathology images.

Deep HT: A deep neural network for diagnose on MR images of tumors of the hand

BACKGROUND

There are many types of hand tumors, and it is often difficult for imaging diagnosticians to make a correct diagnosis, which can easily lead to misdiagnosis and delay in treatment. Thus in this paper, we propose a deep neural network for diagnose on MR Images of tumors of the hand in order to better define preoperative diagnosis and standardize surgical treatment.

METHODS

We collected MRI figures of 221 patients with hand tumors from one medical center from 2016 to 2019, invited medical experts to annotate the images to form the annotation data set. Then the original image is preprocessed to get the image data set. The data set is randomly divided into ten parts, nine for training and one for test. Next, the data set is input into the neural network system for testing. Finally, average the results of ten experiments as an estimate of the accuracy of the algorithm.

RESULTS

This research uses 221 images as dataset and the system shows an average confidence level of 71.6% in segmentation of hand tumors. The segmented tumor regions are validated through ground truth analysis and manual analysis by a radiologist.

CONCLUSIONS

With the recent advances in convolutional neural networks, vast improvements have been made for image segmentation, mainly based on the skip-connection-linked encoder decoder deep architectures. Therefore, in this paper, we propose an automatic segmentation method based on DeepLab v3+ and achieved a good diagnostic accuracy rate.

Artificial Intelligence-Based Diagnosis of Cardiac and Related Diseases

Automatic chest anatomy segmentation plays a key role in computer-aided disease diagnosis, such as for cardiomegaly, pleural effusion, emphysema, and pneumothorax. Among these diseases, cardiomegaly is considered a perilous disease, involving a high risk of sudden cardiac death. It can be diagnosed early by an expert medical practitioner using a chest X-Ray (CXR) analysis. The cardiothoracic ratio (CTR) and transverse cardiac diameter (TCD) are the clinical criteria used to estimate the heart size for diagnosing cardiomegaly. Manual estimation of CTR and other diseases is a time-consuming process and requires significant work by the medical expert. Cardiomegaly and related diseases can be automatically estimated by accurate anatomical semantic segmentation of CXRs using artificial intelligence. Automatic segmentation of the lungs and heart from the CXRs is considered an intensive task owing to inferior quality images and intensity variations using nonideal imaging conditions. Although there are a few deep learning-based techniques for chest anatomy segmentation, most of them only consider single class lung segmentation with deep complex architectures that require a lot of trainable parameters. To address these issues, this study presents two multiclass residual mesh-based CXR segmentation networks, X-RayNet-1 and X-RayNet-2, which are specifically designed to provide fine segmentation performance with a few trainable parameters compared to conventional deep learning schemes. The proposed methods utilize semantic segmentation to support the diagnostic procedure of related diseases. To evaluate X-RayNet-1 and X-RayNet-2, experiments were performed with a publicly available Japanese Society of Radiological Technology (JSRT) dataset for multiclass segmentation of the lungs, heart, and clavicle bones; two other publicly available datasets, Montgomery County (MC) and Shenzhen X-Ray sets (SC), were evaluated for lung segmentation. The experimental results showed that X-RayNet-1 achieved fine performance for all datasets and X-RayNet-2 achieved competitive performance with a 75% parameter reduction.

Mathematical Modelling of Ground Truth Image for 3D Microscopic Objects Using Cascade of Convolutional Neural Networks Optimized with Parameters’ Combinations Generators

Mathematical modelling to compute ground truth from 3D images is an area of research that can strongly benefit from machine learning methods. Deep neural networks (DNNs) are state-of-the-art methods design for solving these kinds of difficulties. Convolutional neural networks (CNNs), as one class of DNNs, can overcome special requirements of quantitative analysis especially when image segmentation is needed. This article presents a system that uses a cascade of CNNs with symmetric blocks of layers in chain, dedicated to 3D image segmentation from microscopic images of 3D nuclei. The system is designed through eight experiments that differ in following aspects: number of training slices and 3D samples for training, usage of pre-trained CNNs and number of slices and 3D samples for validation. CNNs parameters are optimized using linear, brute force, and random combinatorics, followed by voter and median operations. Data augmentation techniques such as reflection, translation and rotation are used in order to produce sufficient training set for CNNs. Optimal CNN parameters are reached by defining 11 standard and two proposed metrics. Finally, benchmarking demonstrates that CNNs improve segmentation accuracy, reliability and increased annotation accuracy, confirming the relevance of CNNs to generate high-throughput mathematical ground truth 3D images.

Channel-Unet: A Spatial Channel-Wise Convolutional Neural Network for Liver and Tumors Segmentation

It is a challenge to automatically and accurately segment the liver and tumors in computed tomography (CT) images, as the problem of over-segmentation or under-segmentation often appears when the Hounsfield unit (Hu) of liver and tumors is close to the Hu of other tissues or background. In this paper, we propose the spatial channel-wise convolution, a convolutional operation along the direction of the channel of feature maps, to extract mapping relationship of spatial information between pixels, which facilitates learning the mapping relationship between pixels in the feature maps and distinguishing the tumors from the liver tissue. In addition, we put forward an iterative extending learning strategy, which optimizes the mapping relationship of spatial information between pixels at different scales and enables spatial channel-wise convolution to map the spatial information between pixels in high-level feature maps. Finally, we propose an end-to-end convolutional neural network called Channel-UNet, which takes UNet as the main structure of the network and adds spatial channel-wise convolution in each up-sampling and down-sampling module. The network can converge the optimized mapping relationship of spatial information between pixels extracted by spatial channel-wise convolution and information extracted by feature maps and realizes multi-scale information fusion. The proposed ChannelUNet is validated by the segmentation task on the 3Dircadb dataset. The Dice values of liver and tumors segmentation were 0.984 and 0.940, which is slightly superior to current best performance. Besides, compared with the current best method, the number of parameters of our method reduces by 25.7%, and the training time of our method reduces by 33.3%. The experimental results demonstrate the efficiency and high accuracy of Channel-UNet in liver and tumors segmentation in CT images.