Generative adversarial networks in medical image segmentation: A review

Generative adversarial networks in medical image reconstruction: A systematic literature review

PURPOSE

Recent advancements in generative adversarial networks (GANs) have demonstrated substantial potential in medical image processing. Despite this progress, reconstructing images from incomplete data remains a challenge, impacting image quality. This systematic literature review explores the use of GANs in enhancing and reconstructing medical imaging data.

METHOD

A document survey of computing literature was conducted using the ACM Digital Library to identify relevant articles from journals and conference proceedings using keyword combinations, such as "generative adversarial networks or generative adversarial network," "medical image or medical imaging," and "image reconstruction."

RESULTS

Across the reviewed articles, there were 122 datasets used in 175 instances, 89 top metrics employed 335 times, 10 different tasks with a total count of 173, 31 distinct organs featured in 119 instances, and 18 modalities utilized in 121 instances, collectively depicting significant utilization of GANs in medical imaging. The adaptability and efficacy of GANs were showcased across diverse medical tasks, organs, and modalities, utilizing top public as well as private/synthetic datasets for disease diagnosis, including the identification of conditions like cancer in different anatomical regions. The study emphasized GAN's increasing integration and adaptability in diverse radiology modalities, showcasing their transformative impact on diagnostic techniques, including cross-modality tasks. The intricate interplay between network size, batch size, and loss function refinement significantly impacts GAN's performance, although challenges in training persist.

CONCLUSIONS

The study underscores GANs as dynamic tools shaping medical imaging, contributing significantly to image quality, training methodologies, and overall medical advancements, positioning them as substantial components driving medical advancements.

Advances in Deep Learning for Semantic Segmentation of Low-Contrast Images: A Systematic Review of Methods, Challenges, and Future Directions

The semantic segmentation (SS) of low-contrast images (LCIs) remains a significant challenge in computer vision, particularly for sensor-driven applications like medical imaging, autonomous navigation, and industrial defect detection, where accurate object delineation is critical. This systematic review develops a comprehensive evaluation of state-of-the-art deep learning (DL) techniques to improve segmentation accuracy in LCI scenarios by addressing key challenges such as diffuse boundaries and regions with similar pixel intensities. It tackles primary challenges, such as diffuse boundaries and regions with similar pixel intensities, which limit conventional methods. Key advancements include attention mechanisms, multi-scale feature extraction, and hybrid architectures combining Convolutional Neural Networks (CNNs) with Vision Transformers (ViTs), which expand the Effective Receptive Field (ERF), improve feature representation, and optimize information flow. We compare the performance of 25 models, evaluating accuracy (e.g., mean Intersection over Union (mIoU), Dice Similarity Coefficient (DSC)), computational efficiency, and robustness across benchmark datasets relevant to automation and robotics. This review identifies limitations, including the scarcity of diverse, annotated LCI datasets and the high computational demands of transformer-based models. Future opportunities emphasize lightweight architectures, advanced data augmentation, integration with multimodal sensor data (e.g., LiDAR, thermal imaging), and ethically transparent AI to build trust in automation systems. This work contributes a practical guide for enhancing LCI segmentation, improving mean accuracy metrics like mIoU by up to 15% in sensor-based applications, as evidenced by benchmark comparisons. It serves as a concise, comprehensive guide for researchers and practitioners advancing DL-based LCI segmentation in real-world sensor applications.

ECP-GAN: Generating Endometrial Cancer Pathology Images and Segmentation Labels via Two-Stage Generative Adversarial Networks

BACKGROUND

Endometrial cancer is one of the most common tumors of the female reproductive system and ranks third in the world list of gynecological malignancies that cause death. However, due to the privacy and complexity of pathology images, it is difficult to obtain pathology images and corresponding annotation, which affect the accuracy of pathology image segmentation and analysis.

METHODS

To address this issue, this paper proposes a two-stage endometrial cancer pathology images- and labels-generating network, which can generate pathology images and corresponding segmentation labels. In the images-to-images network, a pathological style feature information fusion normalization module is proposed, which decouples the original style feature into style feature vectors to provide independent style feature information. In the images-to-labels network, a pathological prior features guidance loss block is proposed, which improves the ability of the model in feature extraction, the segmentation label-generation accuracy, and the boundary sensitivity to the target region.

RESULTS

Training ECP-GAN in the solid tumor endometrial cancer pathological dataset, by physician recognition and experiments on the medical image segmentation tasks, shows that the ECP-GAN network generates realistic images and significantly improves the accuracy of segmentation tasks, which improves about 20% of the segmentation evaluation indicators.

CONCLUSIONS

Through comparative analysis, the experimental results show that the proposed method effectively improves the robustness and accuracy of the model in segmentation tasks. Particularly when dealing with the complex morphological features of pathology images, this method enhances the model's ability to adapt to various changes, significantly improving.

Landscape of 2D Deep Learning Segmentation Networks Applied to CT Scan from Lung Cancer Patients: A Systematic Review

BACKGROUND

The increasing rates of lung cancer emphasize the need for early detection through computed tomography (CT) scans, enhanced by deep learning (DL) to improve diagnosis, treatment, and patient survival. This review examines current and prospective applications of 2D- DL networks in lung cancer CT segmentation, summarizing research, highlighting essential concepts and gaps; Methods: Following the Preferred Reporting Items for Systematic Reviews and Meta-Analysis guidelines, a systematic search of peer-reviewed studies from 01/2020 to 12/2024 on data-driven population segmentation using structured data was conducted across databases like Google Scholar, PubMed, Science Direct, IEEE (Institute of Electrical and Electronics Engineers) and ACM (Association for Computing Machinery) library. 124 studies met the inclusion criteria and were analyzed.

RESULTS

The LIDC-LIDR dataset was the most frequently used; The finding particularly relies on supervised learning with labeled data. The UNet model and its variants were the most frequently used models in medical image segmentation, achieving Dice Similarity Coefficients (DSC) of up to 0.9999. The reviewed studies primarily exhibit significant gaps in addressing class imbalances (67%), underuse of cross-validation (21%), and poor model stability evaluations (3%). Additionally, 88% failed to address the missing data, and generalizability concerns were only discussed in 34% of cases.

CONCLUSIONS

The review emphasizes the importance of Convolutional Neural Networks, particularly UNet, in lung CT analysis and advocates for a combined 2D/3D modeling approach. It also highlights the need for larger, diverse datasets and the exploration of semi-supervised and unsupervised learning to enhance automated lung cancer diagnosis and early detection.

Rim learning framework based on TS-GAN: A new paradigm of automated glaucoma screening from fundus images

Glaucoma detection from fundus images often relies on biomarkers such as the Cup-to-Disc Ratio (CDR) and Rim-to-Disc Ratio (RDR). However, precise segmentation of the optic cup and disc is challenging due to low-contrast boundaries and the interference of blood vessels and optic nerves. This article presents a novel automated framework for glaucoma detection that focuses on the rim structure as a biomarker, excluding the conventional reliance on CDR and RDR. The proposed framework introduces a Teacher-Student Generative Adversarial Network (TS-GAN) for precise segmentation of the optic cup and disc, along with a SqueezeNet for glaucoma detection. The Teacher model uses an attention-based CNN encoder-decoder, while the Student model incorporates Expectation Maximization to enhance segmentation performance. By combining implicit generative modeling and explicit probability density modeling, the TS-GAN effectively addresses the mode collapse problem seen in existing GANs. A rim generator processes the segmented cup and disc to produce the rim, which serves as input to SqueezeNet for glaucoma classification. The framework has been extensively tested on diverse fundus image datasets, including a private dataset, demonstrating superior segmentation and detection accuracy compared to state-of-the-art models. Results show its effectiveness in early glaucoma detection, offering higher accuracy and reliability. This innovative framework provides a robust tool for ophthalmologists, enabling efficient glaucoma management and reducing the risk of vision loss.

Artificial intelligence for treatment delivery: image-guided radiotherapy

Radiation therapy (RT) is a highly digitized field relying heavily on computational methods and, as such, has a high affinity for the automation potential afforded by modern artificial intelligence (AI). This is particularly relevant where imaging is concerned and is especially so during image-guided RT (IGRT). With the advent of online adaptive RT (ART) workflows at magnetic resonance (MR) linear accelerators (linacs) and at cone-beam computed tomography (CBCT) linacs, the need for automation is further increased. AI as applied to modern IGRT is thus one area of RT where we can expect important developments in the near future. In this review article, after outlining modern IGRT and online ART workflows, we cover the role of AI in CBCT and MRI correction for dose calculation, auto-segmentation on IGRT imaging, motion management, and response assessment based on in-room imaging.

Advanced deep learning algorithms in oral cancer detection: Techniques and applications

As the 16th most common cancer globally, oral cancer yearly accounts for some 355,000 new cases. This study underlines that an early diagnosis can improve the prognosis and cut down on mortality. It discloses a multifaceted approach to the detection of oral cancer, including clinical examination, biopsies, imaging techniques, and the incorporation of artificial intelligence and deep learning methods. This study is distinctive in that it provides a thorough analysis of the most recent AI-based methods for detecting oral cancer, including deep learning models and machine learning algorithms that use convolutional neural networks. By improving the precision and effectiveness of cancer cell detection, these models eventually make early diagnosis and therapy possible. This study also discusses the importance of techniques in image pre-processing and segmentation in improving image quality and feature extraction, an essential component of accurate diagnosis. These techniques have shown promising results, with classification accuracies reaching up to 97.66% in some models. Integrating the conventional methods with the cutting-edge AI technologies, this study seeks to advance early diagnosis of oral cancer, thus enhancing patient outcomes and cutting down on the burden this disease is imposing on healthcare systems.

Deep learning and generative artificial intelligence in aging research and healthy longevity medicine

With the global population aging at an unprecedented rate, there is a need to extend healthy productive life span. This review examines how Deep Learning (DL) and Generative Artificial Intelligence (GenAI) are used in biomarker discovery, deep aging clock development, geroprotector identification and generation of dual-purpose therapeutics targeting aging and disease. The paper explores the emergence of multimodal, multitasking research systems highlighting promising future directions for GenAI in human and animal aging research, as well as clinical application in healthy longevity medicine.

Deep learning-based hyperspectral image correction and unmixing for brain tumor surgery

Hyperspectral imaging for fluorescence-guided brain tumor resection improves visualization of tissue differences, which can ameliorate patient outcomes. However, current methods do not effectively correct for heterogeneous optical and geometric tissue properties, leading to less accurate results. We propose two deep learning models for correction and unmixing that can capture these effects. While one is trained with protoporphyrin IX (PpIX) concentration labels, the other is semi-supervised. The models were evaluated on phantom and pig brain data with known PpIX concentration; the supervised and semi-supervised models achieved Pearson correlation coefficients (phantom, pig brain) between known and computed PpIX concentrations of (0.997, 0.990) and (0.98, 0.91), respectively. The classical approach achieved (0.93, 0.82). The semi-supervised approach also generalizes better to human data, achieving a 36% lower false-positive rate for PpIX detection and giving qualitatively more realistic results than existing methods. These results show promise for using deep learning to improve hyperspectral fluorescence-guided neurosurgery.

GC2: Generalizable Continual Classification of Medical Images

Deep learning models have achieved remarkable success in medical image classification. These models are typically trained once on the available annotated images and thus lack the ability of continually learning new tasks (i.e., new classes or data distributions) due to the problem of catastrophic forgetting. Recently, there has been more interest in designing continual learning methods to learn different tasks presented sequentially over time while preserving previously acquired knowledge. However, these methods focus mainly on preventing catastrophic forgetting and are tested under a closed-world assumption; i.e., assuming the test data is drawn from the same distribution as the training data. In this work, we advance the state-of-the-art in continual learning by proposing GC2 for medical image classification, which learns a sequence of tasks while simultaneously enhancing its out-of-distribution robustness. To alleviate forgetting, GC2 employs a gradual culpability-based network pruning to identify an optimal subnetwork for each task. To improve generalization, GC2 incorporates adversarial image augmentation and knowledge distillation approaches for learning generalized and robust representations for each subnetwork. Our extensive experiments on multiple benchmarks in a task-agnostic inference demonstrate that GC2 significantly outperforms baselines and other continual learning methods in reducing forgetting and enhancing generalization. Our code is publicly available at the following link: https://github.com/nourhanb/TMI2024-GC2.

B-mode ultrasound-based CAD by learning using privileged information with dual-level missing modality completion

Learning using privileged information (LUPI) has shown its effectiveness to improve the B-mode ultrasound (BUS) based computer-aided diagnosis (CAD) by transferring knowledge from the elasticity ultrasound (EUS). However, LUPI only performs transfer learning between the paired data with shared labels, and cannot handle the scenario of modality imbalance. In order to conduct the supervised transfer learning between the paired ultrasound data together with the additional single-modal BUS images, a novel multi-view LUPI algorithm with Dual-Level Modality Completion, named DLMC-LUPI, is proposed to improve the performance of BUS-based CAD. The DLMC-LUPI implements both image-level and feature-level (dual-level) completions of missing EUS modality, and then performs multi-view LUPI for knowledge transfer. Specifically, in the dual-level modality completion stage, a variational autoencoder (VAE) model for feature generation and a novel generative adversarial network (VAE-based GAN) model for image generation are sequentially trained. The proposed VAE-based GAN can improve the synthesis quality of EUS images by adopting the features generated by VAE from the BUS images as the model constrain to make the features generated from the synthesized EUS images more similar to them. In the multi-view LUPI stage, two feature vectors are generated from the real or pseudo images as two source domains, and then fed them to the multi-view support vector machine plus classifier for model training. The experiments on two ultrasound datasets indicate that the DLMC-LUPI outperforms all the compared algorithms, and it can effectively improve the performance of single-modal BUS-based CAD.

Automatic Segmentation of 2-D Echocardiography Ultrasound Images by Means of Generative Adversarial Network

Automated cardiac segmentation from 2-D echocardiographic images is a crucial step toward improving clinical diagnosis. Anatomical heterogeneity and inherent noise, however, present technical challenges and lower segmentation accuracy. The objective of this study is to propose a method for the automatic segmentation of the ventricular endocardium, the myocardium, and the left atrium (LA), in order to accurately determine clinical indices. Specifically, we suggest using the recently introduced pixel-to-pixel generative adversarial network (Pix2Pix GAN) model for accurate segmentation. To accomplish this, we integrate the backbone PatchGAN model for the discriminator and the UNET for the generator, for building the Pix2Pix GAN. The resulting model produces precisely segmented images because of UNET's capability for precise segmentation and PatchGAN's capability for fine-grained discrimination. For the experimental validation, we use the cardiac acquisitions for multistructure ultrasound segmentation (CAMUS) dataset, which consists of echocardiographic images from 500 patients in two-chamber (2CH) and four-chamber (4CH) views at the end-diastolic (ED) and end-systolic (ES) phases. Similar to state-of-the-art studies on the same dataset, we followed the same train-test splits. Our results demonstrate that the proposed generative adversarial network (GAN)-based technique improves segmentation performance for clinical and geometrical parameters compared with the state-of-the-art methods. More precisely, throughout the ED and ES phases, the mean Dice values for the left ventricular endocardium ( ) reached 0.961 and 0.930 for 2CH, and 0.959 and 0.950 for 4CH, respectively. Furthermore, the average ejection fraction (EF) correlation and mean absolute error (MAE) obtained were 0.95 and 3.2 mL for 2CH, and 0.98 and 2.1 mL for 4CH, outperforming the state-of-the-art results.

WSSS-CRAM: precise segmentation of histopathological images via class region activation mapping

Introduction

Fast, accurate, and automatic analysis of histopathological images using digital image processing and deep learning technology is a necessary task. Conventional histopathological image analysis algorithms require the manual design of features, while deep learning methods can achieve fast prediction and accurate analysis, but rely on the drive of a large amount of labeled data.

Methods

In this work, we introduce WSSS-CRAM a weakly-supervised semantic segmentation that can obtain detailed pixel-level labels from image-level annotated data. Specifically, we use a discriminative activation strategy to generate category-specific image activation maps via class labels. The category-specific activation maps are then post-processed using conditional random fields to obtain reliable regions that are directly used as ground-truth labels for the segmentation branch. Critically, the two steps of the pseudo-label acquisition and training segmentation model are integrated into an end-to-end model for joint training in this method.

Results

Through quantitative evaluation and visualization results, we demonstrate that the framework can predict pixel-level labels from image-level labels, and also perform well when testing images without image-level annotations.

Discussion

Future, we consider extending the algorithm to different pathological datasets and types of tissue images to validate its generalization capability.

Dynamic Contrast Enhanced MRI Mapping of Vascular Permeability for Evaluation of Breast Cancer Neoadjuvant Chemotherapy Response Using Image-to-Image Conditional Generative Adversarial Networks

Dynamic contrast enhanced (DCE) MRI is a non-invasive imaging technique that has become a quantitative standard for assessing tumor microvascular permeability. Through the application of a pharmacokinetic (PK) model to a series of T1-weighed MR images acquired after an injection of a contrast agent, several vascular permeability parameters can be quantitatively estimated. These parameters, including Ktrans, a measure of capillary permeability, have been widely implemented for assessing tumor vascular function as well as tumor therapeutic response. However, conventional PK modeling for translation of DCE MRI to PK vascular permeability parameter maps is complex and time-consuming for dynamic scans with thousands of pixels per image. In recent years, image-to-image conditional generative adversarial network (cGAN) is emerging as a robust approach in computer vision for complex cross-domain translation tasks. Through a sophisticated adversarial training process between two neural networks, image-to-image cGANs learn to effectively translate images from one domain to another, producing images that are indistinguishable from those in the target domain. In the present study, we have developed a novel image-to-image cGAN approach for mapping DCE MRI data to PK vascular permeability parameter maps. The DCE-to-PK cGAN not only generates high-quality parameter maps that closely resemble the ground truth, but also significantly reduces computation time over 1000-fold. The utility of the cGAN approach to map vascular permeability is validated using open-source breast cancer patient DCE MRI data provided by The Cancer Imaging Archive (TCIA). This data collection includes images and pathological analyses of breast cancer patients acquired before and after the first cycle of neoadjuvant chemotherapy (NACT). Importantly, in good agreement with previous studies leveraging this dataset, the percentage change of vascular permeability Ktrans derived from the DCE-to-PK cGAN enables early prediction of responders to NACT.

Insights and Considerations in Development and Performance Evaluation of Generative Adversarial Networks (GANs): What Radiologists Need to Know

The rapid development of deep learning in medical imaging has significantly enhanced the capabilities of artificial intelligence while simultaneously introducing challenges, including the need for vast amounts of training data and the labor-intensive tasks of labeling and segmentation. Generative adversarial networks (GANs) have emerged as a solution, offering synthetic image generation for data augmentation and streamlining medical image processing tasks through models such as cGAN, CycleGAN, and StyleGAN. These innovations not only improve the efficiency of image augmentation, reconstruction, and segmentation, but also pave the way for unsupervised anomaly detection, markedly reducing the reliance on labeled datasets. Our investigation into GANs in medical imaging addresses their varied architectures, the considerations for selecting appropriate GAN models, and the nuances of model training and performance evaluation. This paper aims to provide radiologists who are new to GAN technology with a thorough understanding, guiding them through the practical application and evaluation of GANs in brain imaging with two illustrative examples using CycleGAN and pixel2style2pixel (pSp)-combined StyleGAN. It offers a comprehensive exploration of the transformative potential of GANs in medical imaging research. Ultimately, this paper strives to equip radiologists with the knowledge to effectively utilize GANs, encouraging further research and application within the field.

A review: artificial intelligence in image-guided spinal surgery

INTRODUCTION

Due to the complex anatomy of the spine and the intricate surgical procedures involved, spinal surgery demands a high level of technical expertise from surgeons. The clinical application of image-guided spinal surgery has significantly enhanced lesion visualization, reduced operation time, and improved surgical outcomes.

AREAS COVERED

This article reviews the latest advancements in deep learning and artificial intelligence in image-guided spinal surgery, aiming to provide references and guidance for surgeons, engineers, and researchers involved in this field.

EXPERT OPINION

Our analysis indicates that image-guided spinal surgery, augmented by artificial intelligence, outperforms traditional spinal surgery techniques. Moving forward, it is imperative to collect a more expansive dataset to further ensure the procedural safety of such surgeries. These insights carry significant implications for the integration of artificial intelligence in the medical field, ultimately poised to enhance the proficiency of surgeons and improve surgical outcomes.

Semi-supervised multi-modal medical image segmentation with unified translation

The two major challenges to deep-learning-based medical image segmentation are multi-modality and a lack of expert annotations. Existing semi-supervised segmentation models can mitigate the problem of insufficient annotations by utilizing a small amount of labeled data. However, most of these models are limited to single-modal data and cannot exploit the complementary information from multi-modal medical images. A few semi-supervised multi-modal models have been proposed recently, but they have rigid structures and require additional training steps for each modality. In this work, we propose a novel flexible method, semi-supervised multi-modal medical image segmentation with unified translation (SMSUT), and a unique semi-supervised procedure that can leverage multi-modal information to improve the semi-supervised segmentation performance. Our architecture capitalizes on unified translation to extract complementary information from multi-modal data which compels the network to focus on the disparities and salient features among each modality. Furthermore, we impose constraints on the model at both pixel and feature levels, to cope with the lack of annotation information and the diverse representations within semi-supervised multi-modal data. We introduce a novel training procedure tailored for semi-supervised multi-modal medical image analysis, by integrating the concept of conditional translation. Our method has a remarkable ability for seamless adaptation to varying numbers of distinct modalities in the training data. Experiments show that our model exceeds the semi-supervised segmentation counterparts in the public datasets which proves our network's high-performance capabilities and the transferability of our proposed method. The code of our method will be openly available at https://github.com/Sue1347/SMSUT-MedicalImgSegmentation.

Hi-gMISnet: generalized medical image segmentation using DWT based multilayer fusion and dual mode attention into high resolutionpGAN

Objective.Automatic medical image segmentation is crucial for accurately isolating target tissue areas in the image from background tissues, facilitating precise diagnoses and procedures. While the proliferation of publicly available clinical datasets led to the development of deep learning-based medical image segmentation methods, a generalized, accurate, robust, and reliable approach across diverse imaging modalities remains elusive.Approach.This paper proposes a novel high-resolution parallel generative adversarial network (pGAN)-based generalized deep learning method for automatic segmentation of medical images from diverse imaging modalities. The proposed method showcases better performance and generalizability by incorporating novel components such as partial hybrid transfer learning, discrete wavelet transform (DWT)-based multilayer and multiresolution feature fusion in the encoder, and a dual mode attention gate in the decoder of the multi-resolution U-Net-based GAN. With multi-objective adversarial training loss functions including a unique reciprocal loss for enforcing cooperative learning inpGANs, it further enhances the robustness and accuracy of the segmentation map.Main results.Experimental evaluations conducted on nine diverse publicly available medical image segmentation datasets, including PhysioNet ICH, BUSI, CVC-ClinicDB, MoNuSeg, GLAS, ISIC-2018, DRIVE, Montgomery, and PROMISE12, demonstrate the proposed method's superior performance. The proposed method achieves mean F1 scores of 79.53%, 88.68%, 82.50%, 93.25%, 90.40%, 94.19%, 81.65%, 98.48%, and 90.79%, respectively, on the above datasets, surpass state-of-the-art segmentation methods. Furthermore, our proposed method demonstrates robust multi-domain segmentation capabilities, exhibiting consistent and reliable performance. The assessment of the model's proficiency in accurately identifying small details indicates that the high-resolution generalized medical image segmentation network (Hi-gMISnet) is more precise in segmenting even when the target area is very small.Significance.The proposed method provides robust and reliable segmentation performance on medical images, and thus it has the potential to be used in a clinical setting for the diagnosis of patients.

A lightweight bladder tumor segmentation method based on attention mechanism

In the endoscopic images of bladder, accurate segmentation of different grade bladder tumor from blurred boundary regions and highly variable shapes is of great significance for doctors' diagnosis and patients' later treatment. We propose a nested attentional feature fusion segmentation network (NAFF-Net) based on the encoder-decoder structure formed by the combination of weighted pyramid pooling module (WPPM) and nested attentional feature fusion (NAFF). Among them, WPPM applies the cascade of atrous convolution to enhance the overall perceptual field while introducing adaptive weights to optimize multi-scale feature extraction, NAFF integrates deep semantic information into shallow feature maps, effectively focusing on edge and detail information in bladder tumor images. Additionally, a weighted mixed loss function is constructed to alleviate the impact of imbalance between positive and negative sample distribution on segmentation accuracy. Experiments illustrate the proposed NAFF-Net achieves better segmentation results compared to other mainstream models, with a MIoU of 84.05%, MPrecision of 91.52%, MRecall of 90.81%, and F1-score of 91.16%, and also achieves good results on the public datasets Kvasir-SEG and CVC-ClinicDB. Compared to other models, NAFF-Net has a smaller number of parameters, which is a significant advantage in model deployment.

A recurrent positional encoding circular attention mechanism network for biomedical image segmentation

Deep-learning-based medical image segmentation techniques can assist doctors in disease diagnosis and rapid treatment. However, existing medical image segmentation models do not fully consider the dependence between feature segments in the feature extraction process, and the correlated features can be further extracted. Therefore, a recurrent positional encoding circular attention mechanism network (RPECAMNet) is proposed based on relative positional encoding for medical image segmentation. Multiple residual modules are used to extract the primary features of the medical images, which are thereafter converted into one-dimensional data for relative positional encoding. The recursive former is used to further extract features from medical images, and decoding is performed using deconvolution. An adaptive loss function is designed to train the model and achieve accurate medical-image segmentation. Finally, the proposed model is used to conduct comparative experiments on the synapse and self-constructed kidney datasets to verify the accuracy of the proposed model for medical image segmentation.

Generative adversarial networks in dental imaging: a systematic review

OBJECTIVES

This systematic review on generative adversarial network (GAN) architectures for dental image analysis provides a comprehensive overview to readers regarding current GAN trends in dental imagery and potential future applications.

METHODS

Electronic databases (PubMed/MEDLINE, Scopus, Embase, and Cochrane Library) were searched to identify studies involving GANs for dental image analysis. Eighteen full-text articles describing the applications of GANs in dental imagery were reviewed. Risk of bias and applicability concerns were assessed using the QUADAS-2 tool.

RESULTS

GANs were used for various imaging modalities, including two-dimensional and three-dimensional images. In dental imaging, GANs were utilized for tasks such as artifact reduction, denoising, and super-resolution, domain transfer, image generation for augmentation, outcome prediction, and identification. The generated images were incorporated into tasks such as landmark detection, object detection and classification. Because of heterogeneity among the studies, a meta-analysis could not be conducted. Most studies (72%) had a low risk of bias in all four domains. However, only three (17%) studies had a low risk of applicability concerns.

CONCLUSIONS

This extensive analysis of GANs in dental imaging highlighted their broad application potential within the dental field. Future studies should address limitations related to the stability, repeatability, and overall interpretability of GAN architectures. By overcoming these challenges, the applicability of GANs in dentistry can be enhanced, ultimately benefiting the dental field in its use of GANs and artificial intelligence.

Impact of bias field correction on 0.35 T pelvic MR images: evaluation on generative adversarial network-based OARs' auto-segmentation and visual grading assessment

Purpose

Magnetic resonance imaging (MRI)-guided radiotherapy enables adaptive treatment plans based on daily anatomical changes and accurate organ visualization. However, the bias field artifact can compromise image quality, affecting diagnostic accuracy and quantitative analyses. This study aims to assess the impact of bias field correction on 0.35 T pelvis MRIs by evaluating clinical anatomy visualization and generative adversarial network (GAN) auto-segmentation performance.

Materials and methods

3D simulation MRIs from 60 prostate cancer patients treated on MR-Linac (0.35 T) were collected and preprocessed with the N4ITK algorithm for bias field correction. A 3D GAN architecture was trained, validated, and tested on 40, 10, and 10 patients, respectively, to auto-segment the organs at risk (OARs) rectum and bladder. The GAN was trained and evaluated either with the original or the bias-corrected MRIs. The Dice similarity coefficient (DSC) and 95th percentile Hausdorff distance (HD95th) were computed for the segmented volumes of each patient. The Wilcoxon signed-rank test assessed the statistical difference of the metrics within OARs, both with and without bias field correction. Five radiation oncologists blindly scored 22 randomly chosen patients in terms of overall image quality and visibility of boundaries (prostate, rectum, bladder, seminal vesicles) of the original and bias-corrected MRIs. Bennett's S score and Fleiss' kappa were used to assess the pairwise interrater agreement and the interrater agreement among all the observers, respectively.

Results

In the test set, the GAN trained and evaluated on original and bias-corrected MRIs showed DSC/HD95th of 0.92/5.63 mm and 0.92/5.91 mm for the bladder and 0.84/10.61 mm and 0.83/9.71 mm for the rectum. No statistical differences in the distribution of the evaluation metrics were found neither for the bladder (DSC: p = 0.07; HD95th: p = 0.35) nor for the rectum (DSC: p = 0.32; HD95th: p = 0.63). From the clinical visual grading assessment, the bias-corrected MRI resulted mostly in either no change or an improvement of the image quality and visualization of the organs' boundaries compared with the original MRI.

Conclusion

The bias field correction did not improve the anatomy visualization from a clinical point of view and the OARs' auto-segmentation outputs generated by the GAN.

Applications of AI in multi-modal imaging for cardiovascular disease

Data for healthcare is diverse and includes many different modalities. Traditional approaches to Artificial Intelligence for cardiovascular disease were typically limited to single modalities. With the proliferation of diverse datasets and new methods in AI, we are now able to integrate different modalities, such as magnetic resonance scans, computerized tomography scans, echocardiography, x-rays, and electronic health records. In this paper, we review research from the last 5 years in applications of AI to multi-modal imaging. There have been many promising results in registration, segmentation, and fusion of different magnetic resonance imaging modalities with each other and computer tomography scans, but there are still many challenges that need to be addressed. Only a few papers have addressed modalities such as x-ray, echocardiography, or non-imaging modalities. As for prediction or classification tasks, there have only been a couple of papers that use multiple modalities in the cardiovascular domain. Furthermore, no models have been implemented or tested in real world cardiovascular clinical settings.

Radiomics and Deep Learning in Nasopharyngeal Carcinoma: A Review

Nasopharyngeal carcinoma is a common head and neck malignancy with distinct clinical management compared to other types of cancer. Precision risk stratification and tailored therapeutic interventions are crucial to improving the survival outcomes. Artificial intelligence, including radiomics and deep learning, has exhibited considerable efficacy in various clinical tasks for nasopharyngeal carcinoma. These techniques leverage medical images and other clinical data to optimize clinical workflow and ultimately benefit patients. In this review, we provide an overview of the technical aspects and basic workflow of radiomics and deep learning in medical image analysis. We then conduct a detailed review of their applications to seven typical tasks in the clinical diagnosis and treatment of nasopharyngeal carcinoma, covering various aspects of image synthesis, lesion segmentation, diagnosis, and prognosis. The innovation and application effects of cutting-edge research are summarized. Recognizing the heterogeneity of the research field and the existing gap between research and clinical translation, potential avenues for improvement are discussed. We propose that these issues can be gradually addressed by establishing standardized large datasets, exploring the biological characteristics of features, and technological upgrades.

Machine and Deep Learning in Hyperspectral Fluorescence-Guided Brain Tumor Surgery

Malignant glioma resection is often the first line of treatment in neuro-oncology. During glioma surgery, the discrimination of tumor's edges can be challenging at the infiltration zone, even by using surgical adjuncts such as fluorescence guidance (e.g., with 5-aminolevulinic acid). Challenging cases in which there is no visible fluorescence include lower-grade gliomas, tumor cells infiltrating beyond the margin as visualized on pre- and/or intraoperative MRI, and even some high-grade tumors. One field of research aiming to address this problem involves inspecting in detail the light emission spectra from different tissues (e.g., tumor vs. normal brain vs. brain parenchyma infiltrated by tumor cells). Hyperspectral imaging measures the emission spectrum at every image pixel level, thus combining spatial and spectral information. Assuming that different tissue types have different "spectral footprints," eventually related to higher or lower abundances of fluorescent dyes or auto-fluorescing molecules, the tissue can then be segmented according to type, providing surgeons a detailed spatial map of what they see. However, processing from raw hyperspectral data cubes to maps or overlays of tissue labels and potentially further molecular information is complex. This chapter will explore some of the classical methods for the various steps of this process and examine how they can be improved with machine learning approaches. While preliminary work on machine learning in hyperspectral imaging has had relatively limited success in brain tumor surgery, more recent research combines this with fluorescence to obtain promising results. In particular, this chapter describes a pipeline that isolates biopsies in ex vivo hyperspectral fluorescence images for efficient labeling, extracts all the relevant emission spectra, preprocesses them to correct for various optical properties, and determines the abundance of fluorophores in each pixel, which correspond directly with the presence of cancerous tissue. Each step contains a combination of classical and deep learning-based methods. Furthermore, the fluorophore abundances are then used in four machine learning models to classify tumor type, WHO grade, margin tissue type, and isocitrate dehydrogenase (IDH) mutation status in brain tumors. The classifiers achieved average test accuracies of 87%, 96.1%, 86%, and 93%, respectively, thus greatly outperforming prior work both with and without fluorescence. This field is new, but these early results show great promise for the feasibility of data-driven hyperspectral imaging for intraoperative classification of brain tumors during fluorescence-guided surgery.

ExS-GAN: Synthesizing Anti-Forensics Images via Extra Supervised GAN

So far, researchers have proposed many forensics tools to protect the authenticity and integrity of digital information. However, with the explosive development of machine learning, existing forensics tools may compromise against new attacks anytime. Hence, it is always necessary to investigate anti-forensics to expose the vulnerabilities of forensics tools. It is beneficial for forensics researchers to develop new tools as countermeasures. To date, one of the potential threats is the generative adversarial networks (GANs), which could be employed for fabricating or forging falsified data to attack forensics detectors. In this article, we investigate the anti-forensics performance of GANs by proposing a novel model, the ExS-GAN, which features an extra supervision system. After training, the proposed model could launch anti-forensics attacks on various manipulated images. Evaluated by experiments, the proposed method could achieve high anti-forensics performance while preserving satisfying image quality. We also justify the proposed extra supervision via an ablation study.

K. SK, Jayaraman S. Automatic foot ulcer segmentation using conditional generative adversarial network (AFSegGAN): A wound management system

Effective wound care is essential to prevent further complications, promote healing, and reduce the risk of infection and other health issues. Chronic wounds, particularly in older adults, patients with disabilities, and those with pressure, venous, or diabetic foot ulcers, cause significant morbidity and mortality. Due to the positive trend in the number of individuals with chronic wounds, particularly among the growing elderly and diabetes populations, it is imperative to develop novel technologies and practices for the best practice clinical management of chronic wounds to minimize the potential health and economic burdens on society. As wound care is managed in hospitals and community care, it is crucial to have quantitative metrics like wound boundary and morphological features. The traditional visual inspection technique is purely subjective and error-prone, and digitization provides an appealing alternative. Various deep-learning models have earned confidence; however, their accuracy primarily relies on the image quality, the dataset size to learn the features, and experts' annotation. This work aims to develop a wound management system that automates wound segmentation using a conditional generative adversarial network (cGAN) and estimate the wound morphological parameters. AFSegGAN was developed and validated on the MICCAI 2021-foot ulcer segmentation dataset. In addition, we use adversarial loss and patch-level comparison at the discriminator network to improve the segmentation performance and balance the GAN network training. Our model outperformed state-of-the-art methods with a Dice score of 93.11% and IoU of 99.07%. The proposed wound management system demonstrates its abilities in wound segmentation and parameter estimation, thereby reducing healthcare workers' efforts to diagnose or manage wounds and facilitating remote healthcare.

MUE-CoT: multi-scale uncertainty entropy-aware co-training framework for left atrial segmentation

Objective.Accurate left atrial segmentation is the basis of the recognition and clinical analysis of atrial fibrillation. Supervised learning has achieved some competitive segmentation results, but the high annotation cost often limits its performance. Semi-supervised learning is implemented from limited labeled data and a large amount of unlabeled data and shows good potential in solving practical medical problems.Approach. In this study, we proposed a collaborative training framework for multi-scale uncertain entropy perception (MUE-CoT) and achieved efficient left atrial segmentation from a small amount of labeled data. Based on the pyramid feature network, learning is implemented from unlabeled data by minimizing the pyramid prediction difference. In addition, novel loss constraints are proposed for co-training in the study. The diversity loss is defined as a soft constraint so as to accelerate the convergence and a novel multi-scale uncertainty entropy calculation method and a consistency regularization term are proposed to measure the consistency between prediction results. The quality of pseudo-labels cannot be guaranteed in the pre-training period, so a confidence-dependent empirical Gaussian function is proposed to weight the pseudo-supervised loss.Main results.The experimental results of a publicly available dataset and an in-house clinical dataset proved that our method outperformed existing semi-supervised methods. For the two datasets with a labeled ratio of 5%, the Dice similarity coefficient scores were 84.94% ± 4.31 and 81.24% ± 2.4, the HD95values were 4.63 mm ± 2.13 and 3.94 mm ± 2.72, and the Jaccard similarity coefficient scores were 74.00% ± 6.20 and 68.49% ± 3.39, respectively.Significance.The proposed model effectively addresses the challenges of limited data samples and high costs associated with manual annotation in the medical field, leading to enhanced segmentation accuracy.

High-Density Functional Near-Infrared Spectroscopy and Machine Learning for Visual Perception Quantification

The main application scenario for wearable sensors involves the generation of data and monitoring metrics. fNIRS (functional near-infrared spectroscopy) allows the nonintrusive monitoring of human visual perception. The quantification of visual perception by fNIRS facilitates applications in engineering-related fields. This study designed a set of experimental procedures to effectively induce visible alterations and to quantify visual perception in conjunction with the acquisition of Hbt (total hemoglobin), Hb (hemoglobin), and HbO2 (oxygenated hemoglobin) data obtained from HfNIRS (high-density functional near-infrared spectroscopy). Volunteers completed the visual task separately in response to different visible changes in the simulated scene. HfNIRS recorded the changes in Hbt, Hb, and HbO2 during the study, the time point of the visual difference, and the time point of the task change. This study consisted of one simulated scene, two visual variations, and four visual tasks. The simulation scene featured a car driving location. The visible change suggested that the brightness and saturation of the car operator interface would change. The visual task represented the completion of the layout, color, design, and information questions answered in response to the visible change. This study collected data from 29 volunteers. The volunteers completed the visual task separately in response to different visual changes in the same simulated scene. HfNIRS recorded the changes in Hbt, Hb, and HbO2 during the study, the time point of the visible difference, and the time point of the task change. The data analysis methods in this study comprised a combination of channel dimensionality reduction, feature extraction, task classification, and score correlation. Channel downscaling: This study used the data of 15 channels in HfNIRS to calculate the mutual information between different channels to set a threshold, and to retain the data of the channels that were higher than those of the mutual information. Feature extraction: The statistics derived from the visual task, including time, mean, median, variance, extreme variance, kurtosis, bias, information entropy, and approximate entropy were computed. Task classification: This study used the KNN (K-Nearest Neighbors) algorithm to classify different visual tasks and to calculate the accuracy, precision, recall, and F1 scores. Scoring correlation: This study matched the visual task scores with the fluctuations of Hbt, Hb, and HbO2 and observed the changes in Hbt, Hb, and HbO2 under different scoring levels. Mutual information was used to downscale the channels, and seven channels were retained for analysis under each visual task. The average accuracy was 96.3% ± 1.99%; the samples that correctly classified the visual task accounted for 96.3% of the total; and the classification accuracy was high. By analyzing the correlation between the scores on different visual tasks and the fluctuations of Hbt, Hb, and HbO2, it was found that the higher the score, the more obvious, significant, and higher the fluctuations of Hbt, Hb, and HbO2. Experiments found that changes in visual perception triggered changes in Hbt, Hb, and HbO2. HfNIRS combined with Hbt, Hb, and HbO2 recorded by machine learning algorithms can effectively quantify visual perception. However, the related research in this paper still needs to be further refined, and the mathematical relationship between HfNIRS and visual perception needs to be further explored to realize the quantitative study of subjective and objective visual perception supported by the mathematical relationship.

Evaluating the performance of generative adversarial network-synthesized periapical images in classifying C-shaped root canals

This study evaluated the performance of generative adversarial network (GAN)-synthesized periapical images for classifying C-shaped root canals, which are challenging to diagnose because of their complex morphology. GANs have emerged as a promising technique for generating realistic images, offering a potential solution for data augmentation in scenarios with limited training datasets. Periapical images were synthesized using the StyleGAN2-ADA framework, and their quality was evaluated based on the average Frechet inception distance (FID) and the visual Turing test. The average FID was found to be 35.353 (± 4.386) for synthesized C-shaped canal images and 25.471 (± 2.779) for non C-shaped canal images. The visual Turing test conducted by two radiologists on 100 randomly selected images revealed that distinguishing between real and synthetic images was difficult. These results indicate that GAN-synthesized images exhibit satisfactory visual quality. The classification performance of the neural network, when augmented with GAN data, showed improvements compared with using real data alone, and could be advantageous in addressing data conditions with class imbalance. GAN-generated images have proven to be an effective data augmentation method, addressing the limitations of limited training data and computational resources in diagnosing dental anomalies.

Placental Vessel Segmentation Using Pix2pix Compared to U-Net

Computer-assisted technologies have made significant progress in fetoscopic laser surgery, including placental vessel segmentation. However, the intra- and inter-procedure variabilities in the state-of-the-art segmentation methods remain a significant hurdle. To address this, we investigated the use of conditional generative adversarial networks (cGANs) for fetoscopic image segmentation and compared their performance with the benchmark U-Net technique for placental vessel segmentation. Two deep-learning models, U-Net and pix2pix (a popular cGAN model), were trained and evaluated using a publicly available dataset and an internal validation set. The overall results showed that the pix2pix model outperformed the U-Net model, with a Dice score of 0.80 [0.70; 0.86] versus 0.75 [0.0.60; 0.84] (p-value < 0.01) and an Intersection over Union (IoU) score of 0.70 [0.61; 0.77] compared to 0.66 [0.53; 0.75] (p-value < 0.01), respectively. The internal validation dataset further validated the superiority of the pix2pix model, achieving Dice and IoU scores of 0.68 [0.53; 0.79] and 0.59 [0.49; 0.69] (p-value < 0.01), respectively, while the U-Net model obtained scores of 0.53 [0.49; 0.64] and 0.49 [0.17; 0.56], respectively. This study successfully compared U-Net and pix2pix models for placental vessel segmentation in fetoscopic images, demonstrating improved results with the cGAN-based approach. However, the challenge of achieving generalizability still needs to be addressed.

Multichannel One-Dimensional Data Augmentation with Generative Adversarial Network

Data augmentation is one of the most important problems in deep learning. There have been many algorithms proposed to solve this problem, such as simple noise injection, the generative adversarial network (GAN), and diffusion models. However, to the best of our knowledge, these works mainly focused on computer vision-related tasks, and there have not been many proposed works for one-dimensional data. This paper proposes a GAN-based data augmentation for generating multichannel one-dimensional data given single-channel inputs. Our architecture consists of multiple discriminators that adapt deep convolution GAN (DCGAN) and patchGAN to extract the overall pattern of the multichannel generated data while also considering the local information of each channel. We conducted an experiment with website fingerprinting data. The result for the three channels' data augmentation showed that our proposed model obtained FID scores of 0.005,0.017,0.051 for each channel, respectively, compared to 0.458,0.551,0.521 when using the vanilla GAN.

CHSNet: Automatic lesion segmentation network guided by CT image features for acute cerebral hemorrhage

Stroke is a cerebrovascular disease that can lead to severe sequelae such as hemiplegia and mental retardation with a mortality rate of up to 40%. In this paper, we proposed an automatic segmentation network (CHSNet) to segment the lesions in cranial CT images based on the characteristics of acute cerebral hemorrhage images, such as high density, multi-scale, and variable location, and realized the three-dimensional (3D) visualization and localization of the cranial lesions after the segmentation was completed. To enhance the feature representation of high-density regions, and capture multi-scale and up-down information on the target location, we constructed a convolutional neural network with encoding-decoding backbone, Res-RCL module, Atrous Spatial Pyramid Pooling, and Attention Gate. We collected images of 203 patients with acute cerebral hemorrhage, constructed a dataset containing 5998 cranial CT slices, and conducted comparative and ablation experiments on the dataset to verify the effectiveness of our model. Our model achieved the best results on both test sets with different segmentation difficulties, test1: Dice = 0.918, IoU = 0.853, ASD = 0.476, RVE = 0.113; test2: Dice = 0.716, IoU = 0.604, ASD = 5.402, RVE = 1.079. Based on the segmentation results, we achieved 3D visualization and localization of hemorrhage in CT images of stroke patients. The study has important implications for clinical adjuvant diagnosis.

Influence of Data Augmentation Strategies on the Segmentation of Oral Histological Images Using Fully Convolutional Neural Networks

Segmentation of tumor regions in H &E-stained slides is an important task for a pathologist while diagnosing different types of cancer, including oral squamous cell carcinoma (OSCC). Histological image segmentation is often constrained by the availability of labeled training data since labeling histological images is a highly skilled, complex, and time-consuming task. Thus, data augmentation strategies become essential to train convolutional neural networks models to overcome the overfitting problem when only a few training samples are available. This paper proposes a new data augmentation strategy, named Random Composition Augmentation (RCAug), to train fully convolutional networks (FCN) to segment OSCC tumor regions in H &E-stained histological images. Given the input image and their corresponding label, a pipeline with a random composition of geometric, distortion, color transfer, and generative image transformations is executed on the fly. Experimental evaluations were performed using an FCN-based method to segment OSCC regions through a set of different data augmentation transformations. By using RCAug, we improved the FCN-based segmentation method from 0.51 to 0.81 of intersection-over-union (IOU) in a whole slide image dataset and from 0.65 to 0.69 of IOU in a tissue microarray images dataset.

Retinal Vascular Image Segmentation Using Improved UNet Based on Residual Module

In recent years, deep learning technology for clinical diagnosis has progressed considerably, and the value of medical imaging continues to increase. In the past, clinicians evaluated medical images according to their individual expertise. In contrast, the application of artificial intelligence technology for automatic analysis and diagnostic assistance to support clinicians in evaluating medical information more efficiently has become an important trend. In this study, we propose a machine learning architecture designed to segment images of retinal blood vessels based on an improved U-Net neural network model. The proposed model incorporates a residual module to extract features more effectively, and includes a full-scale skip connection to combine low level details with high-level features at different scales. The results of an experimental evaluation show that the model was able to segment images of retinal vessels accurately. The proposed method also outperformed several existing models on the benchmark datasets DRIVE and ROSE, including U-Net, ResUNet, U-Net3+, ResUNet++, and CaraNet.

An Improved Co-Training and Generative Adversarial Network (Diff-CoGAN) for Semi-Supervised Medical Image Segmentation

Semi-supervised learning is a technique that utilizes a limited set of labeled data and a large amount of unlabeled data to overcome the challenges of obtaining a perfect dataset in deep learning, especially in medical image segmentation. The accuracy of the predicted labels for the unlabeled data is a critical factor that affects the training performance, thus reducing the accuracy of segmentation. To address this issue, a semi-supervised learning method based on the Diff-CoGAN framework was proposed, which incorporates co-training and generative adversarial network (GAN) strategies. The proposed Diff-CoGAN framework employs two generators and one discriminator. The generators work together by providing mutual information guidance to produce predicted maps that are more accurate and closer to the ground truth. To further improve segmentation accuracy, the predicted maps are subjected to an intersection operation to identify a high-confidence region of interest, which reduces boundary segmentation errors. The predicted maps are then fed into the discriminator, and the iterative process of adversarial training enhances the generators’ ability to generate more precise maps, while also improving the discriminator’s ability to distinguish between the predicted maps and the ground truth. This study conducted experiments on the Hippocampus and Spleen images from the Medical Segmentation Decathlon (MSD) dataset using three semi-supervised methods: co-training, semi-GAN, and Diff-CoGAN. The experimental results demonstrated that the proposed Diff-CoGAN approach significantly enhanced segmentation accuracy compared to the other two methods by benefiting on the mutual guidance of the two generators and the adversarial training between the generators and discriminator. The introduction of the intersection operation prior to the discriminator also further reduced boundary segmentation errors.

CellTranspose: Few-shot Domain Adaptation for Cellular Instance Segmentation

Automated cellular instance segmentation is a process utilized for accelerating biological research for the past two decades, and recent advancements have produced higher quality results with less effort from the biologist. Most current endeavors focus on completely cutting the researcher out of the picture by generating highly generalized models. However, these models invariably fail when faced with novel data, distributed differently than the ones used for training. Rather than approaching the problem with methods that presume the availability of large amounts of target data and computing power for retraining, in this work we address the even greater challenge of designing an approach that requires minimal amounts of new annotated data as well as training time. We do so by designing specialized contrastive losses that leverage the few annotated samples very efficiently. A large set of results show that 3 to 5 annotations lead to models with accuracy that: 1) significantly mitigate the covariate shift effects; 2) matches or surpasses other adaptation methods; 3) even approaches methods that have been fully retrained on the target distribution. The adaptation training is only a few minutes, paving a path towards a balance between model performance, computing requirements and expert-level annotation needs.

E-DU: Deep neural network for multimodal medical image segmentation based on semantic gap compensation

BACKGROUND

U-Net includes encoder, decoder and skip connection structures. It has become the benchmark network in medical image segmentation. However, the direct fusion of low-level and high-level convolution features with semantic gaps by traditional skip connections may lead to problems such as fuzzy generated feature maps and target region segmentation errors.

OBJECTIVE

We use spatial enhancement filtering technology to compensate for the semantic gap and propose an enhanced dense U-Net (E-DU), aiming to apply it to multimodal medical image segmentation to improve the segmentation performance and efficiency.

METHODS

Before combining encoder and decoder features, we replace the traditional skip connection with a multiscale denoise enhancement (MDE) module. The encoder features need to be deeply convolved by the spatial enhancement filter and then combined with the decoder features. We propose a simple and efficient deep full convolution network structure E-DU, which can not only fuse semantically various features but also denoise and enhance the feature map.

RESULTS

We performed experiments on medical image segmentation datasets with seven image modalities and combined MDE with various baseline networks to perform ablation studies. E-DU achieved the best segmentation results on evaluation indicators such as DSC on the U-Net family, with DSC values of 97.78, 97.64, 95.31, 94.42, 94.93, 98.85, and 98.38 (%), respectively. The addition of the MDE module to the attention mechanism network improves segmentation performance and efficiency, reflecting its generalization performance. In comparison to advanced methods, our method is also competitive.

CONCLUSION

Our proposed MDE module has a good segmentation effect and operating efficiency, and it can be easily extended to multiple modal medical segmentation datasets. Our idea and method can achieve clinical multimodal medical image segmentation and make full use of image information to provide clinical decision support. It has great application value and promotion prospects.

Introducing AI to the molecular tumor board: one direction toward the establishment of precision medicine using large-scale cancer clinical and biological information

Since U.S. President Barack Obama announced the Precision Medicine Initiative in his New Year's State of the Union address in 2015, the establishment of a precision medicine system has been emphasized worldwide, particularly in the field of oncology. With the advent of next-generation sequencers specifically, genome analysis technology has made remarkable progress, and there are active efforts to apply genome information to diagnosis and treatment. Generally, in the process of feeding back the results of next-generation sequencing analysis to patients, a molecular tumor board (MTB), consisting of experts in clinical oncology, genetic medicine, etc., is established to discuss the results. On the other hand, an MTB currently involves a large amount of work, with humans searching through vast databases and literature, selecting the best drug candidates, and manually confirming the status of available clinical trials. In addition, as personalized medicine advances, the burden on MTB members is expected to increase in the future. Under these circumstances, introducing cutting-edge artificial intelligence (AI) technology and information and communication technology to MTBs while reducing the burden on MTB members and building a platform that enables more accurate and personalized medical care would be of great benefit to patients. In this review, we introduced the latest status of elemental technologies that have potential for AI utilization in MTB, and discussed issues that may arise in the future as we progress with AI implementation.

FetalGAN: Automated Segmentation of Fetal Functional Brain MRI Using Deep Generative Adversarial Learning and Multi-Scale 3D U-Net

An important step in the preprocessing of resting state functional magnetic resonance images (rs-fMRI) is the separation of brain from non-brain voxels. Widely used imaging tools such as FSL's BET2 and AFNI's 3dSkullStrip accomplish this task effectively in children and adults. In fetal functional brain imaging, however, the presence of maternal tissue around the brain coupled with the non-standard position of the fetal head limit the usefulness of these tools. Accurate brain masks are thus generated manually, a time-consuming and tedious process that slows down preprocessing of fetal rs-fMRI. Recently, deep learning-based segmentation models such as convolutional neural networks (CNNs) have been increasingly used for automated segmentation of medical images, including the fetal brain. Here, we propose a computationally efficient end-to-end generative adversarial neural network (GAN) for segmenting the fetal brain. This method, which we call FetalGAN, yielded whole brain masks that closely approximated the manually labeled ground truth. FetalGAN performed better than 3D U-Net model and BET2: FetalGAN, Dice score = 0.973 ± 0.013, precision = 0.977 ± 0.015; 3D U-Net, Dice score = 0.954 ± 0.054, precision = 0.967 ± 0.037; BET2, Dice score = 0.856 ± 0.084, precision = 0.758 ± 0.113. FetalGAN was also faster than 3D U-Net and the manual method (7.35 s vs. 10.25 s vs. ∼5 min/volume). To the best of our knowledge, this is the first successful implementation of 3D CNN with GAN on fetal fMRI brain images and represents a significant advance in fully automating processing of rs-MRI images.

Facial skin disease prediction using StarGAN v2 and transfer learning

Deep learning algorithms have become the most prominent methods for medical image analysis over the past years, leading to enhanced performances in various medical applications. In this paper, we focus on applying intelligent skin disease detection to face images, where the crucial challenge is the low availability of training data. To achieve high disease detection and classification success rates, we adapt the state-of-the-art StarGAN v2 network to augment images of faces and combine it with a transfer learning approach. The experimental results show that the classification accuracies of transfer learning models are in the range of 77.46–99.80% when trained on datasets that are extended with StarGAN v2 augmented data.