Fine Perceptive GANs for Brain MR Image Super-Resolution in Wavelet Domain

A systematic review of generative AI approaches for medical image enhancement: Comparing GANs, transformers, and diffusion models

BACKGROUND

Medical imaging is a vital diagnostic tool that provides detailed insights into human anatomy but faces challenges affecting its accuracy and efficiency. Advanced generative AI models offer promising solutions. Unlike previous reviews with a narrow focus, a comprehensive evaluation across techniques and modalities is necessary.

OBJECTIVE

This systematic review integrates the three state-of-the-art leading approaches, GANs, Diffusion Models, and Transformers, examining their applicability, methodologies, and clinical implications in improving medical image quality.

METHODS

Using the PRISMA framework, 63 studies from 989 were selected via Google Scholar and PubMed, focusing on GANs, Transformers, and Diffusion Models. Articles from ACM, IEEE Xplore, and Springer were analyzed.

RESULTS

Generative AI techniques show promise in improving image resolution, reducing noise, and enhancing fidelity. GANs generate high-quality images, Transformers utilize global context, and Diffusion Models are effective in denoising and reconstruction. Challenges include high computational costs, limited dataset diversity, and issues with generalizability, with a focus on quantitative metrics over clinical applicability.

CONCLUSION

This review highlights the transformative impact of GANs, Transformers, and Diffusion Models in advancing medical imaging. Future research must address computational and generalization challenges, emphasize open science, and validate these techniques in diverse clinical settings to unlock their full potential. These efforts could enhance diagnostic accuracy, lower costs, and improve patient outcome.

TPDC: Point Cloud Completion by Triangular Pyramid Features and Divide-and-Conquer in Complex Environments

Point cloud completion recovers the complete point clouds from partial ones, providing numerous point cloud information for downstream tasks such as 3-D reconstruction and target detection. However, previous methods usually suffer from unstructured prediction of points in local regions and the discrete nature of the point cloud. To resolve these problems, we propose a point cloud completion network called TPDC. Representing the point cloud as a set of unordered features of points with local geometric information, we devise a Triangular Pyramid Extractor (TPE), using the simplest 3-D structure-a triangular pyramid-to convert the point cloud to a sequence of local geometric information. Our insight of revealing local geometric information in a complex environment is to design a Divide-and-Conquer Splitting Module in a Divide-and-Conquer Splitting Decoder (DCSD) to learn point-splitting patterns that can fit local regions the best. This module employs the Divide-and-Conquer approach to parallelly handle tasks related to fitting ground-truth values to base points and predicting the displacement of split points. This approach aims to make the base points align more closely with the ground-truth values while also forecasting the displacement of split points relative to the base points. Furthermore, we propose a more realistic and challenging benchmark, ShapeNetMask, with more random point cloud input, more complex random item occlusion, and more realistic random environmental perturbations. The results show that our method outperforms both widely used benchmarks as well as the new benchmark.

Generative Adversarial Networks for Neuroimage Translation

Image-to-image translation has gained popularity in the medical field to transform images from one domain to another. Medical image synthesis via domain transformation is advantageous in its ability to augment an image dataset where images for a given class are limited. From the learning perspective, this process contributes to the data-oriented robustness of the model by inherently broadening the model's exposure to more diverse visual data and enabling it to learn more generalized features. In the case of generating additional neuroimages, it is advantageous to obtain unidentifiable medical data and augment smaller annotated datasets. This study proposes the development of a cycle-consistent generative adversarial network (CycleGAN) model for translating neuroimages from one field strength to another (e.g., 3 Tesla [T] to 1.5 T). This model was compared with a model based on a deep convolutional GAN model architecture. CycleGAN was able to generate the synthetic and reconstructed images with reasonable accuracy. The mapping function from the source (3 T) to the target domain (1.5 T) performed optimally with an average peak signal-to-noise ratio value of 25.69 ± 2.49 dB and a mean absolute error value of 2106.27 ± 1218.37. The codes for this study have been made publicly available in the following GitHub repository.a.

Accurate Lung Nodule Segmentation With Detailed Representation Transfer and Soft Mask Supervision

Accurate lung lesion segmentation from computed tomography (CT) images is crucial to the analysis and diagnosis of lung diseases, such as COVID-19 and lung cancer. However, the smallness and variety of lung nodules and the lack of high-quality labeling make the accurate lung nodule segmentation difficult. To address these issues, we first introduce a novel segmentation mask named " soft mask," which has richer and more accurate edge details description and better visualization, and develop a universal automatic soft mask annotation pipeline to deal with different datasets correspondingly. Then, a novel network with detailed representation transfer and soft mask supervision (DSNet) is proposed to process the input low-resolution images of lung nodules into high-quality segmentation results. Our DSNet contains a special detailed representation transfer module (DRTM) for reconstructing the detailed representation to alleviate the small size of lung nodules images and an adversarial training framework with soft mask for further improving the accuracy of segmentation. Extensive experiments validate that our DSNet outperforms other state-of-the-art methods for accurate lung nodule segmentation, and has strong generalization ability in other accurate medical segmentation tasks with competitive results. Besides, we provide a new challenging lung nodules segmentation dataset for further studies (https://drive.google.com/file/d/15NNkvDTb_0Ku0IoPsNMHezJRTH1Oi1wm/view?usp=sharing).

Multi-resolution visual Mamba with multi-directional selective mechanism for retinal disease detection

Introduction

Retinal diseases significantly impact patients' quality of life and increase social medical costs. Optical coherence tomography (OCT) offers high-resolution imaging for precise detection and monitoring of these conditions. While deep learning techniques have been employed to extract features from OCT images for classification, convolutional neural networks (CNNs) often fail to capture global context due to their focus on local receptive fields. Transformer-based methods, on the other hand, suffer from quadratic complexity when handling long-range dependencies.

Methods

To overcome these limitations, we introduce the Multi-Resolution Visual Mamba (MRVM) model, which addresses long-range dependencies with linear computational complexity for OCT image classification. The MRVM model initially employs convolution to extract local features and subsequently utilizes the retinal Mamba to capture global dependencies. By integrating multi-scale global features, the MRVM enhances classification accuracy and overall performance. Additionally, the multi-directional selection mechanism (MSM) within the retinal Mamba improves feature extraction by concentrating on various directions, thereby better capturing complex, orientation-specific retinal patterns.

Results

Experimental results demonstrate that the MRVM model excels in differentiating retinal images with various lesions, achieving superior detection accuracy compared to traditional methods, with overall accuracies of 98.98\% and 96.21\% on two public datasets, respectively.

Discussion

This approach offers a novel perspective for accurately identifying retinal diseases and could contribute to the development of more robust artificial intelligence algorithms and recognition systems for medical image-assisted diagnosis.

3-D Brain Reconstruction by Hierarchical Shape-Perception Network From a Single Incomplete Image

3-D shape reconstruction is essential in the navigation of minimally invasive and auto robot-guided surgeries whose operating environments are indirect and narrow, and there have been some works that focused on reconstructing the 3-D shape of the surgical organ through limited 2-D information available. However, the lack and incompleteness of such information caused by intraoperative emergencies (such as bleeding) and risk control conditions have not been considered. In this article, a novel hierarchical shape-perception network (HSPN) is proposed to reconstruct the 3-D point clouds (PCs) of specific brains from one single incomplete image with low latency. A branching predictor and several hierarchical attention pipelines are constructed to generate PCs that accurately describe the incomplete images and then complete these PCs with high quality. Meanwhile, attention gate blocks (AGBs) are designed to efficiently aggregate geometric local features of incomplete PCs transmitted by hierarchical attention pipelines and internal features of reconstructing PCs. With the proposed HSPN, 3-D shape perception and completion can be achieved spontaneously. Comprehensive results measured by Chamfer distance (CD) and PC-to-PC error demonstrate that the performance of the proposed HSPN outperforms other competitive methods in terms of qualitative displays, quantitative experiment, and classification evaluation.

Multicontrast MRI Super-Resolution via Transformer-Empowered Multiscale Contextual Matching and Aggregation

Magnetic resonance imaging (MRI) possesses the unique versatility to acquire images under a diverse array of distinct tissue contrasts, which makes multicontrast super-resolution (SR) techniques possible and needful. Compared with single-contrast MRI SR, multicontrast SR is expected to produce higher quality images by exploiting a variety of complementary information embedded in different imaging contrasts. However, existing approaches still have two shortcomings: 1) most of them are convolution-based methods and, hence, weak in capturing long-range dependencies, which are essential for MR images with complicated anatomical patterns and 2) they ignore to make full use of the multicontrast features at different scales and lack effective modules to match and aggregate these features for faithful SR. To address these issues, we develop a novel multicontrast MRI SR network via transformer-empowered multiscale feature matching and aggregation, dubbed McMRSR ++ . First, we tame transformers to model long-range dependencies in both reference and target images at different scales. Then, a novel multiscale feature matching and aggregation method is proposed to transfer corresponding contexts from reference features at different scales to the target features and interactively aggregate them. Furthermore, a texture-preserving branch and a contrastive constraint are incorporated into our framework for enhancing the textural details in the SR images. Experimental results on both public and clinical in vivo datasets show that McMRSR ++ outperforms state-of-the-art methods under peak signal to noise ratio (PSNR), structure similarity index measure (SSIM), and root mean square error (RMSE) metrics significantly. Visual results demonstrate the superiority of our method in restoring structures, demonstrating its great potential to improve scan efficiency in clinical practice.

LungViT: Ensembling Cascade of Texture Sensitive Hierarchical Vision Transformers for Cross-Volume Chest CT Image-to-Image Translation

Chest computed tomography (CT) at inspiration is often complemented by an expiratory CT to identify peripheral airways disease. Additionally, co-registered inspiratory-expiratory volumes can be used to derive various markers of lung function. Expiratory CT scans, however, may not be acquired due to dose or scan time considerations or may be inadequate due to motion or insufficient exhale; leading to a missed opportunity to evaluate underlying small airways disease. Here, we propose LungViT- a generative adversarial learning approach using hierarchical vision transformers for translating inspiratory CT intensities to corresponding expiratory CT intensities. LungViT addresses several limitations of the traditional generative models including slicewise discontinuities, limited size of generated volumes, and their inability to model texture transfer at volumetric level. We propose a shifted-window hierarchical vision transformer architecture with squeeze-and-excitation decoder blocks for modeling dependencies between features. We also propose a multiview texture similarity distance metric for texture and style transfer in 3D. To incorporate global information into the training process and refine the output of our model, we use ensemble cascading. LungViT is able to generate large 3D volumes of size 320×320×320 . We train and validate our model using a diverse cohort of 1500 subjects with varying disease severity. To assess model generalizability beyond the development set biases, we evaluate our model on an out-of-distribution external validation set of 200 subjects. Clinical validation on internal and external testing sets shows that synthetic volumes could be reliably adopted for deriving clinical endpoints of chronic obstructive pulmonary disease.

Prior-Guided Adversarial Learning With Hypergraph for Predicting Abnormal Connections in Alzheimer's Disease

Alzheimer's disease (AD) is characterized by alterations of the brain's structural and functional connectivity during its progressive degenerative processes. Existing auxiliary diagnostic methods have accomplished the classification task, but few of them can accurately evaluate the changing characteristics of brain connectivity. In this work, a prior-guided adversarial learning with hypergraph (PALH) model is proposed to predict abnormal brain connections using triple-modality medical images. Concretely, a prior distribution from anatomical knowledge is estimated to guide multimodal representation learning using an adversarial strategy. Also, the pairwise collaborative discriminator structure is further utilized to narrow the difference in representation distribution. Moreover, the hypergraph perceptual network is developed to effectively fuse the learned representations while establishing high-order relations within and between multimodal images. Experimental results demonstrate that the proposed model outperforms other related methods in analyzing and predicting AD progression. More importantly, the identified abnormal connections are partly consistent with previous neuroscience discoveries. The proposed model can evaluate the characteristics of abnormal brain connections at different stages of AD, which is helpful for cognitive disease study and early treatment.

U-shaped convolutional transformer GAN with multi-resolution consistency loss for restoring brain functional time-series and dementia diagnosis

Introduction

The blood oxygen level-dependent (BOLD) signal derived from functional neuroimaging is commonly used in brain network analysis and dementia diagnosis. Missing the BOLD signal may lead to bad performance and misinterpretation of findings when analyzing neurological disease. Few studies have focused on the restoration of brain functional time-series data.

Methods

In this paper, a novel U-shaped convolutional transformer GAN (UCT-GAN) model is proposed to restore the missing brain functional time-series data. The proposed model leverages the power of generative adversarial networks (GANs) while incorporating a U-shaped architecture to effectively capture hierarchical features in the restoration process. Besides, the multi-level temporal-correlated attention and the convolutional sampling in the transformer-based generator are devised to capture the global and local temporal features for the missing time series and associate their long-range relationship with the other brain regions. Furthermore, by introducing multi-resolution consistency loss, the proposed model can promote the learning of diverse temporal patterns and maintain consistency across different temporal resolutions, thus effectively restoring complex brain functional dynamics.

Results

We theoretically tested our model on the public Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset, and our experiments demonstrate that the proposed model outperforms existing methods in terms of both quantitative metrics and qualitative assessments. The model's ability to preserve the underlying topological structure of the brain functional networks during restoration is a particularly notable achievement.

Conclusion

Overall, the proposed model offers a promising solution for restoring brain functional time-series and contributes to the advancement of neuroscience research by providing enhanced tools for disease analysis and interpretation.

Longitudinal MRI analysis using a hybrid DenseNet-BiLSTM method for Alzheimer's disease prediction

Alzheimer's disease is a progressive neurological disorder characterized by brain atrophy and cell death, leading to cognitive decline and impaired functioning. Previous research has primarily focused on using cross-sectional data for Alzheimer's disease identification, but analyzing longitudinal sequential MR images is crucial for improved diagnostic accuracy and understanding disease progression. However, existing deep learning models face challenges in learning spatial and temporal features from such data. To address these challenges, this study presents a novel hybrid DenseNet-BiLSTM method for Alzheimer's disease prediction using longitudinal MRI analysis. The proposed framework combines Convolutional DenseNet for spatial information extraction and joined BiLSTM layers for capturing temporal characteristics and relationships between longitudinal images at different time points. This approach overcomes issues like overfitting, vanishing gradients, and incomplete patient data. We evaluated the model on 684 longitudinal MRI images from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database, including normal controls, individuals with mild cognitive impairment, and Alzheimer's disease patients. The results demonstrate high classification accuracy, with 95.28% for AD/CN, 88.19% for NC/MCI, 83.51% for sMCI/pMCI, and 92.14% for MCI/AD. These findings highlight the substantial improvement in Alzheimer's disease diagnosis achieved through the utilization of longitudinal MRI images. The contributions of this study lie in both the deep learning and medical domains. In the deep learning domain, our hybrid framework effectively learns spatial and temporal features from longitudinal data, addressing the challenges associated with multi-dimensional and sequential time series data. In the medical domain, our study emphasizes the importance of analyzing baseline and longitudinal MR images for accurate diagnosis and understanding disease progression.

Addiction-related brain networks identification via Graph Diffusion Reconstruction Network

Functional magnetic resonance imaging (fMRI) provides insights into complex patterns of brain functional changes, making it a valuable tool for exploring addiction-related brain connectivity. However, effectively extracting addiction-related brain connectivity from fMRI data remains challenging due to the intricate and non-linear nature of brain connections. Therefore, this paper proposed the Graph Diffusion Reconstruction Network (GDRN), a novel framework designed to capture addiction-related brain connectivity from fMRI data acquired from addicted rats. The proposed GDRN incorporates a diffusion reconstruction module that effectively maintains the unity of data distribution by reconstructing the training samples, thereby enhancing the model's ability to reconstruct nicotine addiction-related brain networks. Experimental evaluations conducted on a nicotine addiction rat dataset demonstrate that the proposed GDRN effectively explores nicotine addiction-related brain connectivity. The findings suggest that the GDRN holds promise for uncovering and understanding the complex neural mechanisms underlying addiction using fMRI data.

Enhancing the Super-Resolution of Medical Images: Introducing the Deep Residual Feature Distillation Channel Attention Network for Optimized Performance and Efficiency

In the advancement of medical image super-resolution (SR), the Deep Residual Feature Distillation Channel Attention Network (DRFDCAN) marks a significant step forward. This work presents DRFDCAN, a model that innovates traditional SR approaches by introducing a channel attention block that is tailored for high-frequency features-crucial for the nuanced details in medical diagnostics-while streamlining the network structure for enhanced computational efficiency. DRFDCAN's architecture adopts a residual-within-residual design to facilitate faster inference and reduce memory demands without compromising the integrity of the image reconstruction. This design strategy, combined with an innovative feature extraction method that emphasizes the utility of the initial layer features, allows for improved image clarity and is particularly effective in optimizing the peak signal-to-noise ratio (PSNR). The proposed work redefines efficiency in SR models, outperforming established frameworks like RFDN by improving model compactness and accelerating inference. The meticulous crafting of a feature extractor that effectively captures edge and texture information exemplifies the model's capacity to render detailed images, necessary for accurate medical analysis. The implications of this study are two-fold: it presents a viable solution for deploying SR technology in real-time medical applications, and it sets a precedent for future models that address the delicate balance between computational efficiency and high-fidelity image reconstruction. This balance is paramount in medical applications where the clarity of images can significantly influence diagnostic outcomes. The DRFDCAN model thus stands as a transformative contribution to the field of medical image super-resolution.

Lesion-aware attention network for diabetic nephropathy diagnosis with optical coherence tomography images

Purpose

For early screening of diabetic nephropathy patients, we propose a deep learning algorithm to screen high-risk patients with diabetic nephropathy from retinal images of diabetic patients.

Methods

We propose the use of attentional mechanisms to improve the model's focus on lesion-prone regions of retinal OCT images. First, the data is trained using the base network and the Grad-CAM algorithm locates image regions that have a large impact on the model output and generates a rough mask localization map. The mask is used as a auxiliary region to realize the auxiliary attention module. We then inserted the region-guided attention module into the baseline model and trained the CNN model to guide the model to better focus on relevant lesion features. The proposed model improves the recognition of the lesion region.

Results

To evaluate the lesion-aware attention network, we trained and tested it using OCT volumetric data collected from 66 patients with diabetic retinal microangiopathy (89 eyes, male = 43, female = 23). There were 45 patients (60 eyes, male=27, female = 18) in DR group and 21 patients (29 eyes, male = 16, female = 5) in DN group. Our proposed model performs even better in disease classification, specifically, the accuracy of the proposed model was 91.68%, the sensitivity was 89.99%, and the specificity was 92.18%.

Conclusion

The proposed lesion-aware attention model can provide reliable screening of high-risk patients with diabetic nephropathy.

Brain Structure-Function Fusing Representation Learning Using Adversarial Decomposed-VAE for Analyzing MCI

Integrating the brain structural and functional connectivity features is of great significance in both exploring brain science and analyzing cognitive impairment clinically. However, it remains a challenge to effectively fuse structural and functional features in exploring the complex brain network. In this paper, a novel brain structure-function fusing-representation learning (BSFL) model is proposed to effectively learn fused representation from diffusion tensor imaging (DTI) and resting-state functional magnetic resonance imaging (fMRI) for mild cognitive impairment (MCI) analysis. Specifically, the decomposition-fusion framework is developed to first decompose the feature space into the union of the uniform and unique spaces for each modality, and then adaptively fuse the decomposed features to learn MCI-related representation. Moreover, a knowledge-aware transformer module is designed to automatically capture local and global connectivity features throughout the brain. Also, a uniform-unique contrastive loss is further devised to make the decomposition more effective and enhance the complementarity of structural and functional features. The extensive experiments demonstrate that the proposed model achieves better performance than other competitive methods in predicting and analyzing MCI. More importantly, the proposed model could be a potential tool for reconstructing unified brain networks and predicting abnormal connections during the degenerative processes in MCI.

Identifying novel disease categories through divergence optimization: An approach to prevent misdiagnosis in medical imaging

Given the significant changes in human lifestyle, the incidence of colon cancer has rapidly increased. The diagnostic process can often be complicated due to symptom similarities between colon cancer and other colon-related diseases. In an effort to minimize misdiagnosis, deep learning-based approaches for colon cancer diagnosis have notably progressed within the field of clinical medicine, offering more precise detection and improved patient outcomes. Despite these advancements, practical application of these techniques continues to encounter two major challenges: 1) due to the need for expert annotation, only a limited number of labels are utilized for diagnosis; and 2) the existence of diverse disease types can lead to misdiagnosis when the model encounters unfamiliar disease categories. To overcome these hurdles, we present a method incorporating Universal Domain Adaptation (UniDA). By optimizing the divergence of samples in the source domain, our method detects noise. Furthermore, to identify categories that are not present in the source domain, we optimize the divergence of unlabeled samples in the target domain. Experimental validation on two gastrointestinal datasets demonstrates that our method surpasses current state-of-the-art domain adaptation techniques in identifying unknown disease classes. It is worth noting that our proposed method is the first work of medical image diagnosis aimed at the identification of unknown categories of diseases.

Inflating 2D convolution weights for efficient generation of 3D medical images

BACKGROUND AND OBJECTIVE

The generation of three-dimensional (3D) medical images has great application potential since it takes into account the 3D anatomical structure. Two problems prevent effective training of a 3D medical generative model: (1) 3D medical images are expensive to acquire and annotate, resulting in an insufficient number of training images, and (2) a large number of parameters are involved in 3D convolution.

METHODS

We propose a novel GAN model called 3D Split&Shuffle-GAN. To address the 3D data scarcity issue, we first pre-train a two-dimensional (2D) GAN model using abundant image slices and inflate the 2D convolution weights to improve the initialization of the 3D GAN. Novel 3D network architectures are proposed for both the generator and discriminator of the GAN model to significantly reduce the number of parameters while maintaining the quality of image generation. Several weight inflation strategies and parameter-efficient 3D architectures are investigated.

RESULTS

Experiments on both heart (Stanford AIMI Coronary Calcium) and brain (Alzheimer's Disease Neuroimaging Initiative) datasets show that our method leads to improved 3D image generation quality (14.7 improvements on Frchet inception distance) with significantly fewer parameters (only 48.5% of the baseline method).

CONCLUSIONS

We built a parameter-efficient 3D medical image generation model. Due to the efficiency and effectiveness, it has the potential to generate high-quality 3D brain and heart images for real use cases.

A super-resolution network for medical imaging via transformation analysis of wavelet multi-resolution

In recent years, deep learning super-resolution models for progressive reconstruction have achieved great success. However, these models which refer to multi-resolution analysis basically ignore the information contained in the lower subspaces and do not explore the correlation between features in the wavelet and spatial domain, resulting in not fully utilizing the auxiliary information brought by multi-resolution analysis with multiple domains. Therefore, we propose a super-resolution network based on the wavelet multi-resolution framework (WMRSR) to capture the auxiliary information contained in multiple subspaces and to be aware of the interdependencies between spatial domain and wavelet domain features. Initially, the wavelet multi-resolution input (WMRI) is generated by combining wavelet sub-bands obtained from each subspace through wavelet multi-resolution analysis and the corresponding spatial domain image content, which serves as input to the network. Then, the WMRSR captures the corresponding features from the WMRI in the wavelet domain and spatial domain, respectively, and fuses them adaptively, thus learning fully explored features in multi-resolution and multi-domain. Finally, the high-resolution images are gradually reconstructed in the wavelet multi-resolution framework by our convolution-based wavelet transform module which is suitable for deep neural networks. Extensive experiments conducted on two public datasets demonstrate that our method outperforms other state-of-the-art methods in terms of objective and visual qualities.

Brain Functional Network Generation Using Distribution-Regularized Adversarial Graph Autoencoder with Transformer for Dementia Diagnosis

The topological connectivity information derived from the brain functional network can bring new insights for diagnosing and analyzing dementia disorders. The brain functional network is suitable to bridge the correlation between abnormal connectivities and dementia disorders. However, it is challenging to access considerable amounts of brain functional network data, which hinders the widespread application of data-driven models in dementia diagnosis. In this study, a novel distribution-regularized adversarial graph auto-Encoder (DAGAE) with transformer is proposed to generate new fake brain functional networks to augment the brain functional network dataset, improving the dementia diagnosis accuracy of data-driven models. Specifically, the label distribution is estimated to regularize the latent space learned by the graph encoder, which can make the learning process stable and the learned representation robust. Also, the transformer generator is devised to map the node representations into node-to-node connections by exploring the long-term dependence of highly-correlated distant brain regions. The typical topological properties and discriminative features can be preserved entirely. Furthermore, the generated brain functional networks improve the prediction performance using different classifiers, which can be applied to analyze other cognitive diseases. Attempts on the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset demonstrate that the proposed model can generate good brain functional networks. The classification results show adding generated data can achieve the best accuracy value of 85.33%, sensitivity value of 84.00%, specificity value of 86.67%. The proposed model also achieves superior performance compared with other related augmented models. Overall, the proposed model effectively improves cognitive disease diagnosis by generating diverse brain functional networks.

Machine learning for cognitive behavioral analysis: datasets, methods, paradigms, and research directions

Human behaviour reflects cognitive abilities. Human cognition is fundamentally linked to the different experiences or characteristics of consciousness/emotions, such as joy, grief, anger, etc., which assists in effective communication with others. Detection and differentiation between thoughts, feelings, and behaviours are paramount in learning to control our emotions and respond more effectively in stressful circumstances. The ability to perceive, analyse, process, interpret, remember, and retrieve information while making judgments to respond correctly is referred to as Cognitive Behavior. After making a significant mark in emotion analysis, deception detection is one of the key areas to connect human behaviour, mainly in the forensic domain. Detection of lies, deception, malicious intent, abnormal behaviour, emotions, stress, etc., have significant roles in advanced stages of behavioral science. Artificial Intelligence and Machine learning (AI/ML) has helped a great deal in pattern recognition, data extraction and analysis, and interpretations. The goal of using AI and ML in behavioral sciences is to infer human behaviour, mainly for mental health or forensic investigations. The presented work provides an extensive review of the research on cognitive behaviour analysis. A parametric study is presented based on different physical characteristics, emotional behaviours, data collection sensing mechanisms, unimodal and multimodal datasets, modelling AI/ML methods, challenges, and future research directions.

Generative AI for brain image computing and brain network computing: a review

Recent years have witnessed a significant advancement in brain imaging techniques that offer a non-invasive approach to mapping the structure and function of the brain. Concurrently, generative artificial intelligence (AI) has experienced substantial growth, involving using existing data to create new content with a similar underlying pattern to real-world data. The integration of these two domains, generative AI in neuroimaging, presents a promising avenue for exploring various fields of brain imaging and brain network computing, particularly in the areas of extracting spatiotemporal brain features and reconstructing the topological connectivity of brain networks. Therefore, this study reviewed the advanced models, tasks, challenges, and prospects of brain imaging and brain network computing techniques and intends to provide a comprehensive picture of current generative AI techniques in brain imaging. This review is focused on novel methodological approaches and applications of related new methods. It discussed fundamental theories and algorithms of four classic generative models and provided a systematic survey and categorization of tasks, including co-registration, super-resolution, enhancement, classification, segmentation, cross-modality, brain network analysis, and brain decoding. This paper also highlighted the challenges and future directions of the latest work with the expectation that future research can be beneficial.

Computational approaches for the reconstruction of optic nerve fibers along the visual pathway from medical images: a comprehensive review

Optic never fibers in the visual pathway play significant roles in vision formation. Damages of optic nerve fibers are biomarkers for the diagnosis of various ophthalmological and neurological diseases; also, there is a need to prevent the optic nerve fibers from getting damaged in neurosurgery and radiation therapy. Reconstruction of optic nerve fibers from medical images can facilitate all these clinical applications. Although many computational methods are developed for the reconstruction of optic nerve fibers, a comprehensive review of these methods is still lacking. This paper described both the two strategies for optic nerve fiber reconstruction applied in existing studies, i.e., image segmentation and fiber tracking. In comparison to image segmentation, fiber tracking can delineate more detailed structures of optic nerve fibers. For each strategy, both conventional and AI-based approaches were introduced, and the latter usually demonstrates better performance than the former. From the review, we concluded that AI-based methods are the trend for optic nerve fiber reconstruction and some new techniques like generative AI can help address the current challenges in optic nerve fiber reconstruction.

Improved HHT-microstate analysis of EEG in nicotine addicts

Background

Substance addiction is a chronic disease which causes great harm to modern society and individuals. At present, many studies have applied EEG analysis methods to the substance addiction detection and treatment. As a tool to describe the spatio-temporal dynamic characteristics of large-scale electrophysiological data, EEG microstate analysis has been widely used, which is an effective method to study the relationship between EEG electrodynamics and cognition or disease.

Methods

To study the difference of EEG microstate parameters of nicotine addicts at each frequency band, we combine an improved Hilbert Huang Transformation (HHT) decomposition with microstate analysis, which is applied to the EEG of nicotine addicts.

Results

After using improved HHT-Microstate method, we notice that there is significant difference in EEG microstates of nicotine addicts between viewing smoke pictures group (smoke) and viewing neutral pictures group (neutral). Firstly, there is a significant difference in EEG microstates at full-frequency band between smoke and neutral group. Compared with the FIR-Microstate method, the similarity index of microstate topographic maps at alpha and beta bands had significant differences between smoke and neutral group. Secondly, we find significant class × group interactions for microstate parameters at delta, alpha and beta bands. Finally, the microstate parameters at delta, alpha and beta bands obtained by the improved HHT-microstate analysis method are selected as features for classification and detection under the Gaussian kernel support vector machine. The highest accuracy is 92% sensitivity is 94% and specificity is 91%, which can more effectively detect and identify addiction diseases than FIR-Microstate and FIR-Riemann methods.

Conclusion

Thus, the improved HHT-Microstate analysis method can effectively identify substance addiction diseases and provide new ideas and insights for the brain research of nicotine addiction.

Deep Learning Aided Neuroimaging and Brain Regulation

Currently, deep learning aided medical imaging is becoming the hot spot of AI frontier application and the future development trend of precision neuroscience. This review aimed to render comprehensive and informative insights into the recent progress of deep learning and its applications in medical imaging for brain monitoring and regulation. The article starts by providing an overview of the current methods for brain imaging, highlighting their limitations and introducing the potential benefits of using deep learning techniques to overcome these limitations. Then, we further delve into the details of deep learning, explaining the basic concepts and providing examples of how it can be used in medical imaging. One of the key strengths is its thorough discussion of the different types of deep learning models that can be used in medical imaging including convolutional neural networks (CNNs), recurrent neural networks (RNNs), and generative adversarial network (GAN) assisted magnetic resonance imaging (MRI), positron emission tomography (PET)/computed tomography (CT), electroencephalography (EEG)/magnetoencephalography (MEG), optical imaging, and other imaging modalities. Overall, our review on deep learning aided medical imaging for brain monitoring and regulation provides a referrable glance for the intersection of deep learning aided neuroimaging and brain regulation.

Perceptually Motivated Generative Model for Magnetic Resonance Image Denoising

Image denoising is an important preprocessing step in low-level vision problems involving biomedical images. Noise removal techniques can greatly benefit raw corrupted magnetic resonance images (MRI). It has been discovered that the MR data is corrupted by a mixture of Gaussian-impulse noise caused by detector flaws and transmission errors. This paper proposes a deep generative model (GenMRIDenoiser) for dealing with this mixed noise scenario. This work makes four contributions. To begin, Wasserstein generative adversarial network (WGAN) is used in model training to mitigate the problem of vanishing gradient, mode collapse, and convergence issues encountered while training a vanilla GAN. Second, a perceptually motivated loss function is used to guide the training process in order to preserve the low-level details in the form of high-frequency components in the image. Third, batch renormalization is used between the convolutional and activation layers to prevent performance degradation under the assumption of non-independent and identically distributed (non-iid) data. Fourth, global feature attention module (GFAM) is appended at the beginning and end of the parallel ensemble blocks to capture the long-range dependencies that are often lost due to the small receptive field of convolutional filters. The experimental results over synthetic data and MRI stack obtained from real MR scanners indicate the potential utility of the proposed technique across a wide range of degradation scenarios.

An enhanced ant colony optimizer with Cauchy-Gaussian fusion and novel movement strategy for multi-threshold COVID-19 X-ray image segmentation

The novel coronavirus pneumonia (COVID-19) is a respiratory disease of great concern in terms of its dissemination and severity, for which X-ray imaging-based diagnosis is one of the effective complementary diagnostic methods. It is essential to be able to separate and identify lesions from their pathology images regardless of the computer-aided diagnosis techniques. Therefore, image segmentation in the pre-processing stage of COVID-19 pathology images would be more helpful for effective analysis. In this paper, to achieve highly effective pre-processing of COVID-19 pathological images by using multi-threshold image segmentation (MIS), an enhanced version of ant colony optimization for continuous domains (MGACO) is first proposed. In MGACO, not only a new move strategy is introduced, but also the Cauchy-Gaussian fusion strategy is incorporated. It has been accelerated in terms of convergence speed and has significantly enhanced its ability to jump out of the local optimum. Furthermore, an MIS method (MGACO-MIS) based on MGACO is developed, where it applies the non-local means, 2D histogram as the basis, and employs 2D Kapur’s entropy as the fitness function. To demonstrate the performance of MGACO, we qualitatively analyze it in detail and compare it with other peers on 30 benchmark functions from IEEE CEC2014, which proves that it has a stronger capability of solving problems over the original ant colony optimization for continuous domains. To verify the segmentation effect of MGACO-MIS, we conducted a comparison experiment with eight other similar segmentation methods based on real pathology images of COVID-19 at different threshold levels. The final evaluation and analysis results fully demonstrate that the developed MGACO-MIS is sufficient to obtain high-quality segmentation results in the COVID-19 image segmentation and has stronger adaptability to different threshold levels than other methods. Therefore, it has been well-proven that MGACO is an excellent swarm intelligence optimization algorithm, and MGACO-MIS is also an excellent segmentation method.

Prior knowledge-based precise diagnosis of blend sign from head computed tomography

Introduction

Automated diagnosis of intracranial hemorrhage on head computed tomography (CT) plays a decisive role in clinical management. This paper presents a prior knowledge-based precise diagnosis of blend sign network from head CT scans.

Method

We employ the object detection task as an auxiliary task in addition to the classification task, which could incorporate the hemorrhage location as prior knowledge into the detection framework. The auxiliary task could help the model pay more attention to the regions with hemorrhage, which is beneficial for distinguishing the blend sign. Furthermore, we propose a self-knowledge distillation strategy to deal with inaccuracy annotations.

Results

In the experiment, we retrospectively collected 1749 anonymous non-contrast head CT scans from the First Affiliated Hospital of China Medical University. The dataset contains three categories: no intracranial hemorrhage (non-ICH), normal intracranial hemorrhage (normal ICH), and blend sign. The experimental results demonstrate that our method performs better than other methods.

Discussion

Our method has the potential to assist less-experienced head CT interpreters, reduce radiologists' workload, and improve efficiency in natural clinical settings.

Data-Driven Phenotyping of Alzheimer's Disease under Epigenetic Conditions Using Partial Volume Correction of PET Studies and Manifold Learning

Alzheimer's disease (AD) is the most common form of dementia. An increasing number of studies have confirmed epigenetic changes in AD. Consequently, a robust phenotyping mechanism must take into consideration the environmental effects on the patient in the generation of phenotypes. Positron Emission Tomography (PET) is employed for the quantification of pathological amyloid deposition in brain tissues. The objective is to develop a new methodology for the hyperparametric analysis of changes in cognitive scores and PET features to test for there being multiple AD phenotypes. We used a computational method to identify phenotypes in a retrospective cohort study (532 subjects), using PET and Magnetic Resonance Imaging (MRI) images and neuropsychological assessments, to develop a novel computational phenotyping method that uses Partial Volume Correction (PVC) and subsets of neuropsychological assessments in a non-biased fashion. Our pipeline is based on a Regional Spread Function (RSF) method for PVC and a t-distributed Stochastic Neighbor Embedding (t-SNE) manifold. The results presented demonstrate that (1) the approach to data-driven phenotyping is valid, (2) the different techniques involved in the pipelines produce different results, and (3) they permit us to identify the best phenotyping pipeline. The method identifies three phenotypes and permits us to analyze them under epigenetic conditions.

Addictive brain-network identification by spatial attention recurrent network with feature selection

Addiction in the brain is associated with adaptive changes that reshape addiction-related brain regions and lead to functional abnormalities that cause a range of behavioral changes, and functional magnetic resonance imaging (fMRI) studies can reveal complex dynamic patterns of brain functional change. However, it is still a challenge to identify functional brain networks and discover region-level biomarkers between nicotine addiction (NA) and healthy control (HC) groups. To tackle it, we transform the fMRI of the rat brain into a network with biological attributes and propose a novel feature-selected framework to extract and select the features of addictive brain regions and identify these graph-level networks. In this framework, spatial attention recurrent network (SARN) is designed to capture the features with spatial and time-sequential information. And the Bayesian feature selection(BFS) strategy is adopted to optimize the model and improve classification tasks by restricting features. Our experiments on the addiction brain imaging dataset obtain superior identification performance and interpretable biomarkers associated with addiction-relevant brain regions.

Constructing brain functional network by Adversarial Temporal-Spatial Aligned Transformer for early AD analysis

Introduction

The brain functional network can describe the spontaneous activity of nerve cells and reveal the subtle abnormal changes associated with brain disease. It has been widely used for analyzing early Alzheimer's disease (AD) and exploring pathological mechanisms. However, the current methods of constructing functional connectivity networks from functional magnetic resonance imaging (fMRI) heavily depend on the software toolboxes, which may lead to errors in connection strength estimation and bad performance in disease analysis because of many subjective settings.

Methods

To solve this problem, in this paper, a novel Adversarial Temporal-Spatial Aligned Transformer (ATAT) model is proposed to automatically map 4D fMRI into functional connectivity network for early AD analysis. By incorporating the volume and location of anatomical brain regions, the region-guided feature learning network can roughly focus on local features for each brain region. Also, the spatial-temporal aligned transformer network is developed to adaptively adjust boundary features of adjacent regions and capture global functional connectivity patterns of distant regions. Furthermore, a multi-channel temporal discriminator is devised to distinguish the joint distributions of the multi-region time series from the generator and the real sample.

Results

Experimental results on the Alzheimer's Disease Neuroimaging Initiative (ADNI) proved the effectiveness and superior performance of the proposed model in early AD prediction and progression analysis.

Discussion

To verify the reliability of the proposed model, the detected important ROIs are compared with clinical studies and show partial consistency. Furthermore, the most significant altered connectivity reflects the main characteristics associated with AD.

Conclusion

Generally, the proposed ATAT provides a new perspective in constructing functional connectivity networks and is able to evaluate the disease-related changing characteristics at different stages for neuroscience exploration and clinical disease analysis.

A Systematic Literature Review on Applications of GAN-Synthesized Images for Brain MRI

With the advances in brain imaging, magnetic resonance imaging (MRI) is evolving as a popular radiological tool in clinical diagnosis. Deep learning (DL) methods can detect abnormalities in brain images without an extensive manual feature extraction process. Generative adversarial network (GAN)-synthesized images have many applications in this field besides augmentation, such as image translation, registration, super-resolution, denoising, motion correction, segmentation, reconstruction, and contrast enhancement. The existing literature was reviewed systematically to understand the role of GAN-synthesized dummy images in brain disease diagnosis. Web of Science and Scopus databases were extensively searched to find relevant studies from the last 6 years to write this systematic literature review (SLR). Predefined inclusion and exclusion criteria helped in filtering the search results. Data extraction is based on related research questions (RQ). This SLR identifies various loss functions used in the above applications and software to process brain MRIs. A comparative study of existing evaluation metrics for GAN-synthesized images helps choose the proper metric for an application. GAN-synthesized images will have a crucial role in the clinical sector in the coming years, and this paper gives a baseline for other researchers in the field.

Multi-threshold image segmentation for melanoma based on Kapur’s entropy using enhanced ant colony optimization

Melanoma is a malignant tumor formed by the cancerous transformation of melanocytes, and its medical images contain much information. However, the percentage of the critical information in the image is small, and the noise is non-uniformly distributed. We propose a new multi-threshold image segmentation model based on the two-dimensional histogram approach to the above problem. We present an enhanced ant colony optimization for continuous domains (EACOR) in the proposed model based on the soft besiege and chase strategies. Further, EACOR is combined with two-dimensional Kapur’s entropy to search for the optimal thresholds. An experiment on the IEEE CEC2014 benchmark function was conducted to measure the reliable global search capability of the EACOR algorithm in the proposed model. Moreover, we have also conducted several sets of experiments to test the validity of the image segmentation model proposed in this paper. The experimental results show that the segmented images from the proposed model outperform the comparison method in several evaluation metrics. Ultimately, the model proposed in this paper can provide high-quality samples for subsequent analysis of melanoma pathology images.

Occupational Injury Risk Mitigation: Machine Learning Approach and Feature Optimization for Smart Workplace Surveillance

Forecasting the severity of occupational injuries shall be all industries' top priority. The use of machine learning is theoretically valuable to assist the predictive analysis, thus, this study attempts to propose a feature-optimized predictive model for anticipating occupational injury severity. A public database of 66,405 occupational injury records from OSHA is analyzed using five sets of machine learning models: Support Vector Machine, K-Nearest Neighbors, Naïve Bayes, Decision Tree, and Random Forest. For model comparison, Random Forest outperformed other models with higher accuracy and F1-score. Therefore, it highlighted the potential of ensemble learning as a more accurate prediction model in the field of occupational injury. In constructing the model, this study also proposed the feature optimization technique that revealed the three most important features; 'nature of injury', 'type of event', and 'affected body part' in developing model. The accuracy of the Random Forest model was improved by 0.5% or 0.895 and 0.954 for the prediction of hospitalization and amputation, respectively by redeveloping and optimizing the model with hyperparameter tuning. The feature optimization is essential in providing insight knowledge to the Safety and Health Practitioners for future injury corrective and preventive strategies. This study has shown promising potential for smart workplace surveillance.

Time-sequential graph adversarial learning for brain modularity community detection

Brain community detection is an efficient method to represent the communities of brain networks. However, time-variable functions of the brain and the intricate brain community structure impose a great challenge on it. In this paper, a time-sequential graph adversarial learning (TGAL) framework is proposed to detect brain communities and characterize the structure of communities from brain networks. In the framework, a novel time-sequential graph neural network is designed as an encoder to extract efficient graph representations by spatio-temporal attention mechanism. Since it is difficult to capture the community structure, the measurable modularity loss is used to optimize by maximizing the modularity of the community. In addition, the framework employs an adversarial scheme to guide the learning of representation. The effectiveness of our model is shown through experiments on the real-world brain network datasets, and the great performance of brain community detection demonstrates the advantage of the proposed framework.