Simultaneous single- and multi-contrast super-resolution for brain MRI images based on a convolutional neural network

Dataset augmentation with multiple contrasts images in super-resolution processing of T1-weighted brain magnetic resonance images

This study investigated the effectiveness of augmenting datasets for super-resolution processing of brain Magnetic Resonance Images (MRI) T1-weighted images (T1WIs) using deep learning. By incorporating images with different contrasts from the same subject, this study sought to improve network performance and assess its impact on image quality metrics, such as peak signal-to-noise ratio (PSNR) and structural similarity (SSIM). This retrospective study included 240 patients who underwent brain MRI. Two types of datasets were created: the Pure-Dataset group comprising T1WIs and the Mixed-Dataset group comprising T1WIs, T2-weighted images, and fluid-attenuated inversion recovery images. A U-Net-based network and an Enhanced Deep Super-Resolution network (EDSR) were trained on these datasets. Objective image quality analysis was performed using PSNR and SSIM. Statistical analyses, including paired t test and Pearson's correlation coefficient, were conducted to evaluate the results. Augmenting datasets with images of different contrasts significantly improved training accuracy as the dataset size increased. PSNR values ranged 29.84-30.26 dB for U-Net trained on mixed datasets, and SSIM values ranged 0.9858-0.9868. Similarly, PSNR values ranged 32.34-32.64 dB for EDSR trained on mixed datasets, and SSIM values ranged 0.9941-0.9945. Significant differences in PSNR and SSIM were observed between models trained on pure and mixed datasets. Pearson's correlation coefficient indicated a strong positive correlation between dataset size and image quality metrics. Using diverse image data obtained from the same subject can improve the performance of deep-learning models in medical image super-resolution tasks.

ACGRHA-Net: Accelerated multi-contrast MR imaging with adjacency complementary graph assisted residual hybrid attention network

Multi-contrast magnetic resonance (MR) imaging is an advanced technology used in medical diagnosis, but the long acquisition process can lead to patient discomfort and limit its broader application. Shortening acquisition time by undersampling k-space data introduces noticeable aliasing artifacts. To address this, we propose a method that reconstructs multi-contrast MR images from zero-filled data by utilizing a fully-sampled auxiliary contrast MR image as a prior to learn an adjacency complementary graph. This graph is then combined with a residual hybrid attention network, forming the adjacency complementary graph assisted residual hybrid attention network (ACGRHA-Net) for multi-contrast MR image reconstruction. Specifically, the optimal structural similarity is represented by a graph learned from the fully sampled auxiliary image, where the node features and adjacency matrices are designed to precisely capture structural information among different contrast images. This structural similarity enables effective fusion with the target image, improving the detail reconstruction. Additionally, a residual hybrid attention module is designed in parallel with the graph convolution network, allowing it to effectively capture key features and adaptively emphasize these important features in target contrast MR images. This strategy prioritizes crucial information while preserving shallow features, thereby achieving comprehensive feature fusion at deeper levels to enhance multi-contrast MR image reconstruction. Extensive experiments on the different datasets, using various sampling patterns and accelerated factors demonstrate that the proposed method outperforms the current state-of-the-art reconstruction methods.

Effective deep-learning brain MRI super resolution using simulated training data

BACKGROUND

In the field of medical imaging, high-resolution (HR) magnetic resonance imaging (MRI) is essential for accurate disease diagnosis and analysis. However, HR imaging is prone to artifacts and is not universally available. Consequently, low-resolution (LR) MRI images are typically acquired. Deep learning (DL)-based super-resolution (SR) techniques can transform LR images into HR quality. However, these techniques require paired HR-LR data for training the SR networks.

OBJECTIVE

This research aims to investigate the potential of simulated brain MRI data to train DL-based SR networks.

METHODS

We simulated a large set of anatomically diverse, voxel-aligned, and artifact-free brain MRI data at different resolutions. We utilized this simulated data to train four distinct DL-based SR networks and augment their training. The trained networks were then evaluated using real data from various sources.

RESULTS

With our trained networks, we produced 0.7mm SR images from standard 1mm resolution multi-source T1w brain MRI. Our experimental results demonstrate that the trained networks significantly enhance the sharpness of LR input MR images. For single-source images, the performance of networks trained solely on simulated data is slightly inferior to those trained solely on real data, with an average structural similarity index (SSIM) difference of 0.025. However, networks augmented with simulated data outperform those trained on single-source real data when evaluated across datasets from multiple sources.

CONCLUSION

Paired HR-LR simulated brain MRI data is suitable for training and augmenting diverse brain MRI SR networks. Augmenting the training data with simulated data can enhance the generalizability of the SR networks across real datasets from multiple sources.

Accelerated Musculoskeletal Magnetic Resonance Imaging

With a substantial growth in the use of musculoskeletal MRI, there has been a growing need to improve MRI workflow, and faster imaging has been suggested as one of the solutions for a more efficient examination process. Consequently, there have been considerable advances in accelerated MRI scanning methods. This article aims to review the basic principles and applications of accelerated musculoskeletal MRI techniques including widely used conventional acceleration methods, more advanced deep learning-based techniques, and new approaches to reduce scan time. Specifically, conventional accelerated MRI techniques, including parallel imaging, compressed sensing, and simultaneous multislice imaging, and deep learning-based accelerated MRI techniques, including undersampled MR image reconstruction, super-resolution imaging, artifact correction, and generation of unacquired contrast images, are discussed. Finally, new approaches to reduce scan time, including synthetic MRI, novel sequences, and new coil setups and designs, are also reviewed. We believe that a deep understanding of these fast MRI techniques and proper use of combined acceleration methods will synergistically improve scan time and MRI workflow in daily practice. EVIDENCE LEVEL: 3 TECHNICAL EFFICACY: Stage 1.

Multicontrast MRI Super-Resolution via Transformer-Empowered Multiscale Contextual Matching and Aggregation

Magnetic resonance imaging (MRI) possesses the unique versatility to acquire images under a diverse array of distinct tissue contrasts, which makes multicontrast super-resolution (SR) techniques possible and needful. Compared with single-contrast MRI SR, multicontrast SR is expected to produce higher quality images by exploiting a variety of complementary information embedded in different imaging contrasts. However, existing approaches still have two shortcomings: 1) most of them are convolution-based methods and, hence, weak in capturing long-range dependencies, which are essential for MR images with complicated anatomical patterns and 2) they ignore to make full use of the multicontrast features at different scales and lack effective modules to match and aggregate these features for faithful SR. To address these issues, we develop a novel multicontrast MRI SR network via transformer-empowered multiscale feature matching and aggregation, dubbed McMRSR ++ . First, we tame transformers to model long-range dependencies in both reference and target images at different scales. Then, a novel multiscale feature matching and aggregation method is proposed to transfer corresponding contexts from reference features at different scales to the target features and interactively aggregate them. Furthermore, a texture-preserving branch and a contrastive constraint are incorporated into our framework for enhancing the textural details in the SR images. Experimental results on both public and clinical in vivo datasets show that McMRSR ++ outperforms state-of-the-art methods under peak signal to noise ratio (PSNR), structure similarity index measure (SSIM), and root mean square error (RMSE) metrics significantly. Visual results demonstrate the superiority of our method in restoring structures, demonstrating its great potential to improve scan efficiency in clinical practice.

Self-supervised multicontrast super-resolution for diffusion-weighted prostate MRI

PURPOSE

This study addresses the challenge of low resolution and signal-to-noise ratio (SNR) in diffusion-weighted images (DWI), which are pivotal for cancer detection. Traditional methods increase SNR at high b-values through multiple acquisitions, but this results in diminished image resolution due to motion-induced variations. Our research aims to enhance spatial resolution by exploiting the global structure within multicontrast DWI scans and millimetric motion between acquisitions.

METHODS

We introduce a novel approach employing a "Perturbation Network" to learn subvoxel-size motions between scans, trained jointly with an implicit neural representation (INR) network. INR encodes the DWI as a continuous volumetric function, treating voxel intensities of low-resolution acquisitions as discrete samples. By evaluating this function with a finer grid, our model predicts higher-resolution signal intensities for intermediate voxel locations. The Perturbation Network's motion-correction efficacy was validated through experiments on biological phantoms and in vivo prostate scans.

RESULTS

Quantitative analyses revealed significantly higher structural similarity measures of super-resolution images to ground truth high-resolution images compared to high-order interpolation (p< $$ < $$ 0.005). In blind qualitative experiments,96 . 1 % $$ 96.1\% $$ of super-resolution images were assessed to have superior diagnostic quality compared to interpolated images.

CONCLUSION

High-resolution details in DWI can be obtained without the need for high-resolution training data. One notable advantage of the proposed method is that it does not require a super-resolution training set. This is important in clinical practice because the proposed method can easily be adapted to images with different scanner settings or body parts, whereas the supervised methods do not offer such an option.

MRI super-resolution using similarity distance and multi-scale receptive field based feature fusion GAN and pre-trained slice interpolation network

Challenges arise in achieving high-resolution Magnetic Resonance Imaging (MRI) to improve disease diagnosis accuracy due to limitations in hardware, patient discomfort, long acquisition times, and high costs. While Convolutional Neural Networks (CNNs) have shown promising results in MRI super-resolution, they often don't look into the structural similarity and prior information available in consecutive MRI slices. By leveraging information from sequential slices, more robust features can be obtained, potentially leading to higher-quality MRI slices. We propose a multi-slice two-dimensional (2D) MRI super-resolution network that combines a Generative Adversarial Network (GAN) with feature fusion and a pre-trained slice interpolation network to achieve three-dimensional (3D) super-resolution. The proposed model requires consecutively acquired three low-resolution (LR) MRI slices along a specific axis, and achieves the reconstruction of the MRI slices in the remaining two axes. The network effectively enhances both in-plane and out-of-plane resolution along the sagittal axis while addressing computational and memory constraints in 3D super-resolution. The proposed generator has a in-plane and out-of-plane Attention (IOA) network that fuses both in-plane and out-plane features of MRI dynamically. In terms of out-of-plane attention, the network merges features by considering the similarity distance between features and for in-plane attention, the network employs a two-level pyramid structure with varying receptive fields to extract features at different scales, ensuring the inclusion of both global and local features. Subsequently, to achieve 3D MRI super-resolution, a pre-trained slice interpolation network is used that takes two consecutive super-resolved MRI slices to generate a new intermediate slice. To further enhance the network performance and perceptual quality, we introduce a feature up-sampling layer and a feature extraction block with Scaled Exponential Linear Unit (SeLU). Moreover, our super-resolution network incorporates VGG loss from a fine-tuned VGG-19 network to provide additional enhancement. Through experimental evaluations on the IXI dataset and BRATS dataset, using the peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM) and the number of training parameters, we demonstrate the superior performance of our method compared to the existing techniques. Also, the proposed model can be adapted or modified to achieve super-resolution for both 2D and 3D MRI data.

Generalizing the Enhanced-Deep-Super-Resolution Neural Network to Brain MR Images: A Retrospective Study on the Cam-CAN Dataset

The Enhanced-Deep-Super-Resolution (EDSR) model is a state-of-the-art convolutional neural network suitable for improving image spatial resolution. It was previously trained with general-purpose pictures and then, in this work, tested on biomedical magnetic resonance (MR) images, comparing the network outcomes with traditional up-sampling techniques. We explored possible changes in the model response when different MR sequences were analyzed. T1w and T2w MR brain images of 70 human healthy subjects (F:M, 40:30) from the Cambridge Centre for Ageing and Neuroscience (Cam-CAN) repository were down-sampled and then up-sampled using EDSR model and BiCubic (BC) interpolation. Several reference metrics were used to quantitatively assess the performance of up-sampling operations (RMSE, pSNR, SSIM, and HFEN). Two-dimensional and three-dimensional reconstructions were evaluated. Different brain tissues were analyzed individually. The EDSR model was superior to BC interpolation on the selected metrics, both for two- and three- dimensional reconstructions. The reference metrics showed higher quality of EDSR over BC reconstructions for all the analyzed images, with a significant difference of all the criteria in T1w images and of the perception-based SSIM and HFEN in T2w images. The analysis per tissue highlights differences in EDSR performance related to the gray-level values, showing a relative lack of outperformance in reconstructing hyperintense areas. The EDSR model, trained on general-purpose images, better reconstructs MR T1w and T2w images than BC, without any retraining or fine-tuning. These results highlight the excellent generalization ability of the network and lead to possible applications on other MR measurements.

Improving radiomic modeling for the identification of symptomatic carotid atherosclerotic plaques using deep learning-based 3D super-resolution CT angiography

Rationale and objectives

Radiomic models based on normal-resolution (NR) computed tomography angiography (CTA) images can fail to distinguish between symptomatic and asymptomatic carotid atherosclerotic plaques. This study aimed to explore the effectiveness of a deep learning-based three-dimensional super-resolution (SR) CTA radiomic model for improved identification of symptomatic carotid atherosclerotic plaques.

Materials and methods

A total of 193 patients with carotid atherosclerotic plaques were retrospectively enrolled and allocated into either a symptomatic (n = 123) or an asymptomatic (n = 70) groups. SR CTA images were derived from NR CTA images using deep learning-based three-dimensional SR technology. Handcrafted radiomic features were extracted from both the SR and NR CTA images and three risk models were developed based on manually measured quantitative CTA characteristics and NR and SR radiomic features. Model performances were assessed via receiver operating characteristic, calibration, and decision curve analyses.

Results

The SR model exhibited the optimal performance (area under the curve [AUC] 0.820, accuracy 0.802, sensitivity 0.854, F1 score 0.847) in the testing cohort, outperforming the other two models. The calibration curve analyses and Hosmer-Lemeshow test demonstrated that the SR model exhibited the best goodness of fit, and decision curve analysis revealed that SR model had the highest clinical value and potential patient benefits.

Conclusions

Deep learning-based three-dimensional SR technology could improve the CTA-based radiomic models in identifying symptomatic carotid plaques, potentially providing more accurate and valuable information to guide clinical decision-making to reduce the risk of ischemic stroke.

Contrast-enhanced to non-contrast-enhanced image translation to exploit a clinical data warehouse of T1-weighted brain MRI

BACKGROUND

Clinical data warehouses provide access to massive amounts of medical images, but these images are often heterogeneous. They can for instance include images acquired both with or without the injection of a gadolinium-based contrast agent. Harmonizing such data sets is thus fundamental to guarantee unbiased results, for example when performing differential diagnosis. Furthermore, classical neuroimaging software tools for feature extraction are typically applied only to images without gadolinium. The objective of this work is to evaluate how image translation can be useful to exploit a highly heterogeneous data set containing both contrast-enhanced and non-contrast-enhanced images from a clinical data warehouse.

METHODS

We propose and compare different 3D U-Net and conditional GAN models to convert contrast-enhanced T1-weighted (T1ce) into non-contrast-enhanced (T1nce) brain MRI. These models were trained using 230 image pairs and tested on 77 image pairs from the clinical data warehouse of the Greater Paris area.

RESULTS

Validation using standard image similarity measures demonstrated that the similarity between real and synthetic T1nce images was higher than between real T1nce and T1ce images for all the models compared. The best performing models were further validated on a segmentation task. We showed that tissue volumes extracted from synthetic T1nce images were closer to those of real T1nce images than volumes extracted from T1ce images.

CONCLUSION

We showed that deep learning models initially developed with research quality data could synthesize T1nce from T1ce images of clinical quality and that reliable features could be extracted from the synthetic images, thus demonstrating the ability of such methods to help exploit a data set coming from a clinical data warehouse.

Double Transformer Super-Resolution for Breast Cancer ADC Images

Diffusion-weighted imaging (DWI) has been extensively explored in guiding the clinic management of patients with breast cancer. However, due to the limited resolution, accurately characterizing tumors using DWI and the corresponding apparent diffusion coefficient (ADC) is still a challenging problem. In this paper, we aim to address the issue of super-resolution (SR) of ADC images and evaluate the clinical utility of SR-ADC images through radiomics analysis. To this end, we propose a novel double transformer-based network (DTformer) to enhance the resolution of ADC images. More specifically, we propose a symmetric U-shaped encoder-decoder network with two different types of transformer blocks, named as UTNet, to extract deep features for super-resolution. The basic backbone of UTNet is composed of a locally-enhanced Swin transformer block (LeSwin-T) and a convolutional transformer block (Conv-T), which are responsible for capturing long-range dependencies and local spatial information, respectively. Additionally, we introduce a residual upsampling network (RUpNet) to expand image resolution by leveraging initial residual information from the original low-resolution (LR) images. Extensive experiments show that DTformer achieves superior SR performance. Moreover, radiomics analysis reveals that improving the resolution of ADC images is beneficial for tumor characteristic prediction, such as histological grade and human epidermal growth factor receptor 2 (HER2) status.

Dual contrast attention-guided multi-frequency fusion for multi-contrast MRI super-resolution

Objective. Multi-contrast magnetic resonance (MR) imaging super-resolution (SR) reconstruction is an effective solution for acquiring high-resolution MR images. It utilizes anatomical information from auxiliary contrast images to improve the quality of the target contrast images. However, existing studies have simply explored the relationships between auxiliary contrast and target contrast images but did not fully consider different anatomical information contained in multi-contrast images, resulting in texture details and artifacts unrelated to the target contrast images.Approach. To address these issues, we propose a dual contrast attention-guided multi-frequency fusion (DCAMF) network to reconstruct SR MR images from low-resolution MR images, which adaptively captures relevant anatomical information and processes the texture details and low-frequency information from multi-contrast images in parallel. Specifically, after the feature extraction, a feature selection module based on a dual contrast attention mechanism is proposed to focus on the texture details of the auxiliary contrast images and the low-frequency features of the target contrast images. Then, based on the characteristics of the selected features, a high- and low-frequency fusion decoder is constructed to fuse these features. In addition, a texture-enhancing module is embedded in the high-frequency fusion decoder, to highlight and refine the texture details of the auxiliary contrast and target contrast images. Finally, the high- and low-frequency fusion process is constrained by integrating a deeply-supervised mechanism into the DCAMF network.Main results. The experimental results show that the DCAMF outperforms other state-of-the-art methods. The peak signal-to-noise ratio and structural similarity of DCAMF are 39.02 dB and 0.9771 on the IXI dataset and 37.59 dB and 0.9770 on the BraTS2018 dataset, respectively. The image recovery is further validated in segmentation tasks.Significance. Our proposed SR model can enhance the quality of MR images. The results of the SR study provide a reliable basis for clinical diagnosis and subsequent image-guided treatment.

MGDUN: An interpretable network for multi-contrast MRI image super-resolution reconstruction

Magnetic resonance imaging (MRI) Super-Resolution (SR) aims to obtain high resolution (HR) images with more detailed information for precise diagnosis and quantitative image analysis. Deep unfolding networks outperform general MRI SR reconstruction methods by providing better performance and improved interpretability, which enhance the trustworthiness required in clinical practice. Additionally, current SR reconstruction techniques often rely on a single contrast or a simple multi-contrast fusion mechanism, ignoring the complex relationships between different contrasts. To address these issues, in this paper, we propose a Model-Guided multi-contrast interpretable Deep Unfolding Network (MGDUN) for medical image SR reconstruction, which explicitly incorporates the well-studied multi-contrast MRI observation model into an unfolding iterative network. Specifically, we manually design an objective function for MGDUN that can be iteratively computed by the half-quadratic splitting algorithm. The iterative MGDUN algorithm is unfolded into a special model-guided deep unfolding network that explicitly takes into account both the multi-contrast relationship matrix and the MRI observation matrix during the end-to-end optimization process. Extensive experimental results on the multi-contrast IXI dataset and the BraTs 2019 dataset demonstrate the superiority of our proposed model, with PSNR reaching 37.3366 and 35.9690 respectively. Our proposed MGDUN provides a promising solution for multi-contrast MR image super-resolution reconstruction. Code is available at https://github.com/yggame/MGDUN.

High resolution 0.5mm isotropic T1-weighted and diffusion tensor templates of the brain of non-demented older adults in a common space for the MIITRA atlas

High quality, high resolution T1-weighted (T1w) and diffusion tensor imaging (DTI) brain templates located in a common space can enhance the sensitivity and precision of template-based neuroimaging studies. However, such multimodal templates have not been constructed for the older adult brain. The purpose of this work which is part of the MIITRA atlas project was twofold: (A) to develop 0.5 mm isotropic resolution T1w and DTI templates that are representative of the brain of non-demented older adults and are located in the same space, using advanced multimodal template construction techniques and principles of super resolution on data from a large, diverse, community cohort of 400 non-demented older adults, and (B) to systematically compare the new templates to other standardized templates. It was demonstrated that the new MIITRA-0.5mm T1w and DTI templates are well-matched in space, exhibit good definition of brain structures, including fine structures, exhibit higher image sharpness than other standardized templates, and are free of artifacts. The MIITRA-0.5mm T1w and DTI templates allowed higher intra-modality inter-subject spatial normalization precision as well as higher inter-modality intra-subject spatial matching of older adult T1w and DTI data compared to other available templates. Consequently, MIITRA-0.5mm templates allowed detection of smaller inter-group differences for older adult data compared to other templates. The MIITRA-0.5mm templates were also shown to be most representative of the brain of non-demented older adults compared to other templates with submillimeter resolution. The new templates constructed in this work constitute two of the final products of the MIITRA atlas project and are anticipated to have important implications for the sensitivity and precision of studies on older adults.

Improving Structural MRI Preprocessing with Hybrid Transformer GANs

Magnetic resonance imaging (MRI) is a technique that is widely used in practice to evaluate any pathologies in the human body. One of the areas of interest is the human brain. Naturally, MR images are low-resolution and contain noise due to signal interference, the patient's body's radio-frequency emissions and smaller Tesla coil counts in the machinery. There is a need to solve this problem, as MR tomographs that have the capability of capturing high-resolution images are extremely expensive and the length of the procedure to capture such images increases by the order of magnitude. Vision transformers have lately shown state-of-the-art results in super-resolution tasks; therefore, we decided to evaluate whether we can employ them for structural MRI super-resolution tasks. A literature review showed that similar methods do not focus on perceptual image quality because upscaled images are often blurry and are subjectively of poor quality. Knowing this, we propose a methodology called HR-MRI-GAN, which is a hybrid transformer generative adversarial network capable of increasing resolution and removing noise from 2D T1w MRI slice images. Experiments show that our method quantitatively outperforms other SOTA methods in terms of perceptual image quality and is capable of subjectively generalizing to unseen data. During the experiments, we additionally identified that the visual saliency-induced index metric is not applicable to MRI perceptual quality assessment and that general-purpose denoising networks are effective when removing noise from MR images.

High-Resolution 3D MRI With Deep Generative Networks via Novel Slice-Profile Transformation Super-Resolution

High-resolution magnetic resonance imaging (MRI) sequences, such as 3D turbo or fast spin-echo (TSE/FSE) imaging, are clinically desirable but suffer from long scanning time-related blurring when reformatted into preferred orientations. Instead, multi-slice two-dimensional (2D) TSE imaging is commonly used because of its high in-plane resolution but is limited clinically by poor through-plane resolution due to elongated voxels and the inability to generate multi-planar reformations due to staircase artifacts. Therefore, multiple 2D TSE scans are acquired in various orthogonal imaging planes, increasing the overall MRI scan time. In this study, we propose a novel slice-profile transformation super-resolution (SPTSR) framework with deep generative learning for through-plane super-resolution (SR) of multi-slice 2D TSE imaging. The deep generative networks were trained by synthesized low-resolution training input via slice-profile downsampling (SP-DS), and the trained networks inferred on the slice profile convolved (SP-conv) testing input for 5.5x through-plane SR. The network output was further slice-profile deconvolved (SP-deconv) to achieve an isotropic super-resolution. Compared to SMORE SR method and the networks trained by conventional downsampling, our SPTSR framework demonstrated the best overall image quality from 50 testing cases, evaluated by two abdominal radiologists. The quantitative analysis cross-validated the expert reader study results. 3D simulation experiments confirmed the quantitative improvement of the proposed SPTSR and the effectiveness of the SP-deconv step, compared to 3D ground-truths. Ablation studies were conducted on the individual contributions of SP-DS and SP-conv, networks structure, training dataset size, and different slice profiles.

Exploring the Possibility of Measuring Vertebrae Bone Structure Metrics Using MDCT Images: An Unpaired Image-to-Image Translation Method

Bone structure metrics are vital for the evaluation of vertebral bone strength. However, the gold standard for measuring bone structure metrics, micro-Computed Tomography (micro-CT), cannot be used in vivo, which hinders the early diagnosis of fragility fractures. This paper used an unpaired image-to-image translation method to capture the mapping between clinical multidetector computed tomography (MDCT) and micro-CT images and then generated micro-CT-like images to measure bone structure metrics. MDCT and micro-CT images were scanned from 75 human lumbar spine specimens and formed training and testing sets. The generator in the model focused on learning both the structure and detailed pattern of bone trabeculae and generating micro-CT-like images, and the discriminator determined whether the generated images were micro-CT images or not. Based on similarity metrics (i.e., SSIM and FID) and bone structure metrics (i.e., bone volume fraction, trabecular separation and trabecular thickness), a set of comparisons were performed. The results show that the proposed method can perform better in terms of both similarity metrics and bone structure metrics and the improvement is statistically significant. In particular, we compared the proposed method with the paired image-to-image method and analyzed the pros and cons of the method used.

Mouse brain MR super-resolution using a deep learning network trained with optical imaging data

Introduction

The resolution of magnetic resonance imaging is often limited at the millimeter level due to its inherent signal-to-noise disadvantage compared to other imaging modalities. Super-resolution (SR) of MRI data aims to enhance its resolution and diagnostic value. While deep learning-based SR has shown potential, its applications in MRI remain limited, especially for preclinical MRI, where large high-resolution MRI datasets for training are often lacking.

Methods

In this study, we first used high-resolution mouse brain auto-fluorescence (AF) data acquired using serial two-photon tomography (STPT) to examine the performance of deep learning-based SR for mouse brain images.

Results

We found that the best SR performance was obtained when the resolutions of training and target data were matched. We then applied the network trained using AF data to MRI data of the mouse brain, and found that the performance of the SR network depended on the tissue contrast presented in the MRI data. Using transfer learning and a limited set of high-resolution mouse brain MRI data, we were able to fine-tune the initial network trained using AF to enhance the resolution of MRI data.

Discussion

Our results suggest that deep learning SR networks trained using high-resolution data of a different modality can be applied to MRI data after transfer learning.

Multi-task deep learning for medical image computing and analysis: A review

The renaissance of deep learning has provided promising solutions to various tasks. While conventional deep learning models are constructed for a single specific task, multi-task deep learning (MTDL) that is capable to simultaneously accomplish at least two tasks has attracted research attention. MTDL is a joint learning paradigm that harnesses the inherent correlation of multiple related tasks to achieve reciprocal benefits in improving performance, enhancing generalizability, and reducing the overall computational cost. This review focuses on the advanced applications of MTDL for medical image computing and analysis. We first summarize four popular MTDL network architectures (i.e., cascaded, parallel, interacted, and hybrid). Then, we review the representative MTDL-based networks for eight application areas, including the brain, eye, chest, cardiac, abdomen, musculoskeletal, pathology, and other human body regions. While MTDL-based medical image processing has been flourishing and demonstrating outstanding performance in many tasks, in the meanwhile, there are performance gaps in some tasks, and accordingly we perceive the open challenges and the perspective trends. For instance, in the 2018 Ischemic Stroke Lesion Segmentation challenge, the reported top dice score of 0.51 and top recall of 0.55 achieved by the cascaded MTDL model indicate further research efforts in high demand to escalate the performance of current models.

Simultaneous superresolution reconstruction and distortion correction for single-shot EPI DWI using deep learning

PURPOSE

Single-shot (SS) EPI is widely used for clinical DWI. This study aims to develop an end-to-end deep learning-based method with a novel loss function in an improved network structure to simultaneously increase the resolution and correct distortions for SS-EPI DWI.

THEORY AND METHODS

Point-spread-function (PSF)-encoded EPI can provide high-resolution, distortion-free DWI images. A distorted image from SS-EPI can be described as the convolution between a PSF function with a distortion-free image. The deconvolution process to recover the distortion-free image can be achieved with a convolution neural network, which also learns the mapping function between low-resolution SS-EPI and high-resolution reference PSF-EPI to achieve superresolution. To suppress the oversmoothing effect, we proposed a modified generative adversarial network structure, in which a dense net with gradient map guidance and a multilevel fusion block was used as the generator. A fractional anisotropy loss was proposed to utilize the diffusion anisotropy information among diffusion directions. In vivo brain DWI data were used to test the proposed method.

RESULTS

The results show that distortion-corrected high-resolution DWI images with restored structural details can be obtained from low-resolution SS-EPI images by taking advantage of the high-resolution anatomical images. Additionally, the proposed network can improve the quantitative accuracy of diffusion metrics compared with previously reported networks.

CONCLUSION

Using high-resolution, distortion-free EPI-DWI images as references, a deep learning-based method to simultaneously increase the perceived resolution and correct distortions for low-resolution SS-EPI was proposed. The results show that DWI image quality and diffusion metrics can be improved.

Real-Time 3D Microwave Medical Imaging With Enhanced Variational Born Iterative Method

In this paper, we present a new variational Born iterative method (VBIM) for real-time microwave imaging (MWI) applications. The S-parameter volume integral equation and waveport vector Green's function are implemented to utilize the measured signal of the MWI system. Meanwhile, the real and imaginary separation (RIS) approach is used at each iterative step to simultaneously reconstruct the dielectric permittivity and conductivity of unknown objects. Compared with the Born iterative method and distorted Born iterative method, VBIM requires less computational time to reach the convergence threshold. The graphics processing unit based acceleration technique is implemented for real-time imaging. To demonstrate the efficiency and accuracy of this VBIM-RIS method, synthetic analysis of a complex multi-layer spherical phantom is first conducted. Then, the algorithm is tested with measured data using our new MWI system prototype. Finally, a synthetic brain-tumor phantom model under a thermal therapy procedure is monitored to exemplify the real-time imaging with about 5 seconds per reconstruction frame.

Development of a Super-Resolution Scheme for Pediatric Magnetic Resonance Brain Imaging Through Convolutional Neural Networks

Pediatric medical imaging represents a real challenge for physicians, as children who are patients often move during the examination, and it causes the appearance of different artifacts in the images. Thus, it is not possible to obtain good quality images for this target population limiting the possibility of evaluation and diagnosis in certain pathological conditions. Specifically, magnetic resonance imaging (MRI) is a technique that requires long acquisition times and, therefore, demands the use of sedation or general anesthesia to avoid the movement of the patient, which is really damaging in this specific population. Because ALARA (as low as reasonably achievable) principles should be considered for all imaging studies, one of the most important reasons for establishing novel MRI imaging protocols is to avoid the harmful effects of anesthesia/sedation. In this context, ground-breaking concepts and novel technologies, such as artificial intelligence, can help to find a solution to these challenges while helping in the search for underlying disease mechanisms. The use of new MRI protocols and new image acquisition and/or pre-processing techniques can aid in the development of neuroimaging studies for children evaluation, and their translation to pediatric populations. In this paper, a novel super-resolution method based on a convolutional neural network (CNN) in two and three dimensions to automatically increase the resolution of pediatric brain MRI acquired in a reduced time scheme is proposed. Low resolution images have been generated from an original high resolution dataset and used as the input of the CNN, while several scaling factors have been assessed separately. Apart from a healthy dataset, we also tested our model with pathological pediatric MRI, and it successfully recovers the original image quality in both visual and quantitative ways, even for available examples of dysplasia lesions. We hope then to establish the basis for developing an innovative free-sedation protocol in pediatric anatomical MRI acquisition.

What's new and what's next in diffusion MRI preprocessing

Diffusion MRI (dMRI) provides invaluable information for the study of tissue microstructure and brain connectivity, but suffers from a range of imaging artifacts that greatly challenge the analysis of results and their interpretability if not appropriately accounted for. This review will cover dMRI artifacts and preprocessing steps, some of which have not typically been considered in existing pipelines or reviews, or have only gained attention in recent years: brain/skull extraction, B-matrix incompatibilities w.r.t the imaging data, signal drift, Gibbs ringing, noise distribution bias, denoising, between- and within-volumes motion, eddy currents, outliers, susceptibility distortions, EPI Nyquist ghosts, gradient deviations, B1 bias fields, and spatial normalization. The focus will be on "what's new" since the notable advances prior to and brought by the Human Connectome Project (HCP), as presented in the predecessing issue on "Mapping the Connectome" in 2013. In addition to the development of novel strategies for dMRI preprocessing, exciting progress has been made in the availability of open source tools and reproducible pipelines, databases and simulation tools for the evaluation of preprocessing steps, and automated quality control frameworks, amongst others. Finally, this review will consider practical considerations and our view on "what's next" in dMRI preprocessing.

SOUP-GAN: Super-Resolution MRI Using Generative Adversarial Networks

There is a growing demand for high-resolution (HR) medical images for both clinical and research applications. Image quality is inevitably traded off with acquisition time, which in turn impacts patient comfort, examination costs, dose, and motion-induced artifacts. For many image-based tasks, increasing the apparent spatial resolution in the perpendicular plane to produce multi-planar reformats or 3D images is commonly used. Single-image super-resolution (SR) is a promising technique to provide HR images based on deep learning to increase the resolution of a 2D image, but there are few reports on 3D SR. Further, perceptual loss is proposed in the literature to better capture the textural details and edges versus pixel-wise loss functions, by comparing the semantic distances in the high-dimensional feature space of a pre-trained 2D network (e.g., VGG). However, it is not clear how one should generalize it to 3D medical images, and the attendant implications are unclear. In this paper, we propose a framework called SOUP-GAN: Super-resolution Optimized Using Perceptual-tuned Generative Adversarial Network (GAN), in order to produce thinner slices (e.g., higher resolution in the ‘Z’ plane) with anti-aliasing and deblurring. The proposed method outperforms other conventional resolution-enhancement methods and previous SR work on medical images based on both qualitative and quantitative comparisons. Moreover, we examine the model in terms of its generalization for arbitrarily user-selected SR ratios and imaging modalities. Our model shows promise as a novel 3D SR interpolation technique, providing potential applications for both clinical and research applications.

Development and evaluation of a high resolution 0.5mm isotropic T1-weighted template of the older adult brain

Investigating the structure of the older adult brain at high spatial resolution is of high significance, and a dedicated older adult structural brain template with sub-millimeter resolution is currently lacking. Therefore, the purpose of this work was twofold: (A) to develop a 0.5mm isotropic resolution standardized T1-weighted template of the older adult brain by applying principles of super resolution to high quality MRI data from 222 older adults (65-95 years of age), and (B) to systematically compare the new template to other standardized and study-specific templates in terms of image quality and performance when used as a reference for alignment of older adult data. The new template exhibited higher spatial resolution and improved visualization of fine structural details of the older adult brain compared to a template constructed using a conventional template building approach and the same data. In addition, the new template exhibited higher image sharpness and did not contain image artifacts observed in some of the other templates considered in this work. Due to the above enhancements, the new template provided higher inter-subject spatial normalization precision for older adult data compared to the other templates, and consequently enabled detection of smaller inter-group morphometric differences in older adult data. Finally, the new template was among those that were most representative of older adult brain data. Overall, the new template constructed here is an important resource for studies of aging, and the findings of the present work have important implications in template selection for investigations on older adults.

On the proper use of structural similarity for the robust evaluation of medical image synthesis models

PURPOSE

To propose good practices for using the structural similarity metric (SSIM) and reporting its value. SSIM is one of the most popular image quality metrics in use in the medical image synthesis community because of its alleged superiority over voxel-by-voxel measurements like the average error or the peak signal noise ratio (PSNR). It has seen massive adoption since its introduction, but its limitations are often overlooked. Notably, SSIM is designed to work on a strictly positive intensity scale, which is generally not the case in medical imaging. Common intensity scales such as the Houndsfield units (HU) contain negative numbers, and they can also be introduced by image normalization techniques such as the z-normalization.

METHODS

We created a series of experiments to quantify the impact of negative values in the SSIM computation. Specifically, we trained a 3D U-Net to synthesize T2 weighted MRI from T1 weighted MRI using the BRATS 2018 dataset. SSIM was computed on the synthetic images with a shifted dynamic range. Next, to evaluate the suitability of SSIM as a loss function on images with negative values, it was used as a loss function to synthesize z-normalized images. Finally, the difference between 2D SSIM and 3D SSIM was investigated using multiple 2D U-Nets trained on different planes of the images.

RESULTS

The impact of the misuse of the SSIM was quantified; it was established that it introduces a large downward bias in the computed SSIM. It also introduces a small random error that can change the relative ranking of models. The exact values for this bias and error depend on the quality and the intensity histogram of the synthetic images. Although small, the reported error is significant considering the small SSIM difference between state-of-the-art models. It was shown therefore that SSIM cannot be used as a loss function when images contain negative values due to major errors in the gradient calculation, resulting in under-performing models. 2D SSIM was also found to be overestimated in 2D image synthesis models when computed along the plane of synthesis, due to the discontinuities between slices that is typical of 2D synthesis methods.

CONCLUSION

Various types of misuse of the SSIM were identified and their impact was quantified. Based on the findings, this paper proposes good practices when using SSIM, such as reporting the average over the volume of the image containing tissue and appropriately defining the dynamic range. This article is protected by copyright. All rights reserved.

Chemotherapy response prediction with diffuser elapser network

In solid tumors, elevated fluid pressure and inadequate blood perfusion resulting from unbalanced angiogenesis are the prominent reasons for the ineffective drug delivery inside tumors. To normalize the heterogeneous and tortuous tumor vessel structure, antiangiogenic treatment is an effective approach. Additionally, the combined therapy of antiangiogenic agents and chemotherapy drugs has shown promising effects on enhanced drug delivery. However, the need to find the appropriate scheduling and dosages of the combination therapy is one of the main problems in anticancer therapy. Our study aims to generate a realistic response to the treatment schedule, making it possible for future works to use these patient-specific responses to decide on the optimal starting time and dosages of cytotoxic drug treatment. Our dataset is based on our previous in-silico model with a framework for the tumor microenvironment, consisting of a tumor layer, vasculature network, interstitial fluid pressure, and drug diffusion maps. In this regard, the chemotherapy response prediction problem is discussed in the study, putting forth a proof of concept for deep learning models to capture the tumor growth and drug response behaviors simultaneously. The proposed model utilizes multiple convolutional neural network submodels to predict future tumor microenvironment maps considering the effects of ongoing treatment. Since the model has the task of predicting future tumor microenvironment maps, we use two image quality evaluation metrics, which are structural similarity and peak signal-to-noise ratio, to evaluate model performance. We track tumor cell density values of ground truth and predicted tumor microenvironments. The model predicts tumor microenvironment maps seven days ahead with the average structural similarity score of 0.973 and the average peak signal ratio of 35.41 in the test set. It also predicts tumor cell density at the end day of 7 with the mean absolute percentage error of [Formula: see text].

Deep Learning in Large and Multi-Site Structural Brain MR Imaging Datasets

Large, multi-site, heterogeneous brain imaging datasets are increasingly required for the training, validation, and testing of advanced deep learning (DL)-based automated tools, including structural magnetic resonance (MR) image-based diagnostic and treatment monitoring approaches. When assembling a number of smaller datasets to form a larger dataset, understanding the underlying variability between different acquisition and processing protocols across the aggregated dataset (termed “batch effects”) is critical. The presence of variation in the training dataset is important as it more closely reflects the true underlying data distribution and, thus, may enhance the overall generalizability of the tool. However, the impact of batch effects must be carefully evaluated in order to avoid undesirable effects that, for example, may reduce performance measures. Batch effects can result from many sources, including differences in acquisition equipment, imaging technique and parameters, as well as applied processing methodologies. Their impact, both beneficial and adversarial, must be considered when developing tools to ensure that their outputs are related to the proposed clinical or research question (i.e., actual disease-related or pathological changes) and are not simply due to the peculiarities of underlying batch effects in the aggregated dataset. We reviewed applications of DL in structural brain MR imaging that aggregated images from neuroimaging datasets, typically acquired at multiple sites. We examined datasets containing both healthy control participants and patients that were acquired using varying acquisition protocols. First, we discussed issues around Data Access and enumerated the key characteristics of some commonly used publicly available brain datasets. Then we reviewed methods for correcting batch effects by exploring the two main classes of approaches: Data Harmonization that uses data standardization, quality control protocols or other similar algorithms and procedures to explicitly understand and minimize unwanted batch effects; and Domain Adaptation that develops DL tools that implicitly handle the batch effects by using approaches to achieve reliable and robust results. In this narrative review, we highlighted the advantages and disadvantages of both classes of DL approaches, and described key challenges to be addressed in future studies.

Deep robust residual network for super-resolution of 2D fetal brain MRI

Spatial resolution is a key factor of quantitatively evaluating the quality of magnetic resonance imagery (MRI). Super-resolution (SR) approaches can improve its spatial resolution by reconstructing high-resolution (HR) images from low-resolution (LR) ones to meet clinical and scientific requirements. To increase the quality of brain MRI, we study a robust residual-learning SR network (RRLSRN) to generate a sharp HR brain image from an LR input. Due to the Charbonnier loss can handle outliers well, and Gradient Difference Loss (GDL) can sharpen an image, we combined the Charbonnier loss and GDL to improve the robustness of the model and enhance the texture information of SR results. Two MRI datasets of adult brain, Kirby 21 and NAMIC, were used to train and verify the effectiveness of our model. To further verify the generalizability and robustness of the proposed model, we collected eight clinical fetal brain MRI 2D data for evaluation. The experimental results have shown that the proposed deep residual-learning network achieved superior performance and high efficiency over other compared methods.

Fine-tuning deep learning model parameters for improved super-resolution of dynamic MRI with prior-knowledge

Dynamic imaging is a beneficial tool for interventions to assess physiological changes. Nonetheless during dynamic MRI, while achieving a high temporal resolution, the spatial resolution is compromised. To overcome this spatio-temporal trade-off, this research presents a super-resolution (SR) MRI reconstruction with prior knowledge based fine-tuning to maximise spatial information while reducing the required scan-time for dynamic MRIs. A U-Net based network with perceptual loss is trained on a benchmark dataset and fine-tuned using one subject-specific static high resolution MRI as prior knowledge to obtain high resolution dynamic images during the inference stage. 3D dynamic data for three subjects were acquired with different parameters to test the generalisation capabilities of the network. The method was tested for different levels of in-plane undersampling for dynamic MRI. The reconstructed dynamic SR results after fine-tuning showed higher similarity with the high resolution ground-truth, while quantitatively achieving statistically significant improvement. The average SSIM of the lowest resolution experimented during this research (6.25% of the k-space) before and after fine-tuning were 0.939 ± 0.008 and 0.957 ± 0.006 respectively. This could theoretically result in an acceleration factor of 16, which can potentially be acquired in less than half a second. The proposed approach shows that the super-resolution MRI reconstruction with prior-information can alleviate the spatio-temporal trade-off in dynamic MRI, even for high acceleration factors.

Personalised predictive modelling with brain-inspired spiking neural networks of longitudinal MRI neuroimaging data and the case study of dementia

BACKGROUND

Longitudinal neuroimaging provides spatiotemporal brain data (STBD) measurement that can be utilised to understand dynamic changes in brain structure and/or function underpinning cognitive activities. Making sense of such highly interactive information is challenging, given that the features manifest intricate temporal, causal relations between the spatially distributed neural sources in the brain.

METHODS

The current paper argues for the advancement of deep learning algorithms in brain-inspired spiking neural networks (SNN), capable of modelling structural data across time (longitudinal measurement) and space (anatomical components). The paper proposes a methodology and a computational architecture based on SNN for building personalised predictive models from longitudinal brain data to accurately detect, understand, and predict the dynamics of an individual's functional brain state. The methodology includes finding clusters of similar data to each individual, data interpolation, deep learning in a 3-dimensional brain-template structured SNN model, classification and prediction of individual outcome, visualisation of structural brain changes related to the predicted outcomes, interpretation of results, and individual and group predictive marker discovery.

RESULTS

To demonstrate the functionality of the proposed methodology, the paper presents experimental results on a longitudinal magnetic resonance imaging (MRI) dataset derived from 175 older adults of the internationally recognised community-based cohort Sydney Memory and Ageing Study (MAS) spanning 6 years of follow-up.

SIGNIFICANCE

The models were able to accurately classify and predict 2 years ahead of cognitive decline, such as mild cognitive impairment (MCI) and dementia with 95% and 91% accuracy, respectively. The proposed methodology also offers a 3-dimensional visualisation of the MRI models reflecting the dynamic patterns of regional changes in white matter hyperintensity (WMH) and brain volume over 6 years.

CONCLUSION

The method is efficient for personalised predictive modelling on a wide range of neuroimaging longitudinal data, including also demographic, genetic, and clinical data. As a case study, it resulted in finding predictive markers for MCI and dementia as dynamic brain patterns using MRI data.

MRI super-resolution via realistic downsampling with adversarial learning

Many deep learning (DL) frameworks have demonstrated state-of-the-art performance in the super-resolution (SR) task of magnetic resonance imaging, but most performances have been achieved with simulated low-resolution (LR) images rather than LR images from real acquisition. Due to the limited generalizability of the SR network, enhancement is not guaranteed for real LR images because of the unreality of the training LR images. In this study, we proposed a DL-based SR framework with an emphasis on data construction to achieve better performance on real LR MR images. The framework comprised two steps: (a) downsampling training using a generative adversarial network (GAN) to construct more realistic and perfectly matched LR/high-resolution (HR) pairs. The downsampling GAN input was real LR and HR images. The generator translated the HR images to LR images and the discriminator distinguished the patch-level difference between the synthetic and real LR images. (b) SR training was performed using an enhance4d deep super-resolution network (EDSR). In the controlled experiments, three EDSRs were trained using our proposed method, Gaussian blur, and k-space zero-filling. As for the data, liver MR images were obtained from 24 patients using breath-hold serial LR and HR scans (only HR images were used in the conventional methods). The k-space zero-filling group delivered almost zero enhancement on the real LR images and the Gaussian group produced a considerable number of artifacts. The proposed method exhibited significantly better resolution enhancement and fewer artifacts compared with the other two networks. Our method outperformed the Gaussian method by an improvement of 0.111 ± 0.016 in the structural similarity index and 2.76 ± 0.98 dB in the peak signal-to-noise ratio. The blind/reference-less image spatial quality evaluator metric of the conventional Gaussian method and proposed method were 46.6 ± 4.2 and 34.1 ± 2.4, respectively.

3D dense convolutional neural network for fast and accurate single MR image super-resolution

Super-resolution (SR) MR image reconstruction has shown to be a very promising direction to improve the spatial resolution of low-resolution (LR) MR images. In this paper, we presented a novel MR image SR method based on a dense convolutional neural network (DDSR), and its enhanced version called EDDSR. There are three major innovations: first, we re-designed dense modules to extract hierarchical features directly from LR images and propagate the extracted feature maps through dense connections. Therefore, unlike other CNN-based SR MR techniques that upsample LR patches in the initial phase, our methods take the original LR images or patches as input. This effectively reduces computational complexity and speeds up SR reconstruction. Second, a final deconvolution filter in our model automatically learns filters to fuse and upscale all hierarchical feature maps to generate HR MR images. Using this, EDDSR can perform SR reconstructions at different upscale factors using a single model with one stride fixed deconvolution operation. Third, to further improve SR reconstruction accuracy, we exploited a geometric self-ensemble strategy. Experimental results on three benchmark datasets demonstrate that our methods, DDSR and EDDSR, achieved superior performance compared to state-of-the-art MR image SR methods with less computational load and memory usage.

Deep learning in magnetic resonance image reconstruction

Magnetic resonance (MR) imaging visualises soft tissue contrast in exquisite detail without harmful ionising radiation. In this work, we provide a state-of-the-art review on the use of deep learning in MR image reconstruction from different image acquisition types involving compressed sensing techniques, parallel image acquisition and multi-contrast imaging. Publications with deep learning-based image reconstruction for MR imaging were identified from the literature (PubMed and Google Scholar), and a comprehensive description of each of the works was provided. A detailed comparison that highlights the differences, the data used and the performance of each of these works were also made. A discussion of the potential use cases for each of these methods is provided. The sparse image reconstruction methods were found to be most popular in using deep learning for improved performance, accelerating acquisitions by around 4-8 times. Multi-contrast image reconstruction methods rely on at least one pre-acquired image, but can achieve 16-fold, and even up to 32- to 50-fold acceleration depending on the set-up. Parallel imaging provides frameworks to be integrated in many of these methods for additional speed-up potential. The successful use of compressed sensing techniques and multi-contrast imaging with deep learning and parallel acquisition methods could yield significant MR acquisition speed-ups within clinical routines in the near future.

A deep learning method for automatic segmentation of the bony orbit in MRI and CT images

This paper proposes a fully automatic method to segment the inner boundary of the bony orbit in two different image modalities: magnetic resonance imaging (MRI) and computed tomography (CT). The method, based on a deep learning architecture, uses two fully convolutional neural networks in series followed by a graph-search method to generate a boundary for the orbit. When compared to human performance for segmentation of both CT and MRI data, the proposed method achieves high Dice coefficients on both orbit and background, with scores of 0.813 and 0.975 in CT images and 0.930 and 0.995 in MRI images, showing a high degree of agreement with a manual segmentation by a human expert. Given the volumetric characteristics of these imaging modalities and the complexity and time-consuming nature of the segmentation of the orbital region in the human skull, it is often impractical to manually segment these images. Thus, the proposed method provides a valid clinical and research tool that performs similarly to the human observer.

A Transfer Model Based on Supervised Multi-Layer Dictionary Learning for Brain Tumor MRI Image Recognition

Artificial intelligence (AI) is an effective technology for automatic brain tumor MRI image recognition. The training of an AI model requires a large number of labeled data, but medical data needs to be labeled by professional clinicians, which makes data collection complex and expensive. Moreover, a traditional AI model requires that the training data and test data must follow the independent and identically distributed. To solve this problem, we propose a transfer model based on supervised multi-layer dictionary learning (TSMDL) for brain tumor MRI image recognition in this paper. With the help of the knowledge learned from related domains, the goal of this model is to solve the task of transfer learning where the target domain has only a small number of labeled samples. Based on the framework of multi-layer dictionary learning, the proposed model learns the common shared dictionary of source and target domains in each layer to explore the intrinsic connections and shared information between different domains. At the same time, by making full use of the label information of samples, the Laplacian regularization term is introduced to make the dictionary coding of similar samples as close as possible and the dictionary coding of different class samples as different as possible. The recognition experiments on brain MRI image datasets REMBRANDT and Figshare show that the model performs better than competitive state of-the-art methods.

Brain Tumor MR Image Classification Using Convolutional Dictionary Learning With Local Constraint

Brain tumor image classification is an important part of medical image processing. It assists doctors to make accurate diagnosis and treatment plans. Magnetic resonance (MR) imaging is one of the main imaging tools to study brain tissue. In this article, we propose a brain tumor MR image classification method using convolutional dictionary learning with local constraint (CDLLC). Our method integrates the multi-layer dictionary learning into a convolutional neural network (CNN) structure to explore the discriminative information. Encoding a vector on a dictionary can be considered as multiple projections into new spaces, and the obtained coding vector is sparse. Meanwhile, in order to preserve the geometric structure of data and utilize the supervised information, we construct the local constraint of atoms through a supervised k-nearest neighbor graph, so that the discrimination of the obtained dictionary is strong. To solve the proposed problem, an efficient iterative optimization scheme is designed. In the experiment, two clinically relevant multi-class classification tasks on the Cheng and REMBRANDT datasets are designed. The evaluation results demonstrate that our method is effective for brain tumor MR image classification, and it could outperform other comparisons.

Do Radiographic Assessments of Periodontal Bone Loss Improve with Deep Learning Methods for Enhanced Image Resolution?

Resolution plays an essential role in oral imaging for periodontal disease assessment. Nevertheless, due to limitations in acquisition tools, a considerable number of oral examinations have low resolution, making the evaluation of this kind of lesion difficult. Recently, the use of deep-learning methods for image resolution improvement has seen an increase in the literature. In this work, we performed two studies to evaluate the effects of using different resolution improvement methods (nearest, bilinear, bicubic, Lanczos, SRCNN, and SRGAN). In the first one, specialized dentists visually analyzed the quality of images treated with these techniques. In the second study, we used those methods as different pre-processing steps for inputs of convolutional neural network (CNN) classifiers (Inception and ResNet) and evaluated whether this process leads to better results. The deep-learning methods lead to a substantial improvement in the visual quality of images but do not necessarily promote better classifier performance.

3D Registration of pre-surgical prostate MRI and histopathology images via super-resolution volume reconstruction

The use of MRI for prostate cancer diagnosis and treatment is increasing rapidly. However, identifying the presence and extent of cancer on MRI remains challenging, leading to high variability in detection even among expert radiologists. Improvement in cancer detection on MRI is essential to reducing this variability and maximizing the clinical utility of MRI. To date, such improvement has been limited by the lack of accurately labeled MRI datasets. Data from patients who underwent radical prostatectomy enables the spatial alignment of digitized histopathology images of the resected prostate with corresponding pre-surgical MRI. This alignment facilitates the delineation of detailed cancer labels on MRI via the projection of cancer from histopathology images onto MRI. We introduce a framework that performs 3D registration of whole-mount histopathology images to pre-surgical MRI in three steps. First, we developed a novel multi-image super-resolution generative adversarial network (miSRGAN), which learns information useful for 3D registration by producing a reconstructed 3D MRI. Second, we trained the network to learn information between histopathology slices to facilitate the application of 3D registration methods. Third, we registered the reconstructed 3D histopathology volumes to the reconstructed 3D MRI, mapping the extent of cancer from histopathology images onto MRI without the need for slice-to-slice correspondence. When compared to interpolation methods, our super-resolution reconstruction resulted in the highest PSNR relative to clinical 3D MRI (32.15 dB vs 30.16 dB for BSpline interpolation). Moreover, the registration of 3D volumes reconstructed via super-resolution for both MRI and histopathology images showed the best alignment of cancer regions when compared to (1) the state-of-the-art RAPSODI approach, (2) volumes that were not reconstructed, or (3) volumes that were reconstructed using nearest neighbor, linear, or BSpline interpolations. The improved 3D alignment of histopathology images and MRI facilitates the projection of accurate cancer labels on MRI, allowing for the development of improved MRI interpretation schemes and machine learning models to automatically detect cancer on MRI.

Brain Tumor identification by Convolution Neural Network with Fuzzy C-mean Model Using MR Brain Images

Medical image computing techniques are essential in helping the doctors to support their decision in the diagnosis of the patients. Due to the complexity of the brain structure, we choose to use MR brain images because of their quality and the highest resolution. The objective of this article is to detect brain tumor using convolution neural network with fuzzy c-means model, the advantage of the proposed model is the ability to achieve excellent performance using accuracy, sensitivity, specificity, overall dice and recall values better than the previous models that are already published. In addition, the novel model can identify the brain tumor, using different types of MR images. The proposed model obtained accuracy with 98%.

Deep learning for brain disorders: from data processing to disease treatment

In order to reach precision medicine and improve patients' quality of life, machine learning is increasingly used in medicine. Brain disorders are often complex and heterogeneous, and several modalities such as demographic, clinical, imaging, genetics and environmental data have been studied to improve their understanding. Deep learning, a subpart of machine learning, provides complex algorithms that can learn from such various data. It has become state of the art in numerous fields, including computer vision and natural language processing, and is also growingly applied in medicine. In this article, we review the use of deep learning for brain disorders. More specifically, we identify the main applications, the concerned disorders and the types of architectures and data used. Finally, we provide guidelines to bridge the gap between research studies and clinical routine.

Using super-resolution generative adversarial network models and transfer learning to obtain high resolution digital periapical radiographs

Periapical Radiographs are commonly used to detect several anomalies, like caries, periodontal, and periapical diseases. Even considering that digital imaging systems used nowadays tend to provide high-quality images, external factors, or even system limitations can result in a vast amount of radiographic images with low quality and resolution. Commercial solutions offer tools based on interpolation methods to increase image resolution. However, previous literature shows that these methods may create undesirable effects in the images affecting the diagnosis accuracy. One alternative is using deep learning-based super-resolution methods to achieve better high-resolution images. Nevertheless, the amount of data for training such models is limited, demanding transfer learning approaches. In this work, we propose the use of super-resolution generative adversarial network (SRGAN) models and transfer learning to achieve periapical images with higher quality and resolution. Moreover, we evaluate the influence of using the transfer learning approach and the datasets selected for it in the final generated images. For that, we performed an experiment comparing the performance of the SRGAN models (with and without transfer learning) with other super-resolution methods. Considering Mean Square Error (MSE), Peak Signal to Noise Ratio (PSNR), Structural Similarity Index (SSIM), and Mean Opinion Score (MOS), the results of SRGAN models using transfer learning were better on average. This superiority was also verified statistically using the Wilcoxon paired test. In the visual analysis, the high quality achieved by the SRGAN models, in general, is visible, resulting in more defined edges details and fewer blur effects.