Multimodal super-resolved q-space deep learning

M3S-Net: multi-modality multi-branch multi-self-attention network with structure-promoting loss for low-dose PET/CT enhancement

Objective.PET (Positron Emission Tomography) inherently involves radiotracer injections and long scanning time, which raises concerns about the risk of radiation exposure and patient comfort. Reductions in radiotracer dosage and acquisition time can lower the potential risk and improve patient comfort, respectively, but both will also reduce photon counts and hence degrade the image quality. Therefore, it is of interest to improve the quality of low-dose PET images.Approach.A supervised multi-modality deep learning model, named M3S-Net, was proposed to generate standard-dose PET images (60 s per bed position) from low-dose ones (10 s per bed position) and the corresponding CT images. Specifically, we designed a multi-branch convolutional neural network with multi-self-attention mechanisms, which first extracted features from PET and CT images in two separate branches and then fused the features to generate the final generated PET images. Moreover, a novel multi-modality structure-promoting term was proposed in the loss function to learn the anatomical information contained in CT images.Main results.We conducted extensive numerical experiments on real clinical data collected from local hospitals. Compared with state-of-the-art methods, the proposed M3S-Net not only achieved higher objective metrics and better generated tumors, but also performed better in preserving edges and suppressing noise and artifacts.Significance.The experimental results of quantitative metrics and qualitative displays demonstrate that the proposed M3S-Net can generate high-quality PET images from low-dose ones, which are competable to standard-dose PET images. This is valuable in reducing PET acquisition time and has potential applications in dynamic PET imaging.

Super-resolved q-space learning of diffusion MRI

BACKGROUND

Diffusion magnetic resonance imaging (dMRI) provides a powerful tool to non-invasively investigate neural structures in the living human brain. Nevertheless, its reconstruction performance on neural structures relies on the number of diffusion gradients in the q-space. High-angular (HA) dMRI requires a long scan time, limiting its use in clinical practice, whereas directly reducing the number of diffusion gradients would lead to the underestimation of neural structures.

PURPOSE

We propose a deep compressive sensing-based q-space learning (DCS-qL) approach to estimate HA dMRI from low-angular dMRI.

METHODS

In DCS-qL, we design the deep network architecture by unfolding the proximal gradient descent procedure that addresses the compressive sense problem. In addition, we exploit a lifting scheme to design a network structure with reversible transform properties. For implementation, we apply a self-supervised regression to enhance the signal-to-noise ratio of diffusion data. Then, we utilize a semantic information-guided patch-based mapping strategy for feature extraction, which introduces multiple network branches to handle patches with different tissue labels.

RESULTS

Experimental results show that the proposed approach can yield a promising performance on the tasks of reconstructed HA dMRI images, microstructural indices of neurite orientation dispersion and density imaging, fiber orientation distribution, and fiber bundle estimation.

CONCLUSIONS

The proposed method achieves more accurate neural structures than competing approaches.

Artificial intelligence for diffusion MRI-based tissue microstructure estimation in the human brain: an overview

Artificial intelligence (AI) has made significant advances in the field of diffusion magnetic resonance imaging (dMRI) and other neuroimaging modalities. These techniques have been applied to various areas such as image reconstruction, denoising, detecting and removing artifacts, segmentation, tissue microstructure modeling, brain connectivity analysis, and diagnosis support. State-of-the-art AI algorithms have the potential to leverage optimization techniques in dMRI to advance sensitivity and inference through biophysical models. While the use of AI in brain microstructures has the potential to revolutionize the way we study the brain and understand brain disorders, we need to be aware of the pitfalls and emerging best practices that can further advance this field. Additionally, since dMRI scans rely on sampling of the q-space geometry, it leaves room for creativity in data engineering in such a way that it maximizes the prior inference. Utilization of the inherent geometry has been shown to improve general inference quality and might be more reliable in identifying pathological differences. We acknowledge and classify AI-based approaches for dMRI using these unifying characteristics. This article also highlighted and reviewed general practices and pitfalls involving tissue microstructure estimation through data-driven techniques and provided directions for building on them.

TrGANet: Transforming 3T to 7T dMRI using Trapezoidal Rule and Graph based Attention Modules

Diffusion MRI (dMRI) is a non-invasive tool for assessing the white matter region of the brain by approximating the fiber streamlines, structural connectivity, and estimation of microstructure. This modality can yield useful information for diagnosing several mental diseases as well as for surgical planning. The higher angular resolution diffusion imaging (HARDI) technique is helpful in obtaining more robust fiber tracts by getting a good approximation of regions where fibers cross. Moreover, HARDI is more sensitive to tissue changes and can accurately represent anatomical details in the human brain at higher magnetic strengths. In other words, magnetic strengths affect the quality of the image, and hence high magnetic strength has good tissue contrast with better spatial resolution. However, a higher magnetic strength scanner (like 7T) is costly and unaffordable to most hospitals. Hence, in this work, we have proposed a novel CNN architecture for the transformation of 3T to 7T dMRI. Additionally, we have also reconstructed the multi-shell multi-tissue fiber orientation distribution function (MSMT fODF) at 7T from single-shell 3T. The proposed architecture consists of a CNN-based ODE solver utilizing the Trapezoidal rule and graph-based attention layer alongwith L₁ and total variation loss. Finally, the model has been validated on the HCP data set quantitatively and qualitatively.

Jointly estimating parametric maps of multiple diffusion models from undersampled q-space data: A comparison of three deep learning approaches

PURPOSE

While advanced diffusion techniques have been found valuable in many studies, their clinical availability has been hampered partly due to their long scan times. Moreover, each diffusion technique can only extract a few relevant microstructural features. Using multiple diffusion methods may help to better understand the brain microstructure, which requires multiple expensive model fittings. In this work, we compare deep learning (DL) approaches to jointly estimate parametric maps of multiple diffusion representations/models from highly undersampled q-space data.

METHODS

We implement three DL approaches to jointly estimate parametric maps of diffusion tensor imaging (DTI), diffusion kurtosis imaging (DKI), neurite orientation dispersion and density imaging (NODDI), and multi-compartment spherical mean technique (SMT). A per-voxel q-space deep learning (1D-qDL), a per-slice convolutional neural network (2D-CNN), and a 3D-patch-based microstructure estimation with sparse coding using a separable dictionary (MESC-SD) network are considered.

RESULTS

The accuracy of estimated diffusion maps depends on the q-space undersampling, the selected network architecture, and the region and the parameter of interest. The smallest errors are observed for the MESC-SD network architecture (less than 10 % normalized RMSE in most brain regions).

CONCLUSION

Our experiments show that DL methods are very efficient tools to simultaneously estimate several diffusion maps from undersampled q-space data. These methods can significantly reduce both the scan ( ∼ 6-fold) and processing times ( ∼ 25-fold) for estimating advanced parametric diffusion maps while achieving a reasonable accuracy.

Volumetric Segmentation of White Matter Tracts with Label Embedding

Convolutional neural networks have achieved state-of-the-art performance for white matter (WM) tract segmentation based on diffusion magnetic resonance imaging (dMRI). However, the segmentation can still be difficult for challenging WM tracts with thin bodies or complicated shapes; the segmentation is even more problematic in challenging scenarios with reduced data quality or domain shift between training and test data, which can be easily encountered in clinical settings. In this work, we seek to improve the segmentation of WM tracts, especially for challenging WM tracts in challenging scenarios. In particular, our method is based on volumetric WM tract segmentation, where voxels are directly labeled without performing tractography. To improve the segmentation, we exploit the characteristics of WM tracts that different tracts can cross or overlap and revise the network design accordingly. Specifically, because multiple tracts can co-exist in a voxel, we hypothesize that the different tract labels can be correlated. The tract labels at a single voxel are concatenated as a label vector, the length of which is the number of tract labels. Due to the tract correlation, this label vector can be projected into a lower-dimensional space-referred to as the embedded space-for each voxel, which allows the segmentation network to solve a simpler problem. By predicting the coordinate in the embedded space for the tracts at each voxel and subsequently mapping the coordinate to the label vector with a reconstruction module, the segmentation result can be achieved. To facilitate the learning of the embedded space, an auxiliary label reconstruction loss is integrated with the segmentation accuracy loss during network training, and network training and inference are end-to-end. Our method was validated on two dMRI datasets under various settings. The results show that the proposed method improves the accuracy of WM tract segmentation, and the improvement is more prominent for challenging tracts in challenging scenarios.