An improved deep network for tissue microstructure estimation with uncertainty quantification

DM-CNN: Dynamic Multi-scale Convolutional Neural Network with uncertainty quantification for medical image classification

Convolutional neural network (CNN) has promoted the development of diagnosis technology of medical images. However, the performance of CNN is limited by insufficient feature information and inaccurate attention weight. Previous works have improved the accuracy and speed of CNN but ignored the uncertainty of the prediction, that is to say, uncertainty of CNN has not received enough attention. Therefore, it is still a great challenge for extracting effective features and uncertainty quantification of medical deep learning models In order to solve the above problems, this paper proposes a novel convolutional neural network model named DM-CNN, which mainly contains the four proposed sub-modules : dynamic multi-scale feature fusion module (DMFF), hierarchical dynamic uncertainty quantifies attention (HDUQ-Attention) and multi-scale fusion pooling method (MF Pooling) and multi-objective loss (MO loss). DMFF select different convolution kernels according to the feature maps at different levels, extract different-scale feature information, and make the feature information of each layer have stronger representation ability for information fusion HDUQ-Attention includes a tuning block that adjust the attention weight according to the different information of each layer, and a Monte-Carlo (MC) dropout structure for quantifying uncertainty MF Pooling is a pooling method designed for multi-scale models, which can speed up the calculation and prevent overfitting while retaining the main important information Because the number of parameters in the backbone part of DM-CNN is different from other modules, MO loss is proposed, which has a fast optimization speed and good classification effect DM-CNN conducts experiments on publicly available datasets in four areas of medicine (Dermatology, Histopathology, Respirology, Ophthalmology), achieving state-of-the-art classification performance on all datasets. DM-CNN can not only maintain excellent performance, but also solve the problem of quantification of uncertainty, which is a very important task for the medical field. The code is available: https://github.com/QIANXIN22/DM-CNN.

Application of uncertainty quantification to artificial intelligence in healthcare: A review of last decade (2013-2023)

Uncertainty estimation in healthcare involves quantifying and understanding the inherent uncertainty or variability associated with medical predictions, diagnoses, and treatment outcomes. In this era of Artificial Intelligence (AI) models, uncertainty estimation becomes vital to ensure safe decision-making in the medical field. Therefore, this review focuses on the application of uncertainty techniques to machine and deep learning models in healthcare. A systematic literature review was conducted using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Our analysis revealed that Bayesian methods were the predominant technique for uncertainty quantification in machine learning models, with Fuzzy systems being the second most used approach. Regarding deep learning models, Bayesian methods emerged as the most prevalent approach, finding application in nearly all aspects of medical imaging. Most of the studies reported in this paper focused on medical images, highlighting the prevalent application of uncertainty quantification techniques using deep learning models compared to machine learning models. Interestingly, we observed a scarcity of studies applying uncertainty quantification to physiological signals. Thus, future research on uncertainty quantification should prioritize investigating the application of these techniques to physiological signals. Overall, our review highlights the significance of integrating uncertainty techniques in healthcare applications of machine learning and deep learning models. This can provide valuable insights and practical solutions to manage uncertainty in real-world medical data, ultimately improving the accuracy and reliability of medical diagnoses and treatment recommendations.

Validation of deep learning techniques for quality augmentation in diffusion MRI for clinical studies

The objective of this study is to evaluate the efficacy of deep learning (DL) techniques in improving the quality of diffusion MRI (dMRI) data in clinical applications. The study aims to determine whether the use of artificial intelligence (AI) methods in medical images may result in the loss of critical clinical information and/or the appearance of false information. To assess this, the focus was on the angular resolution of dMRI and a clinical trial was conducted on migraine, specifically between episodic and chronic migraine patients. The number of gradient directions had an impact on white matter analysis results, with statistically significant differences between groups being drastically reduced when using 21 gradient directions instead of the original 61. Fourteen teams from different institutions were tasked to use DL to enhance three diffusion metrics (FA, AD and MD) calculated from data acquired with 21 gradient directions and a b-value of 1000 s/mm2. The goal was to produce results that were comparable to those calculated from 61 gradient directions. The results were evaluated using both standard image quality metrics and Tract-Based Spatial Statistics (TBSS) to compare episodic and chronic migraine patients. The study results suggest that while most DL techniques improved the ability to detect statistical differences between groups, they also led to an increase in false positive. The results showed that there was a constant growth rate of false positives linearly proportional to the new true positives, which highlights the risk of generalization of AI-based tasks when assessing diverse clinical cohorts and training using data from a single group. The methods also showed divergent performance when replicating the original distribution of the data and some exhibited significant bias. In conclusion, extreme caution should be exercised when using AI methods for harmonization or synthesis in clinical studies when processing heterogeneous data in clinical studies, as important information may be altered, even when global metrics such as structural similarity or peak signal-to-noise ratio appear to suggest otherwise.

A microstructure estimation Transformer inspired by sparse representation for diffusion MRI

Diffusion magnetic resonance imaging (dMRI) is an important tool in characterizing tissue microstructure based on biophysical models, which are typically multi-compartmental models with mathematically complex and highly non-linear forms. Resolving microstructures from these models with conventional optimization techniques is prone to estimation errors and requires dense sampling in the q-space with a long scan time. Deep learning based approaches have been proposed to overcome these limitations. Motivated by the superior performance of the Transformer in feature extraction than the convolutional structure, in this work, we present a learning-based framework based on Transformer, namely, a Microstructure Estimation Transformer with Sparse Coding (METSC) for dMRI-based microstructural parameter estimation. To take advantage of the Transformer while addressing its limitation in large training data requirement, we explicitly introduce an inductive bias-model bias into the Transformer using a sparse coding technique to facilitate the training process. Thus, the METSC is composed with three stages, an embedding stage, a sparse representation stage, and a mapping stage. The embedding stage is a Transformer-based structure that encodes the signal in a high-level space to ensure the core voxel of a patch is represented effectively. In the sparse representation stage, a dictionary is constructed by solving a sparse reconstruction problem that unfolds the Iterative Hard Thresholding (IHT) process. The mapping stage is essentially a decoder that computes the microstructural parameters from the output of the second stage, based on the weighted sum of normalized dictionary coefficients where the weights are also learned. We tested our framework on two dMRI models with downsampled q-space data, including the intravoxel incoherent motion (IVIM) model and the neurite orientation dispersion and density imaging (NODDI) model. The proposed method achieved up to 11.25 folds of acceleration while retaining high fitting accuracy for NODDI fitting, reducing the mean squared error (MSE) up to 70% compared with the previous q-space learning approach. METSC outperformed the other state-of-the-art learning-based methods, including the model-free and model-based methods. The network also showed robustness against noise and generalizability across different datasets. The superior performance of METSC indicates its potential to improve dMRI acquisition and model fitting in clinical applications.

Parameter estimation for WMTI-Watson model of white matter using encoder-decoder recurrent neural network

PURPOSE

Biophysical modeling of the diffusion MRI (dMRI) signal provides estimates of specific microstructural tissue properties. Although non-linear least squares (NLLS) is the most widespread fitting method, it suffers from local minima and high computational cost. Deep learning approaches are steadily replacing NLLS, but come with the limitation that the model needs to be retrained for each acquisition protocol and noise level. In this study, a novel fitting approach was proposed based on the encoder-decoder recurrent neural network (RNN) to accelerate model estimation with good generalization to various datasets.

METHODS

The white matter tract integrity (WMTI)-Watson model as an implementation of the Standard Model of diffusion in white matter derives its parameters indirectly from the diffusion and kurtosis tensors (DKI). The RNN-based solver, which estimates the WMTI-Watson model from DKI, is therefore more readily translatable to various data, irrespective of acquisition protocols as long as the DKI was pre-computed from the signal. An embedding approach was also used to render the model insensitive to potential differences in distributions between training data and experimental data. The analytical solution, NLLS, RNN-, and a multilayer perceptron (MLP)-based methods were evaluated on synthetic and in vivo datasets of rat and human brain.

RESULTS

The proposed RNN solver showed highly reduced computation time over the analytical solution and NLLS, with similar accuracy but improved robustness, and superior generalizability over MLP.

CONCLUSION

The RNN estimator can be easily applied to various datasets without retraining, which shows great potential for a widespread use.

Joint estimation of relaxation and diffusion tissue parameters for prostate cancer with relaxation-VERDICT MRI

This work presents a biophysical model of diffusion and relaxation MRI for prostate called relaxation vascular, extracellular and restricted diffusion for cytometry in tumours (rVERDICT). The model includes compartment-specific relaxation effects providing T1/T2 estimates and microstructural parameters unbiased by relaxation properties of the tissue. 44 men with suspected prostate cancer (PCa) underwent multiparametric MRI (mp-MRI) and VERDICT-MRI followed by targeted biopsy. We estimate joint diffusion and relaxation prostate tissue parameters with rVERDICT using deep neural networks for fast fitting. We tested the feasibility of rVERDICT estimates for Gleason grade discrimination and compared with classic VERDICT and the apparent diffusion coefficient (ADC) from mp-MRI. The rVERDICT intracellular volume fraction f_ic discriminated between Gleason 3 + 3 and 3 + 4 (p = 0.003) and Gleason 3 + 4 and ≥ 4 + 3 (p = 0.040), outperforming classic VERDICT and the ADC from mp-MRI. To evaluate the relaxation estimates we compare against independent multi-TE acquisitions, showing that the rVERDICT T2 values are not significantly different from those estimated with the independent multi-TE acquisition (p > 0.05). Also, rVERDICT parameters exhibited high repeatability when rescanning five patients (R² = 0.79-0.98; CV = 1-7%; ICC = 92-98%). The rVERDICT model allows for accurate, fast and repeatable estimation of diffusion and relaxation properties of PCa sensitive enough to discriminate Gleason grades 3 + 3, 3 + 4 and ≥ 4 + 3.

Hybrid Graph Transformer for Tissue Microstructure Estimation with Undersampled Diffusion MRI Data

Advanced contemporary diffusion models for tissue microstructure often require diffusion MRI (DMRI) data with sufficiently dense sampling in the diffusion wavevector space for reliable model fitting, which might not always be feasible in practice. A potential remedy to this problem is by using deep learning techniques to predict high-quality diffusion microstructural indices from sparsely sampled data. However, existing methods are either agnostic to the data geometry in the diffusion wavevector space ( q -space) or limited to leveraging information from only local neighborhoods in the physical coordinate space ( x -space). Here, we propose a hybrid graph transformer (HGT) to explicitly consider the q -space geometric structure with a graph neural network (GNN) and make full use of spatial information with a novel residual dense transformer (RDT). The RDT consists of multiple densely connected transformer layers and a residual connection to facilitate model training. Extensive experiments on the data from the Human Connectome Project (HCP) demonstrate that our method significantly improves the quality of microstructural estimations over existing state-of-the-art methods.

Brain age estimation using multi-feature-based networks

Studying brain aging improves our understanding in differentiating typical and atypical aging. Directly utilizing traditional morphological features for brain age estimation did not show significant performance in healthy controls (HCs), which may be due to the negligence of the information of structural similarities among cortical regions. For this issue, the multi-feature-based network (MFN) built upon morphological features can be employed to describe these similarities. Based on this, we hypothesized that the MFN is more efficient and robust than traditional morphological features in brain age estimating. In this work, we used six different types of morphological features (i.e., cortical volume, cortical thickness, curvature index, folding index, local gyrification index, and surface area) to build individual MFN for brain age estimation. The efficacy of MFN was estimated on 2501 HCs with T1-weighted structural magnetic resonance imaging (sMRI) data and compared with traditional morphological features. We attained a mean absolute error (MAE) of 3.73 years using the proposed method on an independent test set, whereas a mean absolute error of 5.30 years was derived from morphological features. Our experimental results demonstrated that the MFN is an efficient and robust metric for estimating brain age.

Beyond Diffusion Tensor MRI Methods for Improved Characterization of the Brain after Ischemic Stroke: A Review

Ischemic stroke is a worldwide problem, with 15 million people experiencing a stroke annually. MR imaging is a valuable tool for understanding and assessing brain changes after stroke and predicting recovery. Of particular interest is the use of diffusion MR imaging in the nonacute stage 1-30 days poststroke. Thousands of articles have been published on the use of diffusion MR imaging in stroke, including several recent articles reviewing the use of DTI for stroke. The goal of this work was to survey and put into context the recent use of diffusion MR imaging methods beyond DTI, including diffusional kurtosis, generalized fractional anisotropy, spherical harmonics methods, and neurite orientation and dispersion models, in patients poststroke. Early studies report that these types of beyond-DTI methods outperform DTI metrics either in being more sensitive to poststroke changes or by better predicting outcome motor scores. More and larger studies are needed to confirm the improved prediction of stroke recovery with the beyond-DTI methods.

Jointly estimating parametric maps of multiple diffusion models from undersampled q-space data: A comparison of three deep learning approaches

PURPOSE

While advanced diffusion techniques have been found valuable in many studies, their clinical availability has been hampered partly due to their long scan times. Moreover, each diffusion technique can only extract a few relevant microstructural features. Using multiple diffusion methods may help to better understand the brain microstructure, which requires multiple expensive model fittings. In this work, we compare deep learning (DL) approaches to jointly estimate parametric maps of multiple diffusion representations/models from highly undersampled q-space data.

METHODS

We implement three DL approaches to jointly estimate parametric maps of diffusion tensor imaging (DTI), diffusion kurtosis imaging (DKI), neurite orientation dispersion and density imaging (NODDI), and multi-compartment spherical mean technique (SMT). A per-voxel q-space deep learning (1D-qDL), a per-slice convolutional neural network (2D-CNN), and a 3D-patch-based microstructure estimation with sparse coding using a separable dictionary (MESC-SD) network are considered.

RESULTS

The accuracy of estimated diffusion maps depends on the q-space undersampling, the selected network architecture, and the region and the parameter of interest. The smallest errors are observed for the MESC-SD network architecture (less than 10 % normalized RMSE in most brain regions).

CONCLUSION

Our experiments show that DL methods are very efficient tools to simultaneously estimate several diffusion maps from undersampled q-space data. These methods can significantly reduce both the scan ( ∼ 6-fold) and processing times ( ∼ 25-fold) for estimating advanced parametric diffusion maps while achieving a reasonable accuracy.

Dilated densely connected U-Net with uncertainty focus loss for 3D ABUS mass segmentation

BACKGROUND AND OBJECTIVE

Accurate segmentation of breast mass in 3D automated breast ultrasound (ABUS) images plays an important role in qualitative and quantitative ABUS image analysis. Yet this task is challenging due to the low signal to noise ratio and serious artifacts in ABUS images, the large shape and size variation of breast masses, as well as the small training dataset compared with natural images. The purpose of this study is to address these difficulties by designing a dilated densely connected U-Net (D²U-Net) together with an uncertainty focus loss.

METHODS

A lightweight yet effective densely connected segmentation network is constructed to extensively explore feature representations in the small ABUS dataset. In order to deal with the high variation in shape and size of breast masses, a set of hybrid dilated convolutions is integrated into the dense blocks of the D²U-Net. We further suggest an uncertainty focus loss to put more attention on unreliable network predictions, especially the ambiguous mass boundaries caused by low signal to noise ratio and artifacts. Our segmentation algorithm is evaluated on an ABUS dataset of 170 volumes from 107 patients. Ablation analysis and comparison with existing methods are conduct to verify the effectiveness of the proposed method.

RESULTS

Experiment results demonstrate that the proposed algorithm outperforms existing methods on 3D ABUS mass segmentation tasks, with Dice similarity coefficient, Jaccard index and 95% Hausdorff distance of 69.02%, 56.61% and 4.92 mm, respectively.

CONCLUSIONS

The proposed method is effective in segmenting breast masses on our small ABUS dataset, especially breast masses with large shape and size variations.

Multimodal super-resolved q-space deep learning

Super-resolvedq-space deep learning (SR-q-DL) has been developed to estimate high-resolution (HR) tissue microstructure maps from low-quality diffusion magnetic resonance imaging (dMRI) scans acquired with a reduced number of diffusion gradients and low spatial resolution, where deep networks are designed for the estimation. However, existing methods do not exploit HR information from other modalities, which are generally acquired together with dMRI and could provide additional useful information for HR tissue microstructure estimation. In this work, we extend SR-q-DL and propose multimodal SR-q-DL, where information in low-resolution (LR) dMRI is combined with HR information from another modality for HR tissue microstructure estimation. Because the HR modality may not be as sensitive to tissue microstructure as dMRI, direct concatenation of multimodal information does not necessarily lead to improved estimation performance. Since existing deep networks for HR tissue microstructure estimation are patch-based and use redundant information in the spatial domain to enhance the spatial resolution, the HR information in the other modality could inform the deep networks about what input voxels are relevant for the computation of tissue microstructure. Thus, we propose to incorporate the HR information from the HR modality by designing an attention module that guides the computation of HR tissue microstructure from LR dMRI. Specifically, the attention module is integrated with the patch-based SR-q-DL framework that exploits the sparsity of diffusion signals. The sparse representation of the LR diffusion signals in the input patch is first computed with a network component that unrolls an iterative process for sparse reconstruction. Then, the proposed attention module computes a relevance map from the HR modality with sequential convolutional layers. The relevance map indicates the relevance of the LR sparse representation at each voxel for computing the patch of HR tissue microstructure. The relevance is applied to the LR sparse representation with voxelwise multiplication, and the weighted LR sparse representation is used to compute HR tissue microstructure with another network component that allows resolution enhancement. All weights in the proposed network for multimodal SR-q-DL are jointly learned and the estimation is end-to-end. To evaluate the proposed method, we performed experiments on brain dMRI scans together with images of additional HR modalities. In the experiments, the proposed method was applied to the estimation of tissue microstructure measures for different datasets and advanced biophysical models, where the benefit of incorporating multimodal information using the proposed method is shown.

Super-Resolved q-Space deep learning with uncertainty quantification

Diffusion magnetic resonance imaging (dMRI) provides a noninvasive method for measuring brain tissue microstructure. q-Space deep learning(q-DL) methods have been developed to accurately estimate tissue microstructure from dMRI scans acquired with a reduced number of diffusion gradients. In these methods, deep networks are trained to learn the mapping directly from diffusion signals to tissue microstructure. However, the quality of tissue microstructure estimation can be limited not only by the reduced number of diffusion gradients but also by the low spatial resolution of typical dMRI acquisitions. Therefore, in this work we extend q-DL to super-resolved tissue microstructure estimation and propose super-resolvedq-DL (SR-q-DL), where deep networks are designed to map low-resolution diffusion signals undersampled in the q-space to high-resolution tissue microstructure. Specifically, we use a patch-based strategy, where a deep network takes low-resolution patches of diffusion signals as input and outputs high-resolution tissue microstructure patches. The high-resolution patches are then combined to obtain the final high-resolution tissue microstructure map. Motivated by existing q-DL methods, we integrate the sparsity of diffusion signals in the network design, which comprises two functional components. The first component computes sparse representation of diffusion signals for the low-resolution input patch, and the second component maps the low-resolution sparse representation to high-resolution tissue microstructure. The weights in the two components are learned jointly and the trained network performs end-to-end tissue microstructure estimation. In addition to SR-q-DL, we further propose probabilistic SR-q-DL, which can quantify the uncertainty of the network output as well as achieve improved estimation accuracy. In probabilistic SR-q-DL, a deep ensemble strategy is used. Specifically, the deep network for SR-q-DL is revised to produce not only tissue microstructure estimates but also the uncertainty of the estimates. Then, multiple deep networks are trained and their results are fused for the final prediction of high-resolution tissue microstructure and uncertainty quantification. The proposed method was evaluated on two independent datasets of brain dMRI scans. Results indicate that our approach outperforms competing methods in terms of estimation accuracy. In addition, uncertainty measures provided by our method correlate with estimation errors, which indicates potential application of the proposed uncertainty quantification method in brain studies.

Deep learning-based method for reducing residual motion effects in diffusion parameter estimation

PURPOSE

Conventional motion-correction techniques for diffusion MRI can introduce motion-level-dependent bias in derived metrics. To address this challenge, a deep learning-based technique was developed to minimize such residual motion effects.

METHODS

The data-rejection approach was adopted in which motion-corrupted data are discarded before model-fitting. A deep learning-based parameter estimation algorithm, using a hierarchical convolutional neural network (H-CNN), was combined with motion assessment and corrupted volume rejection. The method was designed to overcome the limitations of existing methods of this kind that produce parameter estimations whose quality depends strongly on a proportion of the data discarded. Evaluation experiments were conducted for the estimation of diffusion kurtosis and diffusion-tensor-derived measures at both the individual and group levels. The performance was compared with the robust approach of iteratively reweighted linear least squares (IRLLS) after motion correction with and without outlier replacement.

RESULTS

Compared with IRLLS, the H-CNN-based technique is minimally sensitive to motion effects. It was tested at severe motion levels when 70% to 90% of the data are rejected and when random motion is present. The technique had a stable performance independent of the numbers and schemes of data rejection. A further test on a data set from children with attention-deficit hyperactivity disorder shows the technique can potentially ameliorate spurious group-level difference caused by head motion.

CONCLUSION

This method shows great potential for reducing residual motion effects in motion-corrupted diffusion-weighted-imaging data, bringing benefits that include reduced bias in derived metrics in individual scans and reduced motion-level-dependent bias in population studies employing diffusion MRI.

Challenges for biophysical modeling of microstructure

The biophysical modeling efforts in diffusion MRI have grown considerably over the past 25 years. In this review, we dwell on the various challenges along the journey of bringing a biophysical model from initial design to clinical implementation, identifying both hurdles that have been already overcome and outstanding issues. First, we describe the critical initial task of selecting which features of tissue microstructure can be estimated using a model and which acquisition protocol needs to be implemented to make the estimation possible. The model performance should necessarily be tested in realistic numerical simulations and in experimental data - adapting the fitting strategy accordingly, and parameter estimates should be validated against complementary techniques, when/if available. Secondly, the model performance and validity should be explored in pathological conditions, and, if appropriate, dedicated models for pathology should be developed. We build on examples from tumors, ischemia and demyelinating diseases. We then discuss the challenges associated with clinical translation and added value. Finally, we single out four major unresolved challenges that are related to: the availability of a microstructural ground truth, the validation of model parameters which cannot be accessed with complementary techniques, the development of a generalized standard model for any brain region and pathology, and the seamless communication between different parties involved in the development and application of biophysical models of diffusion.

Estimating Tissue Microstructure with Undersampled Diffusion Data via Graph Convolutional Neural Networks

Advanced diffusion models for tissue microstructure are widely employed to study brain disorders. However, these models usually require diffusion MRI (DMRI) data with densely sampled q-space, which is prohibitive in clinical settings. This problem can be resolved by using deep learning techniques, which learn the mapping between sparsely sampled q-space data and the high-quality diffusion microstructural indices estimated from densely sampled data. However, most existing methods simply view the input DMRI data as a vector without considering data structure in the q-space. In this paper, we propose to overcome this limitation by representing DMRI data using graphs and utilizing graph convolutional neural networks to estimate tissue microstructure. Our method makes full use of the q-space angular neighboring information to improve estimation accuracy. Experimental results based on data from the Baby Connectome Project demonstrate that our method outperforms state-of-the-art methods both qualitatively and quantitatively.