BM3D frames and variational image deblurring

Dual-Domain Self-Supervised Deep Learning with Graph Convolution for Low-Dose Computed Tomography Reconstruction

X-ray computed tomography (CT) is a commonly used imaging modality in clinical practice. Recent years have seen increasing public concern regarding the ionizing radiation from CT. Low-dose CT (LDCT) has been proven to be effective in reducing patients' radiation exposure, but it results in CT images with low signal-to-noise ratio (SNR), failing to meet the image quality required for diagnosis. To enhance the SNR of LDCT images, numerous denoising strategies based on deep learning have been introduced, leading to notable advancements. Despite these advancements, most methods have relied on a supervised training paradigm. The challenge in acquiring aligned pairs of low-dose and normal-dose images in a clinical setting has limited their applicability. Recently, some self-supervised deep learning methods have enabled denoising using only noisy samples. However, these techniques are based on overly simplistic assumptions about noise and focus solely on CT sinogram denoising or image denoising, compromising their effectiveness. To address this, we introduce the Dual-Domain Self-supervised framework, termed DDoS, to accomplish effective LDCT denoising and reconstruction. The framework includes denoising in the sinogram domain, filtered back-projection reconstruction, and denoising in the image domain. By identifying the statistical characteristics of sinogram noise and CT image noise, we develop sinogram-denoising and CT image-denoising networks that are fully adapted to these characteristics. Both networks utilize a unified hybrid architecture that combines graph convolution and incorporates multiple channel attention modules, facilitating the extraction of local and non-local multi-scale features. Comprehensive experiments on two large-scale LDCT datasets demonstrate the superiority of DDoS framework over existing state-of-the-art methods.

MRI recovery with self-calibrated denoisers without fully-sampled data

OBJECTIVE

Acquiring fully sampled training data is challenging for many MRI applications. We present a self-supervised image reconstruction method, termed ReSiDe, capable of recovering images solely from undersampled data.

MATERIALS AND METHODS

ReSiDe is inspired by plug-and-play (PnP) methods, but unlike traditional PnP approaches that utilize pre-trained denoisers, ReSiDe iteratively trains the denoiser on the image or images that are being reconstructed. We introduce two variations of our method: ReSiDe-S and ReSiDe-M. ReSiDe-S is scan-specific and works with a single set of undersampled measurements, while ReSiDe-M operates on multiple sets of undersampled measurements and provides faster inference. Studies I, II, and III compare ReSiDe-S and ReSiDe-M against other self-supervised or unsupervised methods using data from T1- and T2-weighted brain MRI, MRXCAT digital perfusion phantom, and first-pass cardiac perfusion, respectively.

RESULTS

ReSiDe-S and ReSiDe-M outperform other methods in terms of peak signal-to-noise ratio and structural similarity index measure for Studies I and II, and in terms of expert scoring for Study III.

DISCUSSION

We present a self-supervised image reconstruction method and validate it in both static and dynamic MRI applications. These developments can benefit MRI applications where the availability of fully sampled training data is limited.

SeBIR: Semantic-guided burst image restoration

Burst image restoration methods offer the possibility of recovering faithful scene details from multiple low-quality snapshots captured by hand-held devices in adverse scenarios, thereby attracting increasing attention in recent years. However, individual frames in a burst typically suffer from inter-frame misalignments, leading to ghosting artifacts. Besides, existing methods indiscriminately handle all burst frames, struggling to seamlessly remove the corrupted information due to the neglect of multi-frame spatio-temporal varying degradation. To alleviate these limitations, we propose a general semantic-guided model named SeBIR for burst image restoration incorporating the semantic prior knowledge of Segment Anything Model (SAM) to enable adaptive recovery. Specifically, instead of relying solely on a single aligning scheme, we develop a joint implicit and explicit strategy that sufficiently leverages semantic knowledge as guidance to achieve inter-frame alignment. To further adaptively modulate and aggregate aligned features with spatio-temporal disparity, we elaborate a semantic-guided fusion module using the intermediate semantic features of SAM as an explicit guide to weaken the inherent degradation and strengthen the valuable complementary information across frames. Additionally, a semantic-guided local loss is designed to boost local consistency and image quality. Extensive experiments on synthetic and real-world datasets demonstrate the superiority of our method in both quantitative and qualitative evaluations for burst super-resolution, burst denoising, and burst low-light image enhancement tasks.

Dual-channel lightweight GAN for enhancing color retinal images with noise suppression and structural protection

As we all know, suppressing noise while maintaining detailed structure has been a challenging problem in the field of image enhancement, especially for color retinal images. In this paper, a dual-channel lightweight GAN named dilated shuffle generative adversarial network (DS-GAN) is proposed to solve the above problems. The lightweight generator consists of the RB branch used in the red-blue channels and the GN branch used in the green channel. The branches are then integrated with a cat function to generate enhanced images. The RB branch cascades six identical RB-enhanced modules and adds skip connections. The structure of the GN branch is similar to that of the RB branch. The generator simultaneously leverages the local context extraction capability of the normal convolution and the global information extraction capability of the dilated convolution. In addition, it facilitates the fusion and communication of feature information between channels through channel shuffle. Additionally, we utilize the lightweight image classification model ShuffleNetV2 as a discriminator to distinguish between enhanced images and corresponding labels. We also constructed a dataset for color retinal image enhancement by using traditional methods and a hybrid loss function by combining the MS-SSIM and perceptual loss for training the generator. With the proposed dataset and loss function, we train the DS-GAN successfully. We test our method on four publicly available datasets (Messidor, DIARETDB0, DRIVE, and FIRE) and a clinic dataset from the Tianjin Eye Hospital (China), and compare it with six existing image enhancement methods. The results show that the proposed method can simultaneously suppress noise, preserve structure, and enhance contrast in color retinal image enhancement. It gets better results than the compared methods in all cases. Furthermore, the model has fewer parameters, which provides the possibility of real-time image enhancement for portable devices.

Deep generative denoising networks enhance quality and accuracy of gated cardiac PET data

BACKGROUND

Cardiac positron emission tomography (PET) can visualize and quantify the molecular and physiological pathways of cardiac function. However, cardiac and respiratory motion can introduce blurring that reduces PET image quality and quantitative accuracy. Dual cardiac- and respiratory-gated PET reconstruction can mitigate motion artifacts but increases noise as only a subset of data are used for each time frame of the cardiac cycle.

AIM

The objective of this study is to create a zero-shot image denoising framework using a conditional generative adversarial networks (cGANs) for improving image quality and quantitative accuracy in non-gated and dual-gated cardiac PET images.

METHODS

Our study included retrospective list-mode data from 40 patients who underwent an 18F-fluorodeoxyglucose (18F-FDG) cardiac PET study. We initially trained and evaluated a 3D cGAN-known as Pix2Pix-on simulated non-gated low-count PET data paired with corresponding full-count target data, and then deployed the model on an unseen test set acquired on the same PET/CT system including both non-gated and dual-gated PET data.

RESULTS

Quantitative analysis demonstrated that the 3D Pix2Pix network architecture achieved significantly (p value<0.05) enhanced image quality and accuracy in both non-gated and gated cardiac PET images. At 5%, 10%, and 15% preserved count statistics, the model increased peak signal-to-noise ratio (PSNR) by 33.7%, 21.2%, and 15.5%, structural similarity index (SSIM) by 7.1%, 3.3%, and 2.2%, and reduced mean absolute error (MAE) by 61.4%, 54.3%, and 49.7%, respectively. When tested on dual-gated PET data, the model consistently reduced noise, irrespective of cardiac/respiratory motion phases, while maintaining image resolution and accuracy. Significant improvements were observed across all gates, including a 34.7% increase in PSNR, a 7.8% improvement in SSIM, and a 60.3% reduction in MAE.

CONCLUSION

The findings of this study indicate that dual-gated cardiac PET images, which often have post-reconstruction artifacts potentially affecting diagnostic performance, can be effectively improved using a generative pre-trained denoising network.

Real-world video superresolution enhancement method based on the adaptive down-sampling model

With the 5G and the popularity of high-definition and ultrahigh-definition equipment, people have increasingly higher requirements for the resolution of images or videos. However, the transmission pressure on servers is also gradually increasing. Therefore, superresolution technology has attracted much attention in recent years. Simultaneously, with the further development of deep learning techniques, superresolution research is shifting from the calculation of traditional algorithms to the deep learning method, which exhibits a greatly superior final display. First, the traditional block-matching-3D (BM3D) algorithm is formed as the postprocessing module, which can avoid the uneven edge of GAN network recovery, make the picture appear more authentic, and improve the viewer's subjective feelings. Next, the adaptive-downsampling model (ADM) is utilized to train models for specific camera styles. The high-resolution (HR) data sequence is subsequently downsampled to a low-resolution (LR) data sequence, enabling the superresolution algorithm to utilize this training set. This method can obtain better results and improve overall performance by 0.1~0.3 dB.

Anisotropic regularization for sparsely sampled and noise-robust Fourier ptychography

Fourier ptychography (FP) is a powerful computational imaging technique that provides super-resolution and quantitative phase imaging capabilities by scanning samples in Fourier space with angle-varying illuminations. However, the image reconstruction in FP is inherently ill-posed, particularly when the measurements are noisy and have insufficient data redundancy in the Fourier space. To improve FP reconstruction in high-throughput imaging scenarios, we propose a regularized FP reconstruction algorithm utilizing anisotropic total variation (TV) and Tikhonov regularizations for the object and pupil functions, respectively. To solve this regularized FP problem, we formulate a reconstruction algorithm using the alternating direction method of multipliers and show that our approach successfully recovers high-quality images with sparsely sampled and/or noisy measurements. The results are quantitatively and qualitatively compared against various FP reconstruction algorithms to analyze the effect of regularization under harsh imaging conditions. In particular, we demonstrate the effectiveness of our method on the real experimental FP microscopy images, where the TV regularizer effectively suppresses the measurement noise while maintaining the edge information in the biological specimen and helps retrieve the correct amplitude and phase images even under insufficient sampling.

A Simulation Method for Underwater SPAD Depth Imaging Datasets

In recent years, underwater imaging and vision technologies have received widespread attention, and the removal of the backward-scattering interference caused by impurities in the water has become a long-term research focus for scholars. With the advent of new single-photon imaging devices, single-photon avalanche diode (SPAD) devices, with high sensitivity and a high depth resolution, have become cutting-edge research tools in the field of underwater imaging. However, the high production costs and small array areas of SPAD devices make it very difficult to conduct underwater SPAD imaging experiments. To address this issue, we propose a fast and effective underwater SPAD data simulation method and develop a denoising network for the removal of backward-scattering interference in underwater SPAD images based on deep learning and simulated data. The experimental results show that the distribution difference between the simulated and real underwater SPAD data is very small. Moreover, the algorithm based on deep learning and simulated data for the removal of backward-scattering interference in underwater SPAD images demonstrates effectiveness in terms of both metrics and human observation. The model yields improvements in metrics such as the PSNR, SSIM, and entropy of 5.59 dB, 9.03%, and 0.84, respectively, demonstrating its superior performance.

ML3CNet: Non-local means-assisted automatic framework for lung cancer subtypes classification using histopathological images

BACKGROUND AND OBJECTIVE

Lung cancer (LC) has a high fatality rate that continuously affects human lives all over the world. Early detection of LC prolongs human life and helps to prevent the disease. Histopathological inspection is a common method to diagnose LC. Visual inspection of histopathological diagnosis necessitates more inspection time, and the decision depends on the subjective perception of clinicians. Usually, machine learning techniques mostly depend on traditional feature extraction which is labor-intensive and may not be appropriate for enormous data. In this work, a convolutional neural network (CNN)-based architecture is proposed for the more effective classification of lung tissue subtypes using histopathological images.

METHODS

Authors have utilized the first-time nonlocal mean (NLM) filter to suppress the effect of noise from histopathological images. NLM filter efficiently eliminated noise while preserving the edges of images. Then, the obtained denoised images are given as input to the proposed multi-headed lung cancer classification convolutional neural network (ML3CNet). Furthermore, the model quantization technique is utilized to reduce the size of the proposed model for the storage of the data. Reduction in model size requires less memory and speeds up data processing.

RESULTS

The effectiveness of the proposed model is compared with the other existing state-of-the-art methods. The proposed ML3CNet achieved an average classification accuracy of 99.72%, sensitivity of 99.66%, precision of 99.64%, specificity of 99.84%, F-1 score of 0.9965, and area under the curve of 0.9978. The quantized accuracy of 98.92% is attained by the proposed model. To validate the applicability of the proposed ML3CNet, it has also been tested on the colon cancer dataset.

CONCLUSION

The findings reveal that the proposed approach can be beneficial to automatically classify LC subtypes that might assist healthcare workers in making decisions more precisely. The proposed model can be implemented on the hardware using Raspberry Pi for practical realization.

Glaucoma detection model by exploiting multi-region and multi-scan-pattern OCT images with dynamical region score

Currently, deep learning-based methods have achieved success in glaucoma detection. However, most models focus on OCT images captured by a single scan pattern within a given region, holding the high risk of the omission of valuable features in the remaining regions or scan patterns. Therefore, we proposed a multi-region and multi-scan-pattern fusion model to address this issue. Our proposed model exploits comprehensive OCT images from three fundus anatomical regions (macular, middle, and optic nerve head regions) being captured by four scan patterns (radial, volume, single-line, and circular scan patterns). Moreover, to enhance the efficacy of integrating features across various scan patterns within a region and multiple regional features, we employed an attention multi-scan fusion module and an attention multi-region fusion module that auto-assign contribution to distinct scan-pattern features and region features adapting to characters of different samples, respectively. To alleviate the absence of available datasets, we have collected a specific dataset (MRMSG-OCT) comprising OCT images captured by four scan patterns from three regions. The experimental results and visualized feature maps both demonstrate that our proposed model achieves superior performance against the single scan-pattern models and single region-based models. Moreover, compared with the average fusion strategy, our proposed fusion modules yield superior performance, particularly reversing the performance degradation observed in some models relying on fixed weights, validating the efficacy of the proposed dynamic region scores adapted to different samples. Moreover, the derived region contribution scores enhance the interpretability of the model and offer an overview of the model's decision-making process, assisting ophthalmologists in prioritizing regions with heightened scores and increasing efficiency in clinical practice.

Monitoring SF6 Gas Leakage Based on a Customized Binocular System

Sulfur hexafluoride (SF6) gas is extensively utilized as an insulating and arc-quenching medium in the circuit breakers and isolating switches of electrical equipment. It effectively isolates the circuits from the atmosphere and promptly extinguishes arcs. Therefore, the issue of SF6 gas leakage poses a significant threat to the related application fields, and the detection of SF6 gas leakage becomes extremely important. Infrared imaging detection offers advantages including non-contact, high precision, and visualization. However, most existing infrared detection systems are equipped with only one filter to detect SF6 gas. The images captured contain background noise and system noise, making these systems vulnerable to interference from such noises. To address these issues, we propose a method for monitoring SF6 gas leakage based on a customized binocular imaging (CBI) system. The CBI system has two filters, greatly reducing the interference of system noise and background noise. The first filter features the absorption resonant peak of SF6 gas. The second filter is used to record background noise and system noise. One aspect to note is that, in order to avoid the interference of other gases, the central wavelength of this second filter should keep away from the absorption resonant peaks of those gases. Accordingly, the central wavelengths of our customized filters were determined as 10,630 nm and 8370 nm, respectively. Then, two cameras of the same type were separately assembled with a customized filter, and the CBI prototype was accomplished. Finally, we utilized the difference method using two infrared images captured by the CBI system, to monitor the SF6 gas leakage. The results demonstrate that our developed system achieves a high accuracy of over 99.8% in detecting SF6 gas. Furthermore, the CBI system supports a plug-and-play customization to detect various gases for different scenarios.

Phase unwrapping for phase imaging using the plug-and-play proximal algorithm

Phase unwrapping (PU) is essential for various scientific optical applications. This process aims to estimate continuous phase values from acquired wrapped values, which are limited to the interval (-π,π]. However, the PU process can be challenging due to factors such as insufficient sampling, measurement errors, and inadequate equipment calibration, which can introduce excessive noise and unexpected phase discontinuities. This paper presents a robust iterative method based on the plug-and-play (PnP) proximal algorithm to unwrap two-dimensional phase values while simultaneously removing noise at each iteration. Using a least-squares formulation based on local phase differences and reformulating it as a partially differentiable equation, it is possible to employ the fast cosine transform to obtain a closed-form solution for one of the subproblems within the PnP framework. As a result, reliable phase reconstruction can be achieved even in scenarios with extremely high noise levels.

Cost-efficient and glaucoma-specifical model by exploiting normal OCT images with knowledge transfer learning

Monitoring the progression of glaucoma is crucial for preventing further vision loss. However, deep learning-based models emphasize early glaucoma detection, resulting in a significant performance gap to glaucoma-confirmed subjects. Moreover, developing a fully-supervised model is suffering from insufficient annotated glaucoma datasets. Currently, sufficient and low-cost normal OCT images with pixel-level annotations can serve as valuable resources, but effectively transferring shared knowledge from normal datasets is a challenge. To alleviate the issue, we propose a knowledge transfer learning model for exploiting shared knowledge from low-cost and sufficient annotated normal OCT images by explicitly establishing the relationship between the normal domain and the glaucoma domain. Specifically, we directly introduce glaucoma domain information to the training stage through a three-step adversarial-based strategy. Additionally, our proposed model exploits different level shared features in both output space and encoding space with a suitable output size by a multi-level strategy. We have collected and collated a dataset called the TongRen OCT glaucoma dataset, including pixel-level annotated glaucoma OCT images and diagnostic information. The results on the dataset demonstrate our proposed model outperforms the un-supervised model and the mixed training strategy, achieving an increase of 5.28% and 5.77% on mIoU, respectively. Moreover, our proposed model narrows performance gap to the fully-supervised model decreased by only 1.01% on mIoU. Therefore, our proposed model can serve as a valuable tool for extracting glaucoma-related features, facilitating the tracking progression of glaucoma.

Sparsier2Sparse: Self-supervised convolutional neural network-based streak artifacts reduction in sparse-view CT images

BACKGROUND

Sparse-view computed tomography (CT) has attracted a lot of attention for reducing both scanning time and radiation dose. However, sparsely-sampled projection data generate severe streak artifacts in the reconstructed images. In recent decades, many sparse-view CT reconstruction techniques based on fully-supervised learning have been proposed and have shown promising results. However, it is not feasible to acquire pairs of full-view and sparse-view CT images in real clinical practice.

PURPOSE

In this study, we propose a novel self-supervised convolutional neural network (CNN) method to reduce streak artifacts in sparse-view CT images.

METHODS

We generate the training dataset using only sparse-view CT data and train CNN based on self-supervised learning. Since the streak artifacts can be estimated using prior images under the same CT geometry system, we acquire prior images by iteratively applying the trained network to given sparse-view CT images. We then subtract the estimated steak artifacts from given sparse-view CT images to produce the final results.

RESULTS

We validated the imaging performance of the proposed method using extended cardiac-torso (XCAT) and the 2016 AAPM Low-Dose CT Grand Challenge dataset from Mayo Clinic. From the results of visual inspection and modulation transfer function (MTF), the proposed method preserved the anatomical structures effectively and showed higher image resolution compared to the various streak artifacts reduction methods for all projection views.

CONCLUSIONS

We propose a new framework for streak artifacts reduction when only the sparse-view CT data are given. Although we do not use any information of full-view CT data for CNN training, the proposed method achieved the highest performance in preserving fine details. By overcoming the limitation of dataset requirements on fully-supervised-based methods, we expect that our framework can be utilized in the medical imaging field.

Sparse single-pixel imaging via optimization in nonuniform sampling sparsity

Reducing the imaging time while maintaining reconstruction accuracy remains challenging for single-pixel imaging. One cost-effective approach is nonuniform sparse sampling. The existing methods lack intuitive and intrinsic analysis in sparsity. The lack impedes our comprehension of the form's adjustable range and may potentially limit our ability to identify an optimal distribution form within a confined adjustable range, consequently impacting the method's overall performance. In this Letter, we report a sparse sampling method with a wide adjustable range and define a sparsity metric to guide the selection of sampling forms. Through a comprehensive analysis and discussion, we select a sampling form that yields satisfying accuracy. These works will make up for the existing methods' lack of sparsity analysis and help adjust methods to accommodate different situations and needs.

Interpretable neural networks: principles and applications

In recent years, with the rapid development of deep learning technology, great progress has been made in computer vision, image recognition, pattern recognition, and speech signal processing. However, due to the black-box nature of deep neural networks (DNNs), one cannot explain the parameters in the deep network and why it can perfectly perform the assigned tasks. The interpretability of neural networks has now become a research hotspot in the field of deep learning. It covers a wide range of topics in speech and text signal processing, image processing, differential equation solving, and other fields. There are subtle differences in the definition of interpretability in different fields. This paper divides interpretable neural network (INN) methods into the following two directions: model decomposition neural networks, and semantic INNs. The former mainly constructs an INN by converting the analytical model of a conventional method into different layers of neural networks and combining the interpretability of the conventional model-based method with the powerful learning capability of the neural network. This type of INNs is further classified into different subtypes depending on which type of models they are derived from, i.e., mathematical models, physical models, and other models. The second type is the interpretable network with visual semantic information for user understanding. Its basic idea is to use the visualization of the whole or partial network structure to assign semantic information to the network structure, which further includes convolutional layer output visualization, decision tree extraction, semantic graph, etc. This type of method mainly uses human visual logic to explain the structure of a black-box neural network. So it is a post-network-design method that tries to assign interpretability to a black-box network structure afterward, as opposed to the pre-network-design method of model-based INNs, which designs interpretable network structure beforehand. This paper reviews recent progress in these areas as well as various application scenarios of INNs and discusses existing problems and future development directions.

Image denoising in acoustic microscopy using block-matching and 4D filter

Scanning acoustic microscopy (SAM) is a label-free imaging technique used in biomedical imaging, non-destructive testing, and material research to visualize surface and sub-surface structures. In ultrasonic imaging, noises in images can reduce contrast, edge and texture details, and resolution, negatively impacting post-processing algorithms. To reduce the noises in the scanned image, we have employed a 4D block-matching (BM4D) filter that can be used to denoise acoustic volumetric signals. BM4D filter utilizes the transform domain filtering technique with hard thresholding and Wiener filtering stages. The proposed algorithm produces the most suitable denoised output compared to other conventional filtering methods (Gaussian filter, median filter, and Wiener filter) when applied to noisy images. The output from the BM4D-filtered images was compared to the noise level with different conventional filters. Filtered images were qualitatively analyzed using metrics such as structural similarity index matrix (SSIM) and peak signal-to-noise ratio (PSNR). The combined qualitative and quantitative analysis demonstrates that the BM4D technique is the most suitable method for denoising acoustic imaging from the SAM. The proposed block matching filter opens a new avenue in the field of acoustic or photoacoustic image denoising, particularly in scenarios with poor signal-to-noise ratios.

Total-body low-dose CT image denoising using a prior knowledge transfer technique with a contrastive regularization mechanism

PURPOSE

Reducing the radiation exposure experienced by patients in total-body computed tomography (CT) imaging has attracted extensive attention in the medical imaging community. A low radiation dose may result in increased noise and artifacts that greatly affect the subsequent clinical diagnosis. To obtain high-quality total-body low-dose CT (LDCT) images, previous deep learning-based research works developed various network architectures. However, most of these methods only employ normal-dose CT (NDCT) images as ground truths to guide the training process of the constructed denoising network. As a result of this simple restriction, the reconstructed images tend to lose favorable image details and easily generate oversmoothed textures. This study explores how to better utilize the information contained in the feature spaces of NDCT images to guide the LDCT image reconstruction process and achieve high-quality results.

METHODS

We propose a novel intratask knowledge transfer (KT) method that leverages the knowledge distilled from NDCT images as an auxiliary component of the LDCT image reconstruction process. Our proposed architecture is named the teacher-student consistency network (TSC-Net), which consists of teacher and student networks with identical architectures. By employing the designed KT loss, the student network is encouraged to emulate the teacher network in the representation space and gain robust prior content. In addition, to further exploit the information contained in CT scans, a contrastive regularization mechanism (CRM) built upon contrastive learning is introduced. The CRM aims to minimize and maximize the L2 distances from the predicted CT images to the NDCT samples and to the LDCT samples in the latent space, respectively. Moreover, based on attention and the deformable convolution approach, we design a dynamic enhancement module (DEM) to improve the network capability to transform input information flows.

RESULTS

By conducting ablation studies, we prove the effectiveness of the proposed KT loss, CRM, and DEM. Extensive experimental results demonstrate that the TSC-Net outperforms the state-of-the-art methods in both quantitative and qualitative evaluations. Additionally, the excellent results obtained for clinical readings also prove that our proposed method can reconstruct high-quality CT images for clinical applications.

CONCLUSIONS

Based on the experimental results and clinical readings, the TSC-Net has better performance than other approaches. In our future work, we may explore the reconstruction of LDCT images by fusing the positron emission tomography (PET) and CT modalities to further improve the visual quality of the reconstructed CT images.

Optical Flow in the Dark

Optical flow estimation in low-light conditions is a challenging task for existing methods and current optical flow datasets lack low-light samples. Even if the dark images are enhanced before estimation, which could achieve great visual perception, it still leads to suboptimal optical flow results because information like motion consistency may be broken during the enhancement. We propose to apply a novel training policy to learn optical flow directly from new synthetic and real low-light images. Specifically, first, we design a method to collect a new optical flow dataset in multiple exposures with shared optical flow pseudo labels. Then we apply a two-step process to create a synthetic low-light optical flow dataset, based on an existing bright one, by simulating low-light raw features from the multi-exposure raw images we collected. To extend the data diversity, we also include published low-light raw videos without optical flow labels. In our training pipeline, with the three datasets, we create two teacher-student pairs to progressively obtain optical flow labels for all data. Finally, we apply a mix-up training policy with our diversified datasets to produce low-light-robust optical flow models for release. The experiments show that our method can relatively maintain the optical flow accuracy as the image exposure descends and the generalization ability of our method is tested with different cameras in multiple practical scenes.

DWDN: Deep Wiener Deconvolution Network for Non-Blind Image Deblurring

We present a simple and effective approach for non-blind image deblurring, combining classical techniques and deep learning. In contrast to existing methods that deblur the image directly in the standard image space, we propose to perform an explicit deconvolution process in a feature space by integrating a classical Wiener deconvolution framework with learned deep features. A multi-scale cascaded feature refinement module then predicts the deblurred image from the deconvolved deep features, progressively recovering detail and small-scale structures. The proposed model is trained in an end-to-end manner and evaluated on scenarios with simulated Gaussian noise, saturated pixels, or JPEG compression artifacts as well as real-world images. Moreover, we present detailed analyses of the benefit of the feature-based Wiener deconvolution and of the multi-scale cascaded feature refinement as well as the robustness of the proposed approach. Our extensive experimental results show that the proposed deep Wiener deconvolution network facilitates deblurred results with visibly fewer artifacts and quantitatively outperforms state-of-the-art non-blind image deblurring methods by a wide margin.

Iterative deep neural networks based on proximal gradient descent for image restoration

The algorithm unfolding networks with explainability of algorithms and higher efficiency of Deep Neural Networks (DNN) have received considerable attention in solving ill-posed inverse problems. Under the algorithm unfolding network framework, we propose a novel end-to-end iterative deep neural network and its fast network for image restoration. The first one is designed making use of proximal gradient descent algorithm of variational models, which consists of denoiser and reconstruction sub-networks. The second one is its accelerated version with momentum factors. For sub-network of denoiser, we embed the Convolutional Block Attention Module (CBAM) in previous U-Net for adaptive feature refinement. Experiments on image denoising and deblurring demonstrate that competitive performances in quality and efficiency are gained by compared with several state-of-the-art networks for image restoration. Proposed unfolding DNN can be easily extended to solve other similar image restoration tasks, such as image super-resolution, image demosaicking, etc.

Plug-and-Play Regularization Using Linear Solvers

There has been tremendous research on the design of image regularizers over the years, from simple Tikhonov and Laplacian to sophisticated sparsity and CNN-based regularizers. Coupled with a model-based loss function, these are typically used for image reconstruction within an optimization framework. The technical challenge is to develop a regularizer that can accurately model realistic images and be optimized efficiently along with the loss function. Motivated by the recent plug-and-play paradigm for image regularization, we construct a quadratic regularizer whose reconstruction capability is competitive with state-of-the-art regularizers. The novelty of the regularizer is that, unlike classical regularizers, the quadratic objective function is derived from the observed data. Since the regularizer is quadratic, we can reduce the optimization to solving a linear system for applications such as superresolution, deblurring, inpainting, etc. In particular, we show that using iterative Krylov solvers, we can converge to the solution in few iterations, where each iteration requires an application of the forward operator and a linear denoiser. The surprising finding is that we can get close to deep learning methods in terms of reconstruction quality. To the best of our knowledge, the possibility of achieving near state-of-the-art performance using a linear solver is novel.

Plug-and-Play Image Restoration With Deep Denoiser Prior

Recent works on plug-and-play image restoration have shown that a denoiser can implicitly serve as the image prior for model-based methods to solve many inverse problems. Such a property induces considerable advantages for plug-and-play image restoration (e.g., integrating the flexibility of model-based method and effectiveness of learning-based methods) when the denoiser is discriminatively learned via deep convolutional neural network (CNN) with large modeling capacity. However, while deeper and larger CNN models are rapidly gaining popularity, existing plug-and-play image restoration hinders its performance due to the lack of suitable denoiser prior. In order to push the limits of plug-and-play image restoration, we set up a benchmark deep denoiser prior by training a highly flexible and effective CNN denoiser. We then plug the deep denoiser prior as a modular part into a half quadratic splitting based iterative algorithm to solve various image restoration problems. We, meanwhile, provide a thorough analysis of parameter setting, intermediate results and empirical convergence to better understand the working mechanism. Experimental results on three representative image restoration tasks, including deblurring, super-resolution and demosaicing, demonstrate that the proposed plug-and-play image restoration with deep denoiser prior not only significantly outperforms other state-of-the-art model-based methods but also achieves competitive or even superior performance against state-of-the-art learning-based methods. The source code is available at https://github.com/cszn/DPIR.

Nonblind Image Deblurring via Deep Learning in Complex Field

Nonblind image deblurring is about recovering the latent clear image from a blurry one generated by a known blur kernel, which is an often-seen yet challenging inverse problem in imaging. Its key is how to robustly suppress noise magnification during the inversion process. Recent approaches made a breakthrough by exploiting convolutional neural network (CNN)-based denoising priors in the image domain or the gradient domain, which allows using a CNN for noise suppression. The performance of these approaches is highly dependent on the effectiveness of the denoising CNN in removing magnified noise whose distribution is unknown and varies at different iterations of the deblurring process for different images. In this article, we introduce a CNN-based image prior defined in the Gabor domain. The prior not only utilizes the optimal space-frequency resolution and strong orientation selectivity of the Gabor transform but also enables using complex-valued (CV) representations in intermediate processing for better denoising. A CV CNN is developed to exploit the benefits of the CV representations, with better generalization to handle unknown noises over the real-valued ones. Combining our Gabor-domain CV CNN-based prior with an unrolling scheme, we propose a deep-learning-based approach to nonblind image deblurring. Extensive experiments have demonstrated the superior performance of the proposed approach over the state-of-the-art ones.

Blind-noise image denoising with block-matching domain transformation filtering and improved guided filtering

The adaptive block size processing method in different image areas makes block-matching and 3D-filtering (BM3D) have a very good image denoising effect. Based on these observation, in this paper, we improve BM3D in three aspects: adaptive noise variance estimation, domain transformation filtering and nonlinear filtering. First, we improve the noise-variance estimation method of principle component analysis using multilayer wavelet decomposition. Second, we propose compressive sensing based Gaussian sequence Hartley domain transform filtering to reduce noise. Finally, we perform edge-preserving smoothing on the preprocessed image using the guided filtering based on total variation. Experimental results show that the proposed denoising method can be competitive with many representative denoising methods on the evaluation criteria of PSNR. However, it is worth further research on the visual quality of denoised images.

Deformation-Compensated Learning for Image Reconstruction Without Ground Truth

Deep neural networks for medical image reconstruction are traditionally trained using high-quality ground-truth images as training targets. Recent work on Noise2Noise (N2N) has shown the potential of using multiple noisy measurements of the same object as an alternative to having a ground-truth. However, existing N2N-based methods are not suitable for learning from the measurements of an object undergoing nonrigid deformation. This paper addresses this issue by proposing the deformation-compensated learning (DeCoLearn) method for training deep reconstruction networks by compensating for object deformations. A key component of DeCoLearn is a deep registration module, which is jointly trained with the deep reconstruction network without any ground-truth supervision. We validate DeCoLearn on both simulated and experimentally collected magnetic resonance imaging (MRI) data and show that it significantly improves imaging quality.

Hybrid method for improving Tikhonov-based reconstruction quality in electrical impedance tomography

Purpose

Electrical impedance tomography (EIT) has shown its potential in the field of medical imaging. Physiological or pathological variation would cause the change of conductivity. EIT is favorable in reconstructing conductivity distribution inside the detected area. However, due to its ill-posed and nonlinear characteristics, reconstructed images suffer from low spatial resolution.

Approach

Tikhonov regularization method is a popular and effective approach for image reconstruction in EIT. Nevertheless, excessive smoothness is observed when reconstruction is conducted based on Tikhonov method. To improve Tikhonov-based reconstruction quality in EIT, an innovative hybrid iterative optimization method is proposed. An efficient alternating minimization algorithm is introduced to solve the optimization problem.

Results

To verify image reconstruction performance and anti-noise robustness of the proposed method, a series of simulation work and phantom experiments is carried out. Meanwhile, comparison is made with reconstruction results based on Landweber, Newton-Raphson, and Tikhonov methods. The reconstruction performance is also verified by quantitative comparison of blur radius and structural similarity values which further demonstrates the excellent performance of the proposed method.

Conclusions

In contrast to Landweber, Newton-Raphson, and Tikhonov methods, it is found that images reconstructed by the proposed method are more accurate. Even under the impact of noise, the proposed method outperforms comparison methods.

Deep Learning-Based 3D Measurements with Near-Infrared Fringe Projection

Fringe projection profilometry (FPP) is widely applied to 3D measurements, owing to its advantages of high accuracy, non-contact, and full-field scanning. Compared with most FPP systems that project visible patterns, invisible fringe patterns in the spectra of near-infrared demonstrate fewer impacts on human eyes or on scenes where bright illumination may be avoided. However, the invisible patterns, which are generated by a near-infrared laser, are usually captured with severe speckle noise, resulting in 3D reconstructions of limited quality. To cope with this issue, we propose a deep learning-based framework that can remove the effect of the speckle noise and improve the precision of the 3D reconstruction. The framework consists of two deep neural networks where one learns to produce a clean fringe pattern and the other to obtain an accurate phase from the pattern. Compared with traditional denoising methods that depend on complex physical models, the proposed learning-based method is much faster. The experimental results show that the measurement accuracy can be increased effectively by the presented method.

Contrast Enhancement of Scintigraphic Image Using Fuzzy Intensification

Introduction

The objective of this study was to see the effect of fuzzy intensification (INT) operator on enhancement of scintigraphic image.

Materials and Methods

Nuclear medicine physician (NMP) provided 25 scintigraphic images that required enhancement. The image pixels value was converted into fuzzy plane and was subjected to contrast INT operator with parameters of INT operator i.e., cross-over = 0.5 and number of iterations = 1 and 2. The enhanced image was again brought back into spatial domain (de-fuzzification) whose intensity value was in the range 0-255. NMP compared the enhanced image with its input image and labeled it as acceptable or unacceptable. The quality of enhanced image was also accessed objectively using four different image metrics namely: Entropy, edge content, absolute mean brightness error and saturation metrics.

Results

Most of the enhanced images (18 out of 25 images) obtained at cross-over = 0.5 and number of iterations = 1 are acceptable and found to have overall better contrast compared to the corresponding input image. Four images (two brain positron emission tomography scan and two I-131 scan) obtained at cross-over = 0.5 and with iteration = 2 are acceptable. Three input images (one dimercaptosuccinic acid (DMSA), one I-131 and one I-131- metaiodo-benzyl-guanidine (MIBG) scan) were better than their enhanced images.

Conclusions

The enhancement produced by fuzzy INT operator was encouraging. Majority of enhanced images were acceptable at cross-over = 0.5 and number iterations = 1.

MA-GANet: A Multi-Attention Generative Adversarial Network for Defocus Blur Detection

Background clutters pose challenges to defocus blur detection. Existing approaches often produce artifact predictions in background areas with clutter and relatively low confident predictions in boundary areas. In this work, we tackle the above issues from two perspectives. Firstly, inspired by the recent success of self-attention mechanism, we introduce channel-wise and spatial-wise attention modules to attentively aggregate features at different channels and spatial locations to obtain more discriminative features. Secondly, we propose a generative adversarial training strategy to suppress spurious and low reliable predictions. This is achieved by utilizing a discriminator to identify predicted defocus map from ground-truth ones. As such, the defocus network (generator) needs to produce 'realistic' defocus map to minimize discriminator loss. We further demonstrate that the generative adversarial training allows exploiting additional unlabeled data to improve performance, a.k.a. semi-supervised learning, and we provide the first benchmark on semi-supervised defocus detection. Finally, we demonstrate that the existing evaluation metrics for defocus detection generally fail to quantify the robustness with respect to thresholding. For a fair and practical evaluation, we introduce an effective yet efficient AUF_β metric. Extensive experiments on three public datasets verify the superiority of the proposed methods compared against state-of-the-art approaches.

MRI RECOVERY WITH A SELF-CALIBRATED DENOISER

Plug-and-play (PnP) methods that employ application-specific denoisers have been proposed to solve inverse problems, including MRI reconstruction. However, training application-specific denoisers is not feasible for many applications due to the lack of training data. In this work, we propose a PnP-inspired recovery method that does not require data beyond the single, incomplete set of measurements. The proposed self-supervised method, called recovery with a self-calibrated denoiser (ReSiDe), trains the denoiser from the patches of the image being recovered. The denoiser training and a call to the denoising subroutine are performed in each iteration of a PnP algorithm, leading to a progressive refinement of the reconstructed image. For validation, we compare ReSiDe with a compressed sensing-based method and a PnP method with BM3D denoising using single-coil MRI brain data.

Simultaneous Successive Twinning Captured by Atomic Electron Tomography

Shape-controlled synthesis of multiply twinned nanostructures is heavily emphasized in nanoscience, in large part due to the desire to control the size, shape, and terminating facets of metal nanoparticles for applications in catalysis. Direct control of the size and shape of solution-grown nanoparticles relies on an understanding of how synthetic parameters alter nanoparticle structures during synthesis. However, while outcome populations can be effectively studied with standard electron microscopy methods, transient structures that appear during some synthetic routes are difficult to study using conventional high resolution imaging methods due to the high complexity of the 3D nanostructures. Here, we have studied the prevalence of transient structures during growth of multiply twinned particles and employed atomic electron tomography to reveal the atomic-scale three-dimensional structure of a Pd nanoparticle undergoing a shape transition. By identifying over 20 000 atoms within the structure and classifying them according to their local crystallographic environment, we observe a multiply twinned structure consistent with a simultaneous successive twinning from a decahedral to icosahedral structure.

Task-Oriented Convex Bilevel Optimization With Latent Feasibility

This paper firstly proposes a convex bilevel optimization paradigm to formulate and optimize popular learning and vision problems in real-world scenarios. Different from conventional approaches, which directly design their iteration schemes based on given problem formulation, we introduce a task-oriented energy as our latent constraint which integrates richer task information. By explicitly re- characterizing the feasibility, we establish an efficient and flexible algorithmic framework to tackle convex models with both shrunken solution space and powerful auxiliary (based on domain knowledge and data distribution of the task). In theory, we present the convergence analysis of our latent feasibility re- characterization based numerical strategy. We also analyze the stability of the theoretical convergence under computational error perturbation. Extensive numerical experiments are conducted to verify our theoretical findings and evaluate the practical performance of our method on different applications.

Tensor Completion via Complementary Global, Local, and Nonlocal Priors

Completing missing entries in multidimensional visual data is a typical ill-posed problem that requires appropriate exploitation of prior information of the underlying data. Commonly used priors can be roughly categorized into three classes: global tensor low-rankness, local properties, and nonlocal self-similarity (NSS); most existing works utilize one or two of them to implement completion. Naturally, there arises an interesting question: can one concurrently make use of multiple priors in a unified way, such that they can collaborate with each other to achieve better performance? This work gives a positive answer by formulating a novel tensor completion framework which can simultaneously take advantage of the global-local-nonlocal priors. In the proposed framework, the tensor train (TT) rank is adopted to characterize the global correlation; meanwhile, two Plug-and-Play (PnP) denoisers, including a convolutional neural network (CNN) denoiser and the color block-matching and 3 D filtering (CBM3D) denoiser, are incorporated to preserve local details and exploit NSS, respectively. Then, we design a proximal alternating minimization algorithm to efficiently solve this model under the PnP framework. Under mild conditions, we establish the convergence guarantee of the proposed algorithm. Extensive experiments show that these priors organically benefit from each other to achieve state-of-the-art performance both quantitatively and qualitatively.

Deep RED Unfolding Network for Image Restoration

The deep unfolding network (DUN) provides an efficient framework for image restoration. It consists of a regularization module and a data fitting module. In existing DUN models, it is common to directly use a deep convolution neural network (DCNN) as the regularization module, and perform data fitting before regularization in each iteration/stage. In this work, we present a DUN by incorporating a new regularization module, and putting the regularization module before the data fitting module. The proposed regularization model is deducted by using the regularization by denoing (RED) and plugging in it a newly designed DCNN. For the data fitting module, we use the closed-form solution with Faster Fourier Transform (FFT). The resulted DRED-DUN model has some major advantages. First, the regularization model inherits the flexibility of learned image-adaptive and interpretability of RED. Second, the DRED-DUN model is an end-to-end trainable DUN, which learns the regularization network and other parameters jointly, thus leads to better restoration performance than the plug-and-play framework. Third, extensive experiments show that, our proposed model significantly outperforms the-state-of-the-art model-based methods and learning based methods in terms of PSNR indexes as well as the visual effects. In particular, our method has much better capability in recovering salient image components such as edges and small scale textures.

Structure-Aware Motion Deblurring Using Multi-Adversarial Optimized CycleGAN

Recently, Convolutional Neural Networks (CNNs) have achieved great improvements in blind image motion deblurring. However, most existing image deblurring methods require a large amount of paired training data and fail to maintain satisfactory structural information, which greatly limits their application scope. In this paper, we present an unsupervised image deblurring method based on a multi-adversarial optimized cycle-consistent generative adversarial network (CycleGAN). Although original CycleGAN can handle unpaired training data well, the generated high-resolution images are probable to lose content and structure information. To solve this problem, we utilize a multi-adversarial mechanism based on CycleGAN for blind motion deblurring to generate high-resolution images iteratively. In this multi-adversarial manner, the hidden layers of the generator are gradually supervised, and the implicit refinement is carried out to generate high-resolution images continuously. Meanwhile, we also introduce the structure-aware mechanism to enhance the structure and detail retention ability of the multi-adversarial network for deblurring by taking the edge map as guidance information and adding multi-scale edge constraint functions. Our approach not only avoids the strict need for paired training data and the errors caused by blur kernel estimation, but also maintains the structural information better with multi-adversarial learning and structure-aware mechanism. Comprehensive experiments on several benchmarks have shown that our approach prevails the state-of-the-art methods for blind image motion deblurring.

Triply Complementary Priors for Image Restoration

Recent works that utilized deep models have achieved superior results in various image restoration (IR) applications. Such approach is typically supervised, which requires a corpus of training images with distributions similar to the images to be recovered. On the other hand, the shallow methods, which are usually unsupervised remain promising performance in many inverse problems, e.g., image deblurring and image compressive sensing (CS), as they can effectively leverage nonlocal self-similarity priors of natural images. However, most of such methods are patch-based leading to the restored images with various artifacts due to naive patch aggregation in addition to the slow speed. Using either approach alone usually limits performance and generalizability in IR tasks. In this paper, we propose a joint low-rank and deep (LRD) image model, which contains a pair of triply complementary priors, namely, internal and external, shallow and deep, and non-local and local priors. We then propose a novel hybrid plug-and-play (H-PnP) framework based on the LRD model for IR. Following this, a simple yet effective algorithm is developed to solve the proposed H-PnP based IR problems. Extensive experimental results on several representative IR tasks, including image deblurring, image CS and image deblocking, demonstrate that the proposed H-PnP algorithm achieves favorable performance compared to many popular or state-of-the-art IR methods in terms of both objective and visual perception.

Multi-Material Decomposition for Single Energy CT Using Material Sparsity Constraint

Multi-material decomposition (MMD) decomposes CT images into basis material images, and is a promising technique in clinical diagnostic CT to identify material compositions within the human body. MMD could be implemented on measurements obtained from spectral CT protocol, although spectral CT data acquisition is not readily available in most clinical environments. MMD methods using single energy CT (SECT), broadly applied in radiological departments of most hospitals, have been proposed in the literature while challenged by the inferior decomposition accuracy and the limited number of material bases due to the constrained material information in the SECT measurement. In this paper, we propose an image-domain SECT MMD method using material sparsity as an assistance under the condition that each voxel of the CT image contains at most two different elemental materials. L₀ norm represents the material sparsity constraint (MSC) and is integrated into the decomposition objective function with a least-square data fidelity term, total variation term, and a sum-to-one constraint of material volume fractions. An accelerated primal-dual (APD) algorithm with line-search scheme is applied to solve the problem. The pixelwise direct inversion method with the two-material assumption (TMA) is applied to estimate the initials. We validate the proposed method on phantom and patient data. Compared with the TMA method, the proposed MSC method increases the volume fraction accuracy (VFA) from 92.0% to 98.5% in the phantom study. In the patient study, the calcification area can be clearly visualized in the virtual non-contrast image generated by the proposed method, and has a similar shape to that in the ground-truth contrast-free CT image. The high decomposition image quality from the proposed method substantially facilitates the SECT-based MMD clinical applications.

Rotated rectangular aperture imaging through multi-frame blind deconvolution with Hyper-Laplacian priors

Rotated rectangular aperture imaging has many advantages in large aperture telephoto systems due to its lower cost and lower complexity. This technology makes it possible to build super large aperture telescopes. In this paper, we combine the ideas of deblurring with rotated rectangular aperture imaging and propose an image synthesis algorithm based on multi-frame deconvolution. In the specific reconstruction process, Hyper-Laplacian priors and sparse priors are used, and an effective solution is developed. The simulation and real shooting experiments show that our algorithm has excellent performance in visual effect and objective evaluation. The synthetic images are significantly sharper than the results of the existing methods.

Deep learning in electron microscopy

Deep learning is transforming most areas of science and technology, including electron microscopy. This review paper offers a practical perspective aimed at developers with limited familiarity. For context, we review popular applications of deep learning in electron microscopy. Following, we discuss hardware and software needed to get started with deep learning and interface with electron microscopes. We then review neural network components, popular architectures, and their optimization. Finally, we discuss future directions of deep learning in electron microscopy.

Surface-Aware Blind Image Deblurring

Blind image deblurring is a conundrum because there are infinitely many pairs of latent image and blur kernel. To get a stable and reasonable deblurred image, proper prior knowledge of the latent image and the blur kernel is urgently required. Different from the recent works on the statistical observations of the difference between the blurred image and the clean one, our method is built on the surface-aware strategy arising from the intrinsic geometrical consideration. This approach facilitates the blur kernel estimation due to the preserved sharp edges in the intermediate latent image. Extensive experiments demonstrate that our method outperforms the state-of-the-art methods on deblurring the text and natural images. Moreover, our method can achieve attractive results in some challenging cases, such as low-illumination images with large saturated regions and impulse noise. A direct extension of our method to the non-uniform deblurring problem also validates the effectiveness of the surface-aware prior.

Performance of deep learning restoration methods for the extraction of particle dynamics in noisy microscopy image sequences

Particle tracking in living systems requires low light exposure and short exposure times to avoid phototoxicity and photobleaching and to fully capture particle motion with high-speed imaging. Low-excitation light comes at the expense of tracking accuracy. Image restoration methods based on deep learning dramatically improve the signal-to-noise ratio in low-exposure data sets, qualitatively improving the images. However, it is not clear whether images generated by these methods yield accurate quantitative measurements such as diffusion parameters in (single) particle tracking experiments. Here, we evaluate the performance of two popular deep learning denoising software packages for particle tracking, using synthetic data sets and movies of diffusing chromatin as biological examples. With synthetic data, both supervised and unsupervised deep learning restored particle motions with high accuracy in two-dimensional data sets, whereas artifacts were introduced by the denoisers in three-dimensional data sets. Experimentally, we found that, while both supervised and unsupervised approaches improved tracking results compared with the original noisy images, supervised learning generally outperformed the unsupervised approach. We find that nicer-looking image sequences are not synonymous with more precise tracking results and highlight that deep learning algorithms can produce deceiving artifacts with extremely noisy images. Finally, we address the challenge of selecting parameters to train convolutional neural networks by implementing a frugal Bayesian optimizer that rapidly explores multidimensional parameter spaces, identifying networks yielding optimal particle tracking accuracy. Our study provides quantitative outcome measures of image restoration using deep learning. We anticipate broad application of this approach to critically evaluate artificial intelligence solutions for quantitative microscopy.

Quantification of myocardial oxygen extraction fraction: A proof-of-concept study

PURPOSE

To demonstrate a proof of concept for the measurement of myocardial oxygen extraction fraction (mOEF) by a cardiovascular magnetic resonance technique.

METHODS

The mOEF measurement was performed using an electrocardiogram-triggered double-echo asymmetric spin-echo sequence with EPI readout. Seven healthy volunteers (22-37 years old, 5 females) were recruited and underwent the same imaging scans at rest on 2 different days for reproducibility assessment. Another 5 subjects (23-37 years old, 4 females) underwent cardiovascular magnetic resonance studies at rest and during a handgrip isometric exercise with a 25% of maximal voluntary contraction. Both mOEF and myocardial blood volume values were obtained in septal regions from respective maps.

RESULTS

The reproducibility was excellent for the measurements of mOEF in septal myocardium (coefficient of variation: 3.37%) and moderate for myocardial blood volume (coefficient of variation: 19.7%). The average mOEF and myocardial blood volume of 7 subjects at rest were 0.61 ± 0.05 and 11.0 ± 4.3%, respectively. The mOEF agreed well with literature values that were measured by PET in healthy volunteers. In the exercise study, there was no significant change in mOEF (0.61 ± 0.06 vs 0.62 ± 0.07) or myocardial blood volume (12 ± 6% vs 13 ± 4%) from rest to exercise, as expected.

CONCLUSION

The implemented cardiovascular magnetic resonance method shows potential for the quantitative assessment of mOEF in vivo. Future technical work is needed to improve image quality and to further validate mOEF measurements.

Deep Outlier Handling for Image Deblurring

Outlier handling has attracted considerable attention recently but remains challenging for image deblurring. Existing approaches mainly depend on iterative outlier detection steps to explicitly or implicitly reduce the influence of outliers on image deblurring. However, these outlier detection steps usually involve heuristic operations and iterative optimization processes, which are complex and time-consuming. In contrast, we propose to learn a deep convolutional neural network to directly estimate the confidence map, which can identify reliable inliers and outliers from the blurred image and thus facilitates the following deblurring process. We analyze that the proposed algorithm incorporated with the learned confidence map is effective in handling outliers and does not require ad-hoc outlier detection steps which are critical to existing outlier handling methods. Compared to existing approaches, the proposed algorithm is more efficient and can be applied to both non-blind and blind image deblurring. Extensive experimental results demonstrate that the proposed algorithm performs favorably against state-of-the-art methods in terms of accuracy and efficiency.

X-Ray Imaging Calibration for Fuel-Coolant Interaction Experimental Facilities

During a severe accident in either sodium-cooled or water-cooled nuclear reactors, jets of molten nuclear fuel may impinge on the coolant resulting in fuel-coolant interactions (FCI). Experimental programs are being conducted to study this phenomenology and to support the development of severe accident models. Due to the optical opacity of the test section walls, sodium coolant, and the apparent optical opacity of water in the presence of intense ebullition, high-speed X-ray imaging is the preferred technique for FCI visualization. The configuration of these X-ray imaging systems, whereby the test section is installed between a fan-beam X-ray source and a scintillator-image intensifier projecting an image in the visual spectrum onto a high-speed camera, entails certain imaging artefacts and uncertainties. The X-ray imaging configuration requires precise calibration to enable detailed quantitative characterization of the FCI. To this end, ‘phantom’ models have been fabricated using polyethylene, either steel or hafnia powder, and empty cavities to represent sodium, molten fuel and sodium vapor phases respectively. A checkerboard configuration of the phantom enables calibration and correction for lens distortion artefacts which magnify features towards the edge of the field of view. Polydisperse steel ball configurations enable precise determination of the lower limit of detection and the estimation of parallax errors which introduce uncertainty in an object’s silhouette dimensions. Calibration experiments at the MELT facility determined lower limits of detection in the order of 4 mm for steel spheres, and 1.7-3.75 mm for vapor films around a molten jet.

On the Convergence of Learning-Based Iterative Methods for Nonconvex Inverse Problems

Numerous tasks at the core of statistics, learning and vision areas are specific cases of ill-posed inverse problems. Recently, learning-based (e.g., deep) iterative methods have been empirically shown to be useful for these problems. Nevertheless, integrating learnable structures into iterations is still a laborious process, which can only be guided by intuitions or empirical insights. Moreover, there is a lack of rigorous analysis about the convergence behaviors of these reimplemented iterations, and thus the significance of such methods is a little bit vague. This paper moves beyond these limits and proposes Flexible Iterative Modularization Algorithm (FIMA), a generic and provable paradigm for nonconvex inverse problems. Our theoretical analysis reveals that FIMA allows us to generate globally convergent trajectories for learning-based iterative methods. Meanwhile, the devised scheduling policies on flexible modules should also be beneficial for classical numerical methods in the nonconvex scenario. Extensive experiments on real applications verify the superiority of FIMA.