Framing U-Net via Deep Convolutional Framelets: Application to Sparse-View CT

Modality specific U-Net variants for biomedical image segmentation: a survey

With the advent of advancements in deep learning approaches, such as deep convolution neural network, residual neural network, adversarial network; U-Net architectures are most widely utilized in biomedical image segmentation to address the automation in identification and detection of the target regions or sub-regions. In recent studies, U-Net based approaches have illustrated state-of-the-art performance in different applications for the development of computer-aided diagnosis systems for early diagnosis and treatment of diseases such as brain tumor, lung cancer, alzheimer, breast cancer, etc., using various modalities. This article contributes in presenting the success of these approaches by describing the U-Net framework, followed by the comprehensive analysis of the U-Net variants by performing (1) inter-modality, and (2) intra-modality categorization to establish better insights into the associated challenges and solutions. Besides, this article also highlights the contribution of U-Net based frameworks in the ongoing pandemic, severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) also known as COVID-19. Finally, the strengths and similarities of these U-Net variants are analysed along with the challenges involved in biomedical image segmentation to uncover promising future research directions in this area.

Self-Supervised Bi-channel Transformer Networks for Computer-Aided Diagnosis

Self-supervised learning (SSL) can alleviate the issue of small sample size, which has shown its effectiveness for the computer-aided diagnosis (CAD) models. However, since the conventional SSL methods share the identical backbone in both the pretext and downstream tasks, the pretext network generally cannot be well trained in the pre-training stage, if the pretext task is totally different from the downstream one. In this work, we propose a novel task-driven SSL method, namely Self-Supervised Bi-channel Transformer Networks (SSBTN), to improve the diagnostic accuracy of a CAD model by enhancing SSL flexibility. In SSBTN, we innovatively integrate two different networks for the pretext and downstream tasks, respectively, into a unified framework. Consequently, the pretext task can be flexibly designed based on the data characteristics, and the corresponding designed pretext network thus learns more effective feature representation to be transferred to the downstream network. Furthermore, a transformer-based transfer module is developed to efficiently enhance knowledge transfer by conducting feature alignment between two different networks. The proposed SSBTN is evaluated on two publicly available datasets, namely the full-field digital mammography INbreast dataset and the wireless video capsule CrohnIPI dataset. The experimental results indicate that the proposed SSBTN outperforms all the compared algorithms.

A Review of Deep Learning Methods for Compressed Sensing Image Reconstruction and Its Medical Applications

Compressed sensing (CS) and its medical applications are active areas of research. In this paper, we review recent works using deep learning method to solve CS problem for images or medical imaging reconstruction including computed tomography (CT), magnetic resonance imaging (MRI) and positron-emission tomography (PET). We propose a novel framework to unify traditional iterative algorithms and deep learning approaches. In short, we define two projection operators toward image prior and data consistency, respectively, and any reconstruction algorithm can be decomposed to the two parts. Though deep learning methods can be divided into several categories, they all satisfies the framework. We built the relationship between different reconstruction methods of deep learning, and connect them to traditional methods through the proposed framework. It also indicates that the key to solve CS problem and its medical applications is how to depict the image prior. Based on the framework, we analyze the current deep learning methods and point out some important directions of research in the future.

Learning a Fully Connected U-Net for Spectrum Reconstruction of Fourier Transform Imaging Spectrometers

Fourier transform imaging spectrometers (FTISs) are widely used in global hyperspectral remote sensing due to the advantages of high stability, high throughput, and high spectral resolution. Spectrum reconstruction (SpecR) is a classic problem of FTISs determining the acquired data quality and application potential. However, the state-of-the-art SpecR algorithms were restricted by the length of maximum optical path difference (MOPD) of FTISs and apodization processing, resulting in a decrease in spectral resolution; thus, the applications of FTISs were limited. In this study, a deep learning SpecR method, which directly learned an end-to-end mapping between the interference/spectrum information with limited MOPD and without apodization processing, was proposed. The mapping was represented as a fully connected U-Net (FCUN) that takes the interference fringes as the input and outputs the highly precise spectral curves. We trained the proposed FCUN model using the real spectra and simulated pulse spectra, as well as the corresponding simulated interference curves, and achieved good results. Additionally, the performance of the proposed FCUN on real interference and spectral datasets was explored. The FCUN could obtain similar spectral values compared with the state-of-the-art fast Fourier transform (FFT)-based method with only 150 and 200 points in the interferograms. The proposed method could be able to enhance the resolution of the reconstructed spectra in the case of insufficient MOPD. Moreover, the FCUN performed well in visual quality using noisy interferograms and gained nearly 70% to 80% relative improvement over FFT for the coefficient of mean relative error (MRE). All the results based on simulated and real satellite datasets showed that the reconstructed spectra of the FCUN were more consistent with the ideal spectrum compared with that of the traditional method, with higher PSNR and lower values of spectral angle (SA) and relative spectral quadratic error (RQE).

Compound W-Net with Fully Accumulative Residual Connections for Liver Segmentation Using CT Images

Computed tomography (CT) is a common modality for liver diagnosis, treatment, and follow-up process. Providing accurate liver segmentation using CT images is a crucial step towards those tasks. In this paper, we propose a stacked 2-U-Nets model with three different types of skip connections. The proposed connections work to recover the loss of high-level features on the convolutional path of the first U-Net due to the pooling and the loss of low-level features during the upsampling path of the first U-Net. The skip connections concatenate all the features that are generated at the same level from the previous paths to the inputs of the convolutional layers in both paths of the second U-Net in a densely connected manner. We implement two versions of the model with different number of filters at each level of each U-Net by maximising the Dice similarity between the predicted liver region and that of the ground truth. The proposed models were trained with 3Dircadb public dataset that were released for Sliver and 3D liver and tumour segmentation challenges during MICCAI 2007-2008 challenge. The experimental results show that the proposed model outperformed the original U-Net and 2-U-Nets variants, and is comparable to the state-of-the-art mU-Net, DC U-Net, and Cascaded UNET.

Enhancement of 4-D Cone-Beam Computed Tomography (4D-CBCT) Using a Dual-Encoder Convolutional Neural Network (DeCNN)

4D-CBCT is a powerful tool to provide respiration-resolved images for the moving target localization. However, projections in each respiratory phase are intrinsically under-sampled under the clinical scanning time and imaging dose constraints. Images reconstructed by compressed sensing (CS)-based methods suffer from blurred edges. Introducing the average-4D-image constraint to the CS-based reconstruction, such as prior-image-constrained CS (PICCS), can improve the edge sharpness of the stable structures. However, PICCS can lead to motion artifacts in the moving regions. In this study, we proposed a dual-encoder convolutional neural network (DeCNN) to realize the average-image-constrained 4D-CBCT reconstruction. The proposed DeCNN has two parallel encoders to extract features from both the under-sampled target phase images and the average images. The features are then concatenated and fed into the decoder for the high-quality target phase image reconstruction. The reconstructed 4D-CBCT using of the proposed DeCNN from the real lung cancer patient data showed (1) qualitatively, clear and accurate edges for both stable and moving structures; (2) quantitatively, low-intensity errors, high peak signal-to-noise ratio, and high structural similarity compared to the ground truth images; and (3) superior quality to those reconstructed by several other state-of-the-art methods including the back-projection, CS total-variation, PICCS, and the single-encoder CNN. Overall, the proposed DeCNN is effective in exploiting the average-image constraint to improve the 4D-CBCT image quality.

Artificial Intelligence in Radiation Therapy

Artificial intelligence (AI) has great potential to transform the clinical workflow of radiotherapy. Since the introduction of deep neural networks, many AI-based methods have been proposed to address challenges in different aspects of radiotherapy. Commercial vendors have started to release AI-based tools that can be readily integrated to the established clinical workflow. To show the recent progress in AI-aided radiotherapy, we have reviewed AI-based studies in five major aspects of radiotherapy including image reconstruction, image registration, image segmentation, image synthesis, and automatic treatment planning. In each section, we summarized and categorized the recently published methods, followed by a discussion of the challenges, concerns, and future development. Given the rapid development of AI-aided radiotherapy, the efficiency and effectiveness of radiotherapy in the future could be substantially improved through intelligent automation of various aspects of radiotherapy.

Report on the AAPM deep-learning sparse-view CT (DL-sparse-view CT) Grand Challenge

PURPOSE

The purpose of the challenge is to find the deep-learning technique for sparse-view CT image reconstruction that can yield the minimum RMSE under ideal conditions, thereby addressing the question of whether or not deep learning can solve inverse problems in imaging.

METHODS

The challenge set-up involves a 2D breast CT simulation, where the simulated breast phantom has random fibro-glandular structure and high-contrast specks. The phantom allows for arbitrarily large training sets to be generated with perfectly known truth. The training set consists of 4000 cases where each case consists of the truth image, 128-view sinogram data, and the corresponding 128-view filtered back-projection (FBP) image. The networks are trained to predict the truth image from either the sinogram or FBP data. Geometry information is not provided. The participating algorithms are tested on a data set consisting of 100 new cases.

RESULTS

About 60 groups participated in the challenge at the validation phase, and 25 groups submitted test-phase results along with reports on their deep-learning methodology. The winning team improved reconstruction accuracy by two orders of magnitude over our previous CNN-based study on a similar test problem.

CONCLUSIONS

The DL-sparse-view challenge provides a unique opportunity to examine the state-of-the-art in deep-learning techniques for solving the sparse-view CT inverse problem. This article is protected by copyright. All rights reserved.

An Overview of Organs-on-Chips Based on Deep Learning

Microfluidic-based organs-on-chips (OoCs) are a rapidly developing technology in biomedical and chemical research and have emerged as one of the most advanced and promising in vitro models. The miniaturization, stimulated tissue mechanical forces, and microenvironment of OoCs offer unique properties for biomedical applications. However, the large amount of data generated by the high parallelization of OoC systems has grown far beyond the scope of manual analysis by researchers with biomedical backgrounds. Deep learning, an emerging area of research in the field of machine learning, can automatically mine the inherent characteristics and laws of "big data" and has achieved remarkable applications in computer vision, speech recognition, and natural language processing. The integration of deep learning in OoCs is an emerging field that holds enormous potential for drug development, disease modeling, and personalized medicine. This review briefly describes the basic concepts and mechanisms of microfluidics and deep learning and summarizes their successful integration. We then analyze the combination of OoCs and deep learning for image digitization, data analysis, and automation. Finally, the problems faced in current applications are discussed, and future perspectives and suggestions are provided to further strengthen this integration.

Fluoroscopic 3D Image Generation from Patient-Specific PCA Motion Models Derived from 4D-CBCT Patient Datasets: A Feasibility Study

A method for generating fluoroscopic (time-varying) volumetric images using patient-specific motion models derived from four-dimensional cone-beam CT (4D-CBCT) images was developed. 4D-CBCT images acquired immediately prior to treatment have the potential to accurately represent patient anatomy and respiration during treatment. Fluoroscopic 3D image estimation is performed in two steps: (1) deriving motion models and (2) optimization. To derive motion models, every phase in a 4D-CBCT set is registered to a reference phase chosen from the same set using deformable image registration (DIR). Principal components analysis (PCA) is used to reduce the dimensionality of the displacement vector fields (DVFs) resulting from DIR into a few vectors representing organ motion found in the DVFs. The PCA motion models are optimized iteratively by comparing a cone-beam CT (CBCT) projection to a simulated projection computed from both the motion model and a reference 4D-CBCT phase, resulting in a sequence of fluoroscopic 3D images. Patient datasets were used to evaluate the method by estimating the tumor location in the generated images compared to manually defined ground truth positions. Experimental results showed that the average tumor mean absolute error (MAE) along the superior–inferior (SI) direction and the 95th percentile in two patient datasets were 2.29 and 5.79 mm for patient 1, and 1.89 and 4.82 mm for patient 2. This study demonstrated the feasibility of deriving 4D-CBCT-based PCA motion models that have the potential to account for the 3D non-rigid patient motion and localize tumors and other patient anatomical structures on the day of treatment.

Y-Net: a dual-branch deep learning network for nonlinear absorption tomography with wavelength modulation spectroscopy

This paper demonstrates a new method for solving nonlinear tomographic problems, combining calibration-free wavelength modulation spectroscopy (CF-WMS) with a dual-branch deep learning network (Y-Net). The principle of CF-WMS, as well as the architecture, training and performance of Y-Net have been investigated. 20000 samples are randomly generated, with each temperature or H₂O concentration phantom featuring three randomly positioned Gaussian distributions. Non-uniformity coefficient (NUC) method provides quantitative characterizations of the non-uniformity (i.e., the complexity) of the reconstructed fields. Four projections, each with 24 parallel beams are assumed. The average reconstruction errors of temperature and H₂O concentration for the testing dataset with 2000 samples are 1.55% and 2.47%, with standard deviations of 0.46% and 0.75%, respectively. The reconstruction errors for both temperature and species concentration distributions increase almost linearly with increasing NUC from 0.02 to 0.20. The proposed Y-Net shows great advantages over the state-of-the-art simulated annealing algorithm, such as better noise immunity and higher computational efficiency. This is the first time, to the best of our knowledge, that a dual-branch deep learning network (Y-Net) has been applied to WMS-based nonlinear tomography and it opens up opportunities for real-time, in situ monitoring of practical combustion environments.

Methodology for Interactive Labeling of Patched Asphalt Pavement Images Based on U-Net Convolutional Neural Network

Image recognition based on deep learning generally demands a huge sample size for training, for which the image labeling becomes inevitably laborious and time-consuming. In the case of evaluating the pavement quality condition, many pavement distress patching images would need manual screening and labeling, meanwhile the subjectivity of the labeling personnel would greatly affect the accuracy of image labeling. In this study, in order for an accurate and efficient recognition of the pavement patching images, an interactive labeling method is proposed based on the U-Net convolutional neural network, using active learning combined with reverse and correction labeling. According to the calculation results in this paper, the sample size required by the interactive labeling is about half of the traditional labeling method for the same recognition precision. Meanwhile, the accuracy of interactive labeling method based on the mean intersection over union (mean_IOU) index is 6% higher than that of the traditional method using the same sample size and training epochs. In addition, the accuracy analysis of the noise and boundary of the prediction results shows that this method eliminates 92% of the noise in the predictions (the proportion of noise is reduced from 13.85% to 1.06%), and the image definition is improved by 14.1% in terms of the boundary gray area ratio. The interactive labeling is considered as a significantly valuable approach, as it reduces the sample size in each epoch of active learning, greatly alleviates the demand for manpower, and improves learning efficiency and accuracy.

Three-Dimensional Liver Image Segmentation Using Generative Adversarial Networks Based on Feature Restoration

Medical imaging provides a powerful tool for medical diagnosis. In the process of computer-aided diagnosis and treatment of liver cancer based on medical imaging, accurate segmentation of liver region from abdominal CT images is an important step. However, due to defects of liver tissue and limitations of CT imaging procession, the gray level of liver region in CT image is heterogeneous, and the boundary between the liver and those of adjacent tissues and organs is blurred, which makes the liver segmentation an extremely difficult task. In this study, aiming at solving the problem of low segmentation accuracy of the original 3D U-Net network, an improved network based on the three-dimensional (3D) U-Net, is proposed. Moreover, in order to solve the problem of insufficient training data caused by the difficulty of acquiring labeled 3D data, an improved 3D U-Net network is embedded into the framework of generative adversarial networks (GAN), which establishes a semi-supervised 3D liver segmentation optimization algorithm. Finally, considering the problem of poor quality of 3D abdominal fake images generated by utilizing random noise as input, deep convolutional neural networks (DCNN) based on feature restoration method is designed to generate more realistic fake images. By testing the proposed algorithm on the LiTS-2017 and KiTS19 dataset, experimental results show that the proposed semi-supervised 3D liver segmentation method can greatly improve the segmentation performance of liver, with a Dice score of 0.9424 outperforming other methods.

DuDoDR-Net: Dual-domain data consistent recurrent network for simultaneous sparse view and metal artifact reduction in computed tomography

Sparse-view computed tomography (SVCT) aims to reconstruct a cross-sectional image using a reduced number of x-ray projections. While SVCT can efficiently reduce the radiation dose, the reconstruction suffers from severe streak artifacts, and the artifacts are further amplified with the presence of metallic implants, which could adversely impact the medical diagnosis and other downstream applications. Previous methods have extensively explored either SVCT reconstruction without metallic implants, or full-view CT metal artifact reduction (MAR). The issue of simultaneous sparse-view and metal artifact reduction (SVMAR) remains under-explored, and it is infeasible to directly apply previous SVCT and MAR methods to SVMAR which may yield non-ideal reconstruction quality. In this work, we propose a dual-domain data consistent recurrent network, called DuDoDR-Net, for SVMAR. Our DuDoDR-Net aims to reconstruct an artifact-free image by recurrent image domain and sinogram domain restorations. To ensure the metal-free part of acquired projection data is preserved, we also develop the image data consistent layer (iDCL) and sinogram data consistent layer (sDCL) that are interleaved in our recurrent framework. Our experimental results demonstrate that our DuDoDR-Net is able to produce superior artifact-reduced results while preserving the anatomical structures, that outperforming previous SVCT and SVMAR methods, under different sparse-view acquisition settings.

Review and Prospect: Artificial Intelligence in Advanced Medical Imaging

Artificial intelligence (AI) as an emerging technology is gaining momentum in medical imaging. Recently, deep learning-based AI techniques have been actively investigated in medical imaging, and its potential applications range from data acquisition and image reconstruction to image analysis and understanding. In this review, we focus on the use of deep learning in image reconstruction for advanced medical imaging modalities including magnetic resonance imaging (MRI), computed tomography (CT), and positron emission tomography (PET). Particularly, recent deep learning-based methods for image reconstruction will be emphasized, in accordance with their methodology designs and performances in handling volumetric imaging data. It is expected that this review can help relevant researchers understand how to adapt AI for medical imaging and which advantages can be achieved with the assistance of AI.

Degradation-Aware Deep Learning Framework for Sparse-View CT Reconstruction

Sparse-view CT reconstruction is a fundamental task in computed tomography to overcome undesired artifacts and recover the details of textual structure in degraded CT images. Recently, many deep learning-based networks have achieved desirable performances compared to iterative reconstruction algorithms. However, the performance of these methods may severely deteriorate when the degradation strength of the test image is not consistent with that of the training dataset. In addition, these methods do not pay enough attention to the characteristics of different degradation levels, so solely extending the training dataset with multiple degraded images is also not effective. Although training plentiful models in terms of each degradation level can mitigate this problem, extensive parameter storage is involved. Accordingly, in this paper, we focused on sparse-view CT reconstruction for multiple degradation levels. We propose a single degradation-aware deep learning framework to predict clear CT images by understanding the disparity of degradation in both the frequency domain and image domain. The dual-domain procedure can perform particular operations at different degradation levels in frequency component recovery and spatial details reconstruction. The peak signal-to-noise ratio (PSNR), structural similarity (SSIM) and visual results demonstrate that our method outperformed the classical deep learning-based reconstruction methods in terms of effectiveness and scalability.

Generalized deep iterative reconstruction for sparse-view CT imaging

Sparse-view CT is a promising approach in reducing the X-ray radiation dose in clinical CT imaging. However, the CT images reconstructed from the conventional filtered backprojection (FBP) algorithm suffer from severe streaking artifacts. Iterative reconstruction (IR) algorithms have been widely adopted to mitigate these streaking artifacts, but they may prolong the CT imaging time due to the intense data-specific computations. Recently, model-driven deep learning (DL) CT image reconstruction method, which unrolls the iterative optimization procedures into the deep neural network, has shown exciting prospect in improving the image quality and shortening the reconstruction time. In this work, we explore the generalized unrolling scheme for such iterative model to further enhance its performance on sparse-view CT imaging. By using it, the iteration parameters, regularizer term, data-fidelity term and even the mathematical operations are all assumed to be learned and optimized via the network training. Results from the numerical and experimental sparse-view CT imaging demonstrate that the newly proposed network with the maximum generalization provides the best reconstruction performance.

On Hallucinations in Tomographic Image Reconstruction

Tomographic image reconstruction is generally an ill-posed linear inverse problem. Such ill-posed inverse problems are typically regularized using prior knowledge of the sought-after object property. Recently, deep neural networks have been actively investigated for regularizing image reconstruction problems by learning a prior for the object properties from training images. However, an analysis of the prior information learned by these deep networks and their ability to generalize to data that may lie outside the training distribution is still being explored. An inaccurate prior might lead to false structures being hallucinated in the reconstructed image and that is a cause for serious concern in medical imaging. In this work, we propose to illustrate the effect of the prior imposed by a reconstruction method by decomposing the image estimate into generalized measurement and null components. The concept of a hallucination map is introduced for the general purpose of understanding the effect of the prior in regularized reconstruction methods. Numerical studies are conducted corresponding to a stylized tomographic imaging modality. The behavior of different reconstruction methods under the proposed formalism is discussed with the help of the numerical studies.

DRONE: Dual-Domain Residual-based Optimization NEtwork for Sparse-View CT Reconstruction

Deep learning has attracted rapidly increasing attention in the field of tomographic image reconstruction, especially for CT, MRI, PET/SPECT, ultrasound and optical imaging. Among various topics, sparse-view CT remains a challenge which targets a decent image reconstruction from very few projections. To address this challenge, in this article we propose a Dual-domain Residual-based Optimization NEtwork (DRONE). DRONE consists of three modules respectively for embedding, refinement, and awareness. In the embedding module, a sparse sinogram is first extended. Then, sparse-view artifacts are effectively suppressed in the image domain. After that, the refinement module recovers image details in the residual data and image domains synergistically. Finally, the results from the embedding and refinement modules in the data and image domains are regularized for optimized image quality in the awareness module, which ensures the consistency between measurements and images with the kernel awareness of compressed sensing. The DRONE network is trained, validated, and tested on preclinical and clinical datasets, demonstrating its merits in edge preservation, feature recovery, and reconstruction accuracy.

Learning to Reconstruct CT Images From the VVBP-Tensor

Deep learning (DL) is bringing a big movement in the field of computed tomography (CT) imaging. In general, DL for CT imaging can be applied by processing the projection or the image data with trained deep neural networks (DNNs), unrolling the iterative reconstruction as a DNN for training, or training a well-designed DNN to directly reconstruct the image from the projection. In all of these applications, the whole or part of the DNNs work in the projection or image domain alone or in combination. In this study, instead of focusing on the projection or image, we train DNNs to reconstruct CT images from the view-by-view backprojection tensor (VVBP-Tensor). The VVBP-Tensor is the 3D data before summation in backprojection. It contains structures of the scanned object after applying a sorting operation. Unlike the image or projection that provides compressed information due to the integration/summation step in forward or back projection, the VVBP-Tensor provides lossless information for processing, allowing the trained DNNs to preserve fine details of the image. We develop a learning strategy by inputting slices of the VVBP-Tensor as feature maps and outputting the image. Such strategy can be viewed as a generalization of the summation step in conventional filtered backprojection reconstruction. Numerous experiments reveal that the proposed VVBP-Tensor domain learning framework obtains significant improvement over the image, projection, and hybrid projection-image domain learning frameworks. We hope the VVBP-Tensor domain learning framework could inspire algorithm development for DL-based CT imaging.

Data Extrapolation From Learned Prior Images for Truncation Correction in Computed Tomography

Data truncation is a common problem in computed tomography (CT). Truncation causes cupping artifacts inside the field-of-view (FOV) and anatomical structures missing outside the FOV. Deep learning has achieved impressive results in CT reconstruction from limited data. However, its robustness is still a concern for clinical applications. Although the image quality of learning-based compensation schemes may be inadequate for clinical diagnosis, they can provide prior information for more accurate extrapolation than conventional heuristic extrapolation methods. With extrapolated projection, a conventional image reconstruction algorithm can be applied to obtain a final reconstruction. In this work, a general plug-and-play (PnP) method for truncation correction is proposed based on this idea, where various deep learning methods and conventional reconstruction algorithms can be plugged in. Such a PnP method integrates data consistency for measured data and learned prior image information for truncated data. This shows to have better robustness and interpretability than deep learning only. To demonstrate the efficacy of the proposed PnP method, two state-of-the-art deep learning methods, FBPConvNet and Pix2pixGAN, are investigated for truncation correction in cone-beam CT in noise-free and noisy cases. Their robustness is evaluated by showing false negative and false positive lesion cases. With our proposed PnP method, false lesion structures are corrected for both deep learning methods. For FBPConvNet, the root-mean-square error (RMSE) inside the FOV can be improved from 92HU to around 30HU by PnP in the noisy case. Pix2pixGAN solely achieves better image quality than FBPConvNet solely for truncation correction in general. PnP further improves the RMSE inside the FOV from 42HU to around 27HU for Pix2pixGAN. The efficacy of PnP is also demonstrated on real clinical head data.

Analysis and Evaluation of a Deep Learning Reconstruction Approach with Denoising for Orthopedic MRI

PURPOSE

To evaluate two settings (noise reduction of 50% or 75%) of a deep learning (DL) reconstruction model relative to each other and to conventional MR image reconstructions on clinical orthopedic MRI datasets.

MATERIALS AND METHODS

This retrospective study included 54 patients who underwent two-dimensional fast spin-echo MRI for hip (n = 22; mean age, 44 years ± 13 [standard deviation]; nine men) or shoulder (n = 32; mean age, 56 years ± 17; 17 men) conditions between March 2019 and June 2020. MR images were reconstructed with conventional methods and the vendor-provided and commercially available DL model applied with 50% and 75% noise reduction settings (DL 50 and DL 75, respectively). Quantitative analytics, including relative anatomic edge sharpness, relative signal-to-noise ratio (rSNR), and relative contrast-to-noise ratio (rCNR) were computed for each dataset. In addition, the image sets were randomized, blinded, and presented to three board-certified musculoskeletal radiologists for ranking based on overall image quality and diagnostic confidence. Statistical analysis was performed with a nonparametric hypothesis comparing derived quantitative metrics from each reconstruction approach. In addition, inter- and intrarater agreement analysis was performed on the radiologists' rankings.

RESULTS

Both denoising settings of the DL reconstruction showed improved edge sharpness, rSNR, and rCNR relative to the conventional reconstructions. The reader rankings demonstrated strong agreement, with both DL reconstructions outperforming the conventional approach (Gwet agreement coefficient = 0.98). However, there was lower agreement between the readers on which DL reconstruction denoising setting produced higher-quality images (Gwet agreement coefficient = 0.31 for DL 50 and 0.35 for DL 75).

CONCLUSION

The vendor-provided DL MRI reconstruction showed higher edge sharpness, rSNR, and rCNR in comparison with conventional methods; however, optimal levels of denoising may need to be further assessed.Keywords: MRI Reconstruction Method, Deep Learning, Image Analysis, Signal-to-Noise Ratio, MR-Imaging, Neural Networks, Hip, Shoulder, Physics, Observer Performance, Technology Assessment Supplemental material is available for this article. © RSNA, 2021.

CycN-Net: A Convolutional Neural Network Specialized for 4D CBCT Images Refinement

Four-dimensional cone-beam computed tomography (4D CBCT) has been developed to provide a sequence of phase-resolved reconstructions in image-guided radiation therapy. However, 4D CBCT images are degraded by severe streaking artifacts and noise because the phase-resolved image is an extremely sparse-view CT procedure wherein a few under-sampled projections are used for the reconstruction of each phase. Aiming at improving the overall quality of 4D CBCT images, we proposed two CNN models, named N-Net and CycN-Net, respectively, by fully excavating the inherent property of 4D CBCT. To be specific, the proposed N-Net incorporates the prior image reconstructed from entire projection data based on U-Net to boost the image quality for each phase-resolved image. Based on N-Net, a temporal correlation among the phase-resolved images is also considered by the proposed CycN-Net. Extensive experiments on both XCAT simulation data and real patient 4D CBCT datasets were carried out to verify the feasibility of the proposed CNNs. Both networks can effectively suppress streaking artifacts and noise while restoring the distinct features simultaneously, compared with the existing CNN models and two state-of-the-art iterative algorithms. Moreover, the proposed method is robust in handling complicated tasks of various patient datasets and imaging devices, which implies its excellent generalization ability.

Tile-net for undersampled cardiovascular CINE magnetic resonance imaging

We propose the "Tile-net" method based on dividing an image into smaller tiles. Using the tile as the input to the neural network, the network is simplified substantially. The Tile-net learns at a much faster rate than the networks without tiling. The training and reconstruction times for the Tile-net are reduced by 40% and 33%, respectively compared to the networks without tiling. The Tile-net performance is evaluated through the normalized mean square error (NMSE), peak signal to noise ratio (PSNR), structure similarity index measure (SSIM) and the quality of the reconstructed image for test datasets. The Tile-net does not degrade performance; however, it reduces the NMSE by 0.3% compared to the networks without tiling.

Machine learning-based leaky momentum prediction of plasmonic random nanosubstrate

In this work, we explore the use of machine learning for constructing the leakage radiation characteristics of the bright-field images of nanoislands from surface plasmon polariton based on the plasmonic random nanosubstrate. The leakage radiation refers to a leaky wave of surface plasmon polariton (SPP) modes through a dielectric substrate which has drawn interest due to its possibility of direct visualization and analysis of SPP propagation. A fast-learning two-layer neural network has been deployed to learn and predict the relationship between the leakage radiation characteristics and the bright-field images of nanoislands utilizing a limited number of training samples. The proposed learning framework is expected to significantly simplify the process of leaky radiation image construction without the need of sophisticated equipment. Moreover, a wide range of application extensions can be anticipated for the proposed image-to-image prediction.

CT Noise-Reduction Methods for Lower-Dose Scanning: Strengths and Weaknesses of Iterative Reconstruction Algorithms and New Techniques

Iterative reconstruction (IR) algorithms are the most widely used CT noise-reduction method to improve image quality and have greatly facilitated radiation dose reduction within the radiology community. Various IR methods have different strengths and limitations. Because IR algorithms are typically nonlinear, they can modify spatial resolution and image noise texture in different regions of the CT image; hence traditional image-quality metrics are not appropriate to assess the ability of IR to preserve diagnostic accuracy, especially for low-contrast diagnostic tasks. In this review, the authors highlight emerging IR algorithms and CT noise-reduction techniques and summarize how these techniques can be evaluated to help determine the appropriate radiation dose levels for different diagnostic tasks in CT. In addition to advanced IR techniques, we describe novel CT noise-reduction methods based on convolutional neural networks (CNNs). CNN-based noise-reduction techniques may offer the ability to reduce image noise while maintaining high levels of image detail but may have unique drawbacks. Other novel CT noise-reduction methods are being developed to leverage spatial and/or spectral redundancy in multiphase or multienergy CT. Radiologists and medical physicists should be familiar with these different alternatives to adapt available CT technology for different diagnostic tasks. The scope of this article is (a) to review the clinical applications of IR algorithms as well as their strengths, weaknesses, and methods of assessment and (b) to explore new CT image reconstruction and noise-reduction techniques that promise to facilitate radiation dose reduction. ^©RSNA, 2021.

Accurate and robust sparse-view angle CT image reconstruction using deep learning and prior image constrained compressed sensing (DL-PICCS)

BACKGROUND

Sparse-view CT image reconstruction problems encountered in dynamic CT acquisitions are technically challenging. Recently, many deep learning strategies have been proposed to reconstruct CT images from sparse-view angle acquisitions showing promising results. However, two fundamental problems with these deep learning reconstruction methods remain to be addressed: (1) limited reconstruction accuracy for individual patients and (2) limited generalizability for patient statistical cohorts.

PURPOSE

The purpose of this work is to address the previously mentioned challenges in current deep learning methods.

METHODS

A method that combines a deep learning strategy with prior image constrained compressed sensing (PICCS) was developed to address these two problems. In this method, the sparse-view CT data were reconstructed by the conventional filtered backprojection (FBP) method first, and then processed by the trained deep neural network to eliminate streaking artifacts. The outputs of the deep learning architecture were then used as the needed prior image in PICCS to reconstruct the image. If the noise level from the PICCS reconstruction is not satisfactory, another light duty deep neural network can then be used to reduce noise level. Both extensive numerical simulation data and human subject data have been used to quantitatively and qualitatively assess the performance of the proposed DL-PICCS method in terms of reconstruction accuracy and generalizability.

RESULTS

Extensive evaluation studies have demonstrated that: (1) quantitative reconstruction accuracy of DL-PICCS for individual patient is improved when it is compared with the deep learning methods and CS-based methods; (2) the false-positive lesion-like structures and false negative missing anatomical structures in the deep learning approaches can be effectively eliminated in the DL-PICCS reconstructed images; and (3) DL-PICCS enables a deep learning scheme to relax its working conditions to enhance its generalizability.

CONCLUSIONS

DL-PICCS offers a promising opportunity to achieve personalized reconstruction with improved reconstruction accuracy and enhanced generalizability.

A Green Prospective for Learned Post-Processing in Sparse-View Tomographic Reconstruction

Deep Learning is developing interesting tools that are of great interest for inverse imaging applications. In this work, we consider a medical imaging reconstruction task from subsampled measurements, which is an active research field where Convolutional Neural Networks have already revealed their great potential. However, the commonly used architectures are very deep and, hence, prone to overfitting and unfeasible for clinical usages. Inspired by the ideas of the green AI literature, we propose a shallow neural network to perform efficient Learned Post-Processing on images roughly reconstructed by the filtered backprojection algorithm. The results show that the proposed inexpensive network computes images of comparable (or even higher) quality in about one-fourth of time and is more robust than the widely used and very deep ResUNet for tomographic reconstructions from sparse-view protocols.

Advances in micro-CT imaging of small animals

PURPOSE

Micron-scale computed tomography (micro-CT) imaging is a ubiquitous, cost-effective, and non-invasive three-dimensional imaging modality. We review recent developments and applications of micro-CT for preclinical research.

METHODS

Based on a comprehensive review of recent micro-CT literature, we summarize features of state-of-the-art hardware and ongoing challenges and promising research directions in the field.

RESULTS

Representative features of commercially available micro-CT scanners and some new applications for both in vivo and ex vivo imaging are described. New advancements include spectral scanning using dual-energy micro-CT based on energy-integrating detectors or a new generation of photon-counting x-ray detectors (PCDs). Beyond two-material discrimination, PCDs enable quantitative differentiation of intrinsic tissues from one or more extrinsic contrast agents. When these extrinsic contrast agents are incorporated into a nanoparticle platform (e.g. liposomes), novel micro-CT imaging applications are possible such as combined therapy and diagnostic imaging in the field of cancer theranostics. Another major area of research in micro-CT is in x-ray phase contrast (XPC) imaging. XPC imaging opens CT to many new imaging applications because phase changes are more sensitive to density variations in soft tissues than standard absorption imaging. We further review the impact of deep learning on micro-CT. We feature several recent works which have successfully applied deep learning to micro-CT data, and we outline several challenges specific to micro-CT.

CONCLUSIONS

All of these advancements establish micro-CT imaging at the forefront of preclinical research, able to provide anatomical, functional, and even molecular information while serving as a testbench for translational research.

Accurate Transmission-Less Attenuation Correction Method for Amyloid-β Brain PET Using Deep Neural Network

The lack of physically measured attenuation maps (μ-maps) for attenuation and scatter correction is an important technical challenge in brain-dedicated stand-alone positron emission tomography (PET) scanners. The accuracy of the calculated attenuation correction is limited by the nonuniformity of tissue composition due to pathologic conditions and the complex structure of facial bones. The aim of this study is to develop an accurate transmission-less attenuation correction method for amyloid-β (Aβ) brain PET studies. We investigated the validity of a deep convolutional neural network trained to produce a CT-derived μ-map (μ-CT) from simultaneously reconstructed activity and attenuation maps using the MLAA (maximum likelihood reconstruction of activity and attenuation) algorithm for Aβ brain PET. The performance of three different structures of U-net models (2D, 2.5D, and 3D) were compared. The U-net models generated less noisy and more uniform μ-maps than MLAA μ-maps. Among the three different U-net models, the patch-based 3D U-net model reduced noise and cross-talk artifacts more effectively. The Dice similarity coefficients between the μ-map generated using 3D U-net and μ-CT in bone and air segments were 0.83 and 0.67. All three U-net models showed better voxel-wise correlation of the μ-maps compared to MLAA. The patch-based 3D U-net model was the best. While the uptake value of MLAA yielded a high percentage error of 20% or more, the uptake value of 3D U-nets yielded the lowest percentage error within 5%. The proposed deep learning approach that requires no transmission data, anatomic image, or atlas/template for PET attenuation correction remarkably enhanced the quantitative accuracy of the simultaneously estimated MLAA μ-maps from Aβ brain PET.

Limited View Tomographic Reconstruction Using a Cascaded Residual Dense Spatial-Channel Attention Network With Projection Data Fidelity Layer

Limited view tomographic reconstruction aims to reconstruct a tomographic image from a limited number of projection views arising from sparse view or limited angle acquisitions that reduce radiation dose or shorten scanning time. However, such a reconstruction suffers from severe artifacts due to the incompleteness of sinogram. To derive quality reconstruction, previous methods use UNet-like neural architectures to directly predict the full view reconstruction from limited view data; but these methods leave the deep network architecture issue largely intact and cannot guarantee the consistency between the sinogram of the reconstructed image and the acquired sinogram, leading to a non-ideal reconstruction. In this work, we propose a cascaded residual dense spatial-channel attention network consisting of residual dense spatial-channel attention networks and projection data fidelity layers. We evaluate our methods on two datasets. Our experimental results on AAPM Low Dose CT Grand Challenge datasets demonstrate that our algorithm achieves a consistent and substantial improvement over the existing neural network methods on both limited angle reconstruction and sparse view reconstruction. In addition, our experimental results on Deep Lesion datasets demonstrate that our method is able to generate high-quality reconstruction for 8 major lesion types.

Self-supervised learning for accelerated 3D high-resolution ultrasound imaging

PURPOSE

Ultrasound (US) imaging has been widely used in diagnosis, image-guided intervention, and therapy, where high-quality three-dimensional (3D) images are highly desired from sparsely acquired two-dimensional (2D) images. This study aims to develop a deep learning-based algorithm to reconstruct high-resolution (HR) 3D US images only reliant on the acquired sparsely distributed 2D images.

METHODS

We propose a self-supervised learning framework using cycle-consistent generative adversarial network (cycleGAN), where two independent cycleGAN models are trained with paired original US images and two sets of low-resolution (LR) US images, respectively. The two sets of LR US images are obtained through down-sampling the original US images along the two axes, respectively. In US imaging, in-plane spatial resolution is generally much higher than through-plane resolution. By learning the mapping from down-sampled in-plane LR images to original HR US images, cycleGAN can generate through-plane HR images from original sparely distributed 2D images. Finally, HR 3D US images are reconstructed by combining the generated 2D images from the two cycleGAN models.

RESULTS

The proposed method was assessed on two different datasets. One is automatic breast ultrasound (ABUS) images from 70 breast cancer patients, the other is collected from 45 prostate cancer patients. By applying a spatial resolution enhancement factor of 3 to the breast cases, our proposed method achieved the mean absolute error (MAE) value of 0.90 ± 0.15, the peak signal-to-noise ratio (PSNR) value of 37.88 ± 0.88 dB, and the visual information fidelity (VIF) value of 0.69 ± 0.01, which significantly outperforms bicubic interpolation. Similar performances have been achieved using the enhancement factor of 5 in these breast cases and using the enhancement factors of 5 and 10 in the prostate cases.

CONCLUSIONS

We have proposed and investigated a new deep learning-based algorithm for reconstructing HR 3D US images from sparely acquired 2D images. Significant improvement on through-plane resolution has been achieved by only using the acquired 2D images without any external atlas images. Its self-supervision capability could accelerate HR US imaging.

Prenatal prediction and typing of placental invasion using MRI deep and radiomic features

BACKGROUND

To predict placental invasion (PI) and determine the subtype according to the degree of implantation, and to help physicians develop appropriate therapeutic measures, a prenatal prediction and typing of placental invasion method using MRI deep and radiomic features were proposed.

METHODS

The placental tissue of abdominal magnetic resonance (MR) image was segmented to form the regions of interest (ROI) using U-net. The radiomic features were subsequently extracted from ROI. Simultaneously, a deep dynamic convolution neural network (DDCNN) with codec structure was established, which was trained by an autoencoder model to extract the deep features from ROI. Finally, combining the radiomic features and deep features, a classifier based on the multi-layer perceptron model was designed. The classifier was trained to predict prenatal placental invasion as well as determine the invasion subtype.

RESULTS

The experimental results show that the average accuracy, sensitivity, and specificity of the proposed method are 0.877, 0.857, and 0.954 respectively, and the area under the ROC curve (AUC) is 0.904, which outperforms the traditional radiomic based auxiliary diagnostic methods.

CONCLUSIONS

This work not only labeled the placental tissue of MR image in pregnant women automatically but also realized the objective evaluation of placental invasion, thus providing a new approach for the prenatal diagnosis of placental invasion.

Do CNNs Solve the CT Inverse Problem?

OBJECTIVE

This work examines the claim made in the literature that the inverse problem associated with image reconstruction in sparse-view computed tomography (CT) can be solved with a convolutional neural network (CNN).

METHODS

Training, and testing image/data pairs are generated in a dedicated breast CT simulation for sparse-view sampling, using two different object models. The trained CNN is tested to see if images can be accurately recovered from their corresponding sparse-view data. For reference, the same sparse-view CT data is reconstructed by the use of constrained total-variation (TV) minimization (TVmin), which exploits sparsity in the gradient magnitude image (GMI).

RESULTS

There is a significant discrepancy between the image obtained with the CNN and the image that generated the data. TVmin is able to accurately reconstruct the test images.

CONCLUSION

We find that the sparse-view CT inverse problem cannot be solved for the particular published CNN-based methodology that we chose, and the particular object model that we tested.

SIGNIFICANCE

The inability of the CNN to solve the inverse problem associated with sparse-view CT, for the specific conditions of the presented simulation, draws into question similar unsupported claims being made for the use of CNNs and deep-learning to solve inverse problems in medical imaging.

Evaluating renal lesions using deep-learning based extension of dual-energy FoV in dual-source CT-A retrospective pilot study

PURPOSE

Dual-source (DS) CT, dual-energy (DE) field of view (FoV) is limited to the size of the smaller detector array. The purpose was to establish a deep learning-based approach to DE extrapolation by estimating missing image data using data from both tubes to evaluate renal lesions.

METHOD

A DE extrapolation deep-learning (DEEDL) algorithm had been trained on DECT data of 50 patients using a DSCT with DE-FoV = 33 cm (Somatom Flash). Data from 128 patients with known renal lesions falling within DE-FoV was retrospectively collected (100/140 kVp; reference dataset 1). A smaller DE-FoV = 20 cm was simulated excluding the renal lesion of interest (dataset 2) and the DEEDL was applied to this dataset. Output from the DEEDL algorithm was evaluated using ReconCT v14.1 and Syngo.via. Mean attenuation values in lesions on mixed images (HU) were compared calculating the root-mean-squared-error (RMSE) between the datasets using MATLAB R2019a.

RESULTS

The DEEDL algorithm performed well reproducing the image data of the kidney lesions (Bosniak 1 and 2: 125, Bosniak 2F: 6, Bosniak 3: 1 and Bosniak 4/(partially) solid: 32) with RSME values of 10.59 HU, 15.7 HU for attenuation, virtual non-contrast, respectively. The measurements performed in dataset 1 and 2 showed strong correlation with linear regression (r2: attenuation = 0.89, VNC = 0.63, iodine = 0.75), lesions were classified as enhancing with an accuracy of 0.91.

CONCLUSION

This DEEDL algorithm can be used to reconstruct a full dual-energy FoV from restricted data, enabling reliable HU value measurements in areas not covered by the smaller FoV and evaluation of renal lesions.

Low-dose CT imaging via cascaded ResUnet with spectrum loss

The suppression of artifact noise in computed tomography (CT) with a low-dose scan protocol is challenging. Conventional statistical iterative algorithms can improve reconstruction but cannot substantially eliminate large streaks and strong noise elements. In this paper, we present a 3D cascaded ResUnet neural network (Ca-ResUnet) strategy with modified noise power spectrum loss for reducing artifact noise in low-dose CT imaging. The imaging workflow consists of four components. The first component is filtered backprojection (FBP) reconstruction via a domain transformation module for suppressing artifact noise. The second is a ResUnet neural network that operates on the CT image. The third is an image compensation module that compensates for the loss of tiny structures, and the last is a second ResUnet neural network with modified spectrum loss for fine-tuning the reconstructed image. Verification results based on American Association of Physicists in Medicine (AAPM) and United Image Healthcare (UIH) datasets confirm that the proposed strategy significantly reduces serious artifact noise while retaining desired structures.

FISTA-Net: Learning a Fast Iterative Shrinkage Thresholding Network for Inverse Problems in Imaging

Inverse problems are essential to imaging applications. In this letter, we propose a model-based deep learning network, named FISTA-Net, by combining the merits of interpretability and generality of the model-based Fast Iterative Shrinkage/Thresholding Algorithm (FISTA) and strong regularization and tuning-free advantages of the data-driven neural network. By unfolding the FISTA into a deep network, the architecture of FISTA-Net consists of multiple gradient descent, proximal mapping, and momentum modules in cascade. Different from FISTA, the gradient matrix in FISTA-Net can be updated during iteration and a proximal operator network is developed for nonlinear thresholding which can be learned through end-to-end training. Key parameters of FISTA-Net including the gradient step size, thresholding value and momentum scalar are tuning-free and learned from training data rather than hand-crafted. We further impose positive and monotonous constraints on these parameters to ensure they converge properly. The experimental results, evaluated both visually and quantitatively, show that the FISTA-Net can optimize parameters for different imaging tasks, i.e. Electromagnetic Tomography (EMT) and X-ray Computational Tomography (X-ray CT). It outperforms the state-of-the-art model-based and deep learning methods and exhibits good generalization ability over other competitive learning-based approaches under different noise levels.

SPECTnet: a deep learning neural network for SPECT image reconstruction

Background

Single photon emission computed tomography (SPECT) is an important functional tool for clinical diagnosis and scientific research of brain disorders, but suffers from limited spatial resolution and high noise due to hardware design and imaging physics. The present study is to develop a deep learning technique for SPECT image reconstruction that directly converts raw projection data to image with high resolution and low noise, while an efficient training method specifically applicable to medical image reconstruction is presented.

Methods

A custom software was developed to generate 20,000 2-D brain phantoms, of which 16,000 were used to train the neural network, 2,000 for validation, and the final 2,000 for testing. To reduce development difficulty, a two-step training strategy for network design was adopted. We first compressed full-size activity image (128×128 pixels) to a one-D vector consisting of 256×1 pixels, accomplished by an autoencoder (AE) consisting of an encoder and a decoder. The vector is a good representation of the full-size image in a lower-dimensional space and was used as a compact label to develop the second network that maps between the projection-data domain and the vector domain. Since the label had 256 pixels only, the second network was compact and easy to converge. The second network, when successfully developed, was connected to the decoder (a portion of AE) to decompress the vector to a regular 128×128 image. Therefore, a complex network was essentially divided into two compact neural networks trained separately in sequence but eventually connectable.

Results

A total of 2,000 test examples, a synthetic brain phantom, and de-identified patient data were used to validate SPECTnet. Results obtained from SPECTnet were compared with those obtained from our clinic OS-EM method. Images with lower noise and more accurate information in the uptake areas were obtained by SPECTnet.

Conclusions

The challenge of developing a complex deep neural network is reduced by training two separate compact connectable networks. The combination of the two networks forms the full version of SPECTnet. Results show that the developed neural network can produce more accurate SPECT images.

Bridging the Gap between 2D and 3D Contexts in CT Volume for Liver and Tumor Segmentation

Automatic liver and tumor segmentation remain a challenging topic, which subjects to the exploration of 2D and 3D contexts in CT volume. Existing methods are either only focus on the 2D context by treating the CT volume as many independent image slices (but ignore the useful temporal information between adjacent slices), or just explore the 3D context lied in many little voxels (but damage the spatial detail in each slice). These factors lead an inadequate context exploration together for automatic liver and tumor segmentation. In this paper, we propose a novel full-context convolution neural network to bridge the gap between 2D and 3D contexts. The proposed network can utilize the temporal information along the Z axis in CT volume while retaining the spatial detail in each slice. Specifically, a 2D spatial network for intra-slice features extraction and a 3D temporal network for inter-slice features extraction are proposed separately and then are guided by the squeeze-and-excitation layer that allows the flow of 2D context and 3D temporal information. To address the severe class imbalance issue in the CT volume and meanwhile improve the segmentation performance, a loss function consisting of weighted cross-entropy and jaccard distance is proposed. During the network training, the 2D and 3D contexts are learned jointly in an end-to-end way. The proposed network achieves competitive results on the Liver Tumor Segmentation Challenge (LiTS) and the 3D-IRCADB datasets. This method should be a new promising paradigm to explore the contexts for liver and tumor segmentation.

Deep Learning Image Feature Recognition Algorithm for Judgment on the Rationality of Landscape Planning and Design

This paper uses an improved deep learning algorithm to judge the rationality of the design of landscape image feature recognition. The preprocessing of the image is proposed to enhance the data. The deficiencies in landscape feature extraction are further addressed based on the new model. Then, the two-stage training method of the model is used to solve the problems of long training time and convergence difficulties in deep learning. Innovative methods for zoning and segmentation training of landscape pattern features are proposed, which makes model training faster and generates more creative landscape patterns. Because of the impact of too many types of landscape elements in landscape images, traditional convolutional neural networks can no longer effectively solve this problem. On this basis, a fully convolutional neural network model is designed to perform semantic segmentation of landscape elements in landscape images. Through the method of deconvolution, the pixel-level semantic segmentation is realized. Compared with the 65% accuracy rate of the convolutional neural network, the fully convolutional neural network has an accuracy rate of 90.3% for the recognition of landscape elements. The method is effective, accurate, and intelligent for the classification of landscape element design, which better improves the accuracy of classification, greatly reduces the cost of landscape element design classification, and ensures that the technical method is feasible. This paper classifies landscape behavior based on this model for full convolutional neural network landscape images and demonstrates the effectiveness of using the model. In terms of landscape image processing, the image evaluation provides a certain basis.

Sparse-view imaging of a fiber internal structure in holographic diffraction tomography via a convolutional neural network

Deep learning has recently shown great potential in computational imaging. Here, we propose a deep-learning-based reconstruction method to realize the sparse-view imaging of a fiber internal structure in holographic diffraction tomography. By taking the sparse-view sinogram as the input and the cross-section image obtained by the dense-view sinogram as the ground truth, the neural network can reconstruct the cross-section image from the sparse-view sinogram. It performs better than the corresponding filtered back-projection algorithm with a sparse-view sinogram, both in the case of simulated data and real experimental data.

Deep learning-based solvability of underdetermined inverse problems in medical imaging

Recently, with the significant developments in deep learning techniques, solving underdetermined inverse problems has become one of the major concerns in the medical imaging domain, where underdetermined problems are motivated by the willingness to provide high resolution medical images with as little data as possible, by optimizing data collection in terms of minimal acquisition time, cost-effectiveness, and low invasiveness. Typical examples include undersampled magnetic resonance imaging(MRI), interior tomography, and sparse-view computed tomography(CT), where deep learning techniques have achieved excellent performances. However, there is a lack of mathematical analysis of why the deep learning method is performing well. This study aims to explain about learning the causal relationship regarding the structure of the training data suitable for deep learning, to solve highly underdetermined problems. We present a particular low-dimensional solution model to highlight the advantage of deep learning methods over conventional methods, where two approaches use the prior information of the solution in a completely different way. We also analyze whether deep learning methods can learn the desired reconstruction map from training data in the three models (undersampled MRI, sparse-view CT, interior tomography). This paper also discusses the nonlinearity structure of underdetermined linear systems and conditions of learning (called M-RIP condition).

Automated joint skull-stripping and segmentation with Multi-Task U-Net in large mouse brain MRI databases

Skull-stripping and region segmentation are fundamental steps in preclinical magnetic resonance imaging (MRI) studies, and these common procedures are usually performed manually. We present Multi-task U-Net (MU-Net), a convolutional neural network designed to accomplish both tasks simultaneously. MU-Net achieved higher segmentation accuracy than state-of-the-art multi-atlas segmentation methods with an inference time of 0.35 s and no pre-processing requirements. We trained and validated MU-Net on 128 T2-weighted mouse MRI volumes as well as on the publicly available MRM NeAT dataset of 10 MRI volumes. We tested MU-Net with an unusually large dataset combining several independent studies consisting of 1782 mouse brain MRI volumes of both healthy and Huntington animals, and measured average Dice scores of 0.906 (striati), 0.937 (cortex), and 0.978 (brain mask). Further, we explored the effectiveness of our network in the presence of different architectural features, including skip connections and recently proposed framing connections, and the effects of the age range of the training set animals. These high evaluation scores demonstrate that MU-Net is a powerful tool for segmentation and skull-stripping, decreasing inter and intra-rater variability of manual segmentation. The MU-Net code and the trained model are publicly available at https://github.com/Hierakonpolis/MU-Net.