Deep Learning Computed Tomography: Learning Projection-Domain Weights From Image Domain in Limited Angle Problems

Lag-Net: Lag correction for cone-beam CT via a convolutional neural network

BACKGROUND AND OBJECTIVE

Due to the presence of charge traps in amorphous silicon flat-panel detectors, lag signals are generated in consecutively captured projections. These signals lead to ghosting in projection images and severe lag artifacts in cone-beam computed tomography (CBCT) reconstructions. Traditional Linear Time-Invariant (LTI) correction need to measure lag correction factors (LCF) and may leave residual lag artifacts. This incomplete correction is partly attributed to the lack of consideration for exposure dependency.

METHODS

To measure the lag signals more accurately and suppress lag artifacts, we develop a novel hardware correction method. This method requires two scans of the same object, with adjustments to the operating timing of the CT instrumentation during the second scan to measure the lag signal from the first. While this hardware correction significantly mitigates lag artifacts, it is complex to implement and imposes high demands on the CT instrumentation. To enhance the process, We introduce a deep learning method called Lag-Net to remove lag signal, utilizing the nearly lag-free results from hardware correction as training targets for the network.

RESULTS

Qualitative and quantitative analyses of experimental results on both simulated and real datasets demonstrate that deep learning correction significantly outperforms traditional LTI correction in terms of lag artifact suppression and image quality enhancement. Furthermore, the deep learning method achieves reconstruction results comparable to those obtained from hardware correction while avoiding the operational complexities associated with the hardware correction approach.

CONCLUSION

The proposed hardware correction method, despite its operational complexity, demonstrates superior artifact suppression performance compared to the LTI algorithm, particularly under low-exposure conditions. The introduced Lag-Net, which utilizes the results of the hardware correction method as training targets, leverages the end-to-end nature of deep learning to circumvent the intricate operational drawbacks associated with hardware correction. Furthermore, the network's correction efficacy surpasses that of the LTI algorithm in low-exposure scenarios.

Continuous representation-based reconstruction for computed tomography

BACKGROUND

Computed tomography (CT) imaging has been developed to acquire a higher resolution image for detecting early-stage lesions. However, the lack of spatial resolution of CT images is still a limitation to fully utilize the capabilities of display devices for radiologists.

PURPOSE

This limitation can be addressed by improving the quality of the reconstructed image using super-resolution (SR) techniques without changing data acquisition protocols. In particular, local implicit representation-based techniques proposed in the field of low-level computer vision have shown promising performance, but their integration into CT image reconstruction is limited by considerable memory and runtime requirements due to excessive input data size.

METHODS

To address these limitations, we propose a continuous image representation-based CT image reconstruction (CRET) structure. Our CRET ensures fast and memory-efficient reconstruction for the specific region of interest (ROI) image by adapting our proposed sinogram squeezing and decoding via a set of sinusoidal basis functions. Furthermore, post-restoration step can be employed to mitigate residual artifacts and blurring effects, leading to improve image quality.

RESULTS

Our proposed method shows superior image quality than other local implicit representation methods and can be further improved with additional post-processing. In addition proposed structure achieves superior performance in terms of anthropomorphic observer model evaluation compared to conventional techniques. This results highlights that CRET can be used to improve diagnostic capabilities by setting the reconstruction resolution higher than the ground truth images in training.

CONCLUSIONS

Our proposed CRET method offers a promising solution for improving CT image resolution while addressing excessive memory and runtime consumption. The source code of our proposed CRET is available at https://github.com/minwoo-yu/CRET.

DRACO: differentiable reconstruction for arbitrary CBCT orbits

Objective. This study introduces a novel method for reconstructing cone beam computed tomography (CBCT) images for arbitrary orbits, addressing the computational and memory challenges associated with traditional iterative reconstruction algorithms.Approach. The proposed method employs a differentiable shift-variant filtered backprojection neural network, optimized for arbitrary trajectories. By integrating known operators into the learning model, the approach minimizes the number of trainable parameters while enhancing model interpretability. This framework adapts seamlessly to specific orbit geometries, including non-continuous trajectories such as circular-plus-arc or sinusoidal paths, enabling faster and more accurate CBCT reconstructions.Main results. Experimental validation demonstrates that the method significantly accelerates reconstruction, reducing computation time by over 97% compared to conventional iterative algorithms. It achieves superior or comparable image quality with reduced noise, as evidenced by a 38.6% reduction in mean squared error, a 7.7% increase in peak signal-to-noise ratio, and a 5.0% improvement in the structural similarity index measure. The flexibility and robustness of the approach are confirmed through its ability to handle data from diverse scan geometries.Significance. This method represents a significant advancement in interventional medical imaging, particularly for robotic C-arm CT systems, enabling real-time, high-quality CBCT reconstructions for customized orbits. It offers a transformative solution for clinical applications requiring computational efficiency and precision in imaging.

Artificial Intelligence in Computed Tomography Image Reconstruction: A Review of Recent Advances

The development of novel image reconstruction algorithms has been pivotal in enhancing image quality and reducing radiation dose in computed tomography (CT) imaging. Traditional techniques like filtered back projection perform well under ideal conditions but fail to generate high-quality images under low-dose, sparse-view, and limited-angle conditions. Iterative reconstruction methods improve upon filtered back projection by incorporating system models and assumptions about the patient, yet they can suffer from patchy image textures. The emergence of artificial intelligence (AI), particularly deep learning, has further advanced CT reconstruction. AI techniques have demonstrated great potential in reducing radiation dose while preserving image quality and noise texture. Moreover, AI has exhibited unprecedented performance in addressing challenging CT reconstruction problems, including low-dose CT, sparse-view CT, limited-angle CT, and interior tomography. This review focuses on the latest advances in AI-based CT reconstruction under these challenging conditions.

An efficient deep unrolling network for sparse-view CT reconstruction via alternating optimization of dense-view sinograms and images

Objective. Recently, there have been many advancements in deep unrolling methods for sparse-view computed tomography (SVCT) reconstruction. These methods combine model-based and deep learning-based reconstruction techniques, improving the interpretability and achieving significant results. However, they are often computationally expensive, particularly for clinical raw projection data with large sizes. This study aims to address this issue while maintaining the quality of the reconstructed image.Approach. The SVCT reconstruction task is decomposed into two subproblems using the proximal gradient method: optimizing dense-view sinograms and optimizing images. Then dense-view sinogram inpainting, image-residual learning, and image-refinement modules are performed at each iteration stage using deep neural networks. Unlike previous unrolling methods, the proposed method focuses on optimizing dense-view sinograms instead of full-view sinograms. This approach not only reduces computational resources and runtime but also minimizes the challenge for the network to perform sinogram inpainting when the sparse ratio is extremely small, thereby decreasing the propagation of estimation error from the sinogram domain to the image domain.Main results. The proposed method successfully reconstructs an image (512 × 512 pixels) from real-size (2304 × 736) projection data, with 3.39 M training parameters and an inference time of 0.09 s per slice on a GPU. The proposed method also achieves superior quantitative and qualitative results compared with state-of-the-art deep unrolling methods on datasets with sparse ratios of 1/12 and 1/18, especially in suppressing artifacts and preserving structural details. Additionally, results show that using dense-view sinogram inpainting not only accelerates the computational speed but also leads to faster network convergence and further improvements in reconstruction results.Significance. This research presents an efficient dual-domain deep unrolling technique that produces excellent results in SVCT reconstruction while requiring small computational resources. These findings have important implications for speeding up deep unrolling CT reconstruction methods and making them more practical for processing clinical CT projection data.

Emulating Low-Dose PCCT Image Pairs With Independent Noise for Self-Supervised Spectral Image Denoising

Photon counting CT (PCCT) acquires spectral measurements and enables generation of material decomposition (MD) images that provide distinct advantages in various clinical situations. However, noise amplification is observed in MD images, and denoising is typically applied. Clean or high-quality references are rare in clinical scans, often making supervised learning (Noise2Clean) impractical. Noise2Noise is a self-supervised counterpart, using noisy images and corresponding noisy references with zero-mean, independent noise. PCCT counts transmitted photons separately, and raw measurements are assumed to follow a Poisson distribution in each energy bin, providing the possibility to create noise-independent pairs. The approach is to use binomial selection to split the counts into two low-dose scans with independent noise. We prove that the reconstructed spectral images inherit the noise independence from counts domain through noise propagation analysis and also validated it in numerical simulation and experimental phantom scans. The method offers the flexibility to split measurements into desired dose levels while ensuring the reconstructed images share identical underlying features, thereby strengthening the model's robustness for input dose levels and capability of preserving fine details. In both numerical simulation and experimental phantom scans, we demonstrated that Noise2Noise with binomial selection outperforms other common self-supervised learning methods based on different presumptive conditions.

Reconstructing and analyzing the invariances of low-dose CT image denoising networks

BACKGROUND

Deep learning-based methods led to significant advancements in many areas of medical imaging, most of which are concerned with the reduction of artifacts caused by motion, scatter, or noise. However, with most neural networks being black boxes, they remain notoriously difficult to interpret, hindering their clinical implementation. In particular, it has been shown that networks exhibit invariances w.r.t. input features, that is, they learn to ignore certain information in the input data.

PURPOSE

To improve the interpretability of deep learning-based low-dose CT image denoising networks.

METHODS

We learn a complete data representation of low-dose input images using a conditional variational autoencoder (cVAE). In this representation, invariances of any given denoising network are then disentangled from the information it is not invariant to using a conditional invertible neural network (cINN). At test time, image-space invariances are generated by applying the inverse of the cINN and subsequent decoding using the cVAE. We propose two methods to analyze sampled invariances and to find those that correspond to alterations of anatomical structures.

RESULTS

The proposed method is applied to four popular deep learning-based low-dose CT image denoising networks. We find that the networks are not only invariant to noise amplitude and realizations, but also to anatomical structures.

CONCLUSIONS

The proposed method is capable of reconstructing and analyzing invariances of deep learning-based low-dose CT image denoising networks. This is an important step toward interpreting deep learning-based methods for medical imaging, which is essential for their clinical implementation.

Simulation-informed learning for time-resolved angiographic contrast agent concentration reconstruction

Three-dimensional Digital Subtraction Angiography (3D-DSA) is a well-established X-ray-based technique for visualizing vascular anatomy. Recently, four-dimensional DSA (4D-DSA) reconstruction algorithms have been developed to enable the visualization of volumetric contrast flow dynamics through time-series of volumes. This reconstruction problem is ill-posed mainly due to vessel overlap in the projection direction and geometric vessel foreshortening, which leads to information loss in the recorded projection images. However, knowledge about the underlying fluid dynamics can be leveraged to constrain the solution space. In our work, we implicitly include this information in a neural network-based model that is trained on a dataset of image-based blood flow simulations. The model predicts the spatially averaged contrast agent concentration for each centerline point of the vasculature over time, lowering the overall computational demand. The trained network enables the reconstruction of relative contrast agent concentrations with a mean absolute error of 0.02±0.02 and a mean absolute percentage error of 5.31±9.25 %. Moreover, the network is robust to varying degrees of vessel overlap and vessel foreshortening. Our approach demonstrates the potential of the integration of machine learning and blood flow simulations in time-resolved angiographic contrast agent concentration reconstruction.

An efficient dual-domain deep learning network for sparse-view CT reconstruction

BACKGROUND AND OBJECTIVE

We develop an efficient deep-learning based dual-domain reconstruction method for sparse-view CT reconstruction with small training parameters and comparable running time. We aim to investigate the model's capability and its clinical value by performing objective and subjective quality assessments using clinical CT projection data acquired on commercial scanners.

METHODS

We designed two lightweight networks, namely Sino-Net and Img-Net, to restore the projection and image signal from the DD-Net reconstructed images in the projection and image domains, respectively. The proposed network has small training parameters and comparable running time among dual-domain based reconstruction networks and is easy to train (end-to-end). We prospectively collected clinical thoraco-abdominal CT projection data acquired on a Siemens Biograph 128 Edge CT scanner to train and validate the proposed network. Further, we quantitatively evaluated the CT Hounsfield unit (HU) values on 21 organs and anatomic structures, such as the liver, aorta, and ribcage. We also analyzed the noise properties and compared the signal-to-noise ratio (SNR) and the contrast-to-noise ratio (CNR) of the reconstructed images. Besides, two radiologists conducted the subjective qualitative evaluation including the confidence and conspicuity of anatomic structures, and the overall image quality using a 1-5 likert scoring system.

RESULTS

Objective and subjective evaluation showed that the proposed algorithm achieves competitive results in eliminating noise and artifacts, restoring fine structure details, and recovering edges and contours of anatomic structures using 384 views (1/6 sparse rate). The proposed method exhibited good computational cost performance on clinical projection data.

CONCLUSION

This work presents an efficient dual-domain learning network for sparse-view CT reconstruction on raw projection data from a commercial scanner. The study also provides insights for designing an organ-based image quality assessment pipeline for sparse-view reconstruction tasks, potentially benefiting organ-specific dose reduction by sparse-view imaging.

Non-coplanar CBCT image reconstruction using a generative adversarial network for non-coplanar radiotherapy

PURPOSE

To develop a non-coplanar cone-beam computed tomography (CBCT) image reconstruction method using projections within a limited angle range for non-coplanar radiotherapy.

METHODS

A generative adversarial network (GAN) was utilized to reconstruct non-coplanar CBCT images. Data from 40 patients with brain tumors and two head phantoms were used in this study. In the training stage, the generator of the GAN used coplanar CBCT and non-coplanar projections as the input, and an encoder with a dual-branch structure was utilized to extract features from the coplanar CBCT and non-coplanar projections separately. Non-coplanar CBCT images were then reconstructed using a decoder by combining the extracted features. To improve the reconstruction accuracy of the image details, the generator was adversarially trained using a patch-based convolutional neural network as the discriminator. A newly designed joint loss was used to improve the global structure consistency rather than the conventional GAN loss. The proposed model was evaluated using data from eight patients and two phantoms at four couch angles (±45°, ±90°) that are most commonly used for brain non-coplanar radiotherapy in our department. The reconstructed accuracy was evaluated by calculating the root mean square error (RMSE) and an overall registration error ε, computed by integrating the rigid transformation parameters.

RESULTS

In both patient data and phantom data studies, the qualitative and quantitative metrics results indicated that ± 45° couch angle models performed better than ±90° couch angle models and had statistical differences. In the patient data study, the mean RMSE and ε values of couch angle at 45°, -45°, 90°, and -90° were 58.5 HU and 0.42 mm, 56.8 HU and 0.41 mm, 73.6 HU and 0.48 mm, and 65.3 HU and 0.46 mm, respectively. In the phantom data study, the mean RMSE and ε values of couch angle at 45°, -45°, 90°, and -90° were 91.2 HU and 0.46 mm, 95.0 HU and 0.45 mm, 114.6 HU and 0.58 mm, and 102.9 HU and 0.52 mm, respectively.

CONCLUSIONS

The results show that the reconstructed non-coplanar CBCT images can potentially enable intra-treatment three-dimensional position verification for non-coplanar radiotherapy.

Time-Reversion Fast-Sampling Score-Based Model for Limited-Angle CT Reconstruction

The score-based generative model (SGM) has received significant attention in the field of medical imaging, particularly in the context of limited-angle computed tomography (LACT). Traditional SGM approaches achieved robust reconstruction performance by incorporating a substantial number of sampling steps during the inference phase. However, these established SGM-based methods require large computational cost to reconstruct one case. The main challenge lies in achieving high-quality images with rapid sampling while preserving sharp edges and small features. In this study, we propose an innovative rapid-sampling strategy for SGM, which we have aptly named the time-reversion fast-sampling (TIFA) score-based model for LACT reconstruction. The entire sampling procedure adheres steadfastly to the principles of robust optimization theory and is firmly grounded in a comprehensive mathematical model. TIFA's rapid-sampling mechanism comprises several essential components, including jump sampling, time-reversion with re-sampling, and compressed sampling. In the initial jump sampling stage, multiple sampling steps are bypassed to expedite the attainment of preliminary results. Subsequently, during the time-reversion process, the initial results undergo controlled corruption by introducing small-scale noise. The re-sampling process then diligently refines the initially corrupted results. Finally, compressed sampling fine-tunes the refinement outcomes by imposing regularization term. Quantitative and qualitative assessments conducted on numerical simulations, real physical phantom, and clinical cardiac datasets, unequivocally demonstrate that TIFA method (using 200 steps) outperforms other state-of-the-art methods (using 2000 steps) from available [0°, 90°] and [0°, 60°]. Furthermore, experimental results underscore that our TIFA method continues to reconstruct high-quality images even with 10 steps. Our code at https://github.com/tianzhijiaoziA/TIFADiffusion.

Radiation Detectors and Sensors in Medical Imaging

Medical imaging instrumentation design and construction is based on radiation sources and radiation detectors/sensors. This review focuses on the detectors and sensors of medical imaging systems. These systems are subdivided into various categories depending on their structure, the type of radiation they capture, how the radiation is measured, how the images are formed, and the medical goals they serve. Related to medical goals, detectors fall into two major areas: (i) anatomical imaging, which mainly concerns the techniques of diagnostic radiology, and (ii) functional-molecular imaging, which mainly concerns nuclear medicine. An important parameter in the evaluation of the detectors is the combination of the quality of the diagnostic result they offer and the burden of the patient with radiation dose. The latter has to be minimized; thus, the input signal (radiation photon flux) must be kept at low levels. For this reason, the detective quantum efficiency (DQE), expressing signal-to-noise ratio transfer through an imaging system, is of primary importance. In diagnostic radiology, image quality is better than in nuclear medicine; however, in most cases, the dose is higher. On the other hand, nuclear medicine focuses on the detection of functional findings and not on the accurate spatial determination of anatomical data. Detectors are integrated into projection or tomographic imaging systems and are based on the use of scintillators with optical sensors, photoconductors, or semiconductors. Analysis and modeling of such systems can be performed employing theoretical models developed in the framework of cascaded linear systems analysis (LCSA), as well as within the signal detection theory (SDT) and information theory.

Data-driven rapid 4D cone-beam CT reconstruction for new generation linacs

Objective.Newer generation linear accelerators (Linacs) allow 20 s cone-beam CT (CBCT) acquisition which reduces radiation therapy treatment time. However, the current clinical application of these rapid scans is only 3DCBCT. In this paper we propose a novel data-driven rapid 4DCBCT reconstruction method for new generation linacs.Approach.This method relies on estimating the magnitude of the diaphragm motion from an initial 3D reconstruction. This estimated motion is used to linearly approximate a deformation vector field (DVF) for each respiration phase. These DVFs are then used for motion compensated Feldkamp-Davis-Kress (MCFDK) reconstructions. This method, named MCFDK Data Driven (MCFDK-DD), was compared to a MCFDK reconstruction using a prior motion model (MCFDK-Prior), a 3D-FDK reconstruction, and a conventional acquisition (4 mins) conventional reconstruction 4DCBCT (4D-FDK). The data used in this paper were derived from 4DCT volumes from 12 patients from The Cancer Imaging Archives. Image quality was quantified using RMSE of line plots centred on the tumour, tissue interface width (TIW), the mean square error (MSE) and structural similarity index measurement (SSIM).Main Results.The tumour line plots in the Superior-Inferior direction showed reduced RMSE for the MCFDK-DD compared to the 3D-FDK method, indicating the MCFDK-DD method provided a more accurate tumour location. Similarly, the TIW values from the MCFDK-DD reconstructions (median 8.6 mm) were significantly reduced for the MCFDK-DD method compared to the 3D-FDK reconstructions (median 14.8 mm, (p< 0.001). The MCFDK-DD, MCFDK-Prior and 3D-FDK had median MSE values of1.08×10-6mm-1,1.11×10-6mm-1and1.17×10-6mm-1respectively. The corresponding median SSIM values were 0.93, 0.92 and 0.92 respectively indicating the MCFDK-DD had good agreement with the conventional 4D-FDK reconstructions.Significance.These results demonstrate the feasibility of creating accurate data-driven 4DCBCT images for rapid scans on new generation linacs. These findings could lead to increased clinical usage of 4D information on newer generation linacs.

Recent advances in artificial intelligent strategies for tissue engineering and regenerative medicine

BACKGROUND

Tissue engineering and regenerative medicine (TERM) aim to repair or replace damaged or lost tissues or organs due to accidents, diseases, or aging, by applying different sciences. For this purpose, an essential part of TERM is the designing, manufacturing, and evaluating of scaffolds, cells, tissues, and organs. Artificial intelligence (AI) or the intelligence of machines or software can be effective in all areas where computers play a role.

METHODS

The "artificial intelligence," "machine learning," "tissue engineering," "clinical evaluation," and "scaffold" keywords used for searching in various databases and articles published from 2000 to 2024 were evaluated.

RESULTS

The combination of tissue engineering and AI has created a new generation of technological advancement in the biomedical industry. Experience in TERM has been refined using advanced design and manufacturing techniques. Advances in AI, particularly deep learning, offer an opportunity to improve scientific understanding and clinical outcomes in TERM.

CONCLUSION

The findings of this research show the high potential of AI, machine learning, and robots in the selection, design, and fabrication of scaffolds, cells, tissues, or organs, and their analysis, characterization, and evaluation after their implantation. AI can be a tool to accelerate the introduction of tissue engineering products to the bedside.

HIGHLIGHTS

The capabilities of artificial intelligence (AI) can be used in different ways in all the different stages of TERM and not only solve the existing limitations, but also accelerate the processes, increase efficiency and precision, reduce costs, and complications after transplantation. ML predicts which technologies have the most efficient and easiest path to enter the market and clinic. The use of AI along with these imaging techniques can lead to the improvement of diagnostic information, the reduction of operator errors when reading images, and the improvement of image analysis (such as classification, localization, regression, and segmentation).

Learning a stable approximation of an existing but unknown inverse mapping: application to the half-time circular Radon transform

Supervised deep learning-based methods have inspired a new wave of image reconstruction methods that implicitly learn effective regularization strategies from a set of training data. While they hold potential for improving image quality, they have also raised concerns regarding their robustness. Instabilities can manifest when learned methods are applied to find approximate solutions to ill-posed image reconstruction problems for which a unique and stable inverse mapping does not exist, which is a typical use case. In this study, we investigate the performance of supervised deep learning-based image reconstruction in an alternate use case in which a stable inverse mapping is known to exist but is not yet analytically available in closed form. For such problems, a deep learning-based method can learn a stable approximation of the unknown inverse mapping that generalizes well to data that differ significantly from the training set. The learned approximation of the inverse mapping eliminates the need to employ an implicit (optimization-based) reconstruction method and can potentially yield insights into the unknown analytic inverse formula. The specific problem addressed is image reconstruction from a particular case of radially truncated circular Radon transform (CRT) data, referred to as 'half-time' measurement data. For the half-time image reconstruction problem, we develop and investigate a learned filtered backprojection method that employs a convolutional neural network to approximate the unknown filtering operation. We demonstrate that this method behaves stably and readily generalizes to data that differ significantly from training data. The developed method may find application to wave-based imaging modalities that include photoacoustic computed tomography.

Noise-Generating and Imaging Mechanism Inspired Implicit Regularization Learning Network for Low Dose CT Reconstrution

Low-dose computed tomography (LDCT) helps to reduce radiation risks in CT scanning while maintaining image quality, which involves a consistent pursuit of lower incident rays and higher reconstruction performance. Although deep learning approaches have achieved encouraging success in LDCT reconstruction, most of them treat the task as a general inverse problem in either the image domain or the dual (sinogram and image) domains. Such frameworks have not considered the original noise generation of the projection data and suffer from limited performance improvement for the LDCT task. In this paper, we propose a novel reconstruction model based on noise-generating and imaging mechanism in full-domain, which fully considers the statistical properties of intrinsic noises in LDCT and prior information in sinogram and image domains. To solve the model, we propose an optimization algorithm based on the proximal gradient technique. Specifically, we derive the approximate solutions of the integer programming problem on the projection data theoretically. Instead of hand-crafting the sinogram and image regularizers, we propose to unroll the optimization algorithm to be a deep network. The network implicitly learns the proximal operators of sinogram and image regularizers with two deep neural networks, providing a more interpretable and effective reconstruction procedure. Numerical results demonstrate our proposed method improvements of > 2.9 dB in peak signal to noise ratio, > 1.4% promotion in structural similarity metric, and > 9 HU decrements in root mean square error over current state-of-the-art LDCT methods.

Impact of using sinogram domain data in the super-resolution of CT images on diagnostic information

BACKGROUND

In recent times, deep-learning-based super-resolution (DL-SR) techniques for computed tomography (CT) images have shown outstanding results in terms of full-reference image quality (FR-IQ) metrics (e.g., root mean square error and structural similarity index metric), which assesses IQ by measuring its similarity to the high-resolution (HR) image. In addition, IQ can be evaluated via task-based IQ (Task-IQ) metrics that evaluate the ability to perform specific tasks. Ironically, most proposed image domain-based SR techniques are not possible to improve a Task-IQ metric, which assesses the amount of information related to diagnosis.

PURPOSE

In the case of CT imaging systems, sinogram domain data can be utilized for SR techniques. Therefore, this study aims to investigate the impact of utilizing sinogram domain data on diagnostic information restoration ability.

METHODS

We evaluated three DL-SR techniques: using image domain data (Image-SR), using sinogram domain data (Sinogram-SR), and using sinogram as well as image domain data (Dual-SR). For Task-IQ evaluation, the Rayleigh discrimination task was used to evaluate diagnostic ability by focusing on the resolving power aspect, and an ideal observer (IO) can be used to perform the task. In this study, we used a convolutional neural network (CNN)-based IO that approximates the IO performance. We compared the IO performances of the SR techniques according to the data domain to evaluate the discriminative information restoration ability.

RESULTS

Overall, the low-resolution (LR) and SR exhibit lower IO performances compared with that of HR owing to their degraded discriminative information when detector binning is used. Next, between the SR techniques, Image-SR does not show superior IO performances compared to the LR image, but Sinogram-SR and Dual-SR show superior IO performances than the LR image. Furthermore, in Sinogram-SR, we confirm that FR-IQ and IO performance are positively correlated. These observations demonstrate that sinogram domain upsampling improves the representation ability for discriminative information in the image domain compared to the LR and Image-SR.

CONCLUSIONS

Unlike Image-SR, Sinogram-SR can improve the amount of discriminative information present in the image domain. This demonstrates that to improve the amount of discriminative information on the resolving power aspect, it is necessary to employ sinogram domain processing.

SWISTA-Nets: Subband-adaptive wavelet iterative shrinkage thresholding networks for image reconstruction

Robust and interpretable image reconstruction is central to imageology applications in clinical practice. Prevalent deep networks, with strong learning ability to extract implicit information from data manifold, are still lack of prior knowledge introduced from mathematics or physics, leading to instability, poor structure interpretability and high computation cost. As to this issue, we propose two prior knowledge-driven networks to combine the good interpretability of mathematical methods and the powerful learnability of deep learning methods. Incorporating different kinds of prior knowledge, we propose subband-adaptive wavelet iterative shrinkage thresholding networks (SWISTA-Nets), where almost every network module is in one-to-one correspondence with each step involved in the iterative algorithm. By end-to-end training of proposed SWISTA-Nets, implicit information can be extracted from training data and guide the tuning process of key parameters that possess mathematical definition. The inverse problems associated with two medical imaging modalities, i.e., electromagnetic tomography and X-ray computational tomography are applied to validate the proposed networks. Both visual and quantitative results indicate that the SWISTA-Nets outperform mathematical methods and state-of-the-art prior knowledge-driven networks, especially with fewer training parameters, interpretable network structures and well robustness. We assume that our analysis will support further investigation of prior knowledge-driven networks in the field of ill-posed image reconstruction.

AIRPORT: A Data Consistency Constrained Deep Temporal Extrapolation Method To Improve Temporal Resolution In Contrast Enhanced CT Imaging

Typical tomographic image reconstruction methods require that the imaged object is static and stationary during the time window to acquire a minimally complete data set. The violation of this requirement leads to temporal-averaging errors in the reconstructed images. For a fixed gantry rotation speed, to reduce the errors, it is desired to reconstruct images using data acquired over a narrower angular range, i.e., with a higher temporal resolution. However, image reconstruction with a narrower angular range violates the data sufficiency condition, resulting in severe data-insufficiency-induced errors. The purpose of this work is to decouple the trade-off between these two types of errors in contrast-enhanced computed tomography (CT) imaging. We demonstrated that using the developed data consistency constrained deep temporal extrapolation method (AIRPORT), the entire time-varying imaged object can be accurately reconstructed with 40 frames-per-second temporal resolution, the time window needed to acquire a single projection view data using a typical C-arm cone-beam CT system. AIRPORT is applicable to general non-sparse imaging tasks using a single short-scan data acquisition.

Exploring Fracture Patterns: Assessing Representation Methods for Bone Fracture Simulation

Fracture pattern acquisition and representation in human bones play a crucial role in medical simulation, diagnostics, and treatment planning. This article presents a comprehensive review of methodologies employed in acquiring and representing bone fracture patterns. Several techniques, including segmentation algorithms, curvature analysis, and deep learning-based approaches, are reviewed to determine their effectiveness in accurately identifying fracture zones. Additionally, diverse methods for representing fracture patterns are evaluated. The challenges inherent in detecting accurate fracture zones from medical images, the complexities arising from multifragmentary fractures, and the need to automate fracture reduction processes are elucidated. A detailed analysis of the suitability of each representation method for specific medical applications, such as simulation systems, surgical interventions, and educational purposes, is provided. The study explores insights from a broad spectrum of research articles, encompassing diverse methodologies and perspectives. This review elucidates potential directions for future research and contributes to advancements in comprehending the acquisition and representation of fracture patterns in human bone.

Deep Filtered Back Projection for CT Reconstruction

Filtered back projection (FBP) is a classic analytical algorithm for computed tomography (CT) reconstruction, with high computational efficiency. However, images reconstructed by FBP often suffer from excessive noise and artifacts. The original FBP algorithm uses a window function to smooth signals and a linear interpolation to estimate projection values at un-sampled locations. In this study, we propose a novel framework named DeepFBP in which an optimized filter and an optimized nonlinear interpolation operator are learned with neural networks. Specifically, the learned filter can be considered as the product of an optimized window function and the ramp filter, and the learned interpolation can be considered as an optimized way to utilize projection information of nearby locations through nonlinear combination. The proposed method remains the high computational efficiency of the original FBP and achieves much better reconstruction quality at different noise levels. It also outperforms the TV-based statistical iterative algorithm, with computational time being reduced in an order of two, and state-of-the-art post-processing deep learning methods that have deeper and more complicated network structures.

A Fully Differentiable Framework for 2D/3D Registration and the Projective Spatial Transformers

Image-based 2D/3D registration is a critical technique for fluoroscopic guided surgical interventions. Conventional intensity-based 2D/3D registration approa- ches suffer from a limited capture range due to the presence of local minima in hand-crafted image similarity functions. In this work, we aim to extend the 2D/3D registration capture range with a fully differentiable deep network framework that learns to approximate a convex-shape similarity function. The network uses a novel Projective Spatial Transformer (ProST) module that has unique differentiability with respect to 3D pose parameters, and is trained using an innovative double backward gradient-driven loss function. We compare the most popular learning-based pose regression methods in the literature and use the well-established CMAES intensity-based registration as a benchmark. We report registration pose error, target registration error (TRE) and success rate (SR) with a threshold of 10mm for mean TRE. For the pelvis anatomy, the median TRE of ProST followed by CMAES is 4.4mm with a SR of 65.6% in simulation, and 2.2mm with a SR of 73.2% in real data. The CMAES SRs without using ProST registration are 28.5% and 36.0% in simulation and real data, respectively. Our results suggest that the proposed ProST network learns a practical similarity function, which vastly extends the capture range of conventional intensity-based 2D/3D registration. We believe that the unique differentiable property of ProST has the potential to benefit related 3D medical imaging research applications. The source code is available at https://github.com/gaocong13/Projective-Spatial-Transformers.

A Deep Convolutional Gated Recurrent Unit for CT Image Reconstruction

Computed tomography (CT) is one of the most important medical imaging technologies in use today. Most commercial CT products use a technique known as the filtered backprojection (FBP) that is fast and can produce decent image quality when an X-ray dose is high. However, the FBP is not good enough on low-dose X-ray CT imaging because the CT image reconstruction problem becomes more stochastic. A more effective reconstruction technique proposed recently and implemented in a limited number of CT commercial products is an iterative reconstruction (IR). The IR technique is based on a Bayesian formulation of the CT image reconstruction problem with an explicit model of the CT scanning, including its stochastic nature, and a prior model that incorporates our knowledge about what a good CT image should look like. However, constructing such prior knowledge is more complicated than it seems. In this article, we propose a novel neural network for CT image reconstruction. The network is based on the IR formulation and constructed with a recurrent neural network (RNN). Specifically, we transform the gated recurrent unit (GRU) into a neural network performing CT image reconstruction. We call it "GRU reconstruction." This neural network conducts concurrent dual-domain learning. Many deep learning (DL)-based methods in medical imaging are single-domain learning, but dual-domain learning performs better because it learns from both the sinogram and the image domain. In addition, we propose backpropagation through stage (BPTS) as a new RNN backpropagation algorithm. It is similar to the backpropagation through time (BPTT) of an RNN; however, it is tailored for iterative optimization. Results from extensive experiments indicate that our proposed method outperforms conventional model-based methods, single-domain DL methods, and state-of-the-art DL techniques in terms of the root mean squared error (RMSE), the peak signal-to-noise ratio (PSNR), and the structure similarity (SSIM) and in terms of visual appearance.

Radiation-induced acoustic signal denoising using a supervised deep learning framework for imaging and therapy monitoring

Radiation-induced acoustic (RA) imaging is a promising technique for visualizing the invisible radiation energy deposition in tissues, enabling new imaging modalities and real-time therapy monitoring. However, RA imaging signal often suffers from poor signal-to-noise ratios (SNRs), thus requiring measuring hundreds or even thousands of frames for averaging to achieve satisfactory quality. This repetitive measurement increases ionizing radiation dose and degrades the temporal resolution of RA imaging, limiting its clinical utility. In this study, we developed a general deep inception convolutional neural network (GDI-CNN) to denoise RA signals to substantially reduce the number of frames needed for averaging. The network employs convolutions with multiple dilations in each inception block, allowing it to encode and decode signal features with varying temporal characteristics. This design generalizes GDI-CNN to denoise acoustic signals resulting from different radiation sources. The performance of the proposed method was evaluated using experimental data of x-ray-induced acoustic, protoacoustic, and electroacoustic signals both qualitatively and quantitatively. Results demonstrated the effectiveness of GDI-CNN: it achieved x-ray-induced acoustic image quality comparable to 750-frame-averaged results using only 10-frame-averaged measurements, reducing the imaging dose of x-ray-acoustic computed tomography (XACT) by 98.7%; it realized proton range accuracy parallel to 1500-frame-averaged results using only 20-frame-averaged measurements, improving the range verification frequency in proton therapy from 0.5 to 37.5 Hz; it reached electroacoustic image quality comparable to 750-frame-averaged results using only a single frame signal, increasing the electric field monitoring frequency from 1 fps to 1k fps. Compared to lowpass filter-based denoising, the proposed method demonstrated considerably lower mean-squared-errors, higher peak-SNR, and higher structural similarities with respect to the corresponding high-frame-averaged measurements. The proposed deep learning-based denoising framework is a generalized method for few-frame-averaged acoustic signal denoising, which significantly improves the RA imaging's clinical utilities for low-dose imaging and real-time therapy monitoring.

Enhanced artificial intelligence-based diagnosis using CBCT with internal denoising: Clinical validation for discrimination of fungal ball, sinusitis, and normal cases in the maxillary sinus

BACKGROUND AND OBJECTIVE

The cone-beam computed tomography (CBCT) provides three-dimensional volumetric imaging of a target with low radiation dose and cost compared with conventional computed tomography, and it is widely used in the detection of paranasal sinus disease. However, it lacks the sensitivity to detect soft tissue lesions owing to reconstruction constraints. Consequently, only physicians with expertise in CBCT reading can distinguish between inherent artifacts or noise and diseases, restricting the use of this imaging modality. The development of artificial intelligence (AI)-based computer-aided diagnosis methods for CBCT to overcome the shortage of experienced physicians has attracted substantial attention. However, advanced AI-based diagnosis addressing intrinsic noise in CBCT has not been devised, discouraging the practical use of AI solutions for CBCT. We introduce the development of AI-based computer-aided diagnosis for CBCT considering the intrinsic imaging noise and evaluate its efficacy and implications.

METHODS

We propose an AI-based computer-aided diagnosis method using CBCT with a denoising module. This module is implemented before diagnosis to reconstruct the internal ground-truth full-dose scan corresponding to an input CBCT image and thereby improve the diagnostic performance. The proposed method is model agnostic and compatible with various existing and future AI-based denoising or diagnosis models.

RESULTS

The external validation results for the unified diagnosis of sinus fungal ball, chronic rhinosinusitis, and normal cases show that the proposed method improves the micro-, macro-average area under the curve, and accuracy by 7.4, 5.6, and 9.6% (from 86.2, 87.0, and 73.4 to 93.6, 92.6, and 83.0%), respectively, compared with a baseline while improving human diagnosis accuracy by 11% (from 71.7 to 83.0%), demonstrating technical differentiation and clinical effectiveness. In addition, the physician's ability to evaluate the AI-derived diagnosis results may be enhanced compared with existing solutions.

CONCLUSION

This pioneering study on AI-based diagnosis using CBCT indicates that denoising can improve diagnostic performance and reader interpretability in images from the sinonasal area, thereby providing a new approach and direction to radiographic image reconstruction regarding the development of AI-based diagnostic solutions. Furthermore, we believe that the performance enhancement will expedite the adoption of automated diagnostic solutions using CBCT, especially in locations with a shortage of skilled clinicians and limited access to high-dose scanning.

Mitigation of motion-induced artifacts in cone beam computed tomography using deep convolutional neural networks

BACKGROUND

Cone beam computed tomography (CBCT) is often employed on radiation therapy treatment devices (linear accelerators) used in image-guided radiation therapy (IGRT). For each treatment session, it is necessary to obtain the image of the day in order to accurately position the patient and to enable adaptive treatment capabilities including auto-segmentation and dose calculation. Reconstructed CBCT images often suffer from artifacts, in particular those induced by patient motion. Deep-learning based approaches promise ways to mitigate such artifacts.

PURPOSE

We propose a novel deep-learning based approach with the goal to reduce motion induced artifacts in CBCT images and improve image quality. It is based on supervised learning and includes neural network architectures employed as pre- and/or post-processing steps during CBCT reconstruction.

METHODS

Our approach is based on deep convolutional neural networks which complement the standard CBCT reconstruction, which is performed either with the analytical Feldkamp-Davis-Kress (FDK) method, or with an iterative algebraic reconstruction technique (SART-TV). The neural networks, which are based on refined U-net architectures, are trained end-to-end in a supervised learning setup. Labeled training data are obtained by means of a motion simulation, which uses the two extreme phases of 4D CT scans, their deformation vector fields, as well as time-dependent amplitude signals as input. The trained networks are validated against ground truth using quantitative metrics, as well as by using real patient CBCT scans for a qualitative evaluation by clinical experts.

RESULTS

The presented novel approach is able to generalize to unseen data and yields significant reductions in motion induced artifacts as well as improvements in image quality compared with existing state-of-the-art CBCT reconstruction algorithms (up to +6.3 dB and +0.19 improvements in peak signal-to-noise ratio, PSNR, and structural similarity index measure, SSIM, respectively), as evidenced by validation with an unseen test dataset, and confirmed by a clinical evaluation on real patient scans (up to 74% preference for motion artifact reduction over standard reconstruction).

CONCLUSIONS

For the first time, it is demonstrated, also by means of clinical evaluation, that inserting deep neural networks as pre- and post-processing plugins in the existing 3D CBCT reconstruction and trained end-to-end yield significant improvements in image quality and reduction of motion artifacts.

Noise-resilient deep learning for integrated circuit tomography

X-ray tomography is a non-destructive imaging technique that reveals the interior of an object from its projections at different angles. Under sparse-view and low-photon sampling, regularization priors are required to retrieve a high-fidelity reconstruction. Recently, deep learning has been used in X-ray tomography. The prior learned from training data replaces the general-purpose priors in iterative algorithms, achieving high-quality reconstructions with a neural network. Previous studies typically assume the noise statistics of test data are acquired a priori from training data, leaving the network susceptible to a change in the noise characteristics under practical imaging conditions. In this work, we propose a noise-resilient deep-reconstruction algorithm and apply it to integrated circuit tomography. By training the network with regularized reconstructions from a conventional algorithm, the learned prior shows strong noise resilience without the need for additional training with noisy examples, and allows us to obtain acceptable reconstructions with fewer photons in test data. The advantages of our framework may further enable low-photon tomographic imaging where long acquisition times limit the ability to acquire a large training set.

Medical image super-resolution reconstruction algorithms based on deep learning: A survey

BACKGROUND AND OBJECTIVE

With the high-resolution (HR) requirements of medical images in clinical practice, super-resolution (SR) reconstruction algorithms based on low-resolution (LR) medical images have become a research hotspot. This type of method can significantly improve image SR without improving hardware equipment, so it is of great significance to review it.

METHODS

Aiming at the unique SR reconstruction algorithms in the field of medical images, based on subdivided medical fields such as magnetic resonance (MR) images, computed tomography (CT) images, and ultrasound images. Firstly, we deeply analyzed the research progress of SR reconstruction algorithms, and summarized and compared the different types of algorithms. Secondly, we introduced the evaluation indicators corresponding to the SR reconstruction algorithms. Finally, we prospected the development trend of SR reconstruction technology in the medical field.

RESULTS

The medical image SR reconstruction technology based on deep learning can provide more abundant lesion information, relieve the expert's diagnosis pressure, and improve the diagnosis efficiency and accuracy.

CONCLUSION

The medical image SR reconstruction technology based on deep learning helps to improve the quality of medicine, provides help for the diagnosis of experts, and lays a solid foundation for the subsequent analysis and identification tasks of the computer, which is of great significance for improving the diagnosis efficiency of experts and realizing intelligent medical care.

Location-Dependent Spatiotemporal Antialiasing in Photoacoustic Computed Tomography

Photoacoustic computed tomography (PACT) images optical absorption contrast by detecting ultrasonic waves induced by optical energy deposition in materials such as biological tissues. An ultrasonic transducer array or its scanning equivalent is used to detect ultrasonic waves. The spatial distribution of the transducer elements must satisfy the spatial Nyquist criterion; otherwise, spatial aliasing occurs and causes artifacts in reconstructed images. The spatial Nyquist criterion poses different requirements on the transducer elements' distributions for different locations in the image domain, which has not been studied previously. In this research, we elaborate on the location dependency through spatiotemporal analysis and propose a location-dependent spatiotemporal antialiasing method. By applying this method to PACT in full-ring array geometry, we effectively mitigate aliasing artifacts with minimal effects on image resolution in both numerical simulations and in vivo experiments.

Physics-/Model-Based and Data-Driven Methods for Low-Dose Computed Tomography: A survey

Since 2016, deep learning (DL) has advanced tomographic imaging with remarkable successes, especially in low-dose computed tomography (LDCT) imaging. Despite being driven by big data, the LDCT denoising and pure end-to-end reconstruction networks often suffer from the black box nature and major issues such as instabilities, which is a major barrier to apply deep learning methods in low-dose CT applications. An emerging trend is to integrate imaging physics and model into deep networks, enabling a hybridization of physics/model-based and data-driven elements. In this paper, we systematically review the physics/model-based data-driven methods for LDCT, summarize the loss functions and training strategies, evaluate the performance of different methods, and discuss relevant issues and future directions.

Deep image and feature prior algorithm based on U-ConformerNet structure

PURPOSE

The reconstruction performance of the deep image prior (DIP) approach is limited by the conventional convolutional layer structure and it is difficult to enhance its potential. In order to improve the quality of image reconstruction and suppress artifacts, we propose a DIP algorithm with better performance, and verify its superiority in the latest case.

METHODS

We construct a new U-ConformerNet structure as the DIP algorithm's network, replacing the traditional convolutional layer-based U-net structure, and introduce the 'lpips' deep network based feature distance regularization method. Our algorithm can switch between supervised and unsupervised modes at will to meet different needs.

RESULTS

The reconstruction was performed on the low dose CT dataset (LoDoPaB). Our algorithm attained a PSNR of more than 35 dB under unsupervised conditions, and the PSNR under the supervised condition is greater than 36 dB. Both of which are better than the performance of the DIP-TV. Furthermore, the accuracy of this method is positively connected with the quality of the a priori image with the help of deep networks. In terms of noise eradication and artifact suppression, the DIP algorithm with U-ConformerNet structure outperforms the standard DIP method based on convolutional structure.

CONCLUSIONS

It is known by experimental verification that, in unsupervised mode, the algorithm improves the output PSNR by at least 2-3 dB when compared to the DIP-TV algorithm (proposed in 2020). In supervised mode, our algorithm approaches that of the state-of-the-art end-to-end deep learning algorithms.

Convolutional Encoder-Decoder Networks for Volumetric Computed Tomography Surviews from Single- and Dual-View Topograms

Computed tomography (CT) is an extensively used imaging modality capable of generating detailed images of a patient's internal anatomy for diagnostic and interventional procedures. High-resolution volumes are created by measuring and combining information along many radiographic projection angles. In current medical practice, single and dual-view two-dimensional (2D) topograms are utilized for planning the proceeding diagnostic scans and for selecting favorable acquisition parameters, either manually or automatically, as well as for dose modulation calculations. In this study, we develop modified 2D to three-dimensional (3D) encoder-decoder neural network architectures to generate CT-like volumes from single and dual-view topograms. We validate the developed neural networks on synthesized topograms from publicly available thoracic CT datasets. Finally, we assess the viability of the proposed transformational encoder-decoder architecture on both common image similarity metrics and quantitative clinical use case metrics, a first for 2D-to-3D CT reconstruction research. According to our findings, both single-input and dual-input neural networks are able to provide accurate volumetric anatomical estimates. The proposed technology will allow for improved (i) planning of diagnostic CT acquisitions, (ii) input for various dose modulation techniques, and (iii) recommendations for acquisition parameters and/or automatic parameter selection. It may also provide for an accurate attenuation correction map for positron emission tomography (PET) with only a small fraction of the radiation dose utilized.

Deep learning tomographic reconstruction through hierarchical decomposition of domain transforms

Deep learning (DL) has shown unprecedented performance for many image analysis and image enhancement tasks. Yet, solving large-scale inverse problems like tomographic reconstruction remains challenging for DL. These problems involve non-local and space-variant integral transforms between the input and output domains, for which no efficient neural network models are readily available. A prior attempt to solve tomographic reconstruction problems with supervised learning relied on a brute-force fully connected network and only allowed reconstruction with a 128⁴ system matrix size. This cannot practically scale to realistic data sizes such as 512⁴ and 512⁶ for three-dimensional datasets. Here we present a novel framework to solve such problems with DL by casting the original problem as a continuum of intermediate representations between the input and output domains. The original problem is broken down into a sequence of simpler transformations that can be well mapped onto an efficient hierarchical network architecture, with exponentially fewer parameters than a fully connected network would need. We applied the approach to computed tomography (CT) image reconstruction for a 512⁴ system matrix size. This work introduces a new kind of data-driven DL solver for full-size CT reconstruction without relying on the structure of direct (analytical) or iterative (numerical) inversion techniques. This work presents a feasibility demonstration of full-scale learnt reconstruction, whereas more developments will be needed to demonstrate superiority relative to traditional reconstruction approaches. The proposed approach is also extendable to other imaging problems such as emission and magnetic resonance reconstruction. More broadly, hierarchical DL opens the door to a new class of solvers for general inverse problems, which could potentially lead to improved signal-to-noise ratio, spatial resolution and computational efficiency in various areas.

A dual-domain neural network based on sinogram synthesis for sparse-view CT reconstruction

OBJECTIVE

The dual-domain deep learning-based reconstruction techniques have enjoyed many successful applications in the field of medical image reconstruction. Applying the analytical reconstruction based operator to transfer the data from the projection domain to the image domain, the dual-domain techniques may suffer from the insufficient suppression or removal of streak artifacts in areas with the missing view data, when addressing the sparse-view reconstruction problems. In this work, to overcome this problem, an intelligent sinogram synthesis based back-projection network (iSSBP-Net) was proposed for sparse-view computed tomography (CT) reconstruction. In the iSSBP-Net method, a convolutional neural network (CNN) was involved in the dual-domain method to inpaint the missing view data in the sinogram before CT reconstruction.

METHODS

The proposed iSSBP-Net method fused a sinogram synthesis sub-network (SS-Net), a sinogram filter sub-network (SF-Net), a back-projection layer, and a post-CNN into an end-to-end network. Firstly, to inpaint the missing view data, the SS-Net employed a CNN to synthesize the full-view sinogram in the projection domain. Secondly, to improve the visual quality of the sparse-view CT images, the synthesized sinogram was filtered by a CNN. Thirdly, the filtered sinogram was brought into the image domain through the back-projection layer. Finally, to yield images of high visual sensitivity, the post-CNN was applied to restore the desired images from the outputs of the back-projection layer.

RESULTS

The numerical experiments demonstrate that the proposed iSSBP-Net is superior to all competing algorithms under different scanning condintions for sparse-view CT reconstruction. Compared to the competing algorithms, the proposed iSSBP-Net method improved the peak signal-to-noise ratio of the reconstructed images about 1.21 dB, 0.26 dB, 0.01 dB, and 0.37 dB under the scanning conditions of 360, 180, 90, and 60 views, respectively.

CONCLUSION

The promising reconstruction results indicate that involving the SS-Net in the dual-domain method is could be an effective manner to suppress or remove the streak artifacts in sparse-view CT images. Due to the promising results reconstructed by the iSSBP-Net method, this study is intended to inspire the further development of sparse-view CT reconstruction by involving a SS-Net in the dual-domain method.

Self-Training Strategy Based on Finite Element Method for Adaptive Bioluminescence Tomography Reconstruction

Bioluminescence tomography (BLT) is a promising pre-clinical imaging technique for a wide variety of biomedical applications, which can non-invasively reveal functional activities inside living animal bodies through the detection of visible or near-infrared light produced by bioluminescent reactions. Recently, reconstruction approaches based on deep learning have shown great potential in optical tomography modalities. However, these reports only generate data with stationary patterns of constant target number, shape, and size. The neural networks trained by these data sets are difficult to reconstruct the patterns outside the data sets. This will tremendously restrict the applications of deep learning in optical tomography reconstruction. To address this problem, a self-training strategy is proposed for BLT reconstruction in this paper. The proposed strategy can fast generate large-scale BLT data sets with random target numbers, shapes, and sizes through an algorithm named random seed growth algorithm and the neural network is automatically self-trained. In addition, the proposed strategy uses the neural network to build a map between photon densities on surface and inside the imaged object rather than an end-to-end neural network that directly infers the distribution of sources from the photon density on surface. The map of photon density is further converted into the distribution of sources through the multiplication with stiffness matrix. Simulation, phantom, and mouse studies are carried out. Results show the availability of the proposed self-training strategy.

Sam's Net: A Self-Augmented Multistage Deep-Learning Network for End-to-End Reconstruction of Limited Angle CT

Limited angle reconstruction is a typical ill-posed problem in computed tomography (CT). Given incomplete projection data, images reconstructed by conventional analytical algorithms and iterative methods suffer from severe structural distortions and artifacts. In this paper, we proposed a self-augmented multi-stage deep-learning network (Sam's Net) for end-to-end reconstruction of limited angle CT. With the merit of the alternating minimization technique, Sam's Net integrates multi-stage self-constraints into cross-domain optimization to provide additional constraints on the manifold of neural networks. In practice, a sinogram completion network (SCNet) and artifact suppression network (ASNet), together with domain transformation layers constitute the backbone for cross-domain optimization. An online self-augmentation module was designed following the manner defined by alternating minimization, which enables a self-augmented learning procedure and multi-stage inference manner. Besides, a substitution operation was applied as a hard constraint for the solution space based on the data fidelity and a learnable weighting layer was constructed for data consistency refinement. Sam's Net forms a new framework for ill-posed reconstruction problems. In the training phase, the self-augmented procedure guides the optimization into a tightened solution space with enriched diverse data distribution and enhanced data consistency. In the inference phase, multi-stage prediction can improve performance progressively. Extensive experiments with both simulated and practical projections under 90-degree and 120-degree fan-beam configurations validate that Sam's Net can significantly improve the reconstruction quality with high stability and robustness.

Deep-Learning-Based Ultrasound Sound-Speed Tomography Reconstruction with Tikhonov Pseudo-Inverse Priori

Ultrasound sound-speed tomography (USST) is a promising technology for breast imaging and breast cancer detection. Its reconstruction is a complex non-linear mapping from the projection data to the sound-speed image (SSI). The traditional reconstruction methods include mainly the ray-based methods and the waveform-based methods. The ray-based methods with linear approximation have low computational cost but low reconstruction quality; the full wave-based methods with the complex non-linear model have high quality but high cost. To achieve both high quality and low cost, we introduced traditional linear approximation as prior knowledge into a deep neural network and treated the complex non-linear mapping of USST reconstruction as a combination of linear mapping and non-linear mapping. In the proposed method, the linear mapping was seamlessly implemented with a fully connected layer and initialized using the Tikhonov pseudo-inverse matrix. The non-linear mapping was implemented using a U-shape Net (U-Net). Furthermore, we proposed the Tikhonov U-shape net (TU-Net), in which the linear mapping was done before the non-linear mapping, and the U-shape Tikhonov net (UT-Net), in which the non-linear mapping was done before the linear mapping. Moreover, we conducted simulations and experiments for evaluation. In the numerical simulation, the root-mean-squared error was 6.49 and 4.29 m/s for the UT-Net and TU-Net, the peak signal-to-noise ratio was 49.01 and 52.90 dB, the structural similarity was 0.9436 and 0.9761 and the reconstruction time was 10.8 and 11.3 ms, respectively. In this study, the SSIs obtained with the proposed methods exhibited high sound-speed accuracy. Both the UT-Net and the TU-Net achieved high quality and low computational cost.

Application of medical imaging methods and artificial intelligence in tissue engineering and organ-on-a-chip

Organ-on-a-chip (OOC) is a new type of biochip technology. Various types of OOC systems have been developed rapidly in the past decade and found important applications in drug screening and precision medicine. However, due to the complexity in the structure of both the chip-body itself and the engineered-tissue inside, the imaging and analysis of OOC have still been a big challenge for biomedical researchers. Considering that medical imaging is moving towards higher spatial and temporal resolution and has more applications in tissue engineering, this paper aims to review medical imaging methods, including CT, micro-CT, MRI, small animal MRI, and OCT, and introduces the application of 3D printing in tissue engineering and OOC in which medical imaging plays an important role. The achievements of medical imaging assisted tissue engineering are reviewed, and the potential applications of medical imaging in organoids and OOC are discussed. Moreover, artificial intelligence - especially deep learning - has demonstrated its excellence in the analysis of medical imaging; we will also present the application of artificial intelligence in the image analysis of 3D tissues, especially for organoids developed in novel OOC systems.

A review on the application of deep learning for CT reconstruction, bone segmentation and surgical planning in oral and maxillofacial surgery

Computer-assisted surgery (CAS) allows clinicians to personalize treatments and surgical interventions and has therefore become an increasingly popular treatment modality in maxillofacial surgery. The current maxillofacial CAS consists of three main steps: (1) CT image reconstruction, (2) bone segmentation, and (3) surgical planning. However, each of these three steps can introduce errors that can heavily affect the treatment outcome. As a consequence, tedious and time-consuming manual post-processing is often necessary to ensure that each step is performed adequately. One way to overcome this issue is by developing and implementing neural networks (NNs) within the maxillofacial CAS workflow. These learning algorithms can be trained to perform specific tasks without the need for explicitly defined rules. In recent years, an extremely large number of novel NN approaches have been proposed for a wide variety of applications, which makes it a difficult task to keep up with all relevant developments. This study therefore aimed to summarize and review all relevant NN approaches applied for CT image reconstruction, bone segmentation, and surgical planning. After full text screening, 76 publications were identified: 32 focusing on CT image reconstruction, 33 focusing on bone segmentation and 11 focusing on surgical planning. Generally, convolutional NNs were most widely used in the identified studies, although the multilayer perceptron was most commonly applied in surgical planning tasks. Moreover, the drawbacks of current approaches and promising research avenues are discussed.

Deep learning methods for enhancing cone-beam CT image quality towards adaptive radiation therapy: A systematic review

The use of deep learning (DL) to improve cone-beam CT (CBCT) image quality has gained popularity as computational resources and algorithmic sophistication have advanced in tandem. CBCT imaging has the potential to facilitate online adaptive radiation therapy (ART) by utilizing up-to-date patient anatomy to modify treatment parameters before irradiation. Poor CBCT image quality has been an impediment to realizing ART due to the increased scatter conditions inherent to cone-beam acquisitions. Given the recent interest in DL applications in radiation oncology, and specifically DL for CBCT correction, we provide a systematic theoretical and literature review for future stakeholders. The review encompasses DL approaches for synthetic CT generation, as well as projection domain methods employed in the CBCT correction literature. We review trends pertaining to publications from January 2018 to April 2022 and condense their major findings - with emphasis on study design and deep learning techniques. Clinically relevant endpoints relating to image quality and dosimetric accuracy are summarised, highlighting gaps in the literature. Finally, we make recommendations for both clinicians and DL practitioners based on literature trends and the current DL state of the art methods utilized in radiation oncology. This article is protected by copyright. All rights reserved.

Physics-assisted generative adversarial network for X-ray tomography

X-ray tomography is capable of imaging the interior of objects in three dimensions non-invasively, with applications in biomedical imaging, materials science, electronic inspection, and other fields. The reconstruction process can be an ill-conditioned inverse problem, requiring regularization to obtain satisfactory results. Recently, deep learning has been adopted for tomographic reconstruction. Unlike iterative algorithms which require a distribution that is known a priori, deep reconstruction networks can learn a prior distribution through sampling the training distributions. In this work, we develop a Physics-assisted Generative Adversarial Network (PGAN), a two-step algorithm for tomographic reconstruction. In contrast to previous efforts, our PGAN utilizes maximum-likelihood estimates derived from the measurements to regularize the reconstruction with both known physics and the learned prior. Compared with methods with less physics assisting in training, PGAN can reduce the photon requirement with limited projection angles to achieve a given error rate. The advantages of using a physics-assisted learned prior in X-ray tomography may further enable low-photon nanoscale imaging.

NeRP: Implicit Neural Representation Learning With Prior Embedding for Sparsely Sampled Image Reconstruction

Image reconstruction is an inverse problem that solves for a computational image based on sampled sensor measurement. Sparsely sampled image reconstruction poses additional challenges due to limited measurements. In this work, we propose a methodology of implicit Neural Representation learning with Prior embedding (NeRP) to reconstruct a computational image from sparsely sampled measurements. The method differs fundamentally from previous deep learning-based image reconstruction approaches in that NeRP exploits the internal information in an image prior and the physics of the sparsely sampled measurements to produce a representation of the unknown subject. No large-scale data is required to train the NeRP except for a prior image and sparsely sampled measurements. In addition, we demonstrate that NeRP is a general methodology that generalizes to different imaging modalities such as computed tomography (CT) and magnetic resonance imaging (MRI). We also show that NeRP can robustly capture the subtle yet significant image changes required for assessing tumor progression.

Patient‐specific synthetic magnetic resonance imaging generation from cone beam computed tomography for image guidance in liver stereotactic body radiation therapy

Objective

Despite its prevalence, cone beam computed tomography (CBCT) has poor soft-tissue contrast, making it challenging to localize liver tumors. We propose a patient-specific deep learning model to generate synthetic magnetic resonance imaging (MRI) from CBCT to improve tumor localization.

Methods

A key innovation is using patient-specific CBCT-MRI image pairs to train a deep learning model to generate synthetic MRI from CBCT. Specifically, patient planning CT was deformably registered to prior MRI, and then used to simulate CBCT with simulated projections and Feldkamp, Davis, and Kress reconstruction. These CBCT-MRI images were augmented using translations and rotations to generate enough patient-specific training data. A U-Net-based deep learning model was developed and trained to generate synthetic MRI from CBCT in the liver, and then tested on a different CBCT dataset. Synthetic MRIs were quantitatively evaluated against ground-truth MRI.

Results

The synthetic MRI demonstrated superb soft-tissue contrast with clear tumor visualization. On average, the synthetic MRI achieved 28.01, 0.025, and 0.929 for peak signal-to-noise ratio, mean square error, and structural similarity index, respectively, outperforming CBCT images. The model performance was consistent across all three patients tested.

Conclusion

Our study demonstrated the feasibility of a patient-specific model to generate synthetic MRI from CBCT for liver tumor localization, opening up a potential to democratize MRI guidance in clinics with conventional LINACs.

A joint alignment and reconstruction algorithm for electron tomography to visualize in-depth cell-to-cell interactions

Electron tomography allows one to obtain 3D reconstructions visualizing a tissue's ultrastructure from a series of 2D projection images. An inherent problem with this imaging technique is that its projection images contain unwanted shifts, which must be corrected for to achieve reliable reconstructions. Commonly, the projection images are aligned with each other by means of fiducial markers prior to the reconstruction procedure. In this work, we propose a joint alignment and reconstruction algorithm that iteratively solves for both the unknown reconstruction and the unintentional shift and does not require any fiducial markers. We evaluate the approach first on synthetic phantom data where the focus is not only on the reconstruction quality but more importantly on the shift correction. Subsequently, we apply the algorithm to healthy C57BL/6J mice and then compare it with non-obese diabetic (NOD) mice, with the aim of visualizing the attack of immune cells on pancreatic beta cells within type 1 diabetic mice at a more profound level through 3D analysis. We empirically demonstrate that the proposed algorithm is able to compute the shift with a remaining error at only the sub-pixel level and yields high-quality reconstructions for the limited-angle inverse problem. By decreasing labour and material costs, the algorithm facilitates further research directed towards investigating the immune system's attacks in pancreata of NOD mice for numerous samples at different stages of type 1 diabetes.

Calibration by differentiation - Self-supervised calibration for X-ray microscopy using a differentiable cone-beam reconstruction operator

High-resolution X-ray microscopy (XRM) is gaining interest for biological investigations of extremely small-scale structures. XRM imaging of bones in living mice could provide new insights into the emergence and treatment of osteoporosis by observing osteocyte lacunae, which are holes in the bone of few micrometers in size. Imaging living animals at that resolution, however, is extremely challenging and requires very sophisticated data processing converting the raw XRM detector output into reconstructed images. This paper presents an open-source, differentiable reconstruction pipeline for XRM data which analytically computes the final image from the raw measurements. In contrast to most proprietary reconstruction software, it offers the user full control over each processing step and, additionally, makes the entire pipeline deep learning compatible by ensuring differentiability. This allows fitting trainable modules both before and after the actual reconstruction step in a purely data-driven way using the gradient-based optimizers of common deep learning frameworks. The value of such differentiability is demonstrated by calibrating the parameters of a simple cupping correction module operating on the raw projection images using only a self-supervisory quality metric based on the reconstructed volume and no further calibration measurements. The retrospective calibration directly improves image quality as it avoids cupping artifacts and decreases the difference in gray values between outer and inner bone by 68% to 94%. Furthermore, it makes the reconstruction process entirely independent of the XRM manufacturer and paves the way to explore modern deep learning reconstruction methods for arbitrary XRM and, potentially, other flat-panel CT systems. This exemplifies how differentiable reconstruction can be leveraged in the context of XRM and, hence, is an important step toward the goal of reducing the resolution limit of in-vivo bone imaging to the single micrometer domain. This article is protected by copyright. All rights reserved.

A geometry-guided multi-beamlet deep learning technique for CT reconstruction

Purpose. Previous studies have proposed deep-learning techniques to reconstruct CT images from sinograms. However, these techniques employ large fully-connected (FC) layers for projection-to-image domain transformation, producing large models requiring substantial computation power, potentially exceeding the computation memory limit. Our previous work proposed a geometry-guided-deep-learning (GDL) technique for CBCT reconstruction that reduces model size and GPU memory consumption. This study further develops the technique and proposes a novel multi-beamlet deep learning (GMDL) technique of improved performance. The study compares the proposed technique with the FC layer-based deep learning (FCDL) method and the GDL technique through low-dose real-patient CT image reconstruction.Methods. Instead of using a large FC layer, the GMDL technique learns the projection-to-image domain transformation by constructing many small FC layers. In addition to connecting each pixel in the projection domain to beamlet points along the central beamlet in the image domain as GDL does, these smaller FC layers in GMDL connect each pixel to beamlets peripheral to the central beamlet based on the CT projection geometry. We compare ground truth images with low-dose images reconstructed with the GMDL, the FCDL, the GDL, and the conventional FBP methods. The images are quantitatively analyzed in terms of peak-signal-to-noise-ratio (PSNR), structural-similarity-index-measure (SSIM), and root-mean-square-error (RMSE).Results. Compared to other methods, the GMDL reconstructed low-dose CT images show improved image quality in terms of PSNR, SSIM, and RMSE. The optimal number of peripheral beamlets for the GMDL technique is two beamlets on each side of the central beamlet. The model size and memory consumption of the GMDL model is less than 1/100 of the FCDL model.Conclusion. Compared to the FCDL method, the GMDL technique is demonstrated to be able to reconstruct real patient low-dose CT images of improved image quality with significantly reduced model size and GPU memory requirement.

A Hybrid Recursive Implementation of Broad Learning With Incremental Features

The broad learning system (BLS) paradigm has recently emerged as a computationally efficient approach to supervised learning. Its efficiency arises from a learning mechanism based on the method of least-squares. However, the need for storing and inverting large matrices can put the efficiency of such mechanism at risk in big-data scenarios. In this work, we propose a new implementation of BLS in which the need for storing and inverting large matrices is avoided. The distinguishing features of the designed learning mechanism are as follows: 1) the training process can balance between efficient usage of memory and required iterations (hybrid recursive learning) and 2) retraining is avoided when the network is expanded (incremental learning). It is shown that, while the proposed framework is equivalent to the standard BLS in terms of trained network weights,much larger networks than the standard BLS can be smoothly trained by the proposed solution, projecting BLS toward the big-data frontier.

Convex optimization algorithms in medical image reconstruction-in the age of AI

The past decade has seen the rapid growth of model based image reconstruction (MBIR) algorithms, which are often applications or adaptations of convex optimization algorithms from the optimization community. We review some state-of-the-art algorithms that have enjoyed wide popularity in medical image reconstruction, emphasize known connections between different algorithms, and discuss practical issues such as computation and memory cost. More recently, deep learning (DL) has forayed into medical imaging, where the latest development tries to exploit the synergy between DL and MBIR to elevate the MBIR's performance. We present existing approaches and emerging trends in DL-enhanced MBIR methods, with particular attention to the underlying role of convexity and convex algorithms on network architecture. We also discuss how convexity can be employed to improve the generalizability and representation power of DL networks in general.