NeRP: Implicit Neural Representation Learning With Prior Embedding for Sparsely Sampled Image Reconstruction

Coordinate-based neural representation enabling zero-shot learning for fast 3D multiparametric quantitative MRI

Quantitative magnetic resonance imaging (qMRI) offers tissue-specific physical parameters with significant potential for neuroscience research and clinical practice. However, lengthy scan times for 3D multiparametric qMRI acquisition limit its clinical utility. Here, we propose SUMMIT, an innovative imaging methodology that includes data acquisition and an unsupervised reconstruction for simultaneous multiparametric qMRI. SUMMIT first encodes multiple important quantitative properties into highly undersampled k-space. It further leverages implicit neural representation incorporated with a dedicated physics model to reconstruct the desired multiparametric maps without needing external training datasets. SUMMIT delivers co-registered T1, T2, T2∗, and subvoxel quantitative susceptibility mapping. Extensive simulations, phantom, and in vivo brain imaging demonstrate SUMMIT's high accuracy. Notably, SUMMIT uniquely unravels microstructural alternations in patients with white matter hyperintense lesions with high sensitivity and specificity. Additionally, the proposed unsupervised approach for qMRI reconstruction also introduces a novel zero-shot learning paradigm for multiparametric imaging applicable to various medical imaging modalities.

Implicit neural representation-based method for metal-induced beam hardening artifact reduction in X-ray CT imaging

BACKGROUND

In X-ray computed tomography (CT), metal-induced beam hardening artifacts arise from the complex interactions between polychromatic X-ray beams and metallic objects, leading to degraded image quality and impeding accurate diagnosis. A previously proposed metal-induced beam hardening correction (MBHC) method provides a theoretical framework for addressing nonlinear artifacts through mathematical analysis, with its effectiveness demonstrated by numerical simulations and phantom experiments. However, in practical applications, this method relies on precise segmentation of highly attenuating materials and parameter estimations, which limit its ability to fully correct artifacts caused by the intricate interactions between metals and other dense materials, such as bone or teeth.

PURPOSE

This study aims to develop a parameter-free MBHC method that eliminates the need for accurate segmentation and parameter estimations, thereby addressing the limitations of the original MBHC approach.

METHODS

The proposed method employs implicit neural representations (INR) to generate two tomographic images: one representing the monochromatic attenuation distribution at a specific energy level, and another capturing the nonlinear beam hardening effects caused by the polychromatic nature of X-ray beams. A loss function drives the generation of these images, where the predicted projection data is nonlinearly modeled by the combination of the two images. This approach eliminates the need for geometric and parameter estimation of metals, providing a more generalized solution.

RESULTS

Numerical and phantom experiments demonstrates that the proposed method effectively reduces beam hardening artifacts caused by interactions between highly attenuating materials such as metals, bone, and teeth. Additionally, the proposed INR-based method demonstrates potential in addressing challenges related to data insufficiencies, such as photon starvation and truncated fields of view in CT imaging.

CONCLUSIONS

The proposed generalized MBHC method provides high-quality image reconstructions without requiring parameter estimations and segmentations, offering a robust solution for reducing metal-induced beam hardening artifacts in CT imaging.

Deep prior embedding method for Electrical Impedance Tomography

This paper presents a novel deep learning-based approach for Electrical Impedance Tomography (EIT) reconstruction that effectively integrates image priors to enhance reconstruction quality. Traditional neural network methods often rely on random initialization, which may not fully exploit available prior information. Our method addresses this by using image priors to guide the initialization of the neural network, allowing for a more informed starting point and better utilization of prior knowledge throughout the reconstruction process. We explore three different strategies for embedding prior information: non-prior embedding, implicit prior embedding, and full prior embedding. Through simulations and experimental studies, we demonstrate that the incorporation of accurate image priors significantly improves the fidelity of the reconstructed conductivity distribution. The method is robust across varying levels of noise in the measurement data, and the quality of the reconstruction is notably higher when the prior closely resembles the true distribution. This work highlights the importance of leveraging prior information in EIT and provides a framework that could be extended to other inverse problems where prior knowledge is available.

Continuous implicit neural representation for arbitrary super-resolution of system matrix in magnetic particle imaging

Objective. Magnetic particle imaging (MPI) is a novel imaging technique that uses magnetic fields to detect tracer materials consisting of magnetic nanoparticles. System matrix (SM) based image reconstruction is essential for achieving high image quality in MPI. However, the time-consuming SM calibrations need to be repeated whenever the magnetic field's or nanoparticle's characteristics change. Accelerating this calibration process is therefore crucial. The most common acceleration approach involves undersampling during the SM calibration procedure, followed by super-resolution methods to recover the high-resolution SM. However, these methods typically require separate training of multiple models for different undersampling ratios, leading to increased storage and training time costs.Approach. We propose an arbitrary-scale SM super-resolution method based on continuous implicit neural representation (INR). Using INR, the SM is modeled as a continuous function in space, enabling arbitrary-scale super-resolution by sampling the function at different densities. A cross-frequency encoder is implemented to share SM frequency information and analyze contextual relationships, resulting in a more intelligent and efficient sampling strategy. Convolutional neural networks (CNNs) are utilized to learn and optimize the grid sampling process in INR, leveraging the advantage of CNNs in learning local feature associations and considering surrounding information comprehensively.Main results. Experimental results on OpenMPI demonstrate that our method outperforms existing methods and enables calibration at any scale with a single model. The proposed method achieves high accuracy and efficiency in SM recovery, even at high undersampling rates.Significance. The proposed method significantly reduces the storage and training time costs associated with SM calibration, making it more practical for real-world applications. By enabling arbitrary-scale super-resolution with a single model, our approach enhances the flexibility and efficiency of MPI systems, paving the way for more widespread adoption of MPI technology.

Low-Dose CT Image Super-Resolution With Noise Suppression Based on Prior Degradation Estimator and Self-Guidance Mechanism

The anatomies in low-dose computer tomography (LDCT) are usually distorted during the zooming-in observation process due to the small amount of quantum. Super-resolution (SR) methods have been proposed to enhance qualities of LDCT images as post-processing approaches without increasing radiation damage to patients, but suffered from incorrect prediction of degradation information and incomplete leverage of internal connections within the 3D CT volume, resulting in the imbalance between noise removal and detail sharpening in the super-resolution results. In this paper, we propose a novel LDCT SR network where the degradation information self-parsed from the LDCT slice and the 3D anatomical information captured from the LDCT volume are integrated to guide the backbone network. The prior degradation estimator (PDE) is proposed following the contrastive learning strategy to estimate the degradation features in the LDCT images without paired low-normal dose CT images. The self-guidance fusion module (SGFM) is designed to capture anatomical features with internal 3D consistencies between the squashed images along the coronal, sagittal, and axial views of the CT volume. Finally, the features representing degradation and anatomical structures are integrated to recover the CT images with higher resolutions. We apply the proposed method to the 2016 NIH-AAPM Mayo Clinic LDCT Grand Challenge dataset and our collected LDCT dataset to evaluate its ability to recover LDCT images. Experimental results illustrate the superiority of our network concerning quantitative metrics and qualitative observations, demonstrating its potential in recovering detail-sharp and noise-free CT images with higher resolutions from the practical LDCT images.

Geometry-Aware Attenuation Learning for Sparse-View CBCT Reconstruction

Cone Beam Computed Tomography (CBCT) plays a vital role in clinical imaging. Traditional methods typically require hundreds of 2D X-ray projections to reconstruct a high-quality 3D CBCT image, leading to considerable radiation exposure. This has led to a growing interest in sparse-view CBCT reconstruction to reduce radiation doses. While recent advances, including deep learning and neural rendering algorithms, have made strides in this area, these methods either produce unsatisfactory results or suffer from time inefficiency of individual optimization. In this paper, we introduce a novel geometry-aware encoder-decoder framework to solve this problem. Our framework starts by encoding multi-view 2D features from various 2D X-ray projections with a 2D CNN encoder. Leveraging the geometry of CBCT scanning, it then back-projects the multi-view 2D features into the 3D space to formulate a comprehensive volumetric feature map, followed by a 3D CNN decoder to recover 3D CBCT image. Importantly, our approach respects the geometric relationship between 3D CBCT image and its 2D X-ray projections during feature back projection stage, and enjoys the prior knowledge learned from the data population. This ensures its adaptability in dealing with extremely sparse view inputs without individual training, such as scenarios with only 5 or 10 X-ray projections. Extensive evaluations on two simulated datasets and one real-world dataset demonstrate exceptional reconstruction quality and time efficiency of our method.

Deep Ensemble Transformers for Dimensionality Reduction

We propose deep ensemble transformers (DETs), a fast, scalable approach for dimensionality reduction problems. This method leverages the power of deep neural networks and employs cascade ensemble techniques as its fundamental feature extraction tool. To handle high-dimensional data, our approach employs a flexible number of intermediate layers sequentially. These layers progressively transform the input data into decision tree predictions. To further enhance prediction performance, the output from the final intermediate layer is fed through a feed-forward neural network architecture for final prediction. We derive an upper bound of the disparity between the generalization error and the empirical error and demonstrate that it converges to zero. This highlights the generalizability of our method to parameter estimation and feature selection problems. In our experimental evaluations, DETs outperform existing models in terms of prediction accuracy, representation learning ability, and computational time. Specifically, the method achieves over 95% accuracy in gene expression data and can be trained on average 50% faster than traditional artificial neural networks (ANNs).

Improving Efficiency of Iso-Surface Extraction on Implicit Neural Representations Using Uncertainty Propagation

Implicit Neural representations (INRs) are widely used for scientific data reduction and visualization by modeling the function that maps a spatial location to a data value. Without any prior knowledge about the spatial distribution of values, we are forced to sample densely from INRs to perform visualization tasks like iso-surface extraction which can be very computationally expensive. Recently, range analysis has shown promising results in improving the efficiency of geometric queries, such as ray casting and hierarchical mesh extraction, on INRs for 3D geometries by using arithmetic rules to bound the output range of the network within a spatial region. However, the analysis bounds are often too conservative for complex scientific data. In this article, we present an improved technique for range analysis by revisiting the arithmetic rules and analyzing the probability distribution of the network output within a spatial region. We model this distribution efficiently as a Gaussian distribution by applying the central limit theorem. Excluding low probability values, we are able to tighten the output bounds, resulting in a more accurate estimation of the value range, and hence more accurate identification of iso-surface cells and more efficient iso-surface extraction on INRs. Our approach demonstrates superior performance in terms of the iso-surface extraction time on four datasets compared to the original range analysis method and can also be generalized to other geometric query tasks.

Highly accelerated MRI via implicit neural representation guided posterior sampling of diffusion models

Reconstructing high-fidelity magnetic resonance (MR) images from under-sampled k-space is a commonly used strategy to reduce scan time. The posterior sampling of diffusion models based on the real measurement data holds significant promise of improved reconstruction accuracy. However, traditional posterior sampling methods often lack effective data consistency guidance, leading to inaccurate and unstable reconstructions. Implicit neural representation (INR) has emerged as a powerful paradigm for solving inverse problems by modeling a signal's attributes as a continuous function of spatial coordinates. In this study, we present a novel posterior sampler for diffusion models using INR, named DiffINR. The INR-based component incorporates both the diffusion prior distribution and the MRI physical model to ensure high data fidelity. DiffINR demonstrates superior performance on in-distribution datasets with remarkable accuracy, even under high acceleration factors (up to R = 12 in single-channel reconstruction). Furthermore, DiffINR exhibits excellent generalizability across various tissue contrasts and anatomical structures with low uncertainty. Overall, DiffINR significantly improves MRI reconstruction in terms of accuracy, generalizability and stability, paving the way for further accelerating MRI acquisition. Notably, our proposed framework can be a generalizable framework to solve inverse problems in other medical imaging tasks.

Real-time CBCT imaging and motion tracking via a single arbitrarily-angled x-ray projection by a joint dynamic reconstruction and motion estimation (DREME) framework

Objective.Real-time cone-beam computed tomography (CBCT) provides instantaneous visualization of patient anatomy for image guidance, motion tracking, and online treatment adaptation in radiotherapy. While many real-time imaging and motion tracking methods leveraged patient-specific prior information to alleviate under-sampling challenges and meet the temporal constraint (<500 ms), the prior information can be outdated and introduce biases, thus compromising the imaging and motion tracking accuracy. To address this challenge, we developed a frameworkdynamicreconstruction andmotionestimation (DREME) for real-time CBCT imaging and motion estimation, without relying on patient-specific prior knowledge.Approach.DREME incorporates a deep learning-based real-time CBCT imaging and motion estimation method into a dynamic CBCT reconstruction framework. The reconstruction framework reconstructs a dynamic sequence of CBCTs in a data-driven manner from a standard pre-treatment scan, without requiring patient-specific prior knowledge. Meanwhile, a convolutional neural network-based motion encoder is jointly trained during the reconstruction to learn motion-related features relevant for real-time motion estimation, based on a single arbitrarily-angled x-ray projection. DREME was tested on digital phantom simulations and real patient studies.Main Results.DREME accurately solved 3D respiration-induced anatomical motion in real time (∼1.5 ms inference time for each x-ray projection). For the digital phantom studies, it achieved an average lung tumor center-of-mass localization error of 1.2 ± 0.9 mm (Mean ± SD). For the patient studies, it achieved a real-time tumor localization accuracy of 1.6 ± 1.6 mm in the projection domain.Significance.DREME achieves CBCT and volumetric motion estimation in real time from a single x-ray projection at arbitrary angles, paving the way for future clinical applications in intra-fractional motion management. In addition, it can be used for dose tracking and treatment assessment, when combined with real-time dose calculation.

Efficient and accurate commissioning and quality assurance of radiosurgery beam via prior-embedded implicit neural representation learning

BACKGROUND

Dosimetric commissioning and quality assurance (QA) for linear accelerators (LINACs) present a significant challenge for clinical physicists due to the high measurement workload and stringent precision standards. This challenge is exacerbated for radiosurgery LINACs because of increased measurement uncertainty and more demanding setup accuracy for small-field beams. Optimizing physicists' effort during beam measurements while ensuring the quality of the measured data is crucial for clinical efficiency and patient safety.

PURPOSE

To develop a radiosurgery LINAC beam model that embeds prior knowledge of beam data through implicit neural representation (NeRP) learning and to evaluate the model's effectiveness in guiding beam data sampling, predicting complete beam dataset from sparse samples, and verifying detector choice and setup during commissioning and QA.

MATERIALS AND METHODS

Beam data including lateral profile and tissue-phantom-ratio (TPR), collected from CyberKnife LINACs, were investigated. Multi-layer perceptron (MLP) neural networks were optimized to parameterize a continuous function of the beam data, implicitly defined by the mapping from measurement coordinates to measured dose values. Beam priors were embedded into network weights by first training the network to learn the NeRP of a vendor-provided reference dataset. The prior-embedded network was further fine-tuned with sparse clinical measurements and used to predict unacquired beam data. Prospective and retrospective evaluations of different beam data samples in finetuning the model were performed using the reference beam dataset and clinical testing datasets, respectively. Model prediction accuracy was evaluated over 10 clinical datasets collected from various LINACs with different manufacturing modes and collimation systems. Model sensitivity in detecting beam data acquisition errors including inaccurate detector positioning and inappropriate detector choice was evaluated using two additional datasets with intentionally introduced erroneous samples.

RESULTS

Prospective and retrospective evaluations identified consistent beam data samples that are most effective in fine-tuning the model for complete beam data prediction. Despite of discrepancies between clinical beam and the reference beam, fine-tuning the model with sparse beam profile measured at a single depth or with beam TPR measured at a single collimator size predicted beam data that closely match ground truth water tank measurements. Across the 10 clinical beam datasets, the averaged mean absolute error (MAE) in percentage dose was lower than 0.5% and the averaged 1D Gamma passing rate (1%/0.5  mm for profile and 1%/1  mm for TPR) was higher than 99%. In contrast, the MAE and Gamma passing rates were above 1% and below 95% between the reference beam dataset and clinical beam datasets. Model sensitivity to beam data acquisition errors was demonstrated by significant model prediction changes when fine-tuned with erroneous versus correct beam data samples, as quantified by a Gamma passing rate as low as 18.16% between model predictions.

CONCLUSION

A model for small-field radiosurgery beam was proposed that embeds prior knowledge of beam properties and predicts the entire beam data from sparse measurements. The model can serve as a valuable tool for clinical physicists to verify the accuracy of beam data acquisition and promises to improve commissioning and QA reliability and efficiency with substantially reduced number of beam measurements.

An end-to-end implicit neural representation architecture for medical volume data

Medical volume data are rapidly increasing, growing from gigabytes to petabytes, which presents significant challenges in organisation, storage, transmission, manipulation, and rendering. To address the challenges, we propose an end-to-end architecture for data compression, leveraging advanced deep learning technologies. This architecture consists of three key modules: downsampling, implicit neural representation (INR), and super-resolution (SR). We employ a trade-off point method to optimise each module's performance and achieve the best balance between high compression rates and reconstruction quality. Experimental results on multi-parametric MRI data demonstrate that our method achieves a high compression rate of up to 97.5% while maintaining superior reconstruction accuracy, with a Peak Signal-to-Noise Ratio (PSNR) of 40.05 dB and Structural Similarity Index (SSIM) of 0.96. This approach significantly reduces GPU memory requirements and processing time, making it a practical solution for handling large medical datasets.

Computationally Efficient Implicit Training Strategy for Unrolled Networks (IMUNNE): A Preliminary Analysis Using Accelerated Real-Time Cardiac Cine MRI

OBJECTIVE

Highly-undersampled, dynamic MRI reconstruction, particularly in multi-coil scenarios, is a challenging inverse problem. Unrolled networks achieve state-of-the-art performance in MRI reconstruction but suffer from long training times and extensive GPU memory cost.

METHODS

In this work, we propose a novel training strategy for IMplicit UNrolled NEtworks (IMUNNE) for highly-undersampled, multi-coil dynamic MRI reconstruction. It formulates the MRI reconstruction problem as an implicit fixed-point equation and leverages gradient approximation for backpropagation, enabling training of deep architectures with fixed memory cost. This study represents the first application of implicit network theory in the context of real-time cine MRI. The proposed method is evaluated using a prospectively undersampled, real-time cine dataset using radial k-space sampling, comprising balanced steady-state free precession (b-SSFP) readouts. Experiments include a hyperparameter search, head-to-head comparisons with a complex U-Net (CU-Net) and an alternating unrolled network (Alt-UN), and an analysis of robustness under noise perturbations; peak signal-to-noise ratio, structural similarity index, normalized root mean-square error, spatio-temporal entropic difference, and a blur metric were used.

RESULTS

IMUNNE produced significantly and slightly better image quality compared to CU-Net and Alt-UN, respectively. Compared with Alt-UN, IMUNNE significantly reduced training and inference times, making it a promising approach for highly-accelerated, multi-coil real-time cine MRI reconstruction.

CONCLUSION

IMUNNE strategy successfully applies unrolled networks to image reconstruction of highly-accelerated, real-time radial cine MRI.

SIGNIFICANCE

Implicit training enables rapid, high-quality, and cost-effective CMR exams by reducing training and inference times and lowering memory cost associated with advanced reconstruction methods.

Sparse representation for restoring images by exploiting topological structure of graph of patches

Image restoration poses a significant challenge, aiming to accurately recover damaged images by delving into their inherent characteristics. Various models and algorithms have been explored by researchers to address different types of image distortions, including sparse representation, grouped sparse representation, and low‐rank self‐representation. The grouped sparse representation algorithm leverages the prior knowledge of non‐local self‐similarity and imposes sparsity constraints to maintain texture information within images. To further exploit the intrinsic properties of images, this study proposes a novel low‐rank representation‐guided grouped sparse representation image restoration algorithm. This algorithm integrates self‐representation models and trace optimization techniques to effectively preserve the original image structure, thereby enhancing image restoration performance while retaining the original texture and structural information. The proposed method was evaluated on image denoising and deblocking tasks across several datasets, demonstrating promising results.

Multi-Sensor Learning Enables Information Transfer Across Different Sensory Data and Augments Multi-Modality Imaging

Multi-modality imaging is widely used in clinical practice and biomedical research to gain a comprehensive understanding of an imaging subject. Currently, multi-modality imaging is accomplished by post hoc fusion of independently reconstructed images under the guidance of mutual information or spatially registered hardware, which limits the accuracy and utility of multi-modality imaging. Here, we investigate a data-driven multi-modality imaging (DMI) strategy for synergetic imaging of CT and MRI. We reveal two distinct types of features in multi-modality imaging, namely intra- and inter-modality features, and present a multi-sensor learning (MSL) framework to utilize the crossover inter-modality features for augmented multi-modality imaging. The MSL imaging approach breaks down the boundaries of traditional imaging modalities and allows for optimal hybridization of CT and MRI, which maximizes the use of sensory data. We showcase the effectiveness of our DMI strategy through synergetic CT-MRI brain imaging. The principle of DMI is quite general and holds enormous potential for various DMI applications across disciplines.

NeCA: 3D Coronary Artery Tree Reconstruction from Two 2D Projections via Neural Implicit Representation

Cardiovascular diseases (CVDs) are the most common health threats worldwide. 2D X-ray invasive coronary angiography (ICA) remains the most widely adopted imaging modality for CVD assessment during real-time cardiac interventions. However, it is often difficult for the cardiologists to interpret the 3D geometry of coronary vessels based on 2D planes. Moreover, due to the radiation limit, often only two angiographic projections are acquired, providing limited information of the vessel geometry and necessitating 3D coronary tree reconstruction based only on two ICA projections. In this paper, we propose a self-supervised deep learning method called NeCA, which is based on neural implicit representation using the multiresolution hash encoder and differentiable cone-beam forward projector layer, in order to achieve 3D coronary artery tree reconstruction from two 2D projections. We validate our method using six different metrics on a dataset generated from coronary computed tomography angiography of right coronary artery and left anterior descending artery. The evaluation results demonstrate that our NeCA method, without requiring 3D ground truth for supervision or large datasets for training, achieves promising performance in both vessel topology and branch-connectivity preservation compared to the supervised deep learning model.

Continuous 3D Myocardial Motion Tracking via Echocardiography

Myocardial motion tracking stands as an essential clinical tool in the prevention and detection of cardiovascular diseases (CVDs), the foremost cause of death globally. However, current techniques suffer from incomplete and inaccurate motion estimation of the myocardium in both spatial and temporal dimensions, hindering the early identification of myocardial dysfunction. To address these challenges, this paper introduces the Neural Cardiac Motion Field (NeuralCMF). NeuralCMF leverages implicit neural representation (INR) to model the 3D structure and the comprehensive 6D forward/backward motion of the heart. This method surpasses pixel-wise limitations by offering the capability to continuously query the precise shape and motion of the myocardium at any specific point throughout the cardiac cycle, enhancing the detailed analysis of cardiac dynamics beyond traditional speckle tracking. Notably, NeuralCMF operates without the need for paired datasets, and its optimization is self-supervised through the physics knowledge priors in both space and time dimensions, ensuring compatibility with both 2D and 3D echocardiogram video inputs. Experimental validations across three representative datasets support the robustness and innovative nature of the NeuralCMF, marking significant advantages over existing state-of-the-art methods in cardiac imaging and motion tracking. Code is available at: https://njuvision.github.io/NeuralCMF.

SOUL-Net: A Sparse and Low-Rank Unrolling Network for Spectral CT Image Reconstruction

Spectral computed tomography (CT) is an emerging technology, that generates a multienergy attenuation map for the interior of an object and extends the traditional image volume into a 4-D form. Compared with traditional CT based on energy-integrating detectors, spectral CT can make full use of spectral information, resulting in high resolution and providing accurate material quantification. Numerous model-based iterative reconstruction methods have been proposed for spectral CT reconstruction. However, these methods usually suffer from difficulties such as laborious parameter selection and expensive computational costs. In addition, due to the image similarity of different energy bins, spectral CT usually implies a strong low-rank prior, which has been widely adopted in current iterative reconstruction models. Singular value thresholding (SVT) is an effective algorithm to solve the low-rank constrained model. However, the SVT method requires a manual selection of thresholds, which may lead to suboptimal results. To relieve these problems, in this article, we propose a sparse and low-rank unrolling network (SOUL-Net) for spectral CT image reconstruction, that learns the parameters and thresholds in a data-driven manner. Furthermore, a Taylor expansion-based neural network backpropagation method is introduced to improve the numerical stability. The qualitative and quantitative results demonstrate that the proposed method outperforms several representative state-of-the-art algorithms in terms of detail preservation and artifact reduction.

Scan-Specific Unsupervised Highly Accelerated Non-Cartesian CEST Imaging Using Implicit Neural Representation and Explicit Sparse Prior

OBJECTIVE

Chemical exchange saturation transfer (CEST) is a promising magnetic resonance imaging (MRI) technique. CEST imaging usually requires a long scan time, and reducing acquisition time is highly desirable for clinical applications.

METHODS

A novel scan-specific unsupervised deep learning algorithm is proposed to accelerate steady-state pulsed CEST imaging with golden-angle stack-of-stars trajectory using hybrid-feature hash encoding implicit neural representation. Additionally, imaging quality is further improved by using the explicit prior knowledge of low rank and weighted joint sparsity in the spatial and Z-spectral domain of CEST data.

RESULTS

In the retrospective acceleration experiment, the proposed method outperforms other state-of-the-art algorithms (TDDIP, LRTES, kt-SLR, NeRP, CRNN, and PBCS) for the in vivo human brain dataset under various acceleration rates. In the prospective acceleration experiment, the proposed algorithm can still obtain results close to the fully-sampled images.

CONCLUSION AND SIGNIFICANCE

The hybrid-feature hash encoding implicit neural representation combined with explicit sparse prior (INRESP) can efficiently accelerate CEST imaging. The proposed algorithm achieves reduced error and improved image quality compared to several state-of-the-art algorithms at relatively high acceleration factors. The superior performance and the training database-free characteristic make the proposed algorithm promising for accelerating CEST imaging in various applications.

Real-time CBCT Imaging and Motion Tracking via a Single Arbitrarily-angled X-ray Projection by a Joint Dynamic Reconstruction and Motion Estimation (DREME) Framework

Objective

Real-time cone-beam computed tomography (CBCT) provides instantaneous visualization of patient anatomy for image guidance, motion tracking, and online treatment adaptation in radiotherapy. While many real-time imaging and motion tracking methods leveraged patient-specific prior information to alleviate under-sampling challenges and meet the temporal constraint (< 500 ms), the prior information can be outdated and introduce biases, thus compromising the imaging and motion tracking accuracy. To address this challenge, we developed a framework (DREME) for real-time CBCT imaging and motion estimation, without relying on patient-specific prior knowledge.

Approach

DREME incorporates a deep learning-based real-time CBCT imaging and motion estimation method into a dynamic CBCT reconstruction framework. The reconstruction framework reconstructs a dynamic sequence of CBCTs in a data-driven manner from a standard pre-treatment scan, without utilizing patient-specific knowledge. Meanwhile, a convolutional neural network-based motion encoder is jointly trained during the reconstruction to learn motion-related features relevant for real-time motion estimation, based on a single arbitrarily-angled x-ray projection. DREME was tested on digital phantom simulation and real patient studies.

Main results

DREME accurately solved 3D respiration-induced anatomic motion in real time (~1.5 ms inference time for each x-ray projection). In the digital phantom study, it achieved an average lung tumor center-of-mass localization error of 1.2±0.9 mm (Mean±SD). In the patient study, it achieved a real-time tumor localization accuracy of 1.8±1.6 mm in the projection domain.

Significance

DREME achieves CBCT and volumetric motion estimation in real time from a single x-ray projection at arbitrary angles, paving the way for future clinical applications in intra-fractional motion management. In addition, it can be used for dose tracking and treatment assessment, when combined with real-time dose calculation.

Linear diffusion noise boosted deep image prior for unsupervised sparse-view CT reconstruction

Objective.Deep learning has markedly enhanced the performance of sparse-view computed tomography reconstruction. However, the dependence of these methods on supervised training using high-quality paired datasets, and the necessity for retraining under varied physical acquisition conditions, constrain their generalizability across new imaging contexts and settings.Approach.To overcome these limitations, we propose an unsupervised approach grounded in the deep image prior framework. Our approach advances beyond the conventional single noise level input by incorporating multi-level linear diffusion noise, significantly mitigating the risk of overfitting. Furthermore, we embed non-local self-similarity as a deep implicit prior within a self-attention network structure, improving the model's capability to identify and utilize repetitive patterns throughout the image. Additionally, leveraging imaging physics, gradient backpropagation is performed between the image domain and projection data space to optimize network weights.Main Results.Evaluations with both simulated and clinical cases demonstrate our method's effective zero-shot adaptability across various projection views, highlighting its robustness and flexibility. Additionally, our approach effectively eliminates noise and streak artifacts while significantly restoring intricate image details.Significance. Our method aims to overcome the limitations in current supervised deep learning-based sparse-view CT reconstruction, offering improved generalizability and adaptability without the need for extensive paired training data.

Instance-Wise MRI Reconstruction Based on Self-Supervised Implicit Neural Representation

Accelerated MRI involves a trade-off between sampling sufficiency and acquisition time. Supervised deep learning methods have shown great success in MRI reconstruction from under-sampled measurements, but they typically require a large set of fully-sampled MR images for training, which can be difficult to obtain. In this paper, we present a novel fully self-supervised method based on implicit neural representation, which requires only a single under-sampled MRI instance for training. To effectively guide the self-supervised learning process, we introduced multiple novel supervisory signals in both the image and frequency domains. Experimental results indicate that the proposed method outperforms existing self-supervised methods and even a supervised method, demonstrating its strong reliability and flexibility. Our code is publicly available at https://github.com/YSongxiao/SSLInstanceReconMRI.Clinical relevance- The proposed method can significantly enhance the image quality of under-sampled MR images without the need of ground-truth fully-sampled MR images for supervision and additional prior images for guidance.

A subject-specific unsupervised deep learning method for quantitative susceptibility mapping using implicit neural representation

Quantitative susceptibility mapping (QSM) is an MRI-based technique that estimates the underlying tissue magnetic susceptibility based on phase signal. Deep learning (DL)-based methods have shown promise in handling the challenging ill-posed inverse problem for QSM reconstruction. However, they require extensive paired training data that are typically unavailable and suffer from generalization problems. Recent model-incorporated DL approaches also overlook the non-local effect of the tissue phase in applying the source-to-field forward model due to patch-based training constraint, resulting in a discrepancy between the prediction and measurement and subsequently suboptimal QSM reconstruction. This study proposes an unsupervised and subject-specific DL method for QSM reconstruction based on implicit neural representation (INR), referred to as INR-QSM. INR has emerged as a powerful framework for learning a high-quality continuous representation of the signal (image) by exploiting its internal information without training labels. In INR-QSM, the desired susceptibility map is represented as a continuous function of the spatial coordinates, parameterized by a fully-connected neural network. The weights are learned by minimizing a loss function that includes a data fidelity term incorporated by the physical model and regularization terms. Additionally, a novel phase compensation strategy is proposed for the first time to account for the non-local effect of tissue phase in data consistency calculation to make the physical model more accurate. Our experiments show that INR-QSM outperforms traditional established QSM reconstruction methods and the compared unsupervised DL method both qualitatively and quantitatively, and is competitive against supervised DL methods under data perturbations.

Neural Radiance Fields for Novel View Synthesis in Monocular Gastroscopy

Enabling the synthesis of arbitrarily novel viewpoint images within a patient's stomach from pre-captured monocular gastroscopic images is a promising topic in stomach diagnosis. Typical methods to achieve this objective integrate traditional 3D reconstruction techniques, including structure-from-motion (SfM) and Poisson surface reconstruction. These methods produce explicit 3D representations, such as point clouds and meshes, thereby enabling the rendering of the images from novel viewpoints. However, the existence of low-texture and non-Lambertian regions within the stomach often results in noisy and incomplete reconstructions of point clouds and meshes, hindering the attainment of high-quality image rendering. In this paper, we apply the emerging technique of neural radiance fields (NeRF) to monocular gastroscopic data for synthesizing photo-realistic images for novel viewpoints. To address the performance degradation due to view sparsity in local regions of monocular gastroscopy, we incorporate geometry priors from a pre-reconstructed point cloud into the training of NeRF, which introduces a novel geometry-based loss to both pre-captured observed views and generated unobserved views. Compared to other recent NeRF methods, our approach showcases high-fidelity image renderings from novel viewpoints within the stomach both qualitatively and quantitatively.

Inter-scanner super-resolution of 3D cine MRI using a transfer-learning network for MRgRT

Objective. Deep-learning networks for super-resolution (SR) reconstruction enhance the spatial-resolution of 3D magnetic resonance imaging (MRI) for MR-guided radiotherapy (MRgRT). However, variations between MRI scanners and patients impact the quality of SR for real-time 3D low-resolution (LR) cine MRI. In this study, we present a personalized super-resolution (psSR) network that incorporates transfer-learning to overcome the challenges in inter-scanner SR of 3D cine MRI.Approach: Development of the proposed psSR network comprises two-stages: (1) a cohort-specific SR (csSR) network using clinical patient datasets, and (2) a psSR network using transfer-learning to target datasets. The csSR network was developed by training on breath-hold and respiratory-gated high-resolution (HR) 3D MRIs and their k-space down-sampled LR MRIs from 53 thoracoabdominal patients scanned at 1.5 T. The psSR network was developed through transfer-learning to retrain the csSR network using a single breath-hold HR MRI and a corresponding 3D cine MRI from 5 healthy volunteers scanned at 0.55 T. Image quality was evaluated using the peak-signal-noise-ratio (PSNR) and the structure-similarity-index-measure (SSIM). The clinical feasibility was assessed by liver contouring on the psSR MRI using an auto-segmentation network and quantified using the dice-similarity-coefficient (DSC).Results. Mean PSNR and SSIM values of psSR MRIs were increased by 57.2% (13.8-21.7) and 94.7% (0.38-0.74) compared to cine MRIs, with the reference 0.55 T breath-hold HR MRI. In the contour evaluation, DSC was increased by 15% (0.79-0.91). Average time consumed for transfer-learning was 90 s, psSR was 4.51 ms per volume, and auto-segmentation was 210 ms, respectively.Significance. The proposed psSR reconstruction substantially increased image and segmentation quality of cine MRI in an average of 215 ms across the scanners and patients with less than 2 min of prerequisite transfer-learning. This approach would be effective in overcoming cohort- and scanner-dependency of deep-learning for MRgRT.

Dynamic CBCT imaging using prior model-free spatiotemporal implicit neural representation (PMF-STINR)

Objective. Dynamic cone-beam computed tomography (CBCT) can capture high-spatial-resolution, time-varying images for motion monitoring, patient setup, and adaptive planning of radiotherapy. However, dynamic CBCT reconstruction is an extremely ill-posed spatiotemporal inverse problem, as each CBCT volume in the dynamic sequence is only captured by one or a few x-ray projections, due to the slow gantry rotation speed and the fast anatomical motion (e.g. breathing).Approach. We developed a machine learning-based technique, prior-model-free spatiotemporal implicit neural representation (PMF-STINR), to reconstruct dynamic CBCTs from sequentially acquired x-ray projections. PMF-STINR employs a joint image reconstruction and registration approach to address the under-sampling challenge, enabling dynamic CBCT reconstruction from singular x-ray projections. Specifically, PMF-STINR uses spatial implicit neural representations to reconstruct a reference CBCT volume, and it applies temporal INR to represent the intra-scan dynamic motion of the reference CBCT to yield dynamic CBCTs. PMF-STINR couples the temporal INR with a learning-based B-spline motion model to capture time-varying deformable motion during the reconstruction. Compared with the previous methods, the spatial INR, the temporal INR, and the B-spline model of PMF-STINR are all learned on the fly during reconstruction in a one-shot fashion, without using any patient-specific prior knowledge or motion sorting/binning.Main results. PMF-STINR was evaluated via digital phantom simulations, physical phantom measurements, and a multi-institutional patient dataset featuring various imaging protocols (half-fan/full-fan, full sampling/sparse sampling, different energy and mAs settings, etc). The results showed that the one-shot learning-based PMF-STINR can accurately and robustly reconstruct dynamic CBCTs and capture highly irregular motion with high temporal (∼ 0.1 s) resolution and sub-millimeter accuracy.Significance. PMF-STINR can reconstruct dynamic CBCTs and solve the intra-scan motion from conventional 3D CBCT scans without using any prior anatomical/motion model or motion sorting/binning. It can be a promising tool for motion management by offering richer motion information than traditional 4D-CBCTs.

Iterative reconstruction for limited-angle CT using implicit neural representation

Objective.Limited-angle computed tomography (CT) presents a challenge due to its ill-posed nature. In such scenarios, analytical reconstruction methods often exhibit severe artifacts. To tackle this inverse problem, several supervised deep learning-based approaches have been proposed. However, they are constrained by limitations such as generalization issue and the difficulty of acquiring a large amount of paired CT images.Approach.In this work, we propose an iterative neural reconstruction framework designed for limited-angle CT. By leveraging a coordinate-based neural representation, we formulate tomographic reconstruction as a convex optimization problem involving a deep neural network. We then employ differentiable projection layer to optimize this network by minimizing the discrepancy between the predicted and measured projection data. In addition, we introduce a prior-based weight initialization method to ensure the network starts optimization with an informed initial guess. This strategic initialization significantly improves the quality of iterative reconstruction by stabilizing the divergent behavior in ill-posed neural fields. Our method operates in a self-supervised manner, thereby eliminating the need for extensive data.Main results.The proposed method outperforms other iterative and learning-based methods. Experimental results on XCAT and Mayo Clinic datasets demonstrate the effectiveness of our approach in restoring anatomical features as well as structures. This finding was substantiated by visual inspections and quantitative evaluations using NRMSE, PSNR, and SSIM. Moreover, we conduct a comprehensive investigation into the divergent behavior of iterative neural reconstruction, thus revealing its suboptimal convergence when starting from scratch. In contrast, our method consistently produced accurate images by incorporating an initial estimate as informed initialization.Significance.This work showcases the feasibility to reconstruct high-fidelity CT images from limited-angle x-ray projections. The proposed methodology introduces a novel data-free approach to enhance medical imaging, holding promise across various clinical applications.

3D cine-magnetic resonance imaging using spatial and temporal implicit neural representation learning (STINR-MR)

Objective. 3D cine-magnetic resonance imaging (cine-MRI) can capture images of the human body volume with high spatial and temporal resolutions to study anatomical dynamics. However, the reconstruction of 3D cine-MRI is challenged by highly under-sampled k-space data in each dynamic (cine) frame, due to the slow speed of MR signal acquisition. We proposed a machine learning-based framework, spatial and temporal implicit neural representation learning (STINR-MR), for accurate 3D cine-MRI reconstruction from highly under-sampled data.Approach. STINR-MR used a joint reconstruction and deformable registration approach to achieve a high acceleration factor for cine volumetric imaging. It addressed the ill-posed spatiotemporal reconstruction problem by solving a reference-frame 3D MR image and a corresponding motion model that deforms the reference frame to each cine frame. The reference-frame 3D MR image was reconstructed as a spatial implicit neural representation (INR) network, which learns the mapping from input 3D spatial coordinates to corresponding MR values. The dynamic motion model was constructed via a temporal INR, as well as basis deformation vector fields (DVFs) extracted from prior/onboard 4D-MRIs using principal component analysis. The learned temporal INR encodes input time points and outputs corresponding weighting factors to combine the basis DVFs into time-resolved motion fields that represent cine-frame-specific dynamics. STINR-MR was evaluated using MR data simulated from the 4D extended cardiac-torso (XCAT) digital phantom, as well as two MR datasets acquired clinically from human subjects. Its reconstruction accuracy was also compared with that of the model-based non-rigid motion estimation method (MR-MOTUS) and a deep learning-based method (TEMPEST).Main results. STINR-MR can reconstruct 3D cine-MR images with high temporal (<100 ms) and spatial (3 mm) resolutions. Compared with MR-MOTUS and TEMPEST, STINR-MR consistently reconstructed images with better image quality and fewer artifacts and achieved superior tumor localization accuracy via the solved dynamic DVFs. For the XCAT study, STINR reconstructed the tumors to a mean ± SD center-of-mass error of 0.9 ± 0.4 mm, compared to 3.4 ± 1.0 mm of the MR-MOTUS method. The high-frame-rate reconstruction capability of STINR-MR allows different irregular motion patterns to be accurately captured.Significance. STINR-MR provides a lightweight and efficient framework for accurate 3D cine-MRI reconstruction. It is a 'one-shot' method that does not require external data for pre-training, allowing it to avoid generalizability issues typically encountered in deep learning-based methods.

Volumetric MRI with sparse sampling for MR-guided 3D motion tracking via sparse prior-augmented implicit neural representation learning

BACKGROUND

Volumetric reconstruction of magnetic resonance imaging (MRI) from sparse samples is desirable for 3D motion tracking and promises to improve magnetic resonance (MR)-guided radiation treatment precision. Data-driven sparse MRI reconstruction, however, requires large-scale training datasets for prior learning, which is time-consuming and challenging to acquire in clinical settings.

PURPOSE

To investigate volumetric reconstruction of MRI from sparse samples of two orthogonal slices aided by sparse priors of two static 3D MRI through implicit neural representation (NeRP) learning, in support of 3D motion tracking during MR-guided radiotherapy.

METHODS

A multi-layer perceptron network was trained to parameterize the NeRP model of a patient-specific MRI dataset, where the network takes 4D data coordinates of voxel locations and motion states as inputs and outputs corresponding voxel intensities. By first training the network to learn the NeRP of two static 3D MRI with different breathing motion states, prior information of patient breathing motion was embedded into network weights through optimization. The prior information was then augmented from two motion states to 31 motion states by querying the optimized network at interpolated and extrapolated motion state coordinates. Starting from the prior-augmented NeRP model as an initialization point, we further trained the network to fit sparse samples of two orthogonal MRI slices and the final volumetric reconstruction was obtained by querying the trained network at 3D spatial locations. We evaluated the proposed method using 5-min volumetric MRI time series with 340 ms temporal resolution for seven abdominal patients with hepatocellular carcinoma, acquired using golden-angle radial MRI sequence and reconstructed through retrospective sorting. Two volumetric MRI with inhale and exhale states respectively were selected from the first 30 s of the time series for prior embedding and augmentation. The remaining 4.5-min time series was used for volumetric reconstruction evaluation, where we retrospectively subsampled each MRI to two orthogonal slices and compared model-reconstructed images to ground truth images in terms of image quality and the capability of supporting 3D target motion tracking.

RESULTS

Across the seven patients evaluated, the peak signal-to-noise-ratio between model-reconstructed and ground truth MR images was 38.02 ± 2.60 dB and the structure similarity index measure was 0.98 ± 0.01. Throughout the 4.5-min time period, gross tumor volume (GTV) motion estimated by deforming a reference state MRI to model-reconstructed and ground truth MRI showed good consistency. The 95-percentile Hausdorff distance between GTV contours was 2.41 ± 0.77 mm, which is less than the voxel dimension. The mean GTV centroid position difference between ground truth and model estimation was less than 1 mm in all three orthogonal directions.

CONCLUSION

A prior-augmented NeRP model has been developed to reconstruct volumetric MRI from sparse samples of orthogonal cine slices. Only one exhale and one inhale 3D MRI were needed to train the model to learn prior information of patient breathing motion for sparse image reconstruction. The proposed model has the potential of supporting 3D motion tracking during MR-guided radiotherapy for improved treatment precision and promises a major simplification of the workflow by eliminating the need for large-scale training datasets.

IMJENSE: Scan-Specific Implicit Representation for Joint Coil Sensitivity and Image Estimation in Parallel MRI

Parallel imaging is a commonly used technique to accelerate magnetic resonance imaging (MRI) data acquisition. Mathematically, parallel MRI reconstruction can be formulated as an inverse problem relating the sparsely sampled k-space measurements to the desired MRI image. Despite the success of many existing reconstruction algorithms, it remains a challenge to reliably reconstruct a high-quality image from highly reduced k-space measurements. Recently, implicit neural representation has emerged as a powerful paradigm to exploit the internal information and the physics of partially acquired data to generate the desired object. In this study, we introduced IMJENSE, a scan-specific implicit neural representation-based method for improving parallel MRI reconstruction. Specifically, the underlying MRI image and coil sensitivities were modeled as continuous functions of spatial coordinates, parameterized by neural networks and polynomials, respectively. The weights in the networks and coefficients in the polynomials were simultaneously learned directly from sparsely acquired k-space measurements, without fully sampled ground truth data for training. Benefiting from the powerful continuous representation and joint estimation of the MRI image and coil sensitivities, IMJENSE outperforms conventional image or k-space domain reconstruction algorithms. With extremely limited calibration data, IMJENSE is more stable than supervised calibrationless and calibration-based deep-learning methods. Results show that IMJENSE robustly reconstructs the images acquired at 5× and 6× accelerations with only 4 or 8 calibration lines in 2D Cartesian acquisitions, corresponding to 22.0% and 19.5% undersampling rates. The high-quality results and scanning specificity make the proposed method hold the potential for further accelerating the data acquisition of parallel MRI.

Generative modeling of living cells with SO(3)-equivariant implicit neural representations

Data-driven cell tracking and segmentation methods in biomedical imaging require diverse and information-rich training data. In cases where the number of training samples is limited, synthetic computer-generated data sets can be used to improve these methods. This requires the synthesis of cell shapes as well as corresponding microscopy images using generative models. To synthesize realistic living cell shapes, the shape representation used by the generative model should be able to accurately represent fine details and changes in topology, which are common in cells. These requirements are not met by 3D voxel masks, which are restricted in resolution, and polygon meshes, which do not easily model processes like cell growth and mitosis. In this work, we propose to represent living cell shapes as level sets of signed distance functions (SDFs) which are estimated by neural networks. We optimize a fully-connected neural network to provide an implicit representation of the SDF value at any point in a 3D+time domain, conditioned on a learned latent code that is disentangled from the rotation of the cell shape. We demonstrate the effectiveness of this approach on cells that exhibit rapid deformations (Platynereis dumerilii), cells that grow and divide (C. elegans), and cells that have growing and branching filopodial protrusions (A549 human lung carcinoma cells). A quantitative evaluation using shape features and Dice similarity coefficients of real and synthetic cell shapes shows that our model can generate topologically plausible complex cell shapes in 3D+time with high similarity to real living cell shapes. Finally, we show how microscopy images of living cells that correspond to our generated cell shapes can be synthesized using an image-to-image model.

Super-resolution biomedical imaging via reference-free statistical implicit neural representation

Objective.Supervised deep learning for image super-resolution (SR) has limitations in biomedical imaging due to the lack of large amounts of low- and high-resolution image pairs for model training. In this work, we propose a reference-free statistical implicit neural representation (INR) framework, which needs only a single or a few observed low-resolution (LR) image(s), to generate high-quality SR images.Approach.The framework models the statistics of the observed LR images via maximum likelihood estimation and trains the INR network to represent the latent high-resolution (HR) image as a continuous function in the spatial domain. The INR network is constructed as a coordinate-based multi-layer perceptron, whose inputs are image spatial coordinates and outputs are corresponding pixel intensities. The trained INR not only constrains functional smoothness but also allows an arbitrary scale in SR imaging.Main results.We demonstrate the efficacy of the proposed framework on various biomedical images, including computed tomography (CT), magnetic resonance imaging (MRI), fluorescence microscopy, and ultrasound images, across different SR magnification scales of 2×, 4×, and 8×. A limited number of LR images were used for each of the SR imaging tasks to show the potential of the proposed statistical INR framework.Significance.The proposed method provides an urgently needed unsupervised deep learning framework for numerous biomedical SR applications that lack HR reference images.

Modeling linear accelerator (Linac) beam data by implicit neural representation learning for commissioning and quality assurance applications

BACKGROUND

Linear accelerator (Linac) beam data commissioning and quality assurance (QA) play a vital role in accurate radiation treatment delivery and entail a large number of measurements using a variety of field sizes. How to optimize the effort in data acquisition while maintaining high quality of medical physics practice has been sought after.

PURPOSE

We propose to model Linac beam data through implicit neural representation (NeRP) learning. The potential of the beam model in predicting beam data from sparse measurements and detecting data collection errors was evaluated, with the goal of using the beam model to verify beam data collection accuracy and simplify the commissioning and QA process.

MATERIALS AND METHODS

NeRP models with continuous and differentiable functions parameterized by multilayer perceptrons (MLPs) were used to represent various beam data including percentage depth dose (PDD) and profiles of 6 MV beams with and without flattening filter. Prior knowledge of the beam data was embedded into the MLP network by learning the NeRP of a vendor-provided "golden" beam dataset. The prior-embedded network was then trained to fit clinical beam data collected at one field size and used to predict beam data at other field sizes. We evaluated the prediction accuracy by comparing network-predicted beam data to water tank measurements collected from 14 clinical Linacs. Beam datasets with intentionally introduced errors were used to investigate the potential use of the NeRP model for beam data verification, by evaluating the model performance when trained with erroneous beam data samples.

RESULTS

Linac beam data predicted by the model agreed well with water tank measurements, with averaged Gamma passing rates (1%/1 mm passing criteria) higher than 95% and averaged mean absolute errors less than 0.6%. Beam data samples with measurement errors were revealed by inconsistent beam predictions between networks trained with correct versus erroneous data samples, characterized by a Gamma passing rate lower than 90%.

CONCLUSION

A NeRP beam data modeling technique has been established for predicting beam characteristics from sparse measurements. The model provides a valuable tool to verify beam data collection accuracy and promises to simplify commissioning/QA processes by reducing the number of measurements without compromising the quality of medical physics service.

DiFiR-CT: Distance field representation to resolve motion artifacts in computed tomography

BACKGROUND

Motion during data acquisition leads to artifacts in computed tomography (CT) reconstructions. In cases such as cardiac imaging, not only is motion unavoidable, but evaluating the motion of the object is of clinical interest. Reducing motion artifacts has typically been achieved by developing systems with faster gantry rotation or via algorithms which measure and/or estimate the displacement. However, these approaches have had limited success due to both physical constraints as well as the challenge of estimating non-rigid, temporally varying, and patient-specific motion fields.

PURPOSE

To develop a novel reconstruction method which generates time-resolved, artifact-free images without estimation or explicit modeling of the motion.

METHODS

We describe an analysis-by-synthesis approach which progressively regresses a solution consistent with the acquired sinogram. In our method, we focus on the movement of object boundaries. Not only are the boundaries the source of image artifacts, but object boundaries can simultaneously be used to represent both the object as well as its motion over time without the need for an explicit motion model. We represent the object boundaries via a signed distance function (SDF) which can be efficiently modeled using neural networks. As a result, optimization can be performed under spatial and temporal smoothness constraints without the need for explicit motion estimation.

RESULTS

We illustrate the utility of DiFiR-CT in three imaging scenarios with increasing motion complexity: translation of a small circle, heart-like change in an ellipse's diameter, and a complex topological deformation. Compared to filtered backprojection, DiFiR-CT provides high quality image reconstruction for all three motions without hyperparameter tuning or change to the architecture. We also evaluate DiFiR-CT's robustness to noise in the acquired sinogram and found its reconstruction to be accurate across a wide range of noise levels. Lastly, we demonstrate how the approach could be used for multi-intensity scenes and illustrate the importance of the initial segmentation providing a realistic initialization. Code and supplemental movies are available at https://kunalmgupta.github.io/projects/DiFiR-CT.html.

CONCLUSIONS

Projection data can be used to accurately estimate a temporally-evolving scene without the need for explicit motion estimation using a neural implicit representation and analysis-by-synthesis approach.

A Deep Learning-Based Framework for Uncertainty Quantification in Medical Imaging Using the DropWeak Technique: An Empirical Study with Baresnet

Lung cancer is a leading cause of cancer-related deaths globally. Early detection is crucial for improving patient survival rates. Deep learning (DL) has shown promise in the medical field, but its accuracy must be evaluated, particularly in the context of lung cancer classification. In this study, we conducted uncertainty analysis on various frequently used DL architectures, including Baresnet, to assess the uncertainties in the classification results. This study focuses on the use of deep learning for the classification of lung cancer, which is a critical aspect of improving patient survival rates. The study evaluates the accuracy of various deep learning architectures, including Baresnet, and incorporates uncertainty quantification to assess the level of uncertainty in the classification results. The study presents a novel automatic tumor classification system for lung cancer based on CT images, which achieves a classification accuracy of 97.19% with an uncertainty quantification. The results demonstrate the potential of deep learning in lung cancer classification and highlight the importance of uncertainty quantification in improving the accuracy of classification results. This study's novelty lies in the incorporation of uncertainty quantification in deep learning for lung cancer classification, which can lead to more reliable and accurate diagnoses in clinical settings.

Dynamic cone-beam CT reconstruction using spatial and temporal implicit neural representation learning (STINR)

Objective. Dynamic cone-beam CT (CBCT) imaging is highly desired in image-guided radiation therapy to provide volumetric images with high spatial and temporal resolutions to enable applications including tumor motion tracking/prediction and intra-delivery dose calculation/accumulation. However, dynamic CBCT reconstruction is a substantially challenging spatiotemporal inverse problem, due to the extremely limited projection sample available for each CBCT reconstruction (one projection for one CBCT volume).Approach. We developed a simultaneous spatial and temporal implicit neural representation (STINR) method for dynamic CBCT reconstruction. STINR mapped the unknown image and the evolution of its motion into spatial and temporal multi-layer perceptrons (MLPs), and iteratively optimized the neuron weightings of the MLPs via acquired projections to represent the dynamic CBCT series. In addition to the MLPs, we also introduced prior knowledge, in the form of principal component analysis (PCA)-based patient-specific motion models, to reduce the complexity of the temporal mapping to address the ill-conditioned dynamic CBCT reconstruction problem. We used the extended-cardiac-torso (XCAT) phantom and a patient 4D-CBCT dataset to simulate different lung motion scenarios to evaluate STINR. The scenarios contain motion variations including motion baseline shifts, motion amplitude/frequency variations, and motion non-periodicity. The XCAT scenarios also contain inter-scan anatomical variations including tumor shrinkage and tumor position change.Main results. STINR shows consistently higher image reconstruction and motion tracking accuracy than a traditional PCA-based method and a polynomial-fitting-based neural representation method. STINR tracks the lung target to an average center-of-mass error of 1-2 mm, with corresponding relative errors of reconstructed dynamic CBCTs around 10%.Significance. STINR offers a general framework allowing accurate dynamic CBCT reconstruction for image-guided radiotherapy. It is a one-shot learning method that does not rely on pre-training and is not susceptible to generalizability issues. It also allows natural super-resolution. It can be readily applied to other imaging modalities as well.

NuTracker: a coordinate-based neural network representation of lung motion for intrafraction tumor tracking with various surrogates in radiotherapy

Objective. Tracking tumors and surrounding tissues in real-time is critical for reducing errors and uncertainties during radiotherapy. Existing methods are either limited by the linear representation or scale poorly with the volume resolution. To address both issues, we propose a novel coordinate-based neural network representation of lung motion to predict the instantaneous 3D volume at arbitrary spatial resolution from various surrogates: patient surface, fiducial marker, and single kV projection.Approach. The proposed model, namely NuTracker, decomposes the 4DCT into a template volume and dense displacement fields (DDFs), and uses two coordinate neural networks to predict them from spatial coordinates and surrogate states. The predicted template is spatially warped with the predicted DDF to produce the deformed volume for a given surrogate state. The nonlinear coordinate networks enable representing complex motion at infinite resolution. The decomposition allows imposing different regularizations on the spatial and temporal domains. The meta-learning and multi-task learning are used to train NuTracker across patients and tasks, so that commonalities and differences can be exploited. NuTracker was evaluated on seven patients implanted with markers using a leave-one-phase-out procedure.Main results. The 3D marker localization error is 0.66 mm on average and <1 mm at 95th-percentile, which is about 26% and 32% improvement over the predominant linear methods. The tumor coverage and image quality are improved by 5.7% and 11% in terms of dice and PSNR. The difference in the localization error for different surrogates is small and is not statistically significant. Cross-population learning and multi-task learning contribute to performance. The model tolerates surrogate drift to a certain extent.Significance. NuTracker can provide accurate estimation for entire tumor volume based on various surrogates at infinite resolution. It is of great potential to apply the coordinate network to other imaging modalities, e.g. 4DCBCT and other tasks, e.g. 4D dose calculation.