UHR-DeepFMT: Ultra-High Spatial Resolution Reconstruction of Fluorescence Molecular Tomography Based on 3-D Fusion Dual-Sampling Deep Neural Network

TSPE: Reconstruction of multi-morphological tumors of NIR-II fluorescence molecular tomography based on positional encoding

BACKGROUND AND OBJECTIVE

Fluorescence molecular tomography (FMT) is a noninvasive and highly sensitive imaging modality, which can display 3D visualization of tumors by reconstructing fluorescence probes' distribution. However, existing methods mostly ignore positional information, which includes spatial structure information crucial for the reconstruction of light sources. This limits the reconstruction accuracy of light sources with multiple morphologies. Therefore, to our best knowledge, we for the first time integrated positional encoding into the FMT task, enabling the incorporation of high-frequency spatial structure information.

METHODS

We proposed a three-stage network embedded with a positional encoding module (TSPE) to perform high reconstruction accuracy of tumors with multiple morphologies. Additionally, our study focused on NIR-II which had less severe scattering problems and higher imaging accuracy than NIR-I.

RESULTS

The simulation experiments demonstrated that TSPE achieved high reconstruction accuracy in NIR-II FMT, with the barycenter error (BCE) for single-tumor reconstruction reaching 0.18 mm, representing a 14 % reduction compared to other methods. TSPE more accurately distinguished adjacent multi-morphological tumors with a minimal edge-to-edge distance (EED) of 0.3 mm. In vivo experiments also showed that TSPE could achieve more accurate reconstruction of tumors compared with other methods.

CONCLUSIONS

The proposed method can achieve the best reconstruction performance. It has potential to promote the development of NIR-II FMT and its preclinical application.

Self-supervised hybrid neural network to achieve quantitative bioluminescence tomography for cancer research

Bioluminescence tomography (BLT) improves upon commonly-used 2D bioluminescence imaging by reconstructing 3D distributions of bioluminescence activity within biological tissue, allowing tumor localization and volume estimation-critical for cancer therapy development. Conventional model-based BLT is computationally challenging due to the ill-posed nature of the problem and data noise. We introduce a self-supervised hybrid neural network (SHyNN) that integrates the strengths of both conventional model-based methods and machine learning (ML) techniques to address these challenges. The network structure and converging path of SHyNN are designed to mitigate the effects of ill-posedness for achieving accurate and robust solutions. Through simulated and in vivo data on different disease sites, it is demonstrated to outperform the conventional reconstruction approach, particularly under high noise, in tumor localization, volume estimation, and multi-tumor differentiation, highlighting the potential towards quantitative BLT for cancer research.

A Novel Technique for Fluorescence Lifetime Tomography

Fluorescence lifetime has emerged as a unique imaging modality for quantitatively assessing in vivo the molecular environment of diseased tissues. Although fluorescence lifetime microscopy (in 2D) is a mature field, 3D imaging in deep tissues remains elusive and challenging owing to scattering. Herein, we report on a deep neural network (coined AUTO-FLI) that performs both 3D intensity and quantitative lifetime reconstructions in deep tissues. The proposed Deep Learning (DL)-based approach involves an in silico scheme to generate fluorescence lifetime data accurately. The developed DL model is validated both in silico and on experimental phantoms. Overall, AUTO-FLI provides accurate 3D quantitative estimates of both intensity and lifetime distributions in highly scattering media, demonstrating its unique potential for fluorescence lifetime-based molecular imaging at the mesoscopic and macroscopic scale.

Dual and Multi-Target Cone-Beam X-ray Luminescence Computed Tomography Based on the DeepCB-XLCT Network

BACKGROUND AND OBJECTIVE

Emerging as a hybrid imaging modality, cone-beam X-ray luminescence computed tomography (CB-XLCT) has been developed using X-ray-excitable nanoparticles. In contrast to conventional bio-optical imaging techniques like bioluminescence tomography (BLT) and fluorescence molecular tomography (FMT), CB-XLCT offers the advantage of greater imaging depth while significantly reducing interference from autofluorescence and background fluorescence, owing to its utilization of X-ray-excited nanoparticles. However, due to the intricate excitation process and extensive light scattering within biological tissues, the inverse problem of CB-XLCT is fundamentally ill-conditioned.

METHODS

An end-to-end three-dimensional deep encoder-decoder network, termed DeepCB-XLCT, is introduced to improve the quality of CB-XLCT reconstructions. This network directly establishes a nonlinear mapping between the distribution of internal X-ray-excitable nanoparticles and the corresponding boundary fluorescent signals. To improve the fidelity of target shape restoration, the structural similarity loss (SSIM) was incorporated into the objective function of the DeepCB-XLCT network. Additionally, a loss term specifically for target regions was introduced to improve the network's emphasis on the areas of interest. As a result, the inaccuracies in reconstruction caused by the simplified linear model used in conventional methods can be effectively minimized by the proposed DeepCB-XLCT method.

RESULTS AND CONCLUSIONS

Numerical simulations, phantom experiments, and in vivo experiments with two targets were performed, revealing that the DeepCB-XLCT network enhances reconstruction accuracy regarding contrast-to-noise ratio and shape similarity when compared to traditional methods. In addition, the findings from the XLCT tomographic images involving three targets demonstrate its potential for multi-target CB-XLCT imaging.

Projected algebraic reconstruction technique-network for high-fidelity diffuse fluorescence tomography reconstruction

We propose a model-driven projected algebraic reconstruction technique (PART)-network (PART-Net) that leverages the advantages of the traditional model-based method and the neural network to improve the imaging quality of diffuse fluorescence tomography. In this algorithm, nonnegative prior information is incorporated into the ART iteration process to better guide the optimization process, and thereby improve imaging quality. On this basis, PART in conjunction with a residual convolutional neural network is further proposed to obtain high-fidelity image reconstruction. The numerical simulation results demonstrate that the PART-Net algorithm effectively improves noise robustness and reconstruction accuracy by at least 1-2 times and exhibits superiority in spatial resolution and quantification, especially for a small-sized target (r=2m m), compared with the traditional ART algorithm. Furthermore, the phantom and in vivo experiments verify the effectiveness of the PART-Net, suggesting strong generalization capability and a great potential for practical applications.

Fluorescence molecular tomography based on an online maximum a posteriori estimation algorithm

Fluorescence molecular tomography (FMT) is a non-invasive, radiation-free, and highly sensitive optical molecular imaging technique for early tumor detection. However, inadequate measurement information along with significant scattering of near-infrared light within the tissue leads to high ill-posedness in the inverse problem of FMT. To improve the quality and efficiency of FMT reconstruction, we build a reconstruction model based on log-sum regularization and introduce an online maximum a posteriori estimation (OPE) algorithm to solve the non-convex optimization problem. The OPE algorithm approximates a stationary point by evaluating the gradient of the objective function at each iteration, and its notable strength lies in the remarkable speed of convergence. The results of simulations and experiments demonstrate that the OPE algorithm ensures good reconstruction quality and exhibits outstanding performance in terms of reconstruction efficiency.

Performance enhancement of diffuse fluorescence tomography based on an extended Kalman filtering-long short term memory neural network correction model

To alleviate the ill-posedness of diffuse fluorescence tomography (DFT) reconstruction and improve imaging quality and speed, a model-derived deep-learning method is proposed by combining extended Kalman filtering (EKF) with a long short term memory (LSTM) neural network, where the iterative process parameters acquired by implementing semi-iteration EKF (SEKF) served as inputs to the LSTM neural network correction model for predicting the optimal fluorescence distributions. To verify the effectiveness of the SEKF-LSTM algorithm, a series of numerical simulations, phantom and in vivo experiments are conducted, and the experimental results are quantitatively evaluated and compared with the traditional EKF algorithm. The simulation experimental results show that the proposed new algorithm can effectively improve the reconstructed image quality and reconstruction speed. Importantly, the LSTM correction model trained by the simulation data also obtains satisfactory results in the experimental data, suggesting that the SEKF-LSTM algorithm possesses strong generalization ability and great potential for practical applications.

Highly robust reconstruction framework for three-dimensional optical imaging based on physical model constrained neural networks

Objective. The reconstruction of three-dimensional optical imaging that can quantitatively acquire the target distribution from surface measurements is a serious ill-posed problem. Traditional regularization-based reconstruction can solve such ill-posed problem to a certain extent, but its accuracy is highly dependent ona priorinformation, resulting in a less stable and adaptable method. Data-driven deep learning-based reconstruction avoids the errors of light propagation models and the reliance on experience and a prior by learning the mapping relationship between the surface light distribution and the target directly from the dataset. However, the acquisition of the training dataset and the training of the network itself are time consuming, and the high dependence of the network performance on the training dataset results in a low generalization ability. The objective of this work is to develop a highly robust reconstruction framework to solve the existing problems.Approach. This paper proposes a physical model constrained neural networks-based reconstruction framework. In the framework, the neural networks are to generate a target distribution from surface measurements, while the physical model is used to calculate the surface light distribution based on this target distribution. The mean square error between the calculated surface light distribution and the surface measurements is then used as a loss function to optimize the neural network. To further reduce the dependence ona prioriinformation, a movable region is randomly selected and then traverses the entire solution interval. We reconstruct the target distribution in this movable region and the results are used as the basis for its next movement.Main Results. The performance of the proposed framework is evaluated with a series of simulations andin vivoexperiment, including accuracy robustness of different target distributions, noise immunity, depth robustness, and spatial resolution. The results collectively demonstrate that the framework can reconstruct targets with a high accuracy, stability and versatility.Significance. The proposed framework has high accuracy and robustness, as well as good generalizability. Compared with traditional regularization-based reconstruction methods, it eliminates the need to manually delineate feasible regions and adjust regularization parameters. Compared with emerging deep learning assisted methods, it does not require any training dataset, thus saving a lot of time and resources and solving the problem of poor generalization and robustness of deep learning methods. Thus, the framework opens up a new perspective for the reconstruction of three-dimension optical imaging.

FSMN-Net: a free space matching network based on manifold convolution for optical molecular tomography

Optical molecular tomography (OMT) can monitor glioblastomas in small animals non-invasively. Although deep learning (DL) methods have made remarkable achievements in this field, improving its generalization against diverse reconstruction systems remains a formidable challenge. In this Letter, a free space matching network (FSMN-Net) was presented to overcome the parameter mismatch problem in different reconstruction systems. Specifically, a novel, to the best of our knowledge, manifold convolution operator was designed by considering the mathematical model of OMT as a space matching process. Based on the dynamic domain expansion concept, an end-to-end fully convolutional codec further integrates this operator to realize robust reconstruction with voxel-level accuracy. The results of numerical simulations and in vivo experiments demonstrate that the FSMN-Net can stably generate high-resolution reconstruction volumetric images under different reconstruction systems.

Application of the Powell algorithm for estimating optical properties of a semitransparent medium based on time-domain information

Accurate estimation of the optical properties of a semitransparent medium is crucial in various engineering applications. This study introduces the Powell algorithm to estimate the optical properties of a 2D semitransparent slab. The time-domain radiative transfer equation is solved using the discrete ordinate method. The radiative intensity on the medium's surface serves as the measurement signal for the inverse analysis. The results demonstrate that the Powell algorithm accurately estimates the absorption coefficient, scattering coefficient, and scattering asymmetry factor. For simultaneous reconstruction of these three parameters, it is recommended to use eight signal detectors on both the left and right sides of the medium. Even when the standard measurement error is increased to 15%, the relative errors for these three parameters remain low, at 1.87%, 1.379%, and 0.194%.

3D-deep optical learning: a multimodal and multitask reconstruction framework for optical molecular tomography

Optical molecular tomography (OMT) is an emerging imaging technique. To date, the poor universality of reconstruction algorithms based on deep learning for various imaged objects and optical probes limits the development and application of OMT. In this study, based on a new mapping representation, a multimodal and multitask reconstruction framework-3D deep optical learning (3DOL), was presented to overcome the limitations of OMT in universality by decomposing it into two tasks, optical field recovery and luminous source reconstruction. Specifically, slices of the original anatomy (provided by computed tomography) and boundary optical measurement of imaged objects serve as inputs of a recurrent convolutional neural network encoded parallel to extract multimodal features, and 2D information from a few axial planes within the samples is explicitly incorporated, which enables 3DOL to recognize different imaged objects. Subsequently, the optical field is recovered under the constraint of the object geometry, and then the luminous source is segmented by a learnable Laplace operator from the recovered optical field, which obtains stable and high-quality reconstruction results with extremely few parameters. This strategy enable 3DOL to better understand the relationship between the boundary optical measurement, optical field, and luminous source to improve 3DOL's ability to work in a wide range of spectra. The results of numerical simulations, physical phantoms, and in vivo experiments demonstrate that 3DOL is a compatible deep-learning approach to tomographic imaging diverse objects. Moreover, the fully trained 3DOL under specific wavelengths can be generalized to other spectra in the 620-900 nm NIR-I window.

Structure-fused deep 3D hierarchical network: A bioluminescence tomography scheme for different imaging objects

Monte Carlo eXtreme (MCX) method has a unique advantage for deep neural network based bioluminescence tomography (BLT) reconstruction. However, this method ignores the distribution of sources energy and relies on the determined tissue structure. In this paper, a deep 3D hierarchical reconstruction network for BLT was proposed where the inputs were divided into two parts -- bioluminescence image (BLI) and anatomy of the imaged object by CT. Firstly, a parallel encoder is used to feature the original BLI & CT slices and integrate their features to distinguish the different tissue structure of imaging objects; Secondly, GRU is used to fit the spatial information of different slices and convert it into 3D features; Finally, the 3D features are decoded to the spacial and energy information of source by a symmetrical decoding structure. Our research suggested that this method can effectively compute the radiation intensity and the spatial distribution of the source for different imaging object.

Two-stage deep learning method for sparse-view fluorescence molecular tomography reconstruction

Fluorescence molecular tomography (FMT) is a preclinical optical tomographic imaging technique that can trace various physiological and pathological processes at the cellular or even molecular level. Reducing the number of FMT projection views can improve the data acquisition speed, which is significant in applications such as dynamic problems. However, a reduction in the number of projection views will dramatically aggravate the ill-posedness of the FMT inverse problem and lead to significant degradation of the reconstructed images. To deal with this problem, we have proposed a deep-learning-based reconstruction method for sparse-view FMT that only uses four perpendicular projection views and divides the image reconstruction into two stages: image restoration and inverse Radon transform. In the first stage, the projection views of the surface fluorescence are restored to eliminate the blur derived from photon diffusion through a fully convolutional neural network. In the second stage, another convolutional neural network is used to implement the inverse Radon transform between the restored projections from the first stage and the reconstructed transverse slices. Numerical simulation and phantom and mouse experiments are carried out. The results show that the proposed method can effectively deal with the image reconstruction problem of sparse-view FMT.

Robust transformed l1 metric for fluorescence molecular tomography

BACKGROUND AND OBJECTIVE

Fluorescence molecular tomography (FMT) is a non-invasive molecular imaging modality that can be used to observe the three-dimensional distribution of fluorescent probes in vivo. FMT is a promising imaging technique in clinical and preclinical research that has attracted significant attention. Numerous regularization based reconstruction algorithms have been proposed. However, traditional algorithms that use the squared l₂-norm distance usually exaggerate the influence of noise and measurement and calculation errors, and their robustness cannot be guaranteed.

METHODS

In this study, we propose a novel robust transformed l₁ (TL1) metric that interpolates l₀ and l₁ norms through a nonnegative parameter α∈(0,+∞). The TL1 metric looks like the l_p-norm with p∈(0,1). These are markedly different because TL1 metric has two properties, boundedness and Lipschitz-continuity, which make the TL1 criterion suitable distance metric, particularly for robustness, owing to its stronger noise suppression. Subsequently, we apply the proposed metric to FMT and build a robust model to reduce the influence of noise. The nonconvexity of the proposed model made direct optimization difficult, and a continuous optimization method was developed to solve the model. The problem was converted into a difference in convex programming problem for the TL1 metric (DCATL1), and the corresponding algorithm converged linearly.

RESULTS

Various numerical simulations and in vivo bead-implanted mouse experiments were conducted to verify the performance of the proposed method. The experimental results show that the DCATL1 algorithm is more robust than the state-of-the-art approaches and achieves better source localization and morphology recovery.

CONCLUSIONS

The in vivo experiments showed that DCATL1 can be used to visualize the distribution of fluorescent probes inside biological tissues and promote preclinical application in small animals, demonstrating the feasibility and effectiveness of the proposed method.

Generalized conditional gradient method with adaptive regularization parameters for fluorescence molecular tomography

Fluorescence molecular tomography (FMT) is an optical imaging technology with the ability of visualizing the three-dimensional distribution of fluorescently labelled probes in vivo. However, due to the light scattering effect and ill-posed inverse problems, obtaining satisfactory FMT reconstruction is still a challenging problem. In this work, to improve the performance of FMT reconstruction, we proposed a generalized conditional gradient method with adaptive regularization parameters (GCGM-ARP). In order to make a tradeoff between the sparsity and shape preservation of the reconstruction source, and to maintain its robustness, elastic-net (EN) regularization is introduced. EN regularization combines the advantages of L₁-norm and L₂-norm, and overcomes the shortcomings of traditional L_p-norm regularization, such as over-sparsity, over-smoothness, and non-robustness. Thus, the equivalent optimization formulation of the original problem can be obtained. To further improve the performance of the reconstruction, the L-curve is adopted to adaptively adjust the regularization parameters. Then, the generalized conditional gradient method (GCGM) is used to split the minimization problem based on EN regularization into two simpler sub-problems, which are determining the direction of the gradient and the step size. These sub-problems are addressed efficiently to obtain more sparse solutions. To assess the performance of our proposed method, a series of numerical simulation experiments and in vivo experiments were implemented. The experimental results show that, compared with other mathematical reconstruction methods, GCGM-ARP method has the minimum location error (LE) and relative intensity error (RIE), and the maximum dice coefficient (Dice) in the case of different sources number or shape, or Gaussian noise of 5%-25%. This indicates that GCGM-ARP has superior reconstruction performance in source localization, dual-source resolution, morphology recovery, and robustness. In conclusion, the proposed GCGM-ARP is an effective and robust strategy for FMT reconstruction in biomedical application.

Reconstruction based on adaptive group least angle regression for fluorescence molecular tomography

Fluorescence molecular tomography can combine two-dimensional fluorescence imaging with anatomical information to reconstruct three-dimensional images of tumors. Reconstruction based on traditional regularization with tumor sparsity priors does not take into account that tumor cells form clusters, so it performs poorly when multiple light sources are used. Here we describe reconstruction based on an "adaptive group least angle regression elastic net" (AGLEN) method, in which local spatial structure correlation and group sparsity are integrated with elastic net regularization, followed by least angle regression. The AGLEN method works iteratively using the residual vector and a median smoothing strategy in order to adaptively obtain a robust local optimum. The method was verified using numerical simulations as well as imaging of mice bearing liver or melanoma tumors. AGLEN reconstruction performed better than state-of-the-art methods with different sizes of light sources at different distances from the sample and in the presence of Gaussian noise at 5-25%. In addition, AGLEN-based reconstruction accurately imaged tumor expression of cell death ligand-1, which can guide immunotherapy.

Robust reconstruction of fluorescence molecular tomography based on adaptive adversarial learning strategy

Objective.Fluorescence molecular tomography (FMT) is a promising molecular imaging modality for quantifying the three-dimensional (3D) distribution of tumor probes in small animals. However, traditional deep learning reconstruction methods that aim to minimize the mean squared error (MSE) and iterative regularization algorithms that rely on optimal parameters are typically influenced by strong noise, resulting in poor FMT reconstruction robustness.Approach.In this letter, we propose an adaptive adversarial learning strategy (3D-UR-WGAN) to achieve robust FMT reconstructions. Unlike the pixel-based MSE criterion in traditional CNNs or the regularization strategy in iterative solving schemes, the reconstruction strategy can greatly facilitate the performance of the network models through alternating loop training of the generator and the discriminator. Second, the reconstruction strategy combines the adversarial loss in GANs with the L1 loss to significantly enhance the robustness and preserve image details and textual information.Main results.Both numerical simulations and physical phantom experiments demonstrate that the 3D-UR-WGAN method can adaptively eliminate the effects of different noise levels on the reconstruction results, resulting in robust reconstructed images with reduced artifacts and enhanced image contrast. Compared with the state-of-the-art methods, the proposed method achieves better reconstruction performance in terms of target shape recovery and localization accuracy.Significance.This adaptive adversarial learning reconstruction strategy can provide a possible paradigm for robust reconstruction in complex environments, and also has great potential to provide an alternative solution for solving the problem of poor robustness encountered in other optical imaging modalities such as diffuse optical tomography, bioluminescence imaging, and Cherenkov luminescence imaging.

Selfrec-Net: self-supervised deep learning approach for the reconstruction of Cherenkov-excited luminescence scanned tomography

As an emerging imaging technique, Cherenkov-excited luminescence scanned tomography (CELST) can recover a high-resolution 3D distribution of quantum emission fields within tissue using X-ray excitation for deep penetrance. However, its reconstruction is an ill-posed and under-conditioned inverse problem because of the diffuse optical emission signal. Deep learning based image reconstruction has shown very good potential for solving these types of problems, however they suffer from a lack of ground-truth image data to confirm when used with experimental data. To overcome this, a self-supervised network cascaded by a 3D reconstruction network and the forward model, termed Selfrec-Net, was proposed to perform CELST reconstruction. Under this framework, the boundary measurements are input to the network to reconstruct the distribution of the quantum field and the predicted measurements are subsequently obtained by feeding the reconstructed result to the forward model. The network was trained by minimizing the loss between the input measurements and the predicted measurements rather than the reconstructed distributions and the corresponding ground truths. Comparative experiments were carried out on both numerical simulations and physical phantoms. For singular luminescent targets, the results demonstrate the effectiveness and robustness of the proposed network, and comparable performance can be attained to a state-of-the-art deep supervised learning algorithm, where the accuracy of the emission yield and localization of the objects was far superior to iterative reconstruction methods. Reconstruction of multiple objects is still reasonable with high localization accuracy, although with limits to the emission yield accuracy as the distribution becomes more complex. Overall though the reconstruction of Selfrec-Net provides a self-supervised way to recover the location and emission yield of molecular distributions in murine model tissues.

D2-RecST: Dual-domain joint reconstruction strategy for fluorescence molecular tomography based on image domain and perception domain

BACKGROUND AND OBJECTIVE

Fluorescence molecular tomography (FMT) is a promising molecular imaging modality for quantifying the three-dimensional (3D) distribution of fluorescent probes in small animals. Over the past few years, learning-based FMT reconstruction methods have achieved promising results. However, these methods typically attempt to minimize the mean-squared error (MSE) between the reconstructed image and the ground truth. Although signal-to-noise ratios (SNRs) are improved, they are susceptible to non-uniform artifacts and loss of structural detail, making it extremely challenging to obtain accurate and robust FMT reconstructions under noisy measurements.

METHODS

We propose a novel dual-domain joint strategy based on the image domain and perception domain for accurate and robust FMT reconstruction. First, we formulate an explicit adversarial learning strategy in the image domain, which greatly facilitates training and optimization through two enhanced networks to improve anti-noise ability. Besides, we introduce a novel transfer learning strategy in the perceptual domain to optimize edge details by providing perceptual priors for fluorescent targets. Collectively, the proposed dual-domain joint reconstruction strategy can significantly eliminate the non-uniform artifacts and effectively preserve the structural edge details.

RESULTS

Both numerical simulations and in vivo mouse experiments demonstrate that the proposed method markedly outperforms traditional and cutting-edge methods in terms of positioning accuracy, image contrast, robustness, and target morphological recovery.

CONCLUSIONS

The proposed method achieves the best reconstruction performance and has great potential to facilitate precise localization and 3D visualization of tumors in in vivo animal experiments.

Excitation-based fully connected network for precise NIR-II fluorescence molecular tomography

Fluorescence molecular tomography (FMT) is a novel imaging modality to obtain fluorescence biomarkers' three-dimensional (3D) distribution. However, the simplified mathematical model and complicated inverse problem limit it to achieving precise results. In this study, the second near-infrared (NIR-II) fluorescence imaging was adopted to mitigate tissue scattering and reduce noise interference. An excitation-based fully connected network was proposed to model the inverse process of NIR-II photon propagation and directly obtain the 3D distribution of the light source. An excitation block was embedded in the network allowing it to autonomously pay more attention to neurons related to the light source. The barycenter error was added to the loss function to improve the localization accuracy of the light source. Both numerical simulation and in vivo experiments showed the superiority of the novel NIR-II FMT reconstruction strategy over the baseline methods. This strategy was expected to facilitate the application of machine learning in biomedical research.

Self-Training Strategy Based on Finite Element Method for Adaptive Bioluminescence Tomography Reconstruction

Bioluminescence tomography (BLT) is a promising pre-clinical imaging technique for a wide variety of biomedical applications, which can non-invasively reveal functional activities inside living animal bodies through the detection of visible or near-infrared light produced by bioluminescent reactions. Recently, reconstruction approaches based on deep learning have shown great potential in optical tomography modalities. However, these reports only generate data with stationary patterns of constant target number, shape, and size. The neural networks trained by these data sets are difficult to reconstruct the patterns outside the data sets. This will tremendously restrict the applications of deep learning in optical tomography reconstruction. To address this problem, a self-training strategy is proposed for BLT reconstruction in this paper. The proposed strategy can fast generate large-scale BLT data sets with random target numbers, shapes, and sizes through an algorithm named random seed growth algorithm and the neural network is automatically self-trained. In addition, the proposed strategy uses the neural network to build a map between photon densities on surface and inside the imaged object rather than an end-to-end neural network that directly infers the distribution of sources from the photon density on surface. The map of photon density is further converted into the distribution of sources through the multiplication with stiffness matrix. Simulation, phantom, and mouse studies are carried out. Results show the availability of the proposed self-training strategy.

Multi-branch attention prior based parameterized generative adversarial network for fast and accurate limited-projection reconstruction in fluorescence molecular tomography

Limited-projection fluorescence molecular tomography (FMT) allows rapid reconstruction of the three-dimensional (3D) distribution of fluorescent targets within a shorter data acquisition time. However, the limited-projection FMT is severely ill-posed and ill-conditioned due to insufficient fluorescence measurements and the strong scattering properties of photons in biological tissues. Previously, regularization-based methods, combined with the sparse distribution of fluorescent sources, have been commonly used to alleviate the severe ill-posed nature of the limited-projection FMT. Due to the complex iterative computations, time-consuming solution procedures, and less stable reconstruction results, the limited-projection FMT remains an intractable challenge for achieving fast and accurate reconstructions. In this work, we completely discard the previous iterative solving-based reconstruction themes and propose multi-branch attention prior based parameterized generative adversarial network (MAP-PGAN) to achieve fast and accurate limited-projection FMT reconstruction. Firstly, the multi-branch attention can provide parameterized weighted sparse prior information for fluorescent sources, enabling MAP-PGAN to effectively mitigate the ill-posedness and significantly improve the reconstruction accuracy of limited-projection FMT. Secondly, since the end-to-end direct reconstruction strategy is adopted, the complex iterative computation process in traditional regularization algorithms can be avoided, thus greatly accelerating the 3D visualization process. The numerical simulation results show that the proposed MAP-PGAN method outperforms the state-of-the-art methods in terms of localization accuracy and morphological recovery. Meanwhile, the reconstruction time is only about 0.18s, which is about 100 to 1000 times faster than the conventional iteration-based regularization algorithms. The reconstruction results from the physical phantoms and in vivo experiments further demonstrate the feasibility and practicality of the MAP-PGAN method in achieving fast and accurate limited-projection FMT reconstruction.

Encoder-decoder deep learning network for simultaneous reconstruction of fluorescence yield and lifetime distributions

A time-domain fluorescence molecular tomography in reflective geometry (TD-rFMT) has been proposed to circumvent the penetration limit and reconstruct fluorescence distribution within a 2.5-cm depth regardless of the object size. In this paper, an end-to-end encoder-decoder network is proposed to further enhance the reconstruction performance of TD-rFMT. The network reconstructs both the fluorescence yield and lifetime distributions directly from the time-resolved fluorescent signals. According to the properties of TD-rFMT, proper noise was added to the simulation training data and a customized loss function was adopted for self-supervised and supervised joint training. Simulations and phantom experiments demonstrate that the proposed network can significantly improve the spatial resolution, positioning accuracy, and accuracy of lifetime values.

In vivo accurate detection of the liver tumor with pharmacokinetic parametric images from dynamic fluorescence molecular tomography

SIGNIFICANCE

Pharmacokinetic parametric images in dynamic fluorescence molecular tomography (FMT) can describe three-dimensional (3D) physiological and pathological information inside biological tissues, potentially providing quantitative assessment tools for biological research and drug development.

AIM

In vivo imaging of the liver tumor with pharmacokinetic parametric images from dynamic FMT based on the differences in metabolic properties of indocyanine green (ICG) between normal liver cells and tumor liver cells inside biological tissues.

APPROACH

First, an orthotopic liver tumor mouse model was constructed. Then, with the help of the FMT/computer tomography (CT) dual-modality imaging system and the direct reconstruction algorithm, 3D imaging of liver metabolic parameters in nude mice was achieved to distinguish liver tumors from normal tissues. Finally, pharmacokinetic parametric imaging results were validated against in vitro anatomical results.

RESULTS

This letter demonstrates the ability of dynamic FMT to monitor the pharmacokinetic delivery of the fluorescent dye ICG in vivo, thus, enabling the distinction between normal and tumor tissues based on the pharmacokinetic parametric images derived from dynamic FMT.

CONCLUSIONS

Compared with CT structural imaging technology, dynamic FMT combined with compartmental modeling as an analytical method can obtain quantitative images of pharmacokinetic parameters, thus providing a more powerful research tool for organ function assessment, disease diagnosis and new drug development.

A review of advances in imaging methodology in fluorescence molecular tomography

Objective. Fluorescence molecular tomography (FMT) is a promising non-invasive optical molecular imaging technology with strong specificity and sensitivity that has great potential for preclinical and clinical studies in tumor diagnosis, drug development and therapeutic evaluation. However, the strong scattering of photons and insufficient surface measurements make it very challenging to improve the quality of FMT image reconstruction and its practical application for early tumor detection. Therefore, continuous efforts have been made to explore more effective approaches or solutions in the pursuit of high-quality FMT reconstructions. Approach. This review takes a comprehensive overview of advances in imaging methodology for FMT, mainly focusing on two critical issues in FMT reconstructions: improving the accuracy of solving the forward physical model and mitigating the ill-posed nature of the inverse problem from a methodological point of view. More importantly, numerous impressive and practical strategies and methods for improving the quality of FMT reconstruction are summarized. Notably, deep learning methods are discussed in detail to illustrate their advantages in promoting the imaging performance of FMT thanks to large datasets, the emergence of optimized algorithms and the application of innovative networks. Main results. The results demonstrate that the imaging quality of FMT can be effectively promoted by improving the accuracy of optical parameter modeling, combined with prior knowledge, and reducing dimensionality. In addition, the traditional regularization-based methods and deep neural network-based methods, especially end-to-end deep networks, can enormously alleviate the ill-posedness of the inverse problem and improve the quality of FMT image reconstruction. Significance. This review aims to illustrate a variety of effective and practical methods for the reconstruction of FMT images that may benefit future research. Furthermore, it may provide some valuable research ideas and directions for FMT in the future, and could promote, to a certain extent, the development of FMT and other methods of optical tomography.

Monte Carlo-based data generation for efficient deep learning reconstruction of macroscopic diffuse optical tomography and topography applications

SIGNIFICANCE

Deep learning (DL) models are being increasingly developed to map sensor data to the image domain directly. However, DL methodologies are data-driven and require large and diverse data sets to provide robust and accurate image formation performances. For research modalities such as 2D/3D diffuse optical imaging, the lack of large publicly available data sets and the wide variety of instrumentation designs, data types, and applications leads to unique challenges in obtaining well-controlled data sets for training and validation. Meanwhile, great efforts over the last four decades have focused on developing accurate and computationally efficient light propagation models that are flexible enough to simulate a wide variety of experimental conditions.

AIM

Recent developments in Monte Carlo (MC)-based modeling offer the unique advantage of simulating accurately light propagation spatially, temporally, and over an extensive range of optical parameters, including minimally to highly scattering tissue within a computationally efficient platform. Herein, we demonstrate how such MC platforms, namely "Monte Carlo eXtreme" and "Mesh-based Monte Carlo," can be leveraged to generate large and representative data sets for training the DL model efficiently.

APPROACH

We propose data generator pipeline strategies using these platforms and demonstrate their potential in fluorescence optical topography, fluorescence optical tomography, and single-pixel diffuse optical tomography. These applications represent a large variety in instrumentation design, sample properties, and contrast function.

RESULTS

DL models trained using the MC-based in silico datasets, validated further with experimental data not used during training, show accurate and promising results.

CONCLUSION

Overall, these MC-based data generation pipelines are expected to support the development of DL models for rapid, robust, and user-friendly image formation in a wide variety of applications.

VoxDMRN: a voxelwise deep max-pooling residual network for bioluminescence tomography reconstruction

Bioluminescence tomography (BLT) has extensive applications in preclinical studies for cancer research and drug development. However, the spatial resolution of BLT is inadequate because the numerical methods are limited for solving the physical models of photon propagation and the restriction of using tetrahedral meshes for reconstruction. We conducted a series of theoretical derivations and divided the BLT reconstruction process into two steps: feature extraction and nonlinear mapping. Inspired by deep learning, a voxelwise deep max-pooling residual network (VoxDMRN) is proposed to establish the nonlinear relationship between the internal bioluminescent source and surface boundary density to improve the spatial resolution in BLT reconstruction. The numerical simulation and in vivo experiments both demonstrated that VoxDMRN greatly improves the reconstruction performance regarding location accuracy, shape recovery capability, dual-source resolution, robustness, and in vivo practicability.

Attention mechanism-based locally connected network for accurate and stable reconstruction in Cerenkov luminescence tomography

Cerenkov luminescence tomography (CLT) is a novel and highly sensitive imaging technique, which could obtain the three-dimensional distribution of radioactive probes to achieve accurate tumor detection. However, the simplified radiative transfer equation and ill-conditioned inverse problem cause a reconstruction error. In this study, a novel attention mechanism based locally connected (AMLC) network was proposed to reduce barycenter error and improve morphological restorability. The proposed AMLC network consisted of two main parts: a fully connected sub-network for providing a coarse reconstruction result, and a locally connected sub-network based on an attention matrix for refinement. Both numerical simulations and in vivo experiments were conducted to show the superiority of the AMLC network in accuracy and stability over existing methods (MFCNN, KNN-LC network). This method improved CLT reconstruction performance and promoted the application of machine learning in optical imaging research.