Deep Unfolded Robust PCA With Application to Clutter Suppression in Ultrasound

Interpretable Optimization-Inspired Unfolding Network for Low-Light Image Enhancement

Retinex model-based methods have shown to be effective in layer-wise manipulation with well-designed priors for low-light image enhancement (LLIE). However, the hand-crafted priors and conventional optimization algorithm adopted to solve the layer decomposition problem result in the lack of adaptivity and efficiency. To this end, this paper proposes a Retinex-based deep unfolding network (URetinex-Net++), which unfolds an optimization problem into a learnable network to decompose a low-light image into reflectance and illumination layers. By formulating the decomposition problem as an implicit priors regularized model, three learning-based modules are carefully designed, responsible for data-dependent initialization, high-efficient unfolding optimization, and fairly-flexible component adjustment, respectively. Particularly, the proposed unfolding optimization module, introducing two networks to adaptively fit implicit priors in the data-driven manner, can realize noise suppression and details preservation for decomposed components. URetinex-Net++ is a further augmented version of URetinex-Net, which introduces a cross-stage fusion block to alleviate the color defect in URetinex-Net. Therefore, boosted performance on LLIE can be obtained in both visual quality and quantitative metrics, where only a few parameters are introduced and little time is cost. Extensive experiments on real-world low-light images qualitatively and quantitatively demonstrate the effectiveness and superiority of the proposed URetinex-Net++ over state-of-the-art methods.

Imaging of experimental venous thrombus by means of Doppler and CEUS techniques in dogs

Venous thrombosis has been widely studied in humans, but not in dogs. This study was designed to evaluate a venous thrombus in dogs, from creation to solution, by means of various ultrasonographic techniques. Nine healthy Beagle dogs were included in the study. The venous thrombus was formatted by puncturing the lumen of the external jugular veins and then, the veins were examined with B-mode, color Doppler, pulsed-wave Doppler, and contrast-enhanced ultrasound (CEUS) techniques, at regular intervals, within 210-270 min after venipuncture. Haemodynamic parameters were calculated at two different locations, before and after the site of the thrombus formation. The existence of a thrombus was confirmed by CEUS technique. Thrombus volume and echogenicity were evaluated. The results showed that the visualization of the venous thrombus by color Doppler modality was not feasible in some veins. The blood volume was the parameter that could more precisely indicate the presence or absence of a thrombus. In cases where thrombus volume was less than 0.001 cm3, it was impossible to detect its presence using haemodynamic parameters. The CEUS imaging depicted accurately the size and shape of an anechoic venous thrombus, even when its volume was 0.001cm3.

Deep Learning in Ultrasound Localization Microscopy: Applications and Perspectives

Ultrasound localization microscopy (ULM) is a novel super-resolution imaging technique that can image the vasculature in vivo at depth with resolution far beyond the conventional limit of diffraction. By relying on the localization and tracking of clinically approved microbubbles injected in the blood stream, ULM can provide not only anatomical visualization but also hemodynamic quantification of the microvasculature. Several deep learning approaches have been proposed to address challenges in ULM including denoising, improving microbubble localization, estimating blood flow velocity, or performing aberration correction. Proposed deep learning methods often outperform their conventional counterparts by improving image quality and reducing processing time. In addition, their robustness to high concentrations of microbubbles can lead to reduced acquisition times in ULM, addressing a major hindrance to ULM clinical application. Herein, we propose a comprehensive review of the diversity of deep learning applications in ULM focusing on approaches assuming a sparse microbubble distribution. We first provide an overview of how existing studies vary in the constitution of their datasets or in the tasks targeted by the deep learning model. We also take a deeper look into the numerous approaches that have been proposed to improve the localization of microbubbles since they differ highly in their formulation of the optimization problem, their evaluation, or their network architectures. We finally discuss the current limitations and challenges of these methods, as well as the promises and potential of deep learning for ULM in the future.

Boosting Cardiac Color Doppler Frame Rates With Deep Learning

Color Doppler echocardiography enables visualization of blood flow within the heart. However, the limited frame rate impedes the quantitative assessment of blood velocity throughout the cardiac cycle, thereby compromising a comprehensive analysis of ventricular filling. Concurrently, deep learning is demonstrating promising outcomes in postprocessing of echocardiographic data for various applications. This work explores the use of deep learning models for intracardiac Doppler velocity estimation from a reduced number of filtered I/Q signals. We used a supervised learning approach by simulating patient-based cardiac color Doppler acquisitions and proposed data augmentation strategies to enlarge the training dataset. We implemented architectures based on convolutional neural networks (CNNs). In particular, we focused on comparing the U-Net model and the recent ConvNeXt model, alongside assessing real-valued versus complex-valued representations. We found that both models outperformed the state-of-the-art autocorrelator method, effectively mitigating aliasing and noise. We did not observe significant differences between the use of real and complex data. Finally, we validated the models on in vitro and in vivo experiments. All models produced quantitatively comparable results to the baseline and were more robust to noise. ConvNeXt emerged as the sole model to achieve high-quality results on in vivo aliased samples. These results demonstrate the interest of supervised deep learning methods for Doppler velocity estimation from a reduced number of acquisitions.

Dehazing Ultrasound Using Diffusion Models

Echocardiography has been a prominent tool for the diagnosis of cardiac disease. However, these diagnoses can be heavily impeded by poor image quality. Acoustic clutter emerges due to multipath reflections imposed by layers of skin, subcutaneous fat, and intercostal muscle between the transducer and heart. As a result, haze and other noise artifacts pose a real challenge to cardiac ultrasound imaging. In many cases, especially with difficult-to-image patients such as patients with obesity, a diagnosis from B-Mode ultrasound imaging is effectively rendered unusable, forcing sonographers to resort to contrast-enhanced ultrasound examinations or refer patients to other imaging modalities. Tissue harmonic imaging has been a popular approach to combat haze, but in severe cases is still heavily impacted by haze. Alternatively, denoising algorithms are typically unable to remove highly structured and correlated noise, such as haze. It remains a challenge to accurately describe the statistical properties of structured haze, and develop an inference method to subsequently remove it. Diffusion models have emerged as powerful generative models and have shown their effectiveness in a variety of inverse problems. In this work, we present a joint posterior sampling framework that combines two separate diffusion models to model the distribution of both clean ultrasound and haze in an unsupervised manner. Furthermore, we demonstrate techniques for effectively training diffusion models on radio-frequency ultrasound data and highlight the advantages over image data. Experiments on both in-vitro and in-vivo cardiac datasets show that the proposed dehazing method effectively removes haze while preserving signals from weakly reflected tissue.

Low-rank prior-based Fast-RPCA for clutter filtering and noise suppression in non-contrast ultrasound microvascular imaging

Accurate and real-time separation of blood signal from clutter and noise signals is a critical step in clinical non-contrast ultrasound microvascular imaging. Despite the widespread adoption of singular value decomposition (SVD) and robust principal component analysis (RPCA) for clutter filtering and noise suppression, the SVD's sensitivity to threshold selection, along with the RPCA's limitations in undersampling conditions and heavy computational burden often result in suboptimal performance in complex clinical applications. To address those challenges, this study presents a novel low-rank prior-based fast RPCA (LP-fRPCA) approach to enhance the adaptability and robustness of clutter filtering and noise suppression with reduced computational cost. A low-rank prior constraint is integrated into the non-convex RPCA model to achieve a robust and efficient approximation of clutter subspace, while an accelerated alternating projection iterative algorithm is developed to improve convergence speed and computational efficiency. The performance of the LP-fRPCA method was evaluated against SVD with a tissue/blood threshold (SVD1), SVD with both tissue/blood and blood/noise thresholds (SVD2), and the classical RPCA based on the alternating direction method of multipliers algorithm through phantom and in vivo non-contrast experiments on rabbit kidneys. In the slow flow phantom experiment of 0.2 mm/s, LP-fRPCA achieved an average increase in contrast ratio (CR) of 10.68 dB, 9.37 dB, and 8.66 dB compared to SVD1, SVD2, and RPCA, respectively. In the in vivo rabbit kidney experiment, the power Doppler results demonstrate that the LP-fRPCA method achieved a superior balance in the trade-off between insufficient clutter filtering and excessive suppression of blood flow. Additionally, LP-fRPCA significantly reduced the runtime of RPCA by up to 94-fold. Consequently, the LP-fRPCA method promises to be a potential tool for clinical non-contrast ultrasound microvascular imaging.

Deep Learing for Sparse Domain Kalman Filtering With Applications on ECG Denoising and Motility Estimation

OBJECTIVE

The reconstruction of an input based on a sparse combination of signals, known as sparse coding, has found widespread use in signal processing. In this work, the combination of sparse coding with Kalman filtering is explored and its potential is shown on two use-cases.

METHODS

This work extends the Iterative Shrinkage and Thresholding Algorithm with a Kalman filter in the sparse domain. The resulting method may be implemented as a deep unfolded neural network and may be applied to any signal which has a sparse representation and a known or assumed relation between consecutive measurements. This method is evaluated on the use cases of noise reduction in the electrocardiogram (ECG) and the estimation of object motility.

RESULTS

For ECG denoising, the proposed method achieved an improvement in Signal-to-Noise ratio of 18.6 dB, which is comparable to state-of-the-art. In motility estimation, a correlation of 0.84 with ground truth simulations was found.

CONCLUSION

The proposed method was shown to have advantages over sparse coding and Kalman filtering alone. Due to the low complexity and high generalizability of the proposed method, the implementation of context-specific knowledge or an extension to other applications can be readily made.

SIGNIFICANCE

The presented Kalman-ISTA algorithm is a resource-efficient method combining the promise of both sparse coding and Kalman filtering, making it well-suited for various applications.

Adaptive higher-order singular value decomposition clutter filter for ultrafast Doppler imaging of coronary flow under non-negligible tissue motion

BACKGROUND AND OBJECTIVE

With the development of advanced clutter-filtering techniques by singular value decomposition (SVD) and leveraging favorable acquisition settings such as open-chest imaging by a linear high-frequency probe and plane waves, several studies have shown the feasibility of cardiac flow measurements during the entire cardiac cycle, ranging from coronary flow to myocardial perfusion. When applying these techniques in a routine clinical setting, using transthoracic ultrasound imaging, new challenges emerge. Firstly, a smaller aperture is needed that can fit between ribs. Consequently, diverging waves are employed instead of plane waves to achieve an adequate field of view. Secondly, to ensure imaging at a larger depth, the maximum pulse repetition frequency has to be reduced. Lastly, in comparison to the open-chest scenario, tissue motion induced by the heartbeat is significantly stronger. The latter complicates substantially the distinction between clutter and blood signals.

METHODS

This study investigates a strategy to overcome these challenges by diverging wave imaging with an optimal number of tilt angles, in combination with dedicated clutter-filtering techniques. In particular, a novel, adaptive, higher-order SVD (HOSVD) clutter filter, which utilizes spatial, temporal, and angular information of the received ultrasound signals, is proposed to enhance clutter and blood separation.

RESULTS

When non-negligible tissue motion is present, using fewer tilt angles not only reduces the decorrelation between the received waveforms but also allows for collecting more temporal samples at a given ensemble duration, contributing to improved Doppler performance. The addition of a third angular dimension enables the application of HOSVD, providing greater flexibility in selecting blood separation thresholds from a 3-D tensor. This differs from the conventional threshold selection method in a 2-D spatiotemporal space using SVD. Exhaustive threshold search has shown a significant improvement in Contrast and Contrast-to-Noise ratio for Power Doppler images filtered with HOSVD compared to the SVD-based clutter filter.

CONCLUSION

With the improved settings, the obtained Power Doppler images show the feasibility of measuring coronary flow under the influence of non-negligible tissue motion in both in vitro and ex vivo.

Competitive Swarm Optimized SVD Clutter Filtering for Ultrafast Power Doppler Imaging

Ultrafast power Doppler imaging (uPDI) can significantly increase the sensitivity of resolving small vascular paths in ultrasound. While clutter filtering is a fundamental and essential method to realize uPDI, it commonly uses singular value decomposition (SVD) to suppress clutter signals and noise. However, current SVD-based clutter filters using two cutoffs cannot ensure sufficient separation of tissue, blood, and noise in uPDI. This article proposes a new competitive swarm-optimized SVD clutter filter to improve the quality of uPDI. Specifically, without using two cutoffs, such a new filter introduces competitive swarm optimization (CSO) to search for the counterparts of blood signals in each singular value. We validate the CSO-SVD clutter filter on public in vivo datasets. The experimental results demonstrate that our method can achieve higher contrast-to-noise ratio (CNR), signal-to-noise ratio (SNR), and blood-to-clutter ratio (BCR) than the state-of-the-art SVD-based clutter filters, showing a better balance between suppressing clutter signals and preserving blood signals. Particularly, our CSO-SVD clutter filter improves CNR by 0.99 ± 0.08 dB, SNR by 0.79 ± 0.08 dB, and BCR by 1.95 ± 0.03 dB when comparing a spatial-similarity-based SVD clutter filter in the in vivo dataset of rat brain bolus.

Locating X-Ray Coronary Angiogram Keyframes via Long Short-Term Spatiotemporal Attention With Image-to-Patch Contrastive Learning

Locating the start, apex and end keyframes of moving contrast agents for keyframe counting in X-ray coronary angiography (XCA) is very important for the diagnosis and treatment of cardiovascular diseases. To locate these keyframes from the class-imbalanced and boundary-agnostic foreground vessel actions that overlap complex backgrounds, we propose long short-term spatiotemporal attention by integrating a convolutional long short-term memory (CLSTM) network into a multiscale Transformer to learn the segment- and sequence-level dependencies in the consecutive-frame-based deep features. Image-to-patch contrastive learning is further embedded between the CLSTM-based long-term spatiotemporal attention and Transformer-based short-term attention modules. The imagewise contrastive module reuses the long-term attention to contrast image-level foreground/background of XCA sequence, while patchwise contrastive projection selects the random patches of backgrounds as convolution kernels to project foreground/background frames into different latent spaces. A new XCA video dataset is collected to evaluate the proposed method. The experimental results show that the proposed method achieves a mAP (mean average precision) of 72.45% and a F-score of 0.8296, considerably outperforming the state-of-the-art methods. The source code is available at https://github.com/Binjie-Qin/STA-IPCon.

Deep learning for fast super-resolution ultrasound microvessel imaging

Objective. Ultrasound localization microscopy (ULM) enables microvascular reconstruction by localizing microbubbles (MBs). Although ULM can obtain microvascular images that are beyond the ultimate resolution of the ultrasound (US) diffraction limit, it requires long data processing time, and the imaging accuracy is susceptible to the density of MBs. Deep learning (DL)-based ULM is proposed to alleviate these limitations, which simulated MBs at low-resolution and mapped them to coordinates at high-resolution by centroid localization. However, traditional DL-based ULMs are imprecise and computationally complex. Also, the performance of DL is highly dependent on the training datasets, which are difficult to realistically simulate.Approach. A novel architecture called adaptive matching network (AM-Net) and a dataset generation method named multi-mapping (MMP) was proposed to overcome the above challenges. The imaging performance and processing time of the AM-Net have been assessed by simulation andin vivoexperiments.Main results. Simulation results show that at high density (20 MBs/frame), when compared to other DL-based ULM, AM-Net achieves higher localization accuracy in the lateral/axial direction.In vivoexperiment results show that the AM-Net can reconstruct ∼24.3μm diameter micro-vessels and separate two ∼28.3μm diameter micro-vessels. Furthermore, when processing a 128 × 128 pixels image in simulation experiments and an 896 × 1280 pixels imagein vivoexperiment, the processing time of AM-Net is ∼13 s and ∼33 s, respectively, which are 0.3-0.4 orders of magnitude faster than other DL-based ULM.Significance. We proposes a promising solution for ULM with low computing costs and high imaging performance.

A Convolutional Neural Network for Beamforming and Image Reconstruction in Passive Cavitation Imaging

Convolutional neural networks (CNNs), initially developed for image processing applications, have recently received significant attention within the field of medical ultrasound imaging. In this study, passive cavitation imaging/mapping (PCI/PAM), which is used to map cavitation sources based on the correlation of signals across an array of receivers, is evaluated. Traditional reconstruction techniques in PCI, such as delay-and-sum, yield high spatial resolution at the cost of a substantial computational time. This results from the resource-intensive process of determining sensor weights for individual pixels in these methodologies. Consequently, the use of conventional algorithms for image reconstruction does not meet the speed requirements that are essential for real-time monitoring. Here, we show that a three-dimensional (3D) convolutional network can learn the image reconstruction algorithm for a 16×16 element matrix probe with a receive frequency ranging from 256 kHz up to 1.0 MHz. The network was trained and evaluated using simulated data representing point sources, resulting in the successful reconstruction of volumetric images with high sensitivity, especially for single isolated sources (100% in the test set). As the number of simultaneous sources increased, the network's ability to detect weaker intensity sources diminished, although it always correctly identified the main lobe. Notably, however, network inference was remarkably fast, completing the task in approximately 178 s for a dataset comprising 650 frames of 413 volume images with signal duration of 20μs. This processing speed is roughly thirty times faster than a parallelized implementation of the traditional time exposure acoustics algorithm on the same GPU device. This would open a new door for PCI application in the real-time monitoring of ultrasound ablation.

Interpretable neural networks: principles and applications

In recent years, with the rapid development of deep learning technology, great progress has been made in computer vision, image recognition, pattern recognition, and speech signal processing. However, due to the black-box nature of deep neural networks (DNNs), one cannot explain the parameters in the deep network and why it can perfectly perform the assigned tasks. The interpretability of neural networks has now become a research hotspot in the field of deep learning. It covers a wide range of topics in speech and text signal processing, image processing, differential equation solving, and other fields. There are subtle differences in the definition of interpretability in different fields. This paper divides interpretable neural network (INN) methods into the following two directions: model decomposition neural networks, and semantic INNs. The former mainly constructs an INN by converting the analytical model of a conventional method into different layers of neural networks and combining the interpretability of the conventional model-based method with the powerful learning capability of the neural network. This type of INNs is further classified into different subtypes depending on which type of models they are derived from, i.e., mathematical models, physical models, and other models. The second type is the interpretable network with visual semantic information for user understanding. Its basic idea is to use the visualization of the whole or partial network structure to assign semantic information to the network structure, which further includes convolutional layer output visualization, decision tree extraction, semantic graph, etc. This type of method mainly uses human visual logic to explain the structure of a black-box neural network. So it is a post-network-design method that tries to assign interpretability to a black-box network structure afterward, as opposed to the pre-network-design method of model-based INNs, which designs interpretable network structure beforehand. This paper reviews recent progress in these areas as well as various application scenarios of INNs and discusses existing problems and future development directions.

A Sparse Model-Inspired Deep Thresholding Network for Exponential Signal Reconstruction-Application in Fast Biological Spectroscopy

The nonuniform sampling (NUS) is a powerful approach to enable fast acquisition but requires sophisticated reconstruction algorithms. Faithful reconstruction from partially sampled exponentials is highly expected in general signal processing and many applications. Deep learning (DL) has shown astonishing potential in this field, but many existing problems, such as lack of robustness and explainability, greatly limit its applications. In this work, by combining the merits of the sparse model-based optimization method and data-driven DL, we propose a DL architecture for spectra reconstruction from undersampled data, called MoDern. It follows the iterative reconstruction in solving a sparse model to build the neural network, and we elaborately design a learnable soft-thresholding to adaptively eliminate the spectrum artifacts introduced by undersampling. Extensive results on both synthetic and biological data show that MoDern enables more robust, high-fidelity, and ultrafast reconstruction than the state-of-the-art methods. Remarkably, MoDern has a small number of network parameters and is trained on solely synthetic data while generalizing well to biological data in various scenarios. Furthermore, we extend it to an open-access and easy-to-use cloud computing platform (XCloud-MoDern), contributing a promising strategy for further development of biological applications.

Ultra-fast ultrasound blood flow velocimetry for carotid artery with deep learning

Accurate measurement of blood flow velocity is important for the prevention and early diagnosis of atherosclerosis. However, due to the uncertainty of parameter settings, the autocorrelation velocimetry methods based on clutter filtering are prone to incorrectly filter out the near-wall blood flow signal, resulting in poor velocimetric accuracy. In addition, the Doppler coherent compounding acts as a low-pass filter, which also leads to low values of blood flow velocity estimated by the above methods. Motivated by this status quo, here we propose a deep learning estimator that combines clutter filtering and blood flow velocimetry based on the adaptive property of one-dimensional convolutional neural network (1DCNN). The estimator is operated by first extracting the blood flow signal from the original Doppler echo signal through an affine transformation of the 1D convolution, and then converting the extracted signal into the desired blood flow velocity using a linear transformation function. The effectiveness of the proposed method is verified by simulation as well as in vivo carotid artery data. Compared with typical velocimetry methods such as high-pass filtering (HPF) and singular value decomposition (SVD), the results show that the normalized root means square error (NRMSE) obtained by 1DCNN is reduced by 54.99 % and 53.50 % for forward blood flow velocimetry, and 70.99 % and 69.50 % for reverse blood flow velocimetry, respectively. Consistently, the in vivo measurements demonstrate that the goodness-of-fit of the proposed estimator is improved by 8.72 % and 4.74 % for five subjects. Moreover, the estimation time consumed by 1DCNN is greatly reduced, which costs only 2.91 % of the time of HPF and 12.83 % of the time of SVD. In conclusion, the proposed estimator is a better alternative to the current blood flow velocimetry, and is capable of providing more accurate diagnosis information for vascular diseases in clinical applications.

Exponential Signal Reconstruction With Deep Hankel Matrix Factorization

Exponential function is a basic form of temporal signals, and how to fast acquire this signal is one of the fundamental problems and frontiers in signal processing. To achieve this goal, partial data may be acquired but result in severe artifacts in its spectrum, which is the Fourier transform of exponentials. Thus, reliable spectrum reconstruction is highly expected in the fast data acquisition in many applications, such as chemistry, biology, and medical imaging. In this work, we propose a deep learning method whose neural network structure is designed by imitating the iterative process in the model-based state-of-the-art exponentials' reconstruction method with the low-rank Hankel matrix factorization. With the experiments on synthetic data and realistic biological magnetic resonance signals, we demonstrate that the new method yields much lower reconstruction errors and preserves the low-intensity signals much better than compared methods.

Improved Ultrafast Power Doppler Imaging Using United Spatial-Angular Adaptive Scaling Wiener Postfilter

Ultrafast power Doppler imaging (uPDI) using high-frame-rate plane-wave transmission is a new microvascular imaging modality that offers high Doppler sensitivity. However, due to the unfocused transmission of plane waves, the echo signal is subject to interference from noise and clutter, resulting in a low signal-to-noise ratio (SNR) and poor image quality. Adaptive beamforming techniques are effective in suppressing noise and clutter for improved image quality. In this study, an adaptive beamformer based on a united spatial-angular adaptive scaling Wiener (uSA-ASW) postfilter is proposed to improve the resolution and contrast of uPDI. In the proposed method, the signal power and noise power of the Wiener postfilter are estimated by uniting spatial and angular signals, and a united generalized coherence factor (uGCF) is introduced to dynamically adjust the noise power estimation and enhance the robustness of the method. Simulation and in vivo data were used to verify the effectiveness of the proposed method. The results show that the uSA-ASW can achieve higher resolution and significant improvements in image contrast and background noise suppression compared with conventional delay-and-sum (DAS), coherence factor (CF), spatial-angular CF (SACF), and adaptive scaling Wiener (ASW) postfilter methods. In the simulations, uSA-ASW improves contrast-to-noise ratio (CNR) by 34.7 dB (117.3%) compared with DAS, while reducing background noise power (BNP) by 52 dB (221.4%). The uSA-ASW method provides full-width at half-maximum (FWHM) reductions of [Formula: see text] (59.5%) and [Formula: see text] (56.9%), CNR improvements of 25.6 dB (199.9%) and 42 dB (253%), and BNP reductions of 46.1 dB (319.3%) and 12.9 dB (289.1%) over DAS in the experiments of contrast-free human neonatal brain and contrast-free human liver, respectively. In the contrast-free experiments, uSA-ASW effectively balances the performance of noise and clutter suppression and enhanced microvascular visualization. Overall, the proposed method has the potential to become a reliable microvascular imaging technique for aiding in more accurate diagnosis and detection of vascular-related diseases in clinical contexts.

Applications of Deep Learning Algorithms to Ultrasound Imaging Analysis in Preclinical Studies on In Vivo Animals

BACKGROUND AND AIM

Ultrasound (US) imaging is increasingly preferred over other more invasive modalities in preclinical studies using animal models. However, this technique has some limitations, mainly related to operator dependence. To overcome some of the current drawbacks, sophisticated data processing models are proposed, in particular artificial intelligence models based on deep learning (DL) networks. This systematic review aims to overview the application of DL algorithms in assisting US analysis of images acquired in in vivo preclinical studies on animal models.

METHODS

A literature search was conducted using the Scopus and PubMed databases. Studies published from January 2012 to November 2022 that developed DL models on US images acquired in preclinical/animal experimental scenarios were eligible for inclusion. This review was conducted according to PRISMA guidelines.

RESULTS

Fifty-six studies were enrolled and classified into five groups based on the anatomical district in which the DL models were used. Sixteen studies focused on the cardiovascular system and fourteen on the abdominal organs. Five studies applied DL networks to images of the musculoskeletal system and eight investigations involved the brain. Thirteen papers, grouped under a miscellaneous category, proposed heterogeneous applications adopting DL systems. Our analysis also highlighted that murine models were the most common animals used in in vivo studies applying DL to US imaging.

CONCLUSION

DL techniques show great potential in terms of US images acquired in preclinical studies using animal models. However, in this scenario, these techniques are still in their early stages, and there is room for improvement, such as sample sizes, data preprocessing, and model interpretability.

Deep Null Space Learning Improves Dataset Recovery for High Frame Rate Synthetic Transmit Aperture Imaging

Synthetic transmit aperture (STA) imaging benefits from the two-way dynamic focusing to achieve optimal lateral resolution and contrast resolution in the full field of view, at the cost of low frame rate (FR) and low signal-to-noise ratio (SNR). In our previous studies, compressed sensing-based STA (CS-STA) and minimal ${l}_{{2}}$ -norm least squares (LS-STA) methods were proposed to recover the complete STA dataset from fewer Hadamard-encoded (HE) plane wave (PW) transmissions. Results demonstrated that, compared with STA imaging, CS/LS-STA can maintain the high resolution of STA in the full field of view and improve the contrast in the deep region with increased FR. However, these methods would introduce errors to the recovered STA datasets and subsequently produce severe artifacts to the beamformed images, especially in the shallow region. Recently, we discovered that the theoretical explanation for the error introduced in the LS-STA-based recovery is that the LS-STA method neglects the null space component of the real STA dataset. To deal with this problem, we propose to train a convolutional neural network under the null space learning framework (CNN-Null) to estimate the missing null space component) for high-accuracy recovery of the STA dataset from fewer HE PW transmissions. The mapping between the low-quality STA dataset (i.e., the range space component of the real STA dataset recovered using the LS-STA method) and the missing null space component of the real STA dataset was learned by the network with the high-quality STA dataset (obtained using full HE STA (HE-STA) imaging) as training labels. The performance of the proposed CNN-Null method was compared with the baseline LS-STA, conventional STA, and HE-STA methods, in terms of the visual quality, the normalized root mean square error (NRMSE), the generalized contrast-to-noise ratio (gCNR), and the lateral full-width at half-maximum (FWHM). The results demonstrate that the proposed method can greatly improve the recovery accuracy of the STA datasets (lower NRMSE) and, therefore, effectively suppress the artifacts presented in the images (especially in the shallow region) obtained using the LS-STA method (with a gCNR improvement of 0.4 in the cross-sectional carotid artery images). In addition, the proposed method can maintain the high lateral resolution of STA with fewer (as low as 16) PW transmissions, as LS-STA does.

Ultrasound Signal Processing: From Models to Deep Learning

Medical ultrasound imaging relies heavily on high-quality signal processing to provide reliable and interpretable image reconstructions. Conventionally, reconstruction algorithms have been derived from physical principles. These algorithms rely on assumptions and approximations of the underlying measurement model, limiting image quality in settings where these assumptions break down. Conversely, more sophisticated solutions based on statistical modeling or careful parameter tuning or derived from increased model complexity can be sensitive to different environments. Recently, deep learning-based methods, which are optimized in a data-driven fashion, have gained popularity. These model-agnostic techniques often rely on generic model structures and require vast training data to converge to a robust solution. A relatively new paradigm combines the power of the two: leveraging data-driven deep learning and exploiting domain knowledge. These model-based solutions yield high robustness and require fewer parameters and training data than conventional neural networks. In this work we provide an overview of these techniques from the recent literature and discuss a wide variety of ultrasound applications. We aim to inspire the reader to perform further research in this area and to address the opportunities within the field of ultrasound signal processing. We conclude with a future perspective on model-based deep learning techniques for medical ultrasound.

Tunable image quality control of 3-D ultrasound using switchable CycleGAN

In contrast to 2-D ultrasound (US) for uniaxial plane imaging, a 3-D US imaging system can visualize a volume along three axial planes. This allows for a full view of the anatomy, which is useful for gynecological (GYN) and obstetrical (OB) applications. Unfortunately, the 3-D US has an inherent limitation in resolution compared to the 2-D US. In the case of 3-D US with a 3-D mechanical probe, for example, the image quality is comparable along the beam direction, but significant deterioration in image quality is often observed in the other two axial image planes. To address this, here we propose a novel unsupervised deep learning approach to improve 3-D US image quality. In particular, using unmatched high-quality 2-D US images as a reference, we trained a recently proposed switchable CycleGAN architecture so that every mapping plane in 3-D US can learn the image quality of 2-D US images. Thanks to the switchable architecture, our network can also provide real-time control of image enhancement level based on user preference, which is ideal for a user-centric scanner setup. Extensive experiments with clinical evaluation confirm that our method offers significantly improved image quality as well user-friendly flexibility.

Deep Spatial-Angular Regularization for Light Field Imaging, Denoising, and Super-Resolution

Coded aperture is a promising approach for capturing the 4-D light field (LF), in which the 4-D data are compressively modulated into 2-D coded measurements that are further decoded by reconstruction algorithms. The bottleneck lies in the reconstruction algorithms, resulting in rather limited reconstruction quality. To tackle this challenge, we propose a novel learning-based framework for the reconstruction of high-quality LFs from acquisitions via learned coded apertures. The proposed method incorporates the measurement observation into the deep learning framework elegantly to avoid relying entirely on data-driven priors for LF reconstruction. Specifically, we first formulate the compressive LF reconstruction as an inverse problem with an implicit regularization term. Then, we construct the regularization term with a deep efficient spatial-angular separable convolutional sub-network in the form of local and global residual learning to comprehensively explore the signal distribution free from the limited representation ability and inefficiency of deterministic mathematical modeling. Furthermore, we extend this pipeline to LF denoising and spatial super-resolution, which could be considered as variants of coded aperture imaging equipped with different degradation matrices. Extensive experimental results demonstrate that the proposed methods outperform state-of-the-art approaches to a significant extent both quantitatively and qualitatively, i.e., the reconstructed LFs not only achieve much higher PSNR/SSIM but also preserve the LF parallax structure better on both real and synthetic LF benchmarks. The code will be publicly available at https://github.com/MantangGuo/DRLF.

Deep Unrolled Low-Rank Tensor Completion for High Dynamic Range Imaging

The major challenge in high dynamic range (HDR) imaging for dynamic scenes is suppressing ghosting artifacts caused by large object motions or poor exposures. Whereas recent deep learning-based approaches have shown significant synthesis performance, interpretation and analysis of their behaviors are difficult and their performance is affected by the diversity of training data. In contrast, traditional model-based approaches yield inferior synthesis performance to learning-based algorithms despite their theoretical thoroughness. In this paper, we propose an algorithm unrolling approach to ghost-free HDR image synthesis algorithm that unrolls an iterative low-rank tensor completion algorithm into deep neural networks to take advantage of the merits of both learning- and model-based approaches while overcoming their weaknesses. First, we formulate ghost-free HDR image synthesis as a low-rank tensor completion problem by assuming the low-rank structure of the tensor constructed from low dynamic range (LDR) images and linear dependency among LDR images. We also define two regularization functions to compensate for modeling inaccuracy by extracting hidden model information. Then, we solve the problem efficiently using an iterative optimization algorithm by reformulating it into a series of subproblems. Finally, we unroll the iterative algorithm into a series of blocks corresponding to each iteration, in which the optimization variables are updated by rigorous closed-form solutions and the regularizers are updated by learned deep neural networks. Experimental results on different datasets show that the proposed algorithm provides better HDR image synthesis performance with superior robustness compared with state-of-the-art algorithms, while using significantly fewer training samples.

Application of Ultrasound Combined with Magnetic Resonance Imaging in the Diagnosis and Grading of Patients with Prenatal Placenta Accreta

In order to study the clinical application value of placenta accreta (PIA) diagnosis and grading, the authors propose a method based on ultrasound combined with magnetic resonance imaging in the diagnosis and grading of prenatal placenta accreta patients. This method is adopted in materials and methods: a retrospective analysis of hospital patients with high suspicion of placenta accreta by clinical or ultrasonography between October 2019 and October 2021, the imaging and clinical data of 312 patients who underwent placental MRI examination. The MRI imaging data of all patients were jointly analyzed, and the main observation indicators are as follows: (1) dark zone in the placenta, (2) disruption of the border of the myometrium, (3) disruption of the myometrium, (4) abnormal blood vessels in the placenta, (5) enlargement of the lower part of the uterus, and (6) local bulge of the bladder/or invasion of the adjacent tissues of the uterus. The results show the following: in MRI combined with ultrasonography (P < 0.05), there was no statistical significance in the specificity and accuracy of MRI combined with ultrasound to diagnose PIA (P > 0.05). The comparison of graded diagnostic accuracy showed that in ultrasound alone < MRI alone < MRI combined with ultrasound, the differences were statistically significant (P < 0.05). Ultrasound combined with MRI in the diagnosis of placenta accreta is in good agreement with the clinical and surgical pathological results; MRI examination can be used as an important method for prenatal placenta accreta screening. MRI can classify placenta accreta to some extent.

Multimodal Unrolled Robust PCA for Background Foreground Separation

Background foreground separation (BFS) is a popular computer vision problem where dynamic foreground objects are separated from the static background of a scene. Typically, this is performed using consumer cameras because of their low cost, human interpretability, and high resolution. Yet, cameras and the BFS algorithms that process their data have common failure modes due to lighting changes, highly reflective surfaces, and occlusion. One solution is to incorporate an additional sensor modality that provides robustness to such failure modes. In this paper, we explore the ability of a cost-effective radar system to augment the popular Robust PCA technique for BFS. We apply the emerging technique of algorithm unrolling to yield real-time computation, feedforward inference, and strong generalization in comparison with traditional deep learning methods. We benchmark on the RaDICaL dataset to demonstrate both quantitative improvements of incorporating radar data and qualitative improvements that confirm robustness to common failure modes of image-based methods.

Reverberation clutter signal suppression in ultrasound attenuation estimation using wavelet-based robust principal component analysis

Objective. Ultrasound attenuation coefficient estimation (ACE) has diagnostic potential for clinical applications such as quantifying fat content in the liver. Previously, we have proposed a system-independent ACE technique based on spectral normalization of different frequencies, called the reference frequency method (RFM). This technique does not require a well-calibrated reference phantom for normalization. However, this method may be vulnerable to severe reverberation clutter introduced by the body wall. The clutter superimposed on liver echoes may bias the estimation. Approach. We proposed to use robust principal component analysis, combined with wavelet-based sparsity promotion, to suppress the severe reverberation clutters. The capability to mitigate the reverberation clutters was validated through phantom and in vivo studies. Main Results. In the phantom studies with added reverberation clutters, higher normalized cross-correlation and smaller mean absolute errors were attained as compared to RFM results without the proposed method, demonstrating the capability to reconstruct tissue signals from reverberations. In a pilot patient study, the correlation between ACE and proton density fat fraction (PDFF), a measurement of liver fat by MRI as a reference standard, was investigated. The proposed method showed an improvement of the correlation (coefficient of determination, R = 0.82) as compared with the counterpart without the proposed method (R = 0.69). Significance: The proposed method showed the feasibility of suppressing the reverberation clutters, providing an important basis for the development of a robust ACE with large reverberation clutters.

Deep Unfolding of Iteratively Reweighted ADMM for Wireless RF Sensing

We address the detection of material defects, which are inside a layered material structure using compressive sensing-based multiple-input and multiple-output (MIMO) wireless radar. Here, strong clutter due to the reflection of the layered structure's surface often makes the detection of the defects challenging. Thus, sophisticated signal separation methods are required for improved defect detection. In many scenarios, the number of defects that we are interested in is limited, and the signaling response of the layered structure can be modeled as a low-rank structure. Therefore, we propose joint rank and sparsity minimization for defect detection. In particular, we propose a non-convex approach based on the iteratively reweighted nuclear and ℓ1-norm (a double-reweighted approach) to obtain a higher accuracy compared to the conventional nuclear norm and ℓ1-norm minimization. To this end, an iterative algorithm is designed to estimate the low-rank and sparse contributions. Further, we propose deep learning-based parameter tuning of the algorithm (i.e., algorithm unfolding) to improve the accuracy and the speed of convergence of the algorithm. Our numerical results show that the proposed approach outperforms the conventional approaches in terms of mean squared errors of the recovered low-rank and sparse components and the speed of convergence.

Deep Learning-Based Microbubble Localization for Ultrasound Localization Microscopy

Ultrasound localization microscopy (ULM) is an emerging vascular imaging technique that overcomes the resolution-penetration compromise of ultrasound imaging. Accurate and robust microbubble (MB) localization is essential for successful ULM. In this study, we present a deep learning (DL)-based localization technique that uses both Field-II simulation and in vivo chicken embryo chorioallantoic membrane (CAM) data for training. Both radio frequency (RF) and in-phase and quadrature (IQ) data were tested in this study. The simulation experiment shows that the proposed DL-based localization was able to reduce both missing MB localization rate and MB localization error. In general, RF data showed better performance than IQ. For the in vivo CAM study with high MB concentration, DL-based localization was able to reduce the vessel MB saturation time by more than 50% compared to conventional localization. In addition, we propose a DL-based framework for real-time visualization of the high-resolution microvasculature. The findings of this article support the use of DL for more robust and faster MB localization, especially under high MB concentrations. The results indicate that further improvement could be achieved by incorporating temporal information of the MB data.

Characterization and Removal of RFI Artifacts in Radar Data via Model-Constrained Deep Learning Approach

Microwave remote sensing instruments such as synthetic aperture radar (SAR) play an important role in scientific research applications, while they suffer great measurement distortion with the presence of radio frequency interference (RFI). Existing methods either adopt model−based optimization or follow a data−driven black−box learning scheme, and both have specific limitations in terms of efficiency, accuracy, and interpretability. In this paper, we propose a hybrid model−constrained deep learning approach for RFI extraction and mitigation by fusing the classical model-based and advanced data-driven method. Considering the temporal-spatial correlation of target response, as well as the random sparsity property for time−varying interference, a joint low−rank and sparse optimization framework is established. Instead of applying the iterative optimization process with uncertain convergency, the proposed scheme approximates the iterative process with a stacked recurrent neural network. By adopting this hybrid model−constrained deep learning strategy, the original unsupervised decomposition problem is converted to a supervised learning problem. Experimental results show the validity of the proposed method under diverse RFI scenarios, which could avoid the manual tuning of model hyperparameters as well as speed up the efficiency.

Acceleration of reconstruction for compressed sensing based synthetic transmit aperture imaging by using in-phase/quadrature data

Compressed sensing-based synthetic transmit aperture (CS-STA) was previously proposed to recover the full radio-frequency (RF) channel dataset of synthetic transmit aperture (STA) from that of a smaller number of randomly apodized plane wave (PW) transmissions. In this way, the imaging frame rate (FR) and contrast are improved with maintained spatial resolution, compared with those of STA. Because CS-STA reconstruction is repeated for all receive elements and RF samples (with a high sampling frequency), the recovery of STA dataset in RF domain is time-consuming. In the meantime, a large amount of RF data needs to be transferred and stored, resulting in an increase of system complexity and required memory space. In this study, CS-STA is extended to in-phase/quadrature (IQ) domain (with lower sampling frequency) for the recovery of baseband STA IQ dataset to accelerate the CS-STA reconstruction by reducing the amount of data to be processed. More importantly, CS-STA reconstruction using IQ data is of practical importance, as clinical ultrasound systems typically record baseband IQ signal instead of RF signal. Simulations, phantom and in vivo experiments verify the feasibility of CS-STA in IQ domain for the recovery of STA dataset. More specifically, CS-STA using IQ data achieves similar image quality and appreciably improves reconstruction speed (by ∼3 times) compared with that using RF data. These findings demonstrate that IQ-domain CS-STA is capable of relieving the computational and storage burdens, which may facilitate the implementation of CS-STA in practical ultrasound systems.

ApodNet: Learning for High Frame Rate Synthetic Transmit Aperture Ultrasound Imaging

Two-way dynamic focusing in synthetic transmit aperture (STA) beamforming can benefit high-quality ultrasound imaging with higher lateral spatial resolution and contrast resolution. However, STA requires the complete dataset for beamforming in a relatively low frame rate and transmit power. This paper proposes a deep-learning architecture to achieve high frame rate STA imaging with two-way dynamic focusing. The network consists of an encoder and a joint decoder. The encoder trains a set of binary weights as the apodizations of the high-frame-rate plane wave transmissions. In this respect, we term our network ApodNet. The decoder can recover the complete dataset from the acquired channel data to achieve dynamic transmit focusing. We evaluate the proposed method by simulations at different levels of noise and in-vivo experiments on the human biceps brachii and common carotid artery. The experimental results demonstrate that ApodNet provides a promising strategy for high frame rate STA imaging, obtaining comparable lateral resolution and contrast resolution with four-times higher frame rate than conventional STA imaging in the in-vivo experiments. Particularly, ApodNet improves contrast resolution of the hypoechoic targets with much shorter computational time when compared with other high-frame-rate methods in both simulations and in-vivo experiments.

Deep low-Rank plus sparse network for dynamic MR imaging

In dynamic magnetic resonance (MR) imaging, low-rank plus sparse (L+S) decomposition, or robust principal component analysis (PCA), has achieved stunning performance. However, the selection of the parameters of L+S is empirical, and the acceleration rate is limited, which are common failings of iterative compressed sensing MR imaging (CS-MRI) reconstruction methods. Many deep learning approaches have been proposed to address these issues, but few of them use a low-rank prior. In this paper, a model-based low-rank plus sparse network, dubbed L+S-Net, is proposed for dynamic MR reconstruction. In particular, we use an alternating linearized minimization method to solve the optimization problem with low-rank and sparse regularization. Learned soft singular value thresholding is introduced to ensure the clear separation of the L component and S component. Then, the iterative steps are unrolled into a network in which the regularization parameters are learnable. We prove that the proposed L+S-Net achieves global convergence under two standard assumptions. Experiments on retrospective and prospective cardiac cine datasets show that the proposed model outperforms state-of-the-art CS and existing deep learning methods and has great potential for extremely high acceleration factors (up to 24×).

Fast super-resolution ultrasound microvessel imaging using spatiotemporal data with deep fully convolutional neural network

Ultrasound localization microscopy (ULM) has been proposed to image microvasculature beyond the ultrasound diffraction limit. Although ULM can attain microvascular images with a sub-diffraction resolution, long data acquisition time and processing time are the critical limitations. Deep learning-based ULM (deep-ULM) has been proposed to mitigate these limitations. However, microbubble (MB) localization used in deep-ULMs is currently based on spatial information without the use of temporal information. The highly spatiotemporally coherent MB signals provide a strong feature that can be used to differentiate MB signals from background artifacts. In this study, a deep neural network was employed and trained with spatiotemporal ultrasound datasets to better identify the MB signals by leveraging both the spatial and temporal information of the MB signals. Training, validation and testing datasets were acquired from MB suspension to mimic the realistic intensity-varying and moving MB signals. The performance of the proposed network was first demonstrated in the chicken embryo chorioallantoic membrane dataset with an optical microscopic image as the reference standard. Substantial improvement in spatial resolution was shown for the reconstructed super-resolved images compared with power Doppler images. The full-width-half-maximum (FWHM) of a microvessel was improved from 133μm to 35μm, which is smaller than the ultrasound wavelength (73μm). The proposed method was further tested in anin vivohuman liver data. Results showed the reconstructed super-resolved images could resolve a microvessel of nearly 170μm (FWHM). Adjacent microvessels with a distance of 670μm, which cannot be resolved with power Doppler imaging, can be well-separated with the proposed method. Improved contrast ratios using the proposed method were shown compared with that of the conventional deep-ULM method. Additionally, the processing time to reconstruct a high-resolution ultrasound frame with an image size of 1024 × 512 pixels was around 16 ms, comparable to state-of-the-art deep-ULMs.

Artificial Intelligence and Early Detection of Pancreatic Cancer: 2020 Summative Review

Despite considerable research efforts, pancreatic cancer is associated with a dire prognosis and a 5-year survival rate of only 10%. Early symptoms of the disease are mostly nonspecific. The premise of improved survival through early detection is that more individuals will benefit from potentially curative treatment. Artificial intelligence (AI) methodology has emerged as a successful tool for risk stratification and identification in general health care. In response to the maturity of AI, Kenner Family Research Fund conducted the 2020 AI and Early Detection of Pancreatic Cancer Virtual Summit (www.pdac-virtualsummit.org) in conjunction with the American Pancreatic Association, with a focus on the potential of AI to advance early detection efforts in this disease. This comprehensive presummit article was prepared based on information provided by each of the interdisciplinary participants on one of the 5 following topics: Progress, Problems, and Prospects for Early Detection; AI and Machine Learning; AI and Pancreatic Cancer-Current Efforts; Collaborative Opportunities; and Moving Forward-Reflections from Government, Industry, and Advocacy. The outcome from the robust Summit conversations, to be presented in a future white paper, indicate that significant progress must be the result of strategic collaboration among investigators and institutions from multidisciplinary backgrounds, supported by committed funders.

Ultrasound image reconstruction from plane wave radio-frequency data by self-supervised deep neural network

Image reconstruction from radio-frequency (RF) data is crucial for ultrafast plane wave ultrasound (PWUS) imaging. Compared with the traditional delay-and-sum (DAS) method based on relatively imprecise assumptions, sparse regularization (SR) method directly solves the inverse problem of image reconstruction and has presented significant improvement in the image quality when the frame rate remains high. However, the computational complexity of SR is too high for practical implementation, which is inherently associated with its iterative process. In this work, a deep neural network (DNN), which is trained with an incorporated loss function including sparse regularization terms, is proposed to reconstruct PWUS images from RF data with significantly reduced computational time. It is remarkable that, a self-supervised learning scheme, in which the RF data are utilized as both the inputs and the labels during the training process, is employed to overcome the lack of the "ideal" ultrasound images as the labels for DNN. In addition, it has been also verified that the trained network can be used on the RF data obtained with steered plane waves (PWs), and thus the image quality can be further improved with coherent compounding. Using simulation data, the proposed method has significantly shorter reconstruction time (∼10 ms) than the conventional SR method (∼1-5 mins), with comparable spatial resolution and 1.5-dB higher contrast-to-noise ratio (CNR). Besides, the proposed method with single PW can achieve higher CNR than DAS with 75 PWs in reconstruction of in-vivo images of human carotid arteries.

MetaInv-Net: Meta Inversion Network for Sparse View CT Image Reconstruction

X-ray Computed Tomography (CT) is widely used in clinical applications such as diagnosis and image-guided interventions. In this paper, we propose a new deep learning based model for CT image reconstruction with the backbone network architecture built by unrolling an iterative algorithm. However, unlike the existing strategy to include as many data-adaptive components in the unrolled dynamics model as possible, we find that it is enough to only learn the parts where traditional designs mostly rely on intuitions and experience. More specifically, we propose to learn an initializer for the conjugate gradient (CG) algorithm that involved in one of the subproblems of the backbone model. Other components, such as image priors and hyperparameters, are kept as the original design. Since a hypernetwork is introduced to inference on the initialization of the CG module, it makes the proposed model a certain meta-learning model. Therefore, we shall call the proposed model the meta-inversion network (MetaInv-Net). The proposed MetaInv-Net can be designed with much less trainable parameters while still preserves its superior image reconstruction performance than some state-of-the-art deep models in CT imaging. In simulated and real data experiments, MetaInv-Net performs very well and can be generalized beyond the training setting, i.e., to other scanning settings, noise levels, and data sets.

Detection and Localization of Ultrasound Scatterers Using Convolutional Neural Networks

Delay-and-sum (DAS) beamforming is unable to identify individual scatterers when their density is so high that their point spread functions overlap. This paper proposes a convolutional neural network (CNN)-based method to detect and localize high-density scatterers, some of which are closer than the resolution limit of delay-and-sum (DAS) beamforming. A CNN was designed to take radio frequency channel data and return non-overlapping Gaussian confidence maps. The scatterer positions were estimated from the confidence maps by identifying local maxima. On simulated test sets, the CNN method with three plane waves achieved a precision of 1.00 and a recall of 0.91. Localization uncertainties after excluding outliers were ±46 [Formula: see text] (outlier ratio: 4%) laterally and ±26 [Formula: see text] (outlier ratio: 1%) axially. To evaluate the proposed method on measured data, two phantoms containing cavities were 3-D printed and imaged. For the phantom study, the training data were modified according to the physical properties of the phantoms and a new CNN was trained. On an uniformly spaced scatterer phantom, a precision of 0.98 and a recall of 1.00 were achieved with the localization uncertainties of ±101 [Formula: see text] (outlier ratio: 1%) laterally and ±37 [Formula: see text] (outlier ratio: 1%) axially. On a randomly spaced scatterer phantom, a precision of 0.59 and a recall of 0.63 were achieved. The localization uncertainties were ±132 [Formula: see text] (outlier ratio: 0%) laterally and ±44 [Formula: see text] with a bias of 22 [Formula: see text] (outlier ratio: 0%) axially. This method can potentially be extended to detect highly concentrated microbubbles in order to shorten data acquisition times of super-resolution ultrasound imaging.

Deep Learning for Ultrasound Localization Microscopy

By localizing microbubbles (MBs) in the vasculature, ultrasound localization microscopy (ULM) has recently been proposed, which greatly improves the spatial resolution of ultrasound (US) imaging and will be helpful for clinical diagnosis. Nevertheless, several challenges remain in fast ULM imaging. The main problems are that current localization methods used to implement fast ULM imaging, e.g., a previously reported localization method based on sparse recovery (CS-ULM), suffer from long data-processing time and exhaustive parameter tuning (optimization). To address these problems, in this paper, we propose a ULM method based on deep learning, which is achieved by using a modified sub-pixel convolutional neural network (CNN), termed as mSPCN-ULM. Simulations and in vivo experiments are performed to evaluate the performance of mSPCN-ULM. Simulation results show that even if under high-density condition (6.4 MBs/mm²), a high localization precision ( [Formula: see text] in the lateral direction and [Formula: see text] in the axial direction) and a high localization reliability (Jaccard index of 0.66) can be obtained by mSPCN-ULM, compared to CS-ULM. The in vivo experimental results indicate that with plane wave scan at a transmit center frequency of 15.625 MHz, microvessels with diameters of [Formula: see text] can be detected and adjacent microvessels with a distance of [Formula: see text] can be separated. Furthermore, when using GPU acceleration, the data-processing time of mSPCN-ULM can be shortened to ~6 sec/frame in the simulations and ~23 sec/frame in the in vivo experiments, which is 3-4 orders of magnitude faster than CS-ULM. Finally, once the network is trained, mSPCN-ULM does not need parameter tuning to implement ULM. As a result, mSPCN-ULM opens the door to implement ULM with fast data-processing speed, high imaging accuracy, short data-acquisition time, and high flexibility (robustness to parameters) characteristics.

Learned SPARCOM: unfolded deep super-resolution microscopy

The use of photo-activated fluorescent molecules to create long sequences of low emitter-density diffraction-limited images enables high-precision emitter localization, but at the cost of low temporal resolution. We suggest combining SPARCOM, a recent high-performing classical method, with model-based deep learning, using the algorithm unfolding approach, to design a compact neural network incorporating domain knowledge. Our results show that we can obtain super-resolution imaging from a small number of high emitter density frames without knowledge of the optical system and across different test sets using the proposed learned SPARCOM (LSPARCOM) network. We believe LSPARCOM can pave the way to interpretable, efficient live-cell imaging in many settings, and find broad use in single molecule localization microscopy of biological structures.

Technology trends and applications of deep learning in ultrasonography: image quality enhancement, diagnostic support, and improving workflow efficiency

In this review of the most recent applications of deep learning to ultrasound imaging, the architectures of deep learning networks are briefly explained for the medical imaging applications of classification, detection, segmentation, and generation. Ultrasonography applications for image processing and diagnosis are then reviewed and summarized, along with some representative imaging studies of the breast, thyroid, heart, kidney, liver, and fetal head. Efforts towards workflow enhancement are also reviewed, with an emphasis on view recognition, scanning guide, image quality assessment, and quantification and measurement. Finally some future prospects are presented regarding image quality enhancement, diagnostic support, and improvements in workflow efficiency, along with remarks on hurdles, benefits, and necessary collaborations.

Real time SVD-based clutter filtering using randomized singular value decomposition and spatial downsampling for micro-vessel imaging on a Verasonics ultrasound system

Singular value decomposition (SVD)-based clutter filters can robustly reject the tissue clutter as compared with the conventional high pass filter-based clutter filters. However, the computational burden of SVD makes real time SVD-based clutter filtering challenging (e.g. frame rate at least 10-15 Hz with region of interest of about 4 × 4 cm2). Recently, we proposed an acceleration method based on randomized SVD (rSVD) clutter filtering and randomized spatial downsampling, which can significantly reduce the computational complexity without compromising the clutter rejection capability. However, this method has not been implemented on an ultrasound scanner and tested for its performance. In this study, we implement this acceleration method on a Verasonics scanner using a multi-core CPU architecture, and evaluate the selections of the imaging and processing parameters to enable real time micro-vessel imaging. The Blood-to-Clutter Ratio (BCR) performance was evaluated on a Verasonics machine with different settings of parameters such as block size and ensemble size. The demonstration of real time process was implemented on a 12-core CPU (downsampling factor of 12, 12-threads in this study) host computer. The processing time of the rSVD-based clutter filter was less than 30 ms and BCRs were higher than 20 dB as the block size, ensemble size and the rank of tissue clutter subspace were set as 30 × 30, 45 and 26 respectively. We also demonstrate that the micro-vessel imaging frame rate of the proposed architecture can reach approximately 22 Hz when the block size, ensemble size and the rank of tissue clutter subspace were set as 20 × 20 pixels, 45 and 26 respectively (using both images and supplementary videos). The proposed method may be important for real time 2D scanning of tumor microvessels in 3D to select and store the most representative 2D view with most abnormal micro-vessels for better diagnosis.

Clutter suppression in ultrasound: performance evaluation and review of low-rank and sparse matrix decomposition methods

Vessel diseases are often accompanied by abnormalities related to vascular shape and size. Therefore, a clear visualization of vasculature is of high clinical significance. Ultrasound color flow imaging (CFI) is one of the prominent techniques for flow visualization. However, clutter signals originating from slow-moving tissue are one of the main obstacles to obtain a clear view of the vascular network. Enhancement of the vasculature by suppressing the clutters is a significant and irreplaceable step for many applications of ultrasound CFI. Currently, this task is often performed by singular value decomposition (SVD) of the data matrix. This approach exhibits two well-known limitations. First, the performance of SVD is sensitive to the proper manual selection of the ranks corresponding to clutter and blood subspaces. Second, SVD is prone to failure in the presence of large random noise in the dataset. A potential solution to these issues is using decomposition into low-rank and sparse matrices (DLSM) framework. SVD is one of the algorithms for solving the minimization problem under the DLSM framework. Many other algorithms under DLSM avoid full SVD and use approximated SVD or SVD-free ideas which may have better performance with higher robustness and less computing time. In practice, these models separate blood from clutter based on the assumption that steady clutter represents a low-rank structure and that the moving blood component is sparse. In this paper, we present a comprehensive review of ultrasound clutter suppression techniques and exploit the feasibility of low-rank and sparse decomposition schemes in ultrasound clutter suppression. We conduct this review study by adapting 106 DLSM algorithms and validating them against simulation, phantom, and in vivo rat datasets. Two conventional quality metrics, signal-to-noise ratio (SNR) and contrast-to-noise ratio (CNR), are used for performance evaluation. In addition, computation times required by different algorithms for generating clutter suppressed images are reported. Our extensive analysis shows that the DLSM framework can be successfully applied to ultrasound clutter suppression.