Super-Resolved Microbubble Localization in Single-Channel Ultrasound RF Signals Using Deep Learning

Deep Learning in Ultrasound Localization Microscopy: Applications and Perspectives

Ultrasound localization microscopy (ULM) is a novel super-resolution imaging technique that can image the vasculature in vivo at depth with resolution far beyond the conventional limit of diffraction. By relying on the localization and tracking of clinically approved microbubbles injected in the blood stream, ULM can provide not only anatomical visualization but also hemodynamic quantification of the microvasculature. Several deep learning approaches have been proposed to address challenges in ULM including denoising, improving microbubble localization, estimating blood flow velocity, or performing aberration correction. Proposed deep learning methods often outperform their conventional counterparts by improving image quality and reducing processing time. In addition, their robustness to high concentrations of microbubbles can lead to reduced acquisition times in ULM, addressing a major hindrance to ULM clinical application. Herein, we propose a comprehensive review of the diversity of deep learning applications in ULM focusing on approaches assuming a sparse microbubble distribution. We first provide an overview of how existing studies vary in the constitution of their datasets or in the tasks targeted by the deep learning model. We also take a deeper look into the numerous approaches that have been proposed to improve the localization of microbubbles since they differ highly in their formulation of the optimization problem, their evaluation, or their network architectures. We finally discuss the current limitations and challenges of these methods, as well as the promises and potential of deep learning for ULM in the future.

Volumetric Ultrasound Localization Microscopy

Super-resolution ultrasound (SRUS) has evolved significantly with the advent of ultrasound localization microscopy (ULM). This technique enables subwavelength resolution imaging using microbubble contrast agents. Initially confined to 2-D imaging, ULM has progressed toward volumetric approaches, allowing for comprehensive 3-D visualization of microvascular networks. This review explores the technological advancements and challenges associated with volumetric ULM, focusing on key aspects such as transducer design, acquisition speed, data processing algorithms, or integration into clinical practice. We discuss the limitations of traditional 2-D ULM, including dependence on precise imaging plane selection and compromised resolution in microvasculature quantification. In contrast, volumetric ULM offers enhanced spatial resolution and allows motion correction in all directions, promising transformative insights into microvascular pathophysiology. By examining current research and future directions, this review highlights the potential of volumetric ULM to contribute significantly to diagnostic across various medical conditions, including cancers, arteriosclerosis, strokes, diabetes, and neurodegenerative diseases.

RF-ULM: Ultrasound Localization Microscopy Learned From Radio-Frequency Wavefronts

In Ultrasound Localization Microscopy (ULM), achieving high-resolution images relies on the precise localization of contrast agent particles across a series of beamformed frames. However, our study uncovers an enormous potential: The process of delay-and-sum beamforming leads to an irreversible reduction of Radio-Frequency (RF) channel data, while its implications for localization remain largely unexplored. The rich contextual information embedded within RF wavefronts, including their hyperbolic shape and phase, offers great promise for guiding Deep Neural Networks (DNNs) in challenging localization scenarios. To fully exploit this data, we propose to directly localize scatterers in RF channel data. Our approach involves a custom super-resolution DNN using learned feature channel shuffling, non-maximum suppression, and a semi-global convolutional block for reliable and accurate wavefront localization. Additionally, we introduce a geometric point transformation that facilitates seamless mapping to the B-mode coordinate space. To understand the impact of beamforming on ULM, we validate the effectiveness of our method by conducting an extensive comparison with State-Of-The-Art (SOTA) techniques. We present the inaugural in vivo results from a wavefront-localizing DNN, highlighting its real-world practicality. Our findings show that RF-ULM bridges the domain shift between synthetic and real datasets, offering a considerable advantage in terms of precision and complexity. To enable the broader research community to benefit from our findings, our code and the associated SOTA methods are made available at https://github.com/hahnec/rf-ulm.

In vivo ultrasound localization microscopy for high-density microbubbles

Ultrasound Localization Microscopy (ULM) surpasses the constraints imposed by acoustic diffraction, achieving sub-wavelength resolution visualization of microvasculature through the precise localization of minute microbubbles (MBs). Nonetheless, the analysis of densely populated regions with overlapping MB point spread responses introduces significant localization errors, limiting the use of technique to low-concentration conditions. This raises a trade-off issue between localization efficiency and MB density. In this work, we present a new deep learning framework that combines Transformer and U-Net architectures, termed ULM-TransUNet. As a non-linear model, it is able to learn the complex data patterns of overlapping MBs in dense conditions for accurate localization. To evaluate the performance of ULM-TransUNet, a series of numerical simulations and in vivo experiments are carried out. Numerical simulation results indicate that ULM-TransUNet achieves high-quality ULM imaging, with improvements of 21.93 % in detection rate, 17.36 % in detection precision, and 20.53 % in detection sensitivity, compared to previous state-of-the-art deep learning (DL) method (e.g., ULM-UNet). For the in vivo experiments, ULM-TransUNet achieves the highest spatial resolution (9.4 μm) and rapid inference speed (26.04 ms/frame). Furthermore, it consistently detects more small vessels and resolves closely spaced vessels more effectively. The outcomes of this work imply that ULM-TransUNet can potentially enhance the microvascular imaging performance on high-density MB conditions.

Nonlinear simulation for contrast ultrasound imaging

Contrast ultrasound (CUS) has received much interest because of its sensitivity enhancement for blood flow imaging. However, there is still a lack of nonlinear simulation method for CUS, as conventional simulators cannot deal with the microbubble acoustic nonlinearity. In this paper, a nonlinear simulation method of CUS is developed based on a combination strategy of the k-space pseudospectral method and Rayleigh-Plesset Marmottant model. Different contrast pulse sequence strategies as well as the radial modulation imaging are simulated and compared using the proposed method. For blood flow imaging, simulations under different scenarios such as power Doppler and ultrasound localization microscopy are also carried out. Furthermore, a face-to-face comparison is performed between simulations and phantom experiments to validate the proposed method.

Ultrasound wavelet spectra enable direct tissue recognition and full-color visualization

Traditional brightness-mode ultrasound imaging is primarily constrained by the low specificity among tissues and the inconsistency among sonographers. The major cause is the imaging method that represents the amplitude of echoes as brightness and ignores other detailed information, leaving sonographers to interpret based on organ contours that depend highly on specific imaging planes. Other ultrasound imaging modalities, color Doppler imaging or shear wave elastography, overlay motion or stiffness information to brightness-mode images. However, tissue-specific scattering properties and spectral patterns remain unknown in ultrasound imaging. Here we demonstrate that the distribution (size and average distance) of scattering particles leads to characteristic wavelet spectral patterns, which enables tissue recognition and high-contrast ultrasound imaging. Ultrasonic wavelet spectra from similar particle distributions tend to cluster in the eigenspace according to principal component analysis, whereas those with different distributions tend to be distinguishable from one another. For each distribution, a few wavelet spectra are unique and act as a fingerprint to recognize the corresponding tissue. Illumination of specific tissues and organs with designated colors according to the recognition results yields high-contrast ultrasound imaging. The fully-colorized tissue-specific ultrasound imaging potentially simplifies the interpretation and promotes consistency among sonographers, or even enables the applicability for non-professionals.

Context-aware deep learning enables high-efficacy localization of high concentration microbubbles for super-resolution ultrasound localization microscopy

Ultrasound localization microscopy (ULM) enables deep tissue microvascular imaging by localizing and tracking intravenously injected microbubbles circulating in the bloodstream. However, conventional localization techniques require spatially isolated microbubbles, resulting in prolonged imaging time to obtain detailed microvascular maps. Here, we introduce LOcalization with Context Awareness (LOCA)-ULM, a deep learning-based microbubble simulation and localization pipeline designed to enhance localization performance in high microbubble concentrations. In silico, LOCA-ULM enhanced microbubble detection accuracy to 97.8% and reduced the missing rate to 23.8%, outperforming conventional and deep learning-based localization methods up to 17.4% in accuracy and 37.6% in missing rate reduction. In in vivo rat brain imaging, LOCA-ULM revealed dense cerebrovascular networks and spatially adjacent microvessels undetected by conventional ULM. We further demonstrate the superior localization performance of LOCA-ULM in functional ULM (fULM) where LOCA-ULM significantly increased the functional imaging sensitivity of fULM to hemodynamic responses invoked by whisker stimulations in the rat brain.

Deep learning for fast super-resolution ultrasound microvessel imaging

Objective. Ultrasound localization microscopy (ULM) enables microvascular reconstruction by localizing microbubbles (MBs). Although ULM can obtain microvascular images that are beyond the ultimate resolution of the ultrasound (US) diffraction limit, it requires long data processing time, and the imaging accuracy is susceptible to the density of MBs. Deep learning (DL)-based ULM is proposed to alleviate these limitations, which simulated MBs at low-resolution and mapped them to coordinates at high-resolution by centroid localization. However, traditional DL-based ULMs are imprecise and computationally complex. Also, the performance of DL is highly dependent on the training datasets, which are difficult to realistically simulate.Approach. A novel architecture called adaptive matching network (AM-Net) and a dataset generation method named multi-mapping (MMP) was proposed to overcome the above challenges. The imaging performance and processing time of the AM-Net have been assessed by simulation andin vivoexperiments.Main results. Simulation results show that at high density (20 MBs/frame), when compared to other DL-based ULM, AM-Net achieves higher localization accuracy in the lateral/axial direction.In vivoexperiment results show that the AM-Net can reconstruct ∼24.3μm diameter micro-vessels and separate two ∼28.3μm diameter micro-vessels. Furthermore, when processing a 128 × 128 pixels image in simulation experiments and an 896 × 1280 pixels imagein vivoexperiment, the processing time of AM-Net is ∼13 s and ∼33 s, respectively, which are 0.3-0.4 orders of magnitude faster than other DL-based ULM.Significance. We proposes a promising solution for ULM with low computing costs and high imaging performance.

Deep learning for fast denoising filtering in ultrasound localization microscopy

Objective.Addition of a denoising filter step in ultrasound localization microscopy (ULM) has been shown to effectively reduce the error localizations of microbubbles (MBs) and achieve resolution improvement for super-resolution ultrasound (SR-US) imaging. However, previous image-denoising methods (e.g. block-matching 3D, BM3D) requires long data processing times, making ULM only able to be processed offline. This work introduces a new way to reduce data processing time through deep learning.Approach.In this study, we propose deep learning (DL) denoising based on contrastive semi-supervised network (CS-Net). The neural network is mainly trained with simulated MBs data to extract MB signals from noise. And the performances of CS-Net denoising are evaluated in bothin vitroflow phantom experiment andin vivoexperiment of New Zealand rabbit tumor.Main results.Forin vitroflow phantom experiment, the signal-to-noise ratio (SNR) and contrast-to-noise ratio (CNR) of single microbubble image are 26.91 dB and 4.01 dB, repectively. Forin vivoanimal experiment , the SNR and CNR were 12.29 dB and 6.06 dB. In addition, single microvessel of 24μm and two microvessels separated by 46μm could be clearly displayed. Most importantly,, the CS-Net denoising speeds forin vitroandin vivoexperiments were 0.041 s frame-1and 0.062 s frame-1, respectively.Significance.DL denoising based on CS-Net can improve the resolution of SR-US as well as reducing denoising time, thereby making further contributions to the clinical real-time imaging of ULM.

Acoustic Molecular Imaging Beyond the Diffraction Limit In Vivo

Ultrasound molecular imaging (USMI) is a technique used to noninvasively estimate the distribution of molecular markers in vivo by imaging microbubble contrast agents (MCAs) that have been modified to target receptors of interest on the vascular endothelium. USMI is especially relevant for preclinical and clinical cancer research and has been used to predict tumor malignancy and response to treatment. In the last decade, methods that improve the resolution of contrast-enhanced ultrasound by an order of magnitude and allow researchers to noninvasively image individual capillaries have emerged. However, these approaches do not translate directly to molecular imaging. In this work, we demonstrate super-resolution visualization of biomarker expression in vivo using superharmonic ultrasound imaging (SpHI) with dual-frequency transducers, targeted contrast agents, and localization microscopy processing. We validate and optimize the proposed method in vitro using concurrent optical and ultrasound microscopy and a microvessel phantom. With the same technique, we perform a proof-of-concept experiment in vivo in a rat fibrosarcoma model and create maps of biomarker expression co-registered with images of microvasculature. From these images, we measure a resolution of 23 μm, a nearly fivefold improvement in resolution compared to previous diffraction-limited molecular imaging studies.

Volumetric tri-modal imaging with combined photoacoustic, ultrasound, and shear wave elastography

Photoacoustic imaging is a hybrid imaging approach that combines the advantages of optical and ultrasonic imaging in one modality. However, for comprehensive tissue characterization, optical contrast alone is not always sufficient. In this study, we combined photoacoustic imaging with high-resolution ultrasound and shear wave elastography. The multi-modal system can calculate optical absorption, acoustic reflection, and stiffness volumetrically. We constructed a multi-modal phantom with contrast for each imaging modality to test the system's performance. Experimental results indicate that the system successfully visualizes the embedded structures. We envision that the system will lead to more comprehensive tissue characterization for cancer screening and diagnosis.