Deep Learning-Based Microbubble Localization for Ultrasound Localization Microscopy

Context-aware deep learning enables high-efficacy localization of high concentration microbubbles for super-resolution ultrasound localization microscopy

Ultrasound localization microscopy (ULM) enables deep tissue microvascular imaging by localizing and tracking intravenously injected microbubbles circulating in the bloodstream. However, conventional localization techniques require spatially isolated microbubbles, resulting in prolonged imaging time to obtain detailed microvascular maps. Here, we introduce LOcalization with Context Awareness (LOCA)-ULM, a deep learning-based microbubble simulation and localization pipeline designed to enhance localization performance in high microbubble concentrations. In silico, LOCA-ULM enhanced microbubble detection accuracy to 97.8% and reduced the missing rate to 23.8%, outperforming conventional and deep learning-based localization methods up to 17.4% in accuracy and 37.6% in missing rate reduction. In in vivo rat brain imaging, LOCA-ULM revealed dense cerebrovascular networks and spatially adjacent microvessels undetected by conventional ULM. We further demonstrate the superior localization performance of LOCA-ULM in functional ULM (fULM) where LOCA-ULM significantly increased the functional imaging sensitivity of fULM to hemodynamic responses invoked by whisker stimulations in the rat brain.

Artificial intelligence-based speckle featurization and localization for ultrasound speckle tracking velocimetry

Deep learning-based super-resolution ultrasound (DL-SRU) framework has been successful in improving spatial resolution and measuring the velocity field information of a blood flows by localizing and tracking speckle signals of red blood cells (RBCs) without using any contrast agents. However, DL-SRU can localize only a small part of the speckle signals of blood flow owing to ambiguity problems encountered in the classification of blood flow signals from ultrasound B-mode images and the building up of suitable datasets required for training artificial neural networks, as well as the structural limitations of the neural network itself. An artificial intelligence-based speckle featurization and localization (AI-SFL) framework is proposed in this study. It includes a machine learning-based algorithm for classifying blood flow signals from ultrasound B-mode images, dimensionality reduction for featurizing speckle patterns of the classified blood flow signals by approximating them with quantitative values. A novel and robust neural network (ResSU-net) is trained using the online data generation (ODG) method and the extracted speckle features. The super-resolution performance of the proposed AI-SFL and ODG method is evaluated and compared with the results of previous U-net and conventional data augmentation methods under in silico conditions. The predicted locations of RBCs by the AI-SFL and DL-SRU for speckle patterns of blood flow are applied to a PTV algorithm to measure quantitative velocity fields of the flow. Finally, the feasibility of the proposed AI-SFL framework for measuring real blood flows is verified under in vivo conditions.

A Convolutional Neural Network for Beamforming and Image Reconstruction in Passive Cavitation Imaging

Convolutional neural networks (CNNs), initially developed for image processing applications, have recently received significant attention within the field of medical ultrasound imaging. In this study, passive cavitation imaging/mapping (PCI/PAM), which is used to map cavitation sources based on the correlation of signals across an array of receivers, is evaluated. Traditional reconstruction techniques in PCI, such as delay-and-sum, yield high spatial resolution at the cost of a substantial computational time. This results from the resource-intensive process of determining sensor weights for individual pixels in these methodologies. Consequently, the use of conventional algorithms for image reconstruction does not meet the speed requirements that are essential for real-time monitoring. Here, we show that a three-dimensional (3D) convolutional network can learn the image reconstruction algorithm for a 16×16 element matrix probe with a receive frequency ranging from 256 kHz up to 1.0 MHz. The network was trained and evaluated using simulated data representing point sources, resulting in the successful reconstruction of volumetric images with high sensitivity, especially for single isolated sources (100% in the test set). As the number of simultaneous sources increased, the network's ability to detect weaker intensity sources diminished, although it always correctly identified the main lobe. Notably, however, network inference was remarkably fast, completing the task in approximately 178 s for a dataset comprising 650 frames of 413 volume images with signal duration of 20μs. This processing speed is roughly thirty times faster than a parallelized implementation of the traditional time exposure acoustics algorithm on the same GPU device. This would open a new door for PCI application in the real-time monitoring of ultrasound ablation.

Deep learning for fast denoising filtering in ultrasound localization microscopy

Objective.Addition of a denoising filter step in ultrasound localization microscopy (ULM) has been shown to effectively reduce the error localizations of microbubbles (MBs) and achieve resolution improvement for super-resolution ultrasound (SR-US) imaging. However, previous image-denoising methods (e.g. block-matching 3D, BM3D) requires long data processing times, making ULM only able to be processed offline. This work introduces a new way to reduce data processing time through deep learning.Approach.In this study, we propose deep learning (DL) denoising based on contrastive semi-supervised network (CS-Net). The neural network is mainly trained with simulated MBs data to extract MB signals from noise. And the performances of CS-Net denoising are evaluated in bothin vitroflow phantom experiment andin vivoexperiment of New Zealand rabbit tumor.Main results.Forin vitroflow phantom experiment, the signal-to-noise ratio (SNR) and contrast-to-noise ratio (CNR) of single microbubble image are 26.91 dB and 4.01 dB, repectively. Forin vivoanimal experiment , the SNR and CNR were 12.29 dB and 6.06 dB. In addition, single microvessel of 24μm and two microvessels separated by 46μm could be clearly displayed. Most importantly,, the CS-Net denoising speeds forin vitroandin vivoexperiments were 0.041 s frame-1and 0.062 s frame-1, respectively.Significance.DL denoising based on CS-Net can improve the resolution of SR-US as well as reducing denoising time, thereby making further contributions to the clinical real-time imaging of ULM.

Nanodroplet-Based Super-Resolution Ultrasound Localization Microscopy

Over the past decade, super-resolution ultrasound localization microscopy (SR-ULM) has revolutionized ultrasound imaging with its capability to resolve the microvascular structures below the ultrasound diffraction limit. The introduction of this imaging technique enables the visualization, quantification, and characterization of tissue microvasculature. The early implementations of SR-ULM utilize microbubbles (MBs) that require a long image acquisition time due to the requirement of capturing sparsely isolated microbubble signals. The next-generation SR-ULM employs nanodroplets that have the potential to significantly reduce the image acquisition time without sacrificing the resolution. This review discusses various nanodroplet-based ultrasound localization microscopy techniques and their corresponding imaging mechanisms. A summary is given on the preclinical applications of SR-ULM with nanodroplets, and the challenges in the clinical translation of nanodroplet-based SR-ULM are presented while discussing the future perspectives. In conclusion, ultrasound localization microscopy is a promising microvasculature imaging technology that can provide new diagnostic and prognostic information for a wide range of pathologies, such as cancer, heart conditions, and autoimmune diseases, and enable personalized treatment monitoring at a microlevel.

Applications of Deep Learning Algorithms to Ultrasound Imaging Analysis in Preclinical Studies on In Vivo Animals

BACKGROUND AND AIM

Ultrasound (US) imaging is increasingly preferred over other more invasive modalities in preclinical studies using animal models. However, this technique has some limitations, mainly related to operator dependence. To overcome some of the current drawbacks, sophisticated data processing models are proposed, in particular artificial intelligence models based on deep learning (DL) networks. This systematic review aims to overview the application of DL algorithms in assisting US analysis of images acquired in in vivo preclinical studies on animal models.

METHODS

A literature search was conducted using the Scopus and PubMed databases. Studies published from January 2012 to November 2022 that developed DL models on US images acquired in preclinical/animal experimental scenarios were eligible for inclusion. This review was conducted according to PRISMA guidelines.

RESULTS

Fifty-six studies were enrolled and classified into five groups based on the anatomical district in which the DL models were used. Sixteen studies focused on the cardiovascular system and fourteen on the abdominal organs. Five studies applied DL networks to images of the musculoskeletal system and eight investigations involved the brain. Thirteen papers, grouped under a miscellaneous category, proposed heterogeneous applications adopting DL systems. Our analysis also highlighted that murine models were the most common animals used in in vivo studies applying DL to US imaging.

CONCLUSION

DL techniques show great potential in terms of US images acquired in preclinical studies using animal models. However, in this scenario, these techniques are still in their early stages, and there is room for improvement, such as sample sizes, data preprocessing, and model interpretability.

Localization Free Super-Resolution Microbubble Velocimetry Using a Long Short-Term Memory Neural Network

Ultrasound localization microscopy is a super-resolution imaging technique that exploits the unique characteristics of contrast microbubbles to side-step the fundamental trade-off between imaging resolution and penetration depth. However, the conventional reconstruction technique is confined to low microbubble concentrations to avoid localization and tracking errors. Several research groups have introduced sparsity- and deep learning-based approaches to overcome this constraint to extract useful vascular structural information from overlapping microbubble signals, but these solutions have not been demonstrated to produce blood flow velocity maps of the microcirculation. Here, we introduce Deep-SMV, a localization free super-resolution microbubble velocimetry technique, based on a long short-term memory neural network, that provides high imaging speed and robustness to high microbubble concentrations, and directly outputs blood velocity measurements at a super-resolution. Deep-SMV is trained efficiently using microbubble flow simulation on real in vivo vascular data and demonstrates real-time velocity map reconstruction suitable for functional vascular imaging and pulsatility mapping at super-resolution. The technique is successfully applied to a wide variety of imaging scenarios, include flow channel phantoms, chicken embryo chorioallantoic membranes, and mouse brain imaging. An implementation of Deep-SMV is openly available at https://github.com/chenxiptz/SR_microvessel_velocimetry, with two pre-trained models available at https://doi.org/10.7910/DVN/SECUFD.

Microbubbles for human diagnosis and therapy

Microbubbles (MBs) were observed for the first time in vivo as a curious consequence of quick saline injection during ultrasound (US) imaging of the aortic root, more than 50 years ago. From this serendipitous event, MBs are now widely used as contrast enhancers for US imaging. Their intrinsic properties described in this review, allow a multitude of designs, from shell to gas composition but also from grafting targeting agents to drug payload encapsulation. Indeed, the versatile MBs are deeply studied for their dual potential in imaging and therapy. As presented in this paper, new generations of MBs now opens perspectives for targeted molecular imaging along with the development of new US imaging systems. This review also presents an overview of the different therapeutic strategies with US and MBs for cancer, cardiovascular diseases, and inflammation. The overall aim is to overlap those fields in order to find similarities in the MBs application for treatment enhancement associated with US. To conclude, this review explores the new scales of MBs technologies with nanobubbles development, and along concurrent advances in the US imaging field. This review ends by discussing perspectives for the booming future uses of MBs.

Three-Dimensional Ultrasound Localization Microscopy with Bipartite Graph-Based Microbubble Pairing and Kalman-Filtering-Based Tracking on a 256-Channel Verasonics Ultrasound System with a 32 × 32 Matrix Array

Three-dimensional (3D) ultrasound localization microscopy (ULM) using a 2-D matrix probe and microbubbles (MBs) has been recently proposed to visualize microvasculature beyond the ultrasound diffraction limit in three spatial dimensions. However, 3D ULM suffers from several limitations: (1) high system complexity due to numerous channel counts, (2) complex MB flow dynamics in 3D, and (3) extremely long acquisition time. To reduce the system complexity while maintaining high image quality, we used a sub-aperture process to reduce received channel counts. To address the second issue, a 3D bipartite graph-based method with Kalman filtering-based tracking was used in this study for MB tracking. An MB separation approach was incorporated to separate high concentration MB data into multiple, sparser MB datasets, allowing better MB localization and tracking for a limited acquisition time. The proposed method was first validated in a flow channel phantom, showing improved spatial resolutions compared with the contrasted enhanced power Doppler image. Then the proposed method was evaluated with an in vivo chicken embryo brain dataset. Results showed that the reconstructed 3D super-resolution image achieved a spatial resolution of around 52 μm (smaller than the wavelength of around 200 μm). Microvessels that cannot be resolved clearly using localization only, can be well identified with the tailored 3D pairing and tracking algorithms. To sum up, the feasibility of the 3D ULM is shown, indicating the great possibility in clinical applications.

Deep Learning-Based Image Analysis for the Quantification of Tumor-Induced Angiogenesis in the 3D In Vivo Tumor Model—Establishment and Addition to Laser Speckle Contrast Imaging (LSCI)

(1) Background: angiogenesis plays an important role in the growth and metastasis of tumors. We established the CAM assay application, an image analysis software of the IKOSA platform by KML Vision, for the quantification of blood vessels with the in ovo chorioallantoic membrane (CAM) model. We added this proprietary deep learning algorithm to the already established laser speckle contrast imaging (LSCI). (2) Methods: angiosarcoma cell line tumors were grafted onto the CAM. Angiogenesis was measured at the beginning and at the end of tumor growth with both measurement methods. The CAM assay application was trained to enable the recognition of in ovo CAM vessels. Histological stains of the tissue were performed and gluconate, an anti-angiogenic substance, was applied to the tumors. (3) Results: the angiosarcoma cells formed tumors on the CAM that appeared to stay vital and proliferated. An increase in perfusion was observed using both methods. The CAM assay application was successfully established in the in ovo CAM model and anti-angiogenic effects of gluconate were observed. (4) Conclusions: the CAM assay application appears to be a useful method for the quantification of angiogenesis in the CAM model and gluconate could be a potential treatment of angiosarcomas. Both aspects should be evaluated in further research.