Super-Resolution Ultrasound Localization Microscopy Through Deep Learning

Pruning Sparse Tensor Neural Networks Enables Deep Learning for 3D Ultrasound Localization Microscopy

Ultrasound Localization Microscopy (ULM) is a non-invasive technique that allows for the imaging of micro-vessels in vivo, at depth and with a resolution on the order of ten microns. ULM is based on the sub-resolution localization of individual microbubbles injected in the bloodstream. Mapping the whole angioarchitecture requires the accumulation of microbubbles trajectories from thousands of frames, typically acquired over a few minutes. ULM acquisition times can be reduced by increasing the microbubble concentration, but requires more advanced algorithms to detect them individually. Several deep learning approaches have been proposed for this task, but they remain limited to 2D imaging, in part due to the associated large memory requirements. Herein, we propose the use of sparse tensor neural networks to enable deep learning-based 3D ULM by improving memory scalability with increased dimensionality. We study several approaches to efficiently convert ultrasound data into a sparse format and study the impact of the associated loss of information. When applied in 2D, the sparse formulation reduces the memory requirements by a factor 2 at the cost of a small reduction of performance when compared against dense networks. In 3D, the proposed approach reduces memory requirements by two order of magnitude while largely outperforming conventional ULM in high concentration settings. We show that Sparse Tensor Neural Networks in 3D ULM allow for the same benefits as dense deep learning based method in 2D ULM i.e. the use of higher concentration in silico and reduced acquisition time.

The impact of monodisperse microbubble size on contrast-enhanced ultrasound super-localization imaging

Contrast-enhanced ultrasound (CEUS) super-localization imaging has shown promise for the assessment of microvascular networks by localizing and tracking microbubbles. The size of the available microbubbles for clinical use is polydisperse, but size-tailorable monodisperse microbubbles are now being developed that present a narrow size distribution. Therefore, proper frequency and pressure tuning have the potential to improve the signal-to-noise ratio and resolution of CEUS acquisitions, which can be expected to increase the performance of CEUS super-localization imaging. In this work, the impact of monodisperse microbubble size on CEUS imaging quality and the efficacy of super-localization imaging was investigated by jointly tuning different frequencies and pressures for different monodisperse microbubble size when performing in vitro CEUS imaging of microbubbles flowing through a dedicated sugar-printed dual-bifurcation microvasculature phantom. The obtained CEUS acquisitions were then post-processed to generate a super-localization output using the Gaussian-centroid localization approach. Four metrics, including generalized contrast-to-noise ratio, full-width half-maximum, number of localization events, and localization F1-score, were employed to quantify the CEUS imaging quality and super-localization performance. In general, jointly optimizing the transmit frequency and pressure for monodisperse microbubbles with smaller size leads to improved CEUS imaging and better super-localization performance. Yet, the weaker backscatter of smaller microbubbles must also be considered.

Optimizingin vivodata acquisition for robust clinical microvascular imaging using ultrasound localization microscopy

Objective. Ultrasound localization microscopy (ULM) enables microvascular imaging at spatial resolutions beyond the acoustic diffraction limit, offering significant clinical potentials. However, ULM performance relies heavily on microbubble (MB) signal sparsity, the number of detected MBs, and signal-to-noise ratio (SNR), all of which vary in clinical scenarios involving bolus MB injections. These sources of variations underscore the need to optimize MB dosage, data acquisition timing, and imaging settings in order to standardize and optimize ULM of microvasculature. This pilot study aims to investigate the temporal changes in MB signals during bolus injections in both pig and human models to optimize data acquisition for clinical ULM.Approach.Quantitative indices, mainly including individual MB SNR, normalized cross-correlation (NCC) of the MB signal with the point-spread function, and the number of localizable MBs, were developed to evaluate MB signal quality and guide the selection of acquisition timing. The effects of transmitted voltage and dosage on signal quality for MB localization were also explored.Main results. In both pig and human studies, MB localization quality (primarily indicated by NCC) reached a minimum at peak MB concentration, then improved as MB counts decreased during the wash-out phase. An optimal acquisition window was identified by balancing localization quality (empirically, NCC > 0.57) and MB concentration. In the pig model, a relatively short time window (approximately 10 s) for optimal acquisition was identified during the rapid wash-out phase, highlighting the need for real-time MB signal monitoring during data acquisition. The slower wash-out phase in humans allowed for a more flexible imaging window of 1-2 min, while trade-offs were observed between localization quality and MB density (or acquisition length) at different wash-out phase timings. Guided by these findings, robust ULM imaging was achieved in both pig and human kidneys using a short period of data acquisition (3.6 s and 9.6 s of data), demonstrating its feasibility in clinical practice.Significance.This study provides insights into optimizing data acquisition for consistent and reproducible ULM, paving the way for its standardization and broader clinical applications.

Improved Microbubble Tracking for Super-Resolution Ultrasound Localization Microscopy using a Bi-Directional Long Short-term Memory Neural Network

Ultrasound localization microscopy (ULM) enabled high-accuracy measurements of microvessel flow beyond the resolution limit of conventional ultrasound imaging by utilizing contrast microbubbles (MBs) as point targets. Robust tracking of MBs is an essential task for fast and high-quality ULM image reconstruction. Existing MB tracking methods suffer from challenging imaging scenarios such as high-density MB distributions, fast blood flow, and complex flow dynamics. Here we present a deep learning-based MB pairing and tracking method based on a bi-directional long short-term memory neural network for ULM. The proposed method integrates multiparametric MB characteristics to facilitate more robust and accurate MB pairing and tracking. The method was validated on a simulation data set, a tissue-mimicking flow phantom, and in vivo on a mouse and rat brain.

Relationship between contrast-enhanced ultrasound combined with ultrasound resolution microscopy imaging and histological features of hepatocellular carcinoma

OBJECTIVES

Using contrast-enhanced ultrasound (CEUS) and ultrasound resolution microscopy (URM) imaging, this study aimed to evaluate the relationship between microvascular parameters of small hepatocellular carcinoma (sHCC) (≤ 3 cm) and microscopic histological features, which include vessels encapsulating tumour clusters (VETC), microvascular invasion (MVI), and histological grade.

METHODS

Sixteen patients with solitary resected sHCC were prospectively enrolled. CEUS and URM were performed one week before resection. All "ratio" refers to comparisons between the active area (where CEUS microbubble show visible motion track by URM) and the entire lesion. Blood vessel complexity (ratio), blood vessel density (ratio), area (ratio), flow velocity, blood vessel diameter, and perfusion index ("flow velocity" × "vessel ratio") were analysed using URM. The relationships between URM parameters and microscopic histological features (MVI, VETC, and histological grade) were analysed.

RESULTS

There were 5 (31.3%), 8 (50%), and 7 (43.7%) cases of poorly differentiated, MVI-positive, and VETC-positive HCC, respectively. The mean velocity of the entire lesion was higher in the poorly differentiated group than that in the moderately differentiated group (p = 0.026). The complexity ratio (MVI-positive: 1.07 ± 0.03, MVI-negative: 1.03 ± 0.02, p = 0.012), area ratio (MVI-positive: 0.63 ± 0.18, MVI-negative: 0.39 ± 0.16, p = 0.017), and perfusion index (MVI-positive: 8.67 ± 1.88, MVI-negative: 6.42 ± 0.94, p = 0.009) were greater in MVI-positive HCCs. The density ratio (VETC-positive: 1.30 ± 0.19, VETC-negative: 1.10 ± 0.05, p = 0.006) was larger in VETC-positive HCCs.

CONCLUSION

Higher blood flow velocity and area of HCC lesions, and higher blood vessel complexity and density may be related to microscopic histological features. This relationship might provide a strategy of using URM for preoperative non-invasive diagnostic VETC, MVI, and histological grade in the future.

A Tracking Prior to Localization Workflow for Ultrasound Localization Microscopy

Ultrasound Localization Microscopy (ULM) has proven effective in resolving microvascular structures and local mean velocities at sub-diffraction-limited scales, offering high-resolution imaging capabilities. Dynamic ULM (DULM) enables the creation of angiography or velocity movies throughout cardiac cycles. Currently, these techniques rely on a Localization-and-Tracking (LAT) workflow consisting in detecting microbubbles (MB) in the frames before pairing them to generate tracks. While conventional LAT methods perform well at low concentrations, they suffer from longer acquisition times and degraded localization and tracking accuracy at higher concentrations, leading to biased angiogram reconstruction and velocity estimation. In this study, we propose a novel approach to address these challenges by reversing the current workflow. The proposed method, Tracking-and-Localization (TAL), relies on first tracking the MB and then performing localization. Through comprehensive benchmarking using both in silico and in vivo experiments and employing various metrics to quantify ULM angiography and velocity maps, we demonstrate that the TAL method consistently outperforms the reference LAT workflow. Moreover, when applied to DULM, TAL successfully extracts velocity variations along the cardiac cycle with improved repeatability. The findings of this work highlight the effectiveness of the TAL approach in overcoming the limitations of conventional LAT methods, providing enhanced ULM angiography and velocity imaging.

Ultrasound localization microscopy in the diagnosis of breast tumors and prediction of relevant histologic biomarkers associated with prognosis in humans: the protocol for a prospective, multicenter study

BACKGROUND

Benign and malignant breast tumors differ in their microvasculature morphology and distribution. Histologic biomarkers of malignant breast tumors are also correlated with the microvasculature. There is a lack of imaging technology for evaluating the microvasculature. Ultrasound localization microscopy (ULM) can provide detailed microvascular architecture at super-resolution. The objective of this trial is to explore the role of ULM in distinguishing benign from malignant breast tumors and to explore the correlations between ULM qualitative and quantitative parameters and histologic biomarkers in malignant breast tumors.

METHODS/DESIGN

This prospective and multicenter study will include 83 patients with breast tumors that will undergo ULM. 55 patients will be assigned to the malignant group, and 28 patients will be assigned to the benign group. The primary outcome is the differences in the qualitative parameters (microvasculature morphology, distribution, and flow direction) between benign and malignant breast tumors on ULM. Secondary outcomes include (1) differences in the quantitative parameters (microvasculature density, tortuosity, diameter, and flow velocity) between benign and malignant breast tumors based on ULM; (2) diagnostic performance of the qualitative parameters in distinguishing benign and malignant breast tumors; (3) diagnostic performance of the quantitative parameters in distinguishing benign and malignant breast tumors; (4) relationships between the qualitative parameters and histologic biomarkers in malignant breast tumors; (5) relationships between the quantitative parameters and histologic biomarkers in malignant breast tumors; and (6) the evaluation of inter-reader and intra-reader reproducibility.

DISCUSSION

Detecting vascularity in breast tumors is of great significance to differentiate benign from malignant tumors and to predict histologic biomarkers. These histologic biomarkers, such as ER, PR, HER2 and Ki67, are closely related to prognosis evaluation. This trial will provide maximum information about the microvasculature of breast tumors and thereby will help with the formulation of subsequent differential diagnosis and the prediction of histologic biomarkers.

TRIAL REGISTRATION NUMBER/DATE

Chinese Clinical Trial Registry ChiCTR2100048361/6th/July/2021. This study is a part of that clinical trial.

Microbubble tracking based on partial smoothing-based adaptive generalized labelled Multi-Bernoulli filter for super-resolution imaging

Super-resolution ultrasound (SRUS) can image the vasculature at microscopic resolution according to microbubble (MB) localization, with velocity vector maps obtained based on MB tracking information. High MB concentrations can reduce the acquisition time of SRUS imaging, however adjacent and intersecting vessels are difficult to distinguish, thus decreasing resolution. Low acquisition frame rates affect the precision of flow velocity estimation. This study proposes a partial smoothing-based adaptive generalized labeled multi-Bernoulli filter (SAGLMB) to precisely track the MB motion at different flow velocities. SAGLMB employs a generalized labelled multi-Bernoulli filter (GLMB) for MB trajectory allocation to separate adjacent and intersecting vessels. Furthermore, the nonlinear motion of MB was predicted by an unscented Kalman filter, and a cardinalized probability hypothesis density filter was applied to suppress clutter interference. Finally, the trajectories were smoothed by unscented Rauch-Tung-Striebel to improve the resolution of the SRUS image. The simulation results demonstrate that SAGLMB outperforms the conventional bipartite graph-based tracking at high MB concentrations, achieving at least an 8.55 % improvement in the correctly paired precision, with 3 times increase in the structural similarity index measure. Moreover, SAGLMB can obtain more precise flow velocity estimations with a 4 times improvement than the conventional method. The SRUS results of rabbit kidney show that the proposed method significantly improves resolution of adjacent and intersecting vessels at higher MB concentrations and maintains this performance as the acquisition frame rate decreases. Furthermore, the rat brain microvascular network was reconstructed with 9.21 μm (λ/11.1) resolution. Therefore, SAGLMB can achieve robust SRUS imaging at high concentrations and low acquisition frame rates.

Super-Resolution Ultrasound Imaging for Analysis of Microbubbles Cluster by Acoustic Vortex Tweezers

Using acoustic vortex tweezers (AVTs) to spatially accumulate microbubbles (MBs) shows promise for enhancing drug delivery efficiency and reducing off-target effects. The strong echogenicity of accumulated MBs also improves diagnostics via conventional ultrasound (US) B-mode imaging. However, the annular high-pressure distribution of AVT inhibits MBs inflow at the inlet, reducing MBs collection. The spatial resolution of US B-mode imaging further limits theranostic applications of AVT-mediated MBs accumulation. To address these challenges, we integrated an AVT waveform with volumetric super-resolution imaging (VSRI) to monitor the dynamic growth of MBs cluster during accumulation. We used a 5-MHz 2-D array transducer for VSRI, employing plane wave pulses interleaved with accumulating pulses to retain MBs at a flow rate of 0.023-0.047 mL/s in a 3-mm vessel phantom. An asymmetrical AVT waveform (AVT ) was produced by modulating the pressure at the MBs inlet compared to the outlet. The effectiveness was validated in rat cerebral vessels for real-time volumetric tracking of MBs clusters. Microscopy observations showed that AVT could quickly gather flowing MBs into cluster without repelling them at a flow rate of 0.023 mL/s. Statistical results indicated that microscopic data correlated better with VSRI than with B-mode images, suggesting VSRI suffices to detect the dynamics of AVT -actuated MBs accumulation in real-time. Additionally, VSRI detected a significant increase in MBs cluster size over time during AVT in the superior sagittal sinus (SSS) of the rat brain. These findings demonstrate that the proposed strategy can accumulate the flowing MBs at a desired location and simultaneously observe this phenomenon.

ULM-MbCNRT: In Vivo Ultrafast Ultrasound Localization Microscopy by Combining Multibranch CNN and Recursive Transformer

Ultrasound localization microscopy (ULM) overcomes the acoustic diffraction limit by localizing tiny microbubbles (MBs), thus enabling the microvascular to be rendered at subwavelength resolution. Nevertheless, to obtain such superior spatial resolution, it is necessary to spend tens of seconds gathering numerous ultrasound (US) frames to accumulate the MB events required, resulting in ULM imaging still suffering from tradeoffs between imaging quality, data acquisition time, and data processing speed. In this article, we present a new deep learning (DL) framework combining multibranch convolutional neural network (CNN) and recursive transformer (RT), termed ULM-MbCNRT, that is capable of reconstructing a super-resolution (SR) image directly from a temporal mean low-resolution image generated by averaging much fewer raw US frames, i.e., implement an ultrafast ULM imaging. To evaluate the performance of ULM-MbCNRT, a series of numerical simulations and in vivo experiments are carried out. Numerical simulation results indicate that ULM-MbCNRT achieves high-quality ULM imaging with ~10-fold reduction in data acquisition time and ~130-fold reduction in computation time compared to the previous DL method (e.g., the modified subpixel CNN, ULM-mSPCN). For the in vivo experiments, when comparing to the ULM-mSPCN, ULM-MbCNRT allows ~37-fold reduction in data acquisition time (~0.8 s) and ~2134-fold reduction in computation time (~0.87 s) without sacrificing spatial resolution. It implies that ultrafast ULM imaging holds promise for observing rapid biological activity in vivo, potentially improving the diagnosis and monitoring of clinical conditions.

Deep Learning in Ultrasound Localization Microscopy: Applications and Perspectives

Ultrasound localization microscopy (ULM) is a novel super-resolution imaging technique that can image the vasculature in vivo at depth with resolution far beyond the conventional limit of diffraction. By relying on the localization and tracking of clinically approved microbubbles injected in the blood stream, ULM can provide not only anatomical visualization but also hemodynamic quantification of the microvasculature. Several deep learning approaches have been proposed to address challenges in ULM including denoising, improving microbubble localization, estimating blood flow velocity, or performing aberration correction. Proposed deep learning methods often outperform their conventional counterparts by improving image quality and reducing processing time. In addition, their robustness to high concentrations of microbubbles can lead to reduced acquisition times in ULM, addressing a major hindrance to ULM clinical application. Herein, we propose a comprehensive review of the diversity of deep learning applications in ULM focusing on approaches assuming a sparse microbubble distribution. We first provide an overview of how existing studies vary in the constitution of their datasets or in the tasks targeted by the deep learning model. We also take a deeper look into the numerous approaches that have been proposed to improve the localization of microbubbles since they differ highly in their formulation of the optimization problem, their evaluation, or their network architectures. We finally discuss the current limitations and challenges of these methods, as well as the promises and potential of deep learning for ULM in the future.

Transcutaneous Imaging of Rabbit Kidney Using 3-D Acoustic Wave Sparsely Activated Localization Microscopy With a Row-Column-Addressed Array

OBJECTIVE

Super-resolution ultrasound (SRUS) imaging through localizing and tracking microbubbles, also known as ultrasound localization microscopy (ULM), can produce sub-diffraction resolution images of micro-vessels. We have recently demonstrated 3-D selective SRUS with a matrix array and phase change contrast agents (PCCAs). However, this method is limited to a small field of view (FOV) and by the complex hardware required.

METHOD

This study proposed 3-D acoustic wave sparsely activated localization microscopy (AWSALM) using PCCAs and a 128+128 row-column-addressed (RCA) array, which offers ultrafast acquisition with over 6 times larger FOV and 4 times reduction in hardware complexity than a 1024-element matrix array. We first validated this method on an in-vitro microflow phantom and subsequently demonstrated non-invasively on a rabbit kidney in-vivo.

RESULTS

Our results show that 3-D AWSALM images of the phantom covering a mm volume can be generated under 5 seconds with an 8 times resolution improvement over the system point spread function. The full volume of the rabbit kidney can be covered to generate 3-D microvascular structure, flow speed and direction super-resolution maps under 15 seconds, combining the large FOV of RCA with the high resolution of SRUS. Additionally, 3-D AWSALM is selective and can visualize the microvasculature within the activation volume and downstream vessels in isolation. Sub-sets of the kidney microvasculature can be imaged through selective activation of PCCAs.

CONCLUSION

Our study demonstrates large FOV 3-D AWSALM using an RCA probe.

SIGNIFICANCE

3-D AWSALM offers an unique in-vivo imaging tool for fast, selective and large FOV vascular flow mapping.

Deep Power-Aware Tunable Weighting for Ultrasound Microvascular Imaging

Ultrasound microvascular imaging (UMI), including ultrafast power Doppler imaging (uPDI) and ultrasound localization microscopy (ULM), obtains blood flow information through plane wave (PW) transmissions at high frame rates. However, low signal-to-noise ratio (SNR) of PWs causes low image quality. Adaptive beamformers have been proposed to suppress noise energy to achieve higher image quality accompanied by increasing computational complexity. Deep learning (DL) leverages powerful hardware capabilities to enable rapid implementation of noise suppression at the cost of flexibility. To enhance the applicability of DL-based methods, in this work, we propose a deep power-aware tunable (DPT) weighting (i.e., postfilter) for delay-and-sum (DAS) beamforming to improve UMI by enhancing PW images. The model, called Yformer, is a hybrid structure combining convolution and Transformer. With the DAS beamformed and compounded envelope image as input, Yformer can estimate both noise power and signal power. Furthermore, we utilize the obtained powers to compute pixel-wise weights by introducing a tunable noise control factor (NCF), which is tailored for improving the quality of different UMI applications. In vivo experiments on the rat brain demonstrate that Yformer can accurately estimate the powers of noise and signal with the structural similarity index measure (SSIM) higher than 0.95. The performance of the DPT weighting is comparable to that of superior adaptive beamformer in uPDI with low computational cost. The DPT weighting was then applied to four different datasets of ULM, including public simulation, public rat brain, private rat brain, and private rat liver datasets, showing excellent generalizability using the model trained by the private rat brain dataset only. In particular, our method indirectly improves the resolution of liver ULM from 25.24 to m by highlighting small vessels. In addition, the DPT weighting exhibits more details of blood vessels with faster processing, which has the potential to facilitate the clinical applications of high-quality UMI.

Influence of Image Discretization and Patch Size on Microbubble Localization Precision

For ultrasound localization microscopy, the localization of microbubbles (MBs) is an essential part to obtain super-resolved maps of the vasculature. This article analyzes the impact of image discretization and patch size on the precision of different MB localization methods to reconcile different observations from previous studies, provide an estimate of feasible localization precision, and derive guidelines for an optimal parameter selection. For this purpose, the images of MBs were simulated with Gaussian point-spread functions (PSFs) of varying width parameter at randomly generated subpixel positions, and Rician distributed noise was added. Four localization methods were tested on the patches of different sizes (number of pixels ): Gaussian fit (GF), radial symmetry (RS) method, calculation of center of mass (CoM), and peak detection (PD). Additionally, the Cramér-Rao lower bound (CRLB) for the given estimation problem was calculated. Our results show that the localization precision is strongly influenced by the ratio of the PSF width parameter to the pixel size , as well as the patch size N. The best parameter combination depends on the localization method. Generally, very small ratios as well as large ratios in combination with small N lead to performance degradation. The GF as a representative of a model-based fit comes close to the CRLB and always performs best for the ratios given by image discretization if N is adapted to the PSF. To achieve good results with the GF and the RS method, a good rule of thumb is to set the pixel sizes and the patch sizes .

VoxelMorph-Based Deep Learning Motion Correction for Ultrasound Localization Microscopy of Spinal Cord

Accurate assessment of spinal cord vasculature is important for the urgent diagnosis of injury and subsequent treatment. Ultrasound localization microscopy (ULM) offers super-resolution imaging of microvasculature by localizing and tracking individual microbubbles (MBs) across multiple frames. However, a long data acquisition often involves significant motion artifacts caused by breathing and heartbeat, which further impairs the resolution of ULM. This effect is particularly pronounced in spinal cord imaging due to respiratory movement. We propose a VoxelMorph-based deep learning (DL) motion correction method to enhance the ULM performance in spinal cord imaging. Simulations were conducted to demonstrate the motion estimation accuracy of the proposed method, achieving a mean absolute error of m. Results from in vivo experiments show that the proposed method efficiently compensates for rigid and nonrigid motion, providing improved resolution with smaller vascular diameters and enhanced microvessel reconstruction after motion correction. Nonrigid deformation fields with varying displacement magnitudes were applied to in vivo data for assessing the robustness of the algorithm.

Ultrasound Localization Microscopy Precision of Clinical 3-D Ultrasound Systems

Ultrasound localization microscopy (ULM) is becoming well established in preclinical applications. For its translation into clinical practice, the localization precision achievable with commercial ultrasound (US) scanners is crucial-especially with volume imaging, which is essential for dealing with out-of-plane motion. Here, we propose an easy-to-perform method to estimate the localization precision of 3-D US scanners. With this method, we evaluated imaging sequences of the Philips Epiq 7 US device using the X5-1 and the XL14-3 matrix transducers and also tested different localization methods. For the X5-1 transducer, the best lateral, elevational, and axial precision was 109, 95, and m for one contrast mode, and 29, 22, and m for the other. The higher frequency XL14-3 transducer yielded precisions of 17, 38, and m using the harmonic imaging mode. Although the center of mass was the most robust localization method also often providing the best precision, the localization method has only a minor influence on the localization precision compared to the impact by the imaging sequence and transducer. The results show that with one of the imaging modes of the X5-1 transducer, precisions comparable to the XL14-3 transducer can be achieved. However, due to localization precisions worse than m, reconstruction of the microvasculature at the capillary level will not be possible. These results show the importance of evaluating the localization precision of imaging sequences from different US transducers or scanners in all directions before using them for in vivo measurements.

Super-resolution ultrasound and microvasculomics: a consensus statement

This is a summary of a consensus statement on the introduction of "Ultrasound microvasculomics" produced by The Chinese Artificial Intelligence Alliance for Thyroid and Breast Ultrasound. The evaluation of microvessels is a very important part for the assessment of diseases. Super-resolution ultrasound (SRUS) microvascular imaging surpasses traditional ultrasound imaging in the morphological and functional analysis of microcirculation. SRUS microvascular imaging relies on contrast microbubbles to gain sensitivity to microvessels and improves the spatial resolution of ultrasound blood flow imaging for a more detailed depiction of vascular structures and hemodynamics. This method has been applied in preclinical animal models and pilot clinical studies, involving areas including neurology, oncology, nephrology, and cardiology. However, the current quantitative parameters of SRUS images are not enough for precise evaluation of microvessels. Therefore, by employing omics methods, more quantification indicators can be obtained, enabling a more precise and personalized assessment of microvascular status. Ultrasound microvasculomics - a high-throughput extraction of image features from SRUS images - is one novel approach that holds great promise but needs further validation in both bench and clinical settings. CLINICAL RELEVANCE STATEMENT: Super-resolution Ultrasound (SRUS) blood flow imaging improves spatial resolution. Ultrasound microvasculomics is possible to acquire high-throughput information of features from SRUS images. It provides more precise and abundant micro-blood flow information in clinical medicine. KEY POINTS: This consensus statement reviews the development and application of super-resolution ultrasound (SRUS). The shortcomings of the current quantification indicators of SRUS and strengths of the omics methodology are addressed. "Ultrasound microvasculomics" is introduced for a high-throughput extraction of image features from SRUS images.

Super-resolution ultrasound imaging reveals temporal cerebrovascular changes with disease progression in female 5×FAD mouse model of Alzheimer's disease: correlation with pathological impairments

BACKGROUND

Vascular dysfunction is closely associated with the progression of Alzheimer's disease (AD). A critical research gap exists that no studies have explored the in vivo temporal changes of cerebrovascular alterations with AD progression in mouse models, encompassing both structure and flow dynamics at micron-scale resolution across the early, middle, and late stages of the disease.

METHODS

In this study, ultrasound localisation microscopy (ULM) was applied to image the cerebrovascular alterations of the transgenic female 5×FAD mouse model across different stages of disease progression: early (4 months), moderate (7 months), and late (12 months). Age-matched non-transgenic (non-Tg) littermates were used as controls. Immunohistology examinations were performed to evaluate the influence of disease progression on the β-amyloid (Aβ) load and microvascular alterations, including morphological changes and the blood-brain barrier (BBB) leakage.

FINDINGS

Our findings revealed a significant decline in both vascular density and flow velocity in the retrosplenial cortex of 5×FAD mice at an early stage, which subsequently became more pronounced in the visual cortex and hippocampus as the disease progressed. Additionally, we observed a reduction in vascular length preceding diminished flow velocities in cortical penetrating arterioles during the early stages. The quantification of vascular metrics derived from ULM imaging showed significant correlations with those obtained from vascular histological images. Immunofluorescence staining identified early vascular abnormalities in the retrosplenial cortex. As the disease advanced, there was an exacerbation of Aβ accumulation and BBB disruption in a regionally variable manner. The vascular changes observed through ULM imaging exhibited a negative correlation with amyloid load and were associated with the compromise of the BBB integrity.

INTERPRETATION

Through high-resolution, in vivo imaging of cerebrovasculature, this study reveals significant spatiotemporal dysfunction in cerebrovascular dynamics accompanying disease progression in a mouse model of AD, enhancing our understanding of its pathophysiology.

FUNDING

This study is supported by grants from National Key Research and Development Program of China (2020YFA0908800), National Natural Science Foundation of China (12074269, 82272014, 82327804, 62071310), Shenzhen Basic Science Research (20220808185138001, JCYJ20220818095612027, JCYJ20210324093006017), STI 2030-Major Projects (2021ZD0200500) and Guangdong Natural Science Foundation (2024A1515012591, 2024A1515011342).

Capillary-scale Microvessel Imaging with High-frequency Ultrasound Localization Microscopy in Mouse Brain

Ultrasound localization microscopy is a super-resolution vascular imaging technique which has garnered substantial interest as a tool for small animal neuroimaging, neuroscience research, and the characterization of vascular pathologies. In the pursuit of increasingly high-fidelity reconstructions of microvasculature, there remains several outstanding questions concerning this sub-diffraction imaging technology, including the accurate reconstruction of microvessels approaching the capillary scale and the pragmatic challenges associated with long data acquisition times. In the context of small animal neurovascular imaging, we posit that increasing the ultrasound imaging frequency is a straightforward approach to enable higher concentrations of microbubble contrast agents, thus increasing the likelihood of microvascular/capillary mapping and decreasing the imaging duration. We demonstrate that higher frequency imaging results in improved ULM fidelity and more efficient microbubble localization due to a smaller microbubble point-spread function that is easier to localize, and which can achieve a higher localizable concentration within the same unit volume of tissue. A select example of in vivo capillary-level vascular reconstruction is demonstrated for the highest frequency imaging probe, which has substantial implications for neuroscientists investigating microvascular function in disease states, regulation, and brain development. High frequency ULM yielding a spatial resolution of 7.1μm, as measured by Fourier ring correlation, throughout the entire depth of the brain, highlighting this technology as a highly relevant tool for neuroimaging research.

RF-ULM: Ultrasound Localization Microscopy Learned From Radio-Frequency Wavefronts

In Ultrasound Localization Microscopy (ULM), achieving high-resolution images relies on the precise localization of contrast agent particles across a series of beamformed frames. However, our study uncovers an enormous potential: The process of delay-and-sum beamforming leads to an irreversible reduction of Radio-Frequency (RF) channel data, while its implications for localization remain largely unexplored. The rich contextual information embedded within RF wavefronts, including their hyperbolic shape and phase, offers great promise for guiding Deep Neural Networks (DNNs) in challenging localization scenarios. To fully exploit this data, we propose to directly localize scatterers in RF channel data. Our approach involves a custom super-resolution DNN using learned feature channel shuffling, non-maximum suppression, and a semi-global convolutional block for reliable and accurate wavefront localization. Additionally, we introduce a geometric point transformation that facilitates seamless mapping to the B-mode coordinate space. To understand the impact of beamforming on ULM, we validate the effectiveness of our method by conducting an extensive comparison with State-Of-The-Art (SOTA) techniques. We present the inaugural in vivo results from a wavefront-localizing DNN, highlighting its real-world practicality. Our findings show that RF-ULM bridges the domain shift between synthetic and real datasets, offering a considerable advantage in terms of precision and complexity. To enable the broader research community to benefit from our findings, our code and the associated SOTA methods are made available at https://github.com/hahnec/rf-ulm.

In vivo ultrasound localization microscopy for high-density microbubbles

Ultrasound Localization Microscopy (ULM) surpasses the constraints imposed by acoustic diffraction, achieving sub-wavelength resolution visualization of microvasculature through the precise localization of minute microbubbles (MBs). Nonetheless, the analysis of densely populated regions with overlapping MB point spread responses introduces significant localization errors, limiting the use of technique to low-concentration conditions. This raises a trade-off issue between localization efficiency and MB density. In this work, we present a new deep learning framework that combines Transformer and U-Net architectures, termed ULM-TransUNet. As a non-linear model, it is able to learn the complex data patterns of overlapping MBs in dense conditions for accurate localization. To evaluate the performance of ULM-TransUNet, a series of numerical simulations and in vivo experiments are carried out. Numerical simulation results indicate that ULM-TransUNet achieves high-quality ULM imaging, with improvements of 21.93 % in detection rate, 17.36 % in detection precision, and 20.53 % in detection sensitivity, compared to previous state-of-the-art deep learning (DL) method (e.g., ULM-UNet). For the in vivo experiments, ULM-TransUNet achieves the highest spatial resolution (9.4 μm) and rapid inference speed (26.04 ms/frame). Furthermore, it consistently detects more small vessels and resolves closely spaced vessels more effectively. The outcomes of this work imply that ULM-TransUNet can potentially enhance the microvascular imaging performance on high-density MB conditions.

ULTRA-SR Challenge: Assessment of Ultrasound Localization and TRacking Algorithms for Super-Resolution Imaging

With the widespread interest and uptake of super-resolution ultrasound (SRUS) through localization and tracking of microbubbles, also known as ultrasound localization microscopy (ULM), many localization and tracking algorithms have been developed. ULM can image many centimeters into tissue in-vivo and track microvascular flow non-invasively with sub-diffraction resolution. In a significant community effort, we organized a challenge, Ultrasound Localization and TRacking Algorithms for Super-Resolution (ULTRA-SR). The aims of this paper are threefold: to describe the challenge organization, data generation, and winning algorithms; to present the metrics and methods for evaluating challenge entrants; and to report results and findings of the evaluation. Realistic ultrasound datasets containing microvascular flow for different clinical ultrasound frequencies were simulated, using vascular flow physics, acoustic field simulation and nonlinear bubble dynamics simulation. Based on these datasets, 38 submissions from 24 research groups were evaluated against ground truth using an evaluation framework with six metrics, three for localization and three for tracking. In-vivo mouse brain and human lymph node data were also provided, and performance assessed by an expert panel. Winning algorithms are described and discussed. The publicly available data with ground truth and the defined metrics for both localization and tracking present a valuable resource for researchers to benchmark algorithms and software, identify optimized methods/software for their data, and provide insight into the current limits of the field. In conclusion, Ultra-SR challenge has provided benchmarking data and tools as well as direct comparison and insights for a number of the state-of-the art localization and tracking algorithms.

An Automated Heart Shunt Recognition Pipeline Using Deep Neural Networks

Automated recognition of heart shunts using saline contrast transthoracic echocardiography (SC-TTE) has the potential to transform clinical practice, enabling non-experts to assess heart shunt lesions. This study aims to develop a fully automated and scalable analysis pipeline for distinguishing heart shunts, utilizing a deep neural network-based framework. The pipeline consists of three steps: (1) chamber segmentation, (2) ultrasound microbubble localization, and (3) disease classification model establishment. The study's normal control group included 91 patients with intracardiac shunts, 61 patients with extracardiac shunts, and 84 asymptomatic individuals. Participants' SC-TTE images were segmented using the U-Net model to obtain cardiac chambers. The segmentation results were combined with ultrasound microbubble localization to generate multivariate time series data on microbubble counts in each chamber. A classification model was then trained using this data to distinguish between intracardiac and extracardiac shunts. The proposed framework accurately segmented heart chambers (dice coefficient = 0.92 ± 0.1) and localized microbubbles. The disease classification model achieved high accuracy, sensitivity, specificity, F1 score, kappa value, and AUC value for both intracardiac and extracardiac shunts. For intracardiac shunts, accuracy was 0.875 ± 0.008, sensitivity was 0.891 ± 0.002, specificity was 0.865 ± 0.012, F1 score was 0.836 ± 0.011, kappa value was 0.735 ± 0.017, and AUC value was 0.942 ± 0.014. For extracardiac shunts, accuracy was 0.902 ± 0.007, sensitivity was 0.763 ± 0.014, specificity was 0.966 ± 0.008, F1 score was 0.830 ± 0.012, kappa value was 0.762 ± 0.017, and AUC value was 0.916 ± 0.006. The proposed framework utilizing deep neural networks offers a fast, convenient, and accurate method for identifying intracardiac and extracardiac shunts. It aids in shunt recognition and generates valuable quantitative indices, assisting clinicians in diagnosing these conditions.

Inspiring a convergent engineering approach to measure and model the tissue microenvironment

Understanding the molecular and physical complexity of the tissue microenvironment (TiME) in the context of its spatiotemporal organization has remained an enduring challenge. Recent advances in engineering and data science are now promising the ability to study the structure, functions, and dynamics of the TiME in unprecedented detail; however, many advances still occur in silos that rarely integrate information to study the TiME in its full detail. This review provides an integrative overview of the engineering principles underlying chemical, optical, electrical, mechanical, and computational science to probe, sense, model, and fabricate the TiME. In individual sections, we first summarize the underlying principles, capabilities, and scope of emerging technologies, the breakthrough discoveries enabled by each technology and recent, promising innovations. We provide perspectives on the potential of these advances in answering critical questions about the TiME and its role in various disease and developmental processes. Finally, we present an integrative view that appreciates the major scientific and educational aspects in the study of the TiME.

Microbubble detection on ultrasound imaging by utilizing phase patterned waves

Objective.Super-resolution ultrasonography offers the advantage of visualization of intricate microvasculature, which is crucial for disease diagnosis. Mapping of microvessels is possible by localizing microbubbles (MBs) that act as contrast agents and tracking their location. However, there are limitations such as the low detectability of MBs and the utilization of a diluted concentration of MBs, leading to the extension of the acquisition time. We aim to enhance the detectability of MBs to reduce the acquisition time of acoustic data necessary for mapping the microvessels.Approach.We propose utilizing phase patterned waves (PPWs) characterized by spatially patterned phase distributions in the incident beam to achieve this. In contrast to conventional ultrasound irradiation methods, this irradiation method alters bubble interactions, enhancing the oscillation response of MBs and generating more significant scattered waves from specific MBs. This enhances the detectability of MBs, thereby enabling the detection of MBs that were undetectable by the conventional method. The objective is to maximize the overall detection of bubbles by utilizing ultrasound imaging with additional PPWs, including the conventional method. In this paper, we apply PPWs to ultrasound imaging simulations considering bubble-bubble interactions to elucidate the characteristics of PPWs and demonstrate their efficacy by employing PPWs on MBs fixed in a phantom by the experiment.Main results.By utilizing two types of PPWs in addition to the conventional ultrasound irradiation method, we confirmed the detection of up to 93.3% more MBs compared to those detected using the conventional method alone.Significance.Ultrasound imaging using additional PPWs made it possible to increase the number of detected MBs, which is expected to improve the efficiency of bubble detection.

Super-Resolution Ultrasound Imaging for Monitoring the Therapeutic Efficacy of a Vascular Disrupting Agent in an Animal Model of Breast Cancer

OBJECTIVE

Evaluate the use of super-resolution ultrasound (SRUS) imaging for the early detection of tumor response to treatment using a vascular-disrupting agent (VDA).

METHODS

A population of 28 female nude athymic mice (Charles River Laboratories) were implanted with human breast cancer cells (MDA-MB-231, ATCC) in the mammary fat pad and allowed to grow. Ultrasound imaging was performed using a Vevo 3100 scanner (FUJIFILM VisualSonics Inc) equipped with the MX250 linear array transducer immediately before and after receiving bolus injections of a microbubble (MB) contrast agent (Definity, Lantheus Medical Imaging) via the tail vein. Following baseline ultrasound imaging, VDA drug (combretastatin A4 phosphate, CA4P, Sigma Aldrich) or control saline was injected via the placed catheter. After 4 or 24 hours, repeat ultrasound imaging along the same tumor cross-section occurred. Direct intratumoral pressure measurements were obtained using a calibrated sensor. All raw ultrasound data were saved for offline processing and SRUS image reconstruction using custom MATLAB software (MathWorks Inc). From a region encompassing the tumor space and the entire postprocessed ultrasound image sequence, time MB count (TMC) curves were generated in addition to traditional SRUS maps reflecting MB enumeration at each pixel location. Peak enhancement (PE) and wash-in rate (WIR) were extracted from these TMC curves. At termination, intratumoral microvessel density (MVD) was quantified using tomato lectin labeling of patent blood vessels.

RESULTS

SRUS images exhibited a clear difference between control and treated tumors. While there was no difference in any group parameters at baseline (0 hour, P > .09), both SRUS-derived PE and WIR measurements in tumors treated with VDA exhibited significant decreases by 4 (P = .03 and P = .05, respectively) and 24 hours (P = .02 and P = .01, respectively), but not in control group tumors (P > .22). Similarly, SRUS derived microvascular maps were not different at baseline (P = .81), but measures of vessel density were lower in treated tumors at both 4 and 24 hours (P < .04). An inverse relationship between intratumoral pressure and both PE and WIR parameters were found in control tumors (R2 > .09, P < .03).

CONCLUSION

SRUS imaging is a new modality for assessing tumor response to treatment using a VDA.

A Residual Dense Attention Generative Adversarial Network for Microscopic Image Super-Resolution

With the development of deep learning, the Super-Resolution (SR) reconstruction of microscopic images has improved significantly. However, the scarcity of microscopic images for training, the underutilization of hierarchical features in original Low-Resolution (LR) images, and the high-frequency noise unrelated with the image structure generated during the reconstruction process are still challenges in the Single Image Super-Resolution (SISR) field. Faced with these issues, we first collected sufficient microscopic images through Motic, a company engaged in the design and production of optical and digital microscopes, to establish a dataset. Secondly, we proposed a Residual Dense Attention Generative Adversarial Network (RDAGAN). The network comprises a generator, an image discriminator, and a feature discriminator. The generator includes a Residual Dense Block (RDB) and a Convolutional Block Attention Module (CBAM), focusing on extracting the hierarchical features of the original LR image. Simultaneously, the added feature discriminator enables the network to generate high-frequency features pertinent to the image's structure. Finally, we conducted experimental analysis and compared our model with six classic models. Compared with the best model, our model improved PSNR and SSIM by about 1.5 dB and 0.2, respectively.

Understanding the effects of microbubble concentration on localization accuracy in super-resolution ultrasound imaging

Super-resolution ultrasound (SRUS) through localising and tracking of microbubbles (MBs) can achieve sub-wavelength resolution for imaging microvascular structure and flow dynamics in deep tissuein vivo. The technique assumes that signals from individual MBs can be isolated and localised accurately, but this assumption starts to break down when the MB concentration increases and the signals from neighbouring MBs start to interfere. The aim of this study is to gain understanding of the effect of MB-MB distance on ultrasound images and their localisation. Ultrasound images of two MBs approaching each other were synthesised by simulating both ultrasound field propagation and nonlinear MB dynamics. Besides the distance between MBs, a range of other influencing factors including MB size, ultrasound frequency, transmit pulse sequence, pulse amplitude and localisation methods were studied. The results show that as two MBs approach each other, the interference fringes can lead to significant and oscillating localisation errors, which are affected by both the MB and imaging parameters. When modelling a clinical linear array probe operating at 6 MHz, localisation errors between 20 and 30μm (∼1/10 wavelength) can be generated when MBs are ∼500μm (2 wavelengths or ∼1.7 times the point spread function (PSF)) away from each other. When modelling a cardiac probe operating at 1.5 MHz, the localisation errors were as high as 200μm (∼1/5 wavelength) even when the MBs were more than 10 wavelengths apart (2.9 times the PSF). For both frequencies, at smaller separation distances, the two MBs were misinterpreted as one MB located in between the two true positions. Cross-correlation or Gaussian fitting methods were found to generate slightly smaller localisation errors than centroiding. In conclusion, caution should be taken when generating and interpreting SRUS images obtained using high agent concentration with MBs separated by less than 1.7 to 3 times the PSF, as significant localisation errors can be generated due to interference between neighbouring MBs.

Time Efficient Ultrasound Localization Microscopy Based on A Novel Radial Basis Function 2D Interpolation

Ultrasound localization microscopy (ULM) allows for the generation of super-resolved (SR) images of the vasculature by precisely localizing intravenously injected microbubbles. Although SR images may be useful for diagnosing and treating patients, their use in the clinical context is limited by the need for prolonged acquisition times and high frame rates. The primary goal of our study is to relax the requirement of high frame rates to obtain SR images. To this end, we propose a new time-efficient ULM (TEULM) pipeline built on a cutting-edge interpolation method. More specifically, we suggest employing Radial Basis Functions (RBFs) as interpolators to estimate the missing values in the 2-dimensional (2D) spatio-temporal structures. To evaluate this strategy, we first mimic the data acquisition at a reduced frame rate by applying a down-sampling (DS = 2, 4, 8, and 10) factor to high frame rate ULM data. Then, we up-sample the data to the original frame rate using the suggested interpolation to reconstruct the missing frames. Finally, using both the original high frame rate data and the interpolated one, we reconstruct SR images using the ULM framework steps. We evaluate the proposed TEULM using four in vivo datasets, a Rat brain (dataset A), a Rat kidney (dataset B), a Rat tumor (dataset C) and a Rat brain bolus (dataset D), interpolating at the in-phase and quadrature (IQ) level. Results demonstrate the effectiveness of TEULM in recovering vascular structures, even at a DS rate of 10 (corresponding to a frame rate of sub-100Hz). In conclusion, the proposed technique is successful in reconstructing accurate SR images while requiring frame rates of one order of magnitude lower than standard ULM.

Super-resolution techniques for biomedical applications and challenges

Super-resolution (SR) techniques have revolutionized the field of biomedical applications by detailing the structures at resolutions beyond the limits of imaging or measuring tools. These techniques have been applied in various biomedical applications, including microscopy, magnetic resonance imaging (MRI), computed tomography (CT), X-ray, electroencephalogram (EEG), ultrasound, etc. SR methods are categorized into two main types: traditional non-learning-based methods and modern learning-based approaches. In both applications, SR methodologies have been effectively utilized on biomedical images, enhancing the visualization of complex biological structures. Additionally, these methods have been employed on biomedical data, leading to improvements in computational precision and efficiency for biomedical simulations. The use of SR techniques has resulted in more detailed and accurate analyses in diagnostics and research, essential for early disease detection and treatment planning. However, challenges such as computational demands, data interpretation complexities, and the lack of unified high-quality data persist. The article emphasizes these issues, underscoring the need for ongoing development in SR technologies to further improve biomedical research and patient care outcomes.

Context-aware deep learning enables high-efficacy localization of high concentration microbubbles for super-resolution ultrasound localization microscopy

Ultrasound localization microscopy (ULM) enables deep tissue microvascular imaging by localizing and tracking intravenously injected microbubbles circulating in the bloodstream. However, conventional localization techniques require spatially isolated microbubbles, resulting in prolonged imaging time to obtain detailed microvascular maps. Here, we introduce LOcalization with Context Awareness (LOCA)-ULM, a deep learning-based microbubble simulation and localization pipeline designed to enhance localization performance in high microbubble concentrations. In silico, LOCA-ULM enhanced microbubble detection accuracy to 97.8% and reduced the missing rate to 23.8%, outperforming conventional and deep learning-based localization methods up to 17.4% in accuracy and 37.6% in missing rate reduction. In in vivo rat brain imaging, LOCA-ULM revealed dense cerebrovascular networks and spatially adjacent microvessels undetected by conventional ULM. We further demonstrate the superior localization performance of LOCA-ULM in functional ULM (fULM) where LOCA-ULM significantly increased the functional imaging sensitivity of fULM to hemodynamic responses invoked by whisker stimulations in the rat brain.

Improving sub-pixel accuracy in ultrasound localization microscopy using supervised and self-supervised deep learning

With a spatial resolution of tens of microns, ultrasound localization microscopy (ULM) reconstructs microvascular structures and measures intravascular flows by tracking microbubbles (1-5 μm) in contrast enhanced ultrasound (CEUS) images. Since the size of CEUS bubble traces, e.g. 0.5-1 mm for ultrasound with a wavelength λ = 280 μm, is typically two orders of magnitude larger than the bubble diameter, accurately localizing microbubbles in noisy CEUS data is vital to the fidelity of the ULM results. In this paper, we introduce a residual learning based supervised super-resolution blind deconvolution network (SupBD-net), and a new loss function for a self-supervised blind deconvolution network (SelfBD-net), for detecting bubble centers at a spatial resolution finer than λ/10. Our ultimate purpose is to improve the ability to distinguish closely located microvessels and the accuracy of the velocity profile measurements in macrovessels. Using realistic synthetic data, the performance of these methods is calibrated and compared against several recently introduced deep learning and blind deconvolution techniques. For bubble detection, errors in bubble center location increase with the trace size, noise level, and bubble concentration. For all cases, SupBD-net yields the least error, keeping it below 0.1 λ. For unknown bubble trace morphology, where all the supervised learning methods fail, SelfBD-net can still maintain an error of less than 0.15 λ. SupBD-net also outperforms the other methods in separating closely located bubbles and parallel microvessels. In macrovessels, SupBD-net maintains the least errors in the vessel radius and velocity profile after introducing a procedure that corrects for terminated tracks caused by overlapping traces. Application of these methods is demonstrated by mapping the cerebral microvasculature of a neonatal pig, where neighboring microvessels separated by 0.15 λ can be readily distinguished by SupBD-net and SelfBD-net, but not by the other techniques. Hence, the newly proposed residual learning based methods improve the spatial resolution and accuracy of ULM in micro- and macro-vessels.

Competitive Swarm Optimized SVD Clutter Filtering for Ultrafast Power Doppler Imaging

Ultrafast power Doppler imaging (uPDI) can significantly increase the sensitivity of resolving small vascular paths in ultrasound. While clutter filtering is a fundamental and essential method to realize uPDI, it commonly uses singular value decomposition (SVD) to suppress clutter signals and noise. However, current SVD-based clutter filters using two cutoffs cannot ensure sufficient separation of tissue, blood, and noise in uPDI. This article proposes a new competitive swarm-optimized SVD clutter filter to improve the quality of uPDI. Specifically, without using two cutoffs, such a new filter introduces competitive swarm optimization (CSO) to search for the counterparts of blood signals in each singular value. We validate the CSO-SVD clutter filter on public in vivo datasets. The experimental results demonstrate that our method can achieve higher contrast-to-noise ratio (CNR), signal-to-noise ratio (SNR), and blood-to-clutter ratio (BCR) than the state-of-the-art SVD-based clutter filters, showing a better balance between suppressing clutter signals and preserving blood signals. Particularly, our CSO-SVD clutter filter improves CNR by 0.99 ± 0.08 dB, SNR by 0.79 ± 0.08 dB, and BCR by 1.95 ± 0.03 dB when comparing a spatial-similarity-based SVD clutter filter in the in vivo dataset of rat brain bolus.

Motor Interaction Control Based on Muscle Force Model and Depth Reinforcement Strategy

The current motion interaction model has the problems of insufficient motion fidelity and lack of self-adaptation to complex environments. To address this problem, this study proposed to construct a human motion control model based on the muscle force model and stage particle swarm, and based on this, this study utilized the deep deterministic gradient strategy algorithm to construct a motion interaction control model based on the muscle force model and the deep reinforcement strategy. Empirical analysis of the human motion control model proposed in this study revealed that the joint trajectory correlation and muscle activity correlation of the model were higher than those of other comparative models, and its joint trajectory correlation was up to 0.90, and its muscle activity correlation was up to 0.84. In addition, this study validated the effectiveness of the motion interaction control model using the depth reinforcement strategy and found that in the mixed-obstacle environment, the model's desired results were obtained by training 1.1 × 103 times, and the walking distance was 423 m, which was better than other models. In summary, the proposed motor interaction control model using the muscle force model and deep reinforcement strategy has higher motion fidelity and can realize autonomous decision making and adaptive control in the face of complex environments. It can provide a theoretical reference for improving the effect of motion control and realizing intelligent motion interaction.

Artificial intelligence-based speckle featurization and localization for ultrasound speckle tracking velocimetry

Deep learning-based super-resolution ultrasound (DL-SRU) framework has been successful in improving spatial resolution and measuring the velocity field information of a blood flows by localizing and tracking speckle signals of red blood cells (RBCs) without using any contrast agents. However, DL-SRU can localize only a small part of the speckle signals of blood flow owing to ambiguity problems encountered in the classification of blood flow signals from ultrasound B-mode images and the building up of suitable datasets required for training artificial neural networks, as well as the structural limitations of the neural network itself. An artificial intelligence-based speckle featurization and localization (AI-SFL) framework is proposed in this study. It includes a machine learning-based algorithm for classifying blood flow signals from ultrasound B-mode images, dimensionality reduction for featurizing speckle patterns of the classified blood flow signals by approximating them with quantitative values. A novel and robust neural network (ResSU-net) is trained using the online data generation (ODG) method and the extracted speckle features. The super-resolution performance of the proposed AI-SFL and ODG method is evaluated and compared with the results of previous U-net and conventional data augmentation methods under in silico conditions. The predicted locations of RBCs by the AI-SFL and DL-SRU for speckle patterns of blood flow are applied to a PTV algorithm to measure quantitative velocity fields of the flow. Finally, the feasibility of the proposed AI-SFL framework for measuring real blood flows is verified under in vivo conditions.

Deep learning for fast super-resolution ultrasound microvessel imaging

Objective. Ultrasound localization microscopy (ULM) enables microvascular reconstruction by localizing microbubbles (MBs). Although ULM can obtain microvascular images that are beyond the ultimate resolution of the ultrasound (US) diffraction limit, it requires long data processing time, and the imaging accuracy is susceptible to the density of MBs. Deep learning (DL)-based ULM is proposed to alleviate these limitations, which simulated MBs at low-resolution and mapped them to coordinates at high-resolution by centroid localization. However, traditional DL-based ULMs are imprecise and computationally complex. Also, the performance of DL is highly dependent on the training datasets, which are difficult to realistically simulate.Approach. A novel architecture called adaptive matching network (AM-Net) and a dataset generation method named multi-mapping (MMP) was proposed to overcome the above challenges. The imaging performance and processing time of the AM-Net have been assessed by simulation andin vivoexperiments.Main results. Simulation results show that at high density (20 MBs/frame), when compared to other DL-based ULM, AM-Net achieves higher localization accuracy in the lateral/axial direction.In vivoexperiment results show that the AM-Net can reconstruct ∼24.3μm diameter micro-vessels and separate two ∼28.3μm diameter micro-vessels. Furthermore, when processing a 128 × 128 pixels image in simulation experiments and an 896 × 1280 pixels imagein vivoexperiment, the processing time of AM-Net is ∼13 s and ∼33 s, respectively, which are 0.3-0.4 orders of magnitude faster than other DL-based ULM.Significance. We proposes a promising solution for ULM with low computing costs and high imaging performance.

Three-dimensional super resolution ultrasound imaging with a multi-frequency hemispherical phased array

BACKGROUND

High resolution imaging of the microvasculature plays an important role in both diagnostic and therapeutic applications in the brain. However, ultrasound pulse-echo sonography imaging the brain vasculatures has been limited to narrow acoustic windows and low frequencies due to the distortion of the skull bone, which sacrifices axial resolution since it is pulse length dependent.

PURPOSE

To overcome the detect limit, a large aperture 256-module sparse hemispherical transmit/receive array was used to visualize the acoustic emissions of ultrasound-vaporized lipid-coated decafluorobutane nanodroplets flowing through tube phantoms and within rabbit cerebral vasculature in vivo via passive acoustic mapping and super resolution techniques.

METHODS

Nanodroplets were vaporized with 55 kHz burst-mode ultrasound (burst length = 145 μs, burst repetition frequency = 9-45 Hz, peak negative acoustic pressure = 0.10-0.22 MPa), which propagates through overlying tissues well without suffering from severe distortions. The resulting emissions were received at a higher frequency (612 or 1224 kHz subarray) to improve the resulting spatial resolution during passive beamforming. Normal resolution three-dimensional images were formed using a delay, sum, and integrate beamforming algorithm, and super-resolved images were extracted via Gaussian fitting of the estimated point-spread-function to the normal resolution data.

RESULTS

With super resolution techniques, the mean lateral (axial) full-width-at-half-maximum image intensity was 16 ± 3 (32 ± 6) μm, and 7 ± 1 (15 ± 2) μm corresponding to ∼1/67 of the normal resolution at 612 and 1224 kHz, respectively. The mean positional uncertainties were ∼1/350 (lateral) and ∼1/180 (axial) of the receive wavelength in water. In addition, a temporal correlation between nanodroplet vaporization and the transmit waveform shape was observed, which may provide the opportunity to enhance the signal-to-noise ratio in future studies.

CONCLUSIONS

Here, we demonstrate the feasibility of vaporizing nanodroplets via low frequency ultrasound and simultaneously performing spatial mapping via passive beamforming at higher frequencies to improve the resulting spatial resolution of super resolution imaging techniques. This method may enable complete four-dimensional vascular mapping in organs where a hemispherical array could be positioned to surround the target, such as the brain, breast, or testicles.

Contrast-Free Super-Resolution Power Doppler (CS-PD) Based on Deep Neural Networks

Super-resolution ultrasound microvessel imaging based on ultrasound localization microscopy (ULM) is an emerging imaging modality that is capable of resolving micrometer-scaled vessels deep into tissue. In practice, ULM is limited by the need for contrast injection, long data acquisition, and computationally expensive postprocessing times. In this study, we present a contrast-free super-resolution power Doppler (CS-PD) technique that uses deep networks to achieve super-resolution with short data acquisition. The training dataset is comprised of spatiotemporal ultrafast ultrasound signals acquired from in vivo mouse brains, while the testing dataset includes in vivo mouse brain, chicken embryo chorioallantoic membrane (CAM), and healthy human subjects. The in vivo mouse imaging studies demonstrate that CS-PD could achieve an approximate twofold improvement in spatial resolution when compared with conventional power Doppler. In addition, the microvascular images generated by CS-PD showed good agreement with the corresponding ULM images as indicated by a structural similarity index of 0.7837 and a peak signal-to-noise ratio (PSNR) of 25.52. Moreover, CS-PD was able to preserve the temporal profile of the blood flow (e.g., pulsatility) that is similar to conventional power Doppler. Finally, the generalizability of CS-PD was demonstrated on testing data of different tissues using different imaging settings. The fast inference time of the proposed deep neural network also allows CS-PD to be implemented for real-time imaging. These features of CS-PD offer a practical, fast, and robust microvascular imaging solution for many preclinical and clinical applications of Doppler ultrasound.

Nanodroplet-Based Super-Resolution Ultrasound Localization Microscopy

Over the past decade, super-resolution ultrasound localization microscopy (SR-ULM) has revolutionized ultrasound imaging with its capability to resolve the microvascular structures below the ultrasound diffraction limit. The introduction of this imaging technique enables the visualization, quantification, and characterization of tissue microvasculature. The early implementations of SR-ULM utilize microbubbles (MBs) that require a long image acquisition time due to the requirement of capturing sparsely isolated microbubble signals. The next-generation SR-ULM employs nanodroplets that have the potential to significantly reduce the image acquisition time without sacrificing the resolution. This review discusses various nanodroplet-based ultrasound localization microscopy techniques and their corresponding imaging mechanisms. A summary is given on the preclinical applications of SR-ULM with nanodroplets, and the challenges in the clinical translation of nanodroplet-based SR-ULM are presented while discussing the future perspectives. In conclusion, ultrasound localization microscopy is a promising microvasculature imaging technology that can provide new diagnostic and prognostic information for a wide range of pathologies, such as cancer, heart conditions, and autoimmune diseases, and enable personalized treatment monitoring at a microlevel.

Ultrasound localization microscopy

Ultrasound Localization Microscopy (ULM) is an emerging technique that provides impressive super-resolved images of microvasculature, i.e., images with much better resolution than the conventional diffraction-limited ultrasound techniques and is already taking its first steps from preclinical to clinical applications. In comparison to the established perfusion or flow measurement methods, namely contrast-enhanced ultrasound (CEUS) and Doppler techniques, ULM allows imaging and flow measurements even down to the capillary level. As ULM can be realized as a post-processing method, conventional ultrasound systems can be used for. ULM relies on the localization of single microbubbles (MB) of commercial, clinically approved contrast agents. In general, these very small and strong scatterers with typical radii of 1-3 µm are imaged much larger in ultrasound images than they actually are due to the point spread function of the imaging system. However, by applying appropriate methods, these MBs can be localized with sub-pixel precision. Then, by tracking MBs over successive frames of image sequences, not only the morphology of vascular trees but also functional information such as flow velocities or directions can be obtained and visualized. In addition, quantitative parameters can be derived to describe pathological and physiological changes in the microvasculature. In this review, the general concept of ULM and conditions for its applicability to microvessel imaging are explained. Based on this, various aspects of the different processing steps for a concrete implementation are discussed. The trade-off between complete reconstruction of the microvasculature and the necessary measurement time as well as the implementation in 3D are reviewed in more detail, as they are the focus of current research. Through an overview of potential or already realized preclinical and clinical applications - pathologic angiogenesis or degeneration of vessels, physiological angiogenesis, or the general understanding of organ or tissue function - the great potential of ULM is demonstrated.

Localization Free Super-Resolution Microbubble Velocimetry Using a Long Short-Term Memory Neural Network

Ultrasound localization microscopy is a super-resolution imaging technique that exploits the unique characteristics of contrast microbubbles to side-step the fundamental trade-off between imaging resolution and penetration depth. However, the conventional reconstruction technique is confined to low microbubble concentrations to avoid localization and tracking errors. Several research groups have introduced sparsity- and deep learning-based approaches to overcome this constraint to extract useful vascular structural information from overlapping microbubble signals, but these solutions have not been demonstrated to produce blood flow velocity maps of the microcirculation. Here, we introduce Deep-SMV, a localization free super-resolution microbubble velocimetry technique, based on a long short-term memory neural network, that provides high imaging speed and robustness to high microbubble concentrations, and directly outputs blood velocity measurements at a super-resolution. Deep-SMV is trained efficiently using microbubble flow simulation on real in vivo vascular data and demonstrates real-time velocity map reconstruction suitable for functional vascular imaging and pulsatility mapping at super-resolution. The technique is successfully applied to a wide variety of imaging scenarios, include flow channel phantoms, chicken embryo chorioallantoic membranes, and mouse brain imaging. An implementation of Deep-SMV is openly available at https://github.com/chenxiptz/SR_microvessel_velocimetry, with two pre-trained models available at https://doi.org/10.7910/DVN/SECUFD.

Super-resolution ultrasound microvascular imaging: Is it ready for clinical use?

The field of super-resolution ultrasound microvascular imaging has been rapidly growing over the past decade. By leveraging contrast microbubbles as point targets for localization and tracking, super-resolution ultrasound pinpoints the location of microvessels and measures their blood flow velocity. Super-resolution ultrasound is the first in vivo imaging modality that can image micron-scale vessels at a clinically relevant imaging depth without tissue destruction. These unique capabilities of super-resolution ultrasound provide structural (vessel morphology) and functional (vessel blood flow) assessments of tissue microvasculature on a global and local scale, which opens new doors for many enticing preclinical and clinical applications that benefit from microvascular biomarkers. The goal of this short review is to provide an update on recent advancements in super-resolution ultrasound imaging, with a focus on summarizing existing applications and discussing the prospects of translating super-resolution imaging to clinical practice and research. In this review, we also provide brief introductions of how super-resolution ultrasound works, how does it compare with other imaging modalities, and what are the tradeoffs and limitations for an audience who is not familiar with the technology.

Backscattering amplitude in ultrasound localization microscopy

In the last decade, Ultrafast ultrasound localisation microscopy has taken non-invasive deep vascular imaging down to the microscopic level. By imaging diluted suspensions of circulating microbubbles in the blood stream at kHz frame rate and localizing the center of their individual point spread function with a sub-resolution precision, it enabled to break the unvanquished trade-off between depth of imaging and resolution by microscopically mapping the microbubbles flux and velocities deep into tissue. However, ULM also suffers limitations. Many small vessels are not visible in the ULM images due to the noise level in areas dimly explored by the microbubbles. Moreover, as the vast majority of studies are performed using 2D imaging, quantification is limited to in-plane velocity or flux measurements which hinders the accurate velocity determination and quantification. Here we show that the backscattering amplitude of each individual microbubble can also be exploited to produce backscattering images of the vascularization with a higher sensitivity compared to conventional ULM images. By providing valuable information about the relative distance of the microbubble to the 2D imaging plane in the out-of-plane direction, backscattering ULM images introduces a physically relevant 3D rendering perception in the vascular maps. It also retrieves the missing information about the out-of-plane motion of microbubbles and provides a way to improve 3D flow and velocity quantification using 2D ULM. These results pave the way to improved visualization and quantification for 2D and 3D ULM.

Ultrasound Localization Microscopy Using Deep Neural Network

Noninvasive imaging of microvascular structures in deep tissues provides morphological and functional information for clinical diagnosis and monitoring. Ultrasound localization microscopy (ULM) is an emerging imaging technique that can generate microvascular structures with subwavelength diffraction resolution. However, the clinical utility of ULM is hindered by technical limitations, such as long data acquisition time, high microbubble (MB) concentration, and inaccurate localization. In this article, we propose a Swin transformer-based neural network to perform end-to-end mapping to implement MB localization. The performance of the proposed method was validated using synthetic and in vivo data using different quantitative metrics. The results indicate that our proposed network can achieve higher precision and better imaging capability than previously used methods. Furthermore, the computational cost of processing per frame is 3-4 times faster than traditional methods, which makes the real-time application of this technique feasible in the future.

A Data-Efficient Deep Learning Strategy for Tissue Characterization via Quantitative Ultrasound: Zone Training

Deep learning (DL) powered biomedical ultrasound imaging is an emerging research field where researchers adapt the image analysis capabilities of DL algorithms to biomedical ultrasound imaging settings. A major roadblock to wider adoption of DL powered biomedical ultrasound imaging is that acquisition of large and diverse datasets is expensive in clinical settings, which is a requirement for successful DL implementation. Hence, there is a constant need for developing data-efficient DL techniques to turn DL powered biomedical ultrasound imaging into reality. In this work, we develop a data-efficient DL training strategy for classifying tissues based on the ultrasonic backscattered RF data, i.e., quantitative ultrasound (QUS), which we named zone training. In zone training, we propose to divide the complete field of view of an ultrasound image into multiple zones associated with different regions of a diffraction pattern and then, train separate DL networks for each zone. The main advantage of zone training is that it requires less training data to achieve high accuracy. In this work, three different tissue-mimicking phantoms were classified by a DL network. The results demonstrated that zone training can require a factor of 2-3 less training data in low data regime to achieve similar classification accuracies compared to a conventional training strategy.

Assessment of Transarterial Chemoembolization Using Super-resolution Ultrasound Imaging and a Rat Model of Hepatocellular Carcinoma

OBJECTIVE

Hepatocellular carcinoma (HCC) is a highly prevalent form of liver cancer diagnosed annually in 600,000 people worldwide. A common treatment is transarterial chemoembolization (TACE), which interrupts the blood supply of oxygen and nutrients to the tumor mass. The need for repeat TACE treatments may be assessed in the weeks after therapy with contrast-enhanced ultrasound (CEUS) imaging. Although the spatial resolution of traditional CEUS has been restricted by the diffraction limit of ultrasound (US), this physical barrier has been overcome by a recent innovation known as super-resolution US (SRUS) imaging. In short, SRUS enhances the visible details of smaller microvascular structures on the 10 to 100 µm scale, which unlocks a host of new clinical opportunities for US.

METHODS

In this study, a rat model of orthotopic HCC is introduced and TACE treatment response (to a doxorubicin-lipiodol emulsion) is assessed using longitudinal SRUS and magnetic resonance imaging (MRI) performed at 0, 7 and 14 d. Animals were euthanized at 14 d for histological analysis of excised tumor tissue and determination of TACE response, that is, control, partial response or complete response. CEUS imaging was performed using a pre-clinical US system (Vevo 3100, FUJIFILM VisualSonics Inc.) equipped with an MX201 linear array transducer. After administration of a microbubble contrast agent (Definity, Lantheus Medical Imaging), a series of CEUS images were collected at each tissue cross-section as the transducer was mechanically stepped at 100 μm increments. SRUS images were formed at each spatial position, and a microvascular density metric was calculated. Microscale computed tomography (microCT, OI/CT, MILabs) was used to confirm TACE procedure success, and tumor size was monitored using a small animal MRI system (BioSpec 3T, Bruker Corp.).

RESULTS

Although there were no differences at baseline (p > 0.15), both microvascular density levels and tumor size measures from the complete responder cases at 14 d were considerably lower and smaller, respectively, than those in the partial responder or control group animals. Histological analysis revealed tumor-to-necrosis levels of 8.4%, 51.1% and 100%, for the control, partial responder and complete responder groups, respectively (p < 0.005).

CONCLUSION

SRUS imaging is a promising modality for assessing early changes in microvascular networks in response to tissue perfusion-altering interventions such as TACE treatment of HCC.

Laser diode photoacoustic point source detection: machine learning-based denoising and reconstruction

A new development in photoacoustic (PA) imaging has been the use of compact, portable and low-cost laser diodes (LDs), but LD-based PA imaging suffers from low signal intensity recorded by the conventional transducers. A common method to improve signal strength is temporal averaging, which reduces frame rate and increases laser exposure to patients. To tackle this problem, we propose a deep learning method that will denoise point source PA radio-frequency (RF) data before beamforming with a very few frames, even one. We also present a deep learning method to automatically reconstruct point sources from noisy pre-beamformed data. Finally, we employ a strategy of combined denoising and reconstruction, which can supplement the reconstruction algorithm for very low signal-to-noise ratio inputs.

Ultrasound Signal Processing: From Models to Deep Learning

Medical ultrasound imaging relies heavily on high-quality signal processing to provide reliable and interpretable image reconstructions. Conventionally, reconstruction algorithms have been derived from physical principles. These algorithms rely on assumptions and approximations of the underlying measurement model, limiting image quality in settings where these assumptions break down. Conversely, more sophisticated solutions based on statistical modeling or careful parameter tuning or derived from increased model complexity can be sensitive to different environments. Recently, deep learning-based methods, which are optimized in a data-driven fashion, have gained popularity. These model-agnostic techniques often rely on generic model structures and require vast training data to converge to a robust solution. A relatively new paradigm combines the power of the two: leveraging data-driven deep learning and exploiting domain knowledge. These model-based solutions yield high robustness and require fewer parameters and training data than conventional neural networks. In this work we provide an overview of these techniques from the recent literature and discuss a wide variety of ultrasound applications. We aim to inspire the reader to perform further research in this area and to address the opportunities within the field of ultrasound signal processing. We conclude with a future perspective on model-based deep learning techniques for medical ultrasound.

HD2A-Net: A novel dual gated attention network using comprehensive hybrid dilated convolutions for medical image segmentation

The convolutional neural networks (CNNs) have been widely proposed in the medical image analysis tasks, especially in the image segmentations. In recent years, the encoder-decoder structures, such as the U-Net, were rendered. However, the multi-scale information transmission and effective modeling for long-range feature dependencies in these structures were not sufficiently considered. To improve the performance of the existing methods, we propose a novel hybrid dual dilated attention network (HD²A-Net) to conduct the lesion region segmentations. In the proposed network, we innovatively present the comprehensive hybrid dilated convolution (CHDC) module, which facilitates the transmission of the multi-scale information. Based on the CHDC module and the attention mechanisms, we design a novel dual dilated gated attention (DDGA) block to enhance the saliency of related regions from the multi-scale aspect. Besides, a dilated dense (DD) block is designed to expand the receptive fields. The ablation studies were performed to verify our proposed blocks. Besides, the interpretability of the HD²A-Net was analyzed through the visualization of the attention weight maps from the key blocks. Compared to the state-of-the-art methods including CA-Net, DeepLabV3+, and Attention U-Net, the HD²A-Net outperforms significantly, with the metrics of Dice, Average Symmetric Surface Distance (ASSD), and mean Intersection-over-Union (mIoU) reaching 93.16%, 93.63%, and 94.72%, 0.36 pix, 0.69 pix, and 0.52 pix, and 88.03%, 88.67%, and 90.33% on three publicly available medical image datasets: MAEDE-MAFTOUNI (COVID-19 CT), ISIC-2018 (Melanoma Dermoscopy), and Kvasir-SEG (Gastrointestinal Disease Polyp), respectively.