Learning Sub-Sampling and Signal Recovery With Applications in Ultrasound Imaging

Enhancing single-element compressive ultrasound imaging through novel random aperture masking and data mixing strategy

As ultrasound techniques continue to evolve, the integration of compressed sensing technology has emerged as a pivotal advancement, offering a transformative impact on the landscape of ultrasound imaging. A key attribute of compressed sensing lies in its ability to facilitate a substantial reduction in both machinery size and power consumption. This technological synergy not only addresses crucial practical considerations in the design of ultrasound systems but also opens avenues for enhanced portability and energy efficiency. This study develops a model and introduces an aperture mask with a mixing scheme for compressive ultrasound imaging employing a single transducer, aiming to minimize the loss of information and scrutinize the variables influencing image quality while facilitating computationally efficient system simulation. A detailed procedural guide is presented for generating synthetic data, accompanied by qualitative and quantitative analyses using several sparse recovery methods under different experimental conditions. This study's analysis reveals that the proposed strategy achieves improved metrics, offering advantages for sparse recovery. Specifically, the finite element results demonstrate approximately a 10% improvement in the condition number of the measurement matrix, reflecting enhanced numerical stability.

Branched Convolutional Neural Networks for Receiver Channel Recovery in High-Frame-Rate Sparse-Array Ultrasound Imaging

For high-frame-rate ultrasound imaging, it remains challenging to implement on compact systems as a sparse imaging configuration with limited array channels. One key issue is that the resulting image quality is known to be mediocre not only because unfocused plane-wave excitations are used but also because grating lobes would emerge in sparse-array configurations. In this article, we present the design and use of a new channel recovery framework to infer full-array plane-wave channel datasets for periodically sparse arrays that operate with as few as one-quarter of the full-array aperture. This framework is based on a branched encoder-decoder convolutional neural network (CNN) architecture, which was trained using full-array plane-wave channel data collected from human carotid arteries (59 864 training acquisitions; 5-MHz imaging frequency; 20-MHz sampling rate; plane-wave steering angles between -15° and 15° in 1° increments). Three branched encoder-decoder CNNs were separately trained to recover missing channels after differing degrees of channelwise downsampling (2, 3, and 4 times). The framework's performance was tested on full-array and downsampled plane-wave channel data acquired from an in vitro point target, human carotid arteries, and human brachioradialis muscle. Results show that when inferred full-array plane-wave channel data were used for beamforming, spatial aliasing artifacts in the B-mode images were suppressed for all degrees of channel downsampling. In addition, the image contrast was enhanced compared with B-mode images obtained from beamforming with downsampled channel data. When the recovery framework was implemented on an RTX-2080 GPU, the three investigated degrees of downsampling all achieved the same inference time of 4 ms. Overall, the proposed framework shows promise in enhancing the quality of high-frame-rate ultrasound images generated using a sparse-array imaging setup.

Ultrasound Plane-Wave Compressed Sensing Reconstruction Method Using Intra-frame and Inter-frame Joint Multi-hypothesis and Reference Frame Multi-hypothesis Predictions

OBJECTIVE

Ultrasound plane-wave imaging has the advantage of high frame rate in addition to high data volume. High data sampling rates and large amounts of data storage can become bottlenecks in ultrasound system design. Although compressed sensing technology can help reduce the burden of sampling and transmission, it achieves relatively low image quality because of its reliance solely on signal sparsity. Therefore, we proposed reconstructing the ultrasound signal by applying additional prior knowledge, such as plane-wave imaging and its echo characteristics.

METHODS

Inspired by multi-hypothesis prediction methods in video compression coding, the plane-wave multi-hypothesis prediction compressed sensing reconstruction method was proposed to improve the accuracy of reconstructions. We applied multi-hypothesis prediction and residual reconstruction on the plane wave to enhance the quality of reconstruction and correct predicted values. Also, to acquire high-quality hypotheses, two hypothesis acquisition schemes were evaluated, constructing search windows on both preceding and subsequent frames as well as the reference frame.

RESULTS

Compared with traditional reconstruction methods that rely on sparsity, multi-hypothesis prediction compressed sensing methods can reduce signal reconstruction errors and significantly eliminate image artifacts. Furthermore, by using improved hypotheses, signal reconstruction and image quality can be enhanced, resulting in higher contrast.

CONCLUSION

Comparative simulation experimental results based on the publicly available Plane-Wave Imaging Challenge in Medical Ultrasound (PICMUS) and acoustic radiation force imaging data sets demonstrate that the proposed method outperforms other methods in both reconstruction errors and image quality. This helps to reduce the complexity of sampling and transmission of the ultrasound system.

Extended aperture image reconstruction for plane-wave imaging

B-mode images undergo degradation in the boundary region because of the limited number of elements in the ultrasound probe. Herein, a deep learning-based extended aperture image reconstruction method is proposed to reconstruct a B-mode image with an enhanced boundary region. The proposed network can reconstruct an image using pre-beamformed raw data received from the half-aperture of the probe. To generate a high-quality training target without degradation in the boundary region, the target data were acquired using the full-aperture. Training data were acquired from an experimental study using a tissue-mimicking phantom, vascular phantom, and simulation of random point scatterers. Compared with plane-wave images from delay and sum beamforming, the proposed extended aperture image reconstruction method achieves improvement at the boundary region in terms of the multi-scale structure of similarity and peak signal-to-noise ratio by 8% and 4.10 dB in resolution evaluation phantom, 7% and 3.15 dB in contrast speckle phantom, and 5% and 3 dB in in vivo study of carotid artery imaging. The findings in this study prove the feasibility of a deep learning-based extended aperture image reconstruction method for boundary region improvement.

Photoacoustic imaging with limited sampling: a review of machine learning approaches

Photoacoustic imaging combines high optical absorption contrast and deep acoustic penetration, and can reveal structural, molecular, and functional information about biological tissue non-invasively. Due to practical restrictions, photoacoustic imaging systems often face various challenges, such as complex system configuration, long imaging time, and/or less-than-ideal image quality, which collectively hinder their clinical application. Machine learning has been applied to improve photoacoustic imaging and mitigate the otherwise strict requirements in system setup and data acquisition. In contrast to the previous reviews of learned methods in photoacoustic computed tomography (PACT), this review focuses on the application of machine learning approaches to address the limited spatial sampling problems in photoacoustic imaging, specifically the limited view and undersampling issues. We summarize the relevant PACT works based on their training data, workflow, and model architecture. Notably, we also introduce the recent limited sampling works on the other major implementation of photoacoustic imaging, i.e., photoacoustic microscopy (PAM). With machine learning-based processing, photoacoustic imaging can achieve improved image quality with modest spatial sampling, presenting great potential for low-cost and user-friendly clinical applications.

Ultrasound Signal Processing: From Models to Deep Learning

Medical ultrasound imaging relies heavily on high-quality signal processing to provide reliable and interpretable image reconstructions. Conventionally, reconstruction algorithms have been derived from physical principles. These algorithms rely on assumptions and approximations of the underlying measurement model, limiting image quality in settings where these assumptions break down. Conversely, more sophisticated solutions based on statistical modeling or careful parameter tuning or derived from increased model complexity can be sensitive to different environments. Recently, deep learning-based methods, which are optimized in a data-driven fashion, have gained popularity. These model-agnostic techniques often rely on generic model structures and require vast training data to converge to a robust solution. A relatively new paradigm combines the power of the two: leveraging data-driven deep learning and exploiting domain knowledge. These model-based solutions yield high robustness and require fewer parameters and training data than conventional neural networks. In this work we provide an overview of these techniques from the recent literature and discuss a wide variety of ultrasound applications. We aim to inspire the reader to perform further research in this area and to address the opportunities within the field of ultrasound signal processing. We conclude with a future perspective on model-based deep learning techniques for medical ultrasound.

A Review of the Gumbel-max Trick and its Extensions for Discrete Stochasticity in Machine Learning

The Gumbel-max trick is a method to draw a sample from a categorical distribution, given by its unnormalized (log-)probabilities. Over the past years, the machine learning community has proposed several extensions of this trick to facilitate, e.g., drawing multiple samples, sampling from structured domains, or gradient estimation for error backpropagation in neural network optimization. The goal of this survey article is to present background about the Gumbel-max trick, and to provide a structured overview of its extensions to ease algorithm selection. Moreover, it presents a comprehensive outline of (machine learning) literature in which Gumbel-based algorithms have been leveraged, reviews commonly-made design choices, and sketches a future perspective.

Improving Lateral Resolution in 3-D Imaging With Micro-beamforming Through Adaptive Beamforming by Deep Learning

There is an increased desire for miniature ultrasound probes with small apertures to provide volumetric images at high frame rates for in-body applications. Satisfying these increased requirements makes simultaneous achievement of a good lateral resolution a challenge. As micro-beamforming is often employed to reduce data rate and cable count to acceptable levels, receive processing methods that try to improve spatial resolution will have to compensate the introduced reduction in focusing. Existing beamformers do not realize sufficient improvement and/or have a computational cost that prohibits their use. Here we propose the use of adaptive beamforming by deep learning (ABLE) in combination with training targets generated by a large aperture array, which inherently has better lateral resolution. In addition, we modify ABLE to extend its receptive field across multiple voxels. We illustrate that this method improves lateral resolution both quantitatively and qualitatively, such that image quality is improved compared with that achieved by existing delay-and-sum, coherence factor, filtered-delay-multiplication-and-sum and Eigen-based minimum variance beamformers. We found that only in silica data are required to train the network, making the method easily implementable in practice.

Minimizing Image Quality Loss After Channel Count Reduction for Plane Wave Ultrasound via Deep Learning Inference

High-frame-rate ultrasound imaging uses unfocused transmissions to insonify an entire imaging view for each transmit event, thereby enabling frame rates over 1000 frames per second (fps). At these high frame rates, it is naturally challenging to realize real-time transfer of channel-domain raw data from the transducer to the system back end. Our work seeks to halve the total data transfer rate by uniformly decimating the receive channel count by 50% and, in turn, doubling the array pitch. We show that despite the reduced channel count and the inevitable use of a sparse array aperture, the resulting beamformed image quality can be maintained by designing a custom convolutional encoder-decoder neural network to infer the radio frequency (RF) data of the nullified channels. This deep learning framework was trained with in vivo human carotid data (5-MHz plane wave imaging, 128 channels, 31 steering angles over a 30° span, and 62 799 frames in total). After training, the network was tested on an in vitro point target scenario that was dissimilar to the training data, in addition to in vivo carotid validation datasets. In the point target phantom image beamformed from inferred channel data, spatial aliasing artifacts attributed to array pitch doubling were found to be reduced by up to 10 dB. For carotid imaging, our proposed approach yielded a lumen-to-tissue contrast that was on average within 3 dB compared to the full-aperture image, whereas without channel data inferencing, the carotid lumen was obscured. When implemented on an RTX-2080 GPU, the inference time to apply the trained network was 4 ms, which favors real-time imaging. Overall, our technique shows that with the help of deep learning, channel data transfer rates can be effectively halved with limited impact on the resulting image quality.

B-Spline Parameterized Joint Optimization of Reconstruction and K-Space Trajectories (BJORK) for Accelerated 2D MRI

Optimizing k-space sampling trajectories is a promising yet challenging topic for fast magnetic resonance imaging (MRI). This work proposes to optimize a reconstruction method and sampling trajectories jointly concerning image reconstruction quality in a supervised learning manner. We parameterize trajectories with quadratic B-spline kernels to reduce the number of parameters and apply multi-scale optimization, which may help to avoid sub-optimal local minima. The algorithm includes an efficient non-Cartesian unrolled neural network-based reconstruction and an accurate approximation for backpropagation through the non-uniform fast Fourier transform (NUFFT) operator to accurately reconstruct and back-propagate multi-coil non-Cartesian data. Penalties on slew rate and gradient amplitude enforce hardware constraints. Sampling and reconstruction are trained jointly using large public datasets. To correct for possible eddy-current effects introduced by the curved trajectory, we use a pencil-beam trajectory mapping technique. In both simulations and in- vivo experiments, the learned trajectory demonstrates significantly improved image quality compared to previous model-based and learning-based trajectory optimization methods for 10× acceleration factors. Though trained with neural network-based reconstruction, the proposed trajectory also leads to improved image quality with compressed sensing-based reconstruction.

Deep-fUS: A Deep Learning Platform for Functional Ultrasound Imaging of the Brain Using Sparse Data

Functional ultrasound (fUS) is a rapidly emerging modality that enables whole-brain imaging of neural activity in awake and mobile rodents. To achieve sufficient blood flow sensitivity in the brain microvasculature, fUS relies on long ultrasound data acquisitions at high frame rates, posing high demands on the sampling and processing hardware. Here we develop an image reconstruction method based on deep learning that significantly reduces the amount of data necessary while retaining imaging performance. We trained convolutional neural networks to learn the power Doppler reconstruction function from sparse sequences of ultrasound data with compression factors of up to 95%. High-quality images from in vivo acquisitions in rats were used for training and performance evaluation. We demonstrate that time series of power Doppler images can be reconstructed with sufficient accuracy to detect the small changes in cerebral blood volume (~10%) characteristic of task-evoked cortical activation, even though the network was not formally trained to reconstruct such image series. The proposed platform may facilitate the development of this neuroimaging modality in any setting where dedicated hardware is not available or in clinical scanners.

Requirements and Hardware Limitations of High-Frame-Rate 3-D Ultrasound Imaging Systems

The spread of high frame rate and 3-D imaging techniques has raised pressing requirements for ultrasound systems. In particular, the processing power and data transfer rate requirements may be so demanding to hinder the real-time (RT) implementation of such techniques. This paper first analyzes the general requirements involved in RT ultrasound systems. Then, it identifies the main bottlenecks in the receiving section of a specific RT scanner, the ULA-OP 256, which is one of the most powerful available open scanners and may therefore be assumed as a reference. This case study has evidenced that the “star” topology, used to digitally interconnect the system’s boards, may easily saturate the data transfer bandwidth, thus impacting the achievable frame/volume rates in RT. The architecture of the digital scanner was exploited to tackle the bottlenecks by enabling a new “ring“ communication topology. Experimental 2-D and 3-D high-frame-rate imaging tests were conducted to evaluate the frame rates achievable with both interconnection modalities. It is shown that the ring topology enables up to 4400 frames/s and 510 volumes/s, with mean increments of +230% (up to +620%) compared to the star topology.

Deep-Learning Based Adaptive Ultrasound Imaging From Sub-Nyquist Channel Data

Traditional beamforming of medical ultrasound images relies on sampling rates significantly higher than the actual Nyquist rate of the received signals. This results in large amounts of data to store and process, imposing hardware and software challenges on the development of ultrasound machinery and algorithms, and impacting the resulting performance. In light of the capabilities demonstrated by deep learning methods over the past years across a variety of fields, including medical imaging, it is natural to consider their ability to recover high-quality ultrasound images from partial data. Here, we propose an approach for deep-learning-based reconstruction of B-mode images from temporally and spatially sub-sampled channel data. We begin by considering sub-Nyquist sampled data, time-aligned in the frequency domain and transformed back to the time domain. The data are further sampled spatially so that only a subset of the received signals is acquired. The partial data is used to train an encoder-decoder convolutional neural network (CNN), using as targets minimum-variance (MV) beamformed signals that were generated from the original, fully-sampled data. Our approach yields high-quality B-mode images, with up to two times higher resolution than previously proposed reconstruction approaches (NESTA) from compressed data as well as delay-and-sum (DAS) beamforming of the fully-sampled data. In terms of contrast-to- noise ratio (CNR), our results are comparable to MV beamforming of the fully-sampled data, and provide up to 2 dB higher CNR values than DAS and NESTA, thus enabling better and more efficient imaging than what is used in clinical practice today.

Application of a Nursing Data-Driven Model for Continuous Improvement of PICC Care Quality

A PICC catheter maintenance network was established and managed to monitor the maintenance of catheters in placed patients throughout the process, providing homogeneous PICC catheter continuity of care for patients. Model-driven thinking is an idea for simulation system development. Model-driven architecture (MDA) is a design methodology that implements model-driven thinking and is widely used in simulation system development. Based on the requirements of nursing, the data-driven model is mainly divided into interface layer and functional service layer; this study adopts MDA technology which can detach the functions of the system from the platform, based on domain knowledge, and the metamodel adopts XSD-based data model to generate the PIM model, which is stored in the model library. The results showed that the number of nurses at maintenance sites increased from 79 to 232, the PICC placement rate for oncology patients increased from 35.0% to 76.0%, the nurse maintenance operation pass rate increased from 53.9% to 88.4%, and the maintenance default rate decreased from 40.0% to 10.9%.

Complex Convolutional Neural Networks for Ultrafast Ultrasound Imaging Reconstruction From In-Phase/Quadrature Signal

Ultrafast ultrasound imaging remains an active area of interest in the ultrasound community due to its ultrahigh frame rates. Recently, a wide variety of studies based on deep learning have sought to improve ultrafast ultrasound imaging. Most of these approaches have been performed on radio frequency (RF) signals. However, in- phase/quadrature (I/Q) digital beamformers are now widely used as low-cost strategies. In this work, we used complex convolutional neural networks for reconstruction of ultrasound images from I/Q signals. We recently described a convolutional neural network architecture called ID-Net, which exploited an inception layer designed for reconstruction of RF diverging-wave ultrasound images. In the present study, we derive the complex equivalent of this network, i.e., complex-valued inception for diverging-wave network (CID-Net) that operates on I/Q data. We provide experimental evidence that CID-Net provides the same image quality as that obtained from RF-trained convolutional neural networks, i.e., using only three I/Q images, CID-Net produces high-quality images that can compete with those obtained by coherently compounding 31 RF images. Moreover, we show that CID-Net outperforms the straightforward architecture that consists of processing real and imaginary parts of the I/Q signal separately, which thereby indicates the importance of consistently processing the I/Q signals using a network that exploits the complex nature of such signals.

Evaluating Input Domain and Model Selection for Deep Network Ultrasound Beamforming

Improving ultrasound B-mode image quality remains an important area of research. Recently, there has been increased interest in using deep neural networks (DNNs) to perform beamforming to improve image quality more efficiently. Several approaches have been proposed that use different representations of channel data for network processing, including a frequency-domain approach that we previously developed. We previously assumed that the frequency domain would be more robust to varying pulse shapes. However, frequency- and time-domain implementations have not been directly compared. In addition, because our approach operates on aperture domain data as an intermediate beamforming step, a discrepancy often exists between network performance and image quality on fully reconstructed images, making model selection challenging. Here, we perform a systematic comparison of frequency- and time-domain implementations. In addition, we propose a contrast-to-noise ratio (CNR)-based regularization to address previous challenges with model selection. Training channel data were generated from simulated anechoic cysts. Test channel data were generated from simulated anechoic cysts with and without varied pulse shapes, in addition to physical phantom and in vivo data. We demonstrate that simplified time-domain implementations are more robust than we previously assumed, especially when using phase preserving data representations. Specifically, 0.39- and 0.36-dB median improvements in in vivo CNR compared to DAS were achieved with frequency- and time-domain implementations, respectively. We also demonstrate that CNR regularization improves the correlation between training validation loss and simulated CNR by 0.83 and between simulated and in vivo CNR by 0.35 compared to DNNs trained without CNR regularization.

Self-supervised learning for accelerated 3D high-resolution ultrasound imaging

PURPOSE

Ultrasound (US) imaging has been widely used in diagnosis, image-guided intervention, and therapy, where high-quality three-dimensional (3D) images are highly desired from sparsely acquired two-dimensional (2D) images. This study aims to develop a deep learning-based algorithm to reconstruct high-resolution (HR) 3D US images only reliant on the acquired sparsely distributed 2D images.

METHODS

We propose a self-supervised learning framework using cycle-consistent generative adversarial network (cycleGAN), where two independent cycleGAN models are trained with paired original US images and two sets of low-resolution (LR) US images, respectively. The two sets of LR US images are obtained through down-sampling the original US images along the two axes, respectively. In US imaging, in-plane spatial resolution is generally much higher than through-plane resolution. By learning the mapping from down-sampled in-plane LR images to original HR US images, cycleGAN can generate through-plane HR images from original sparely distributed 2D images. Finally, HR 3D US images are reconstructed by combining the generated 2D images from the two cycleGAN models.

RESULTS

The proposed method was assessed on two different datasets. One is automatic breast ultrasound (ABUS) images from 70 breast cancer patients, the other is collected from 45 prostate cancer patients. By applying a spatial resolution enhancement factor of 3 to the breast cases, our proposed method achieved the mean absolute error (MAE) value of 0.90 ± 0.15, the peak signal-to-noise ratio (PSNR) value of 37.88 ± 0.88 dB, and the visual information fidelity (VIF) value of 0.69 ± 0.01, which significantly outperforms bicubic interpolation. Similar performances have been achieved using the enhancement factor of 5 in these breast cases and using the enhancement factors of 5 and 10 in the prostate cases.

CONCLUSIONS

We have proposed and investigated a new deep learning-based algorithm for reconstructing HR 3D US images from sparely acquired 2D images. Significant improvement on through-plane resolution has been achieved by only using the acquired 2D images without any external atlas images. Its self-supervision capability could accelerate HR US imaging.

Towards practical implementation of the compressed sensing framework for multi-element synthetic transmit aperture imaging

Compressed sensing (CS) has been adapted to synthetic aperture (SA) ultrasound imaging to improve the frame-rate of the system. Recently, we proposed a novel CS framework using Gaussian under-sampling to reduce the number of receive elements in multi-element synthetic transmit aperture (MSTA) imaging. However, that framework requires different receive elements to be chosen randomly for each transmission, which may add to practical implementation challenges. Modifying the scheme to employ the same set of receive elements for all transmissions of MSTA leads to degradation of the recovered image quality. Therefore, this work proposes a novel sampling scheme based on a genetic algorithm (GA), which optimally chooses the receive element positions once and uses it for all the transmission of MSTA. The CS performance using GA sampling schemes is evaluated against the previously proposed CS framework on in-vitro and in-vivo datasets. The obtained results suggest that not only does the GA-based approach allows the use of the same set of sparse receive elements for each transmit, but also leads to the lowest CS recovery error (NRMSE) and 14% overall improvement in image contrast, in comparison to the previously-proposed Gaussian sampling scheme. Thus, using the CS framework along with GA, can potentially reduce the complexity in implementation of CS-framework to MSTA based systems.