Image Quality Enhancement Using a Deep Neural Network for Plane Wave Medical Ultrasound Imaging

Artificial Intelligence Image Recognition System for Preventing Wrong-Site Upper Limb Surgery

Our image recognition system employs a deep learning model to differentiate between the left and right upper limbs in images, allowing doctors to determine the correct surgical position. From the experimental results, it was found that the precision rate and the recall rate of the intelligent image recognition system for preventing wrong-site upper limb surgery proposed in this paper could reach 98% and 93%, respectively. The results proved that our Artificial Intelligence Image Recognition System (AIIRS) could indeed assist orthopedic surgeons in preventing the occurrence of wrong-site left and right upper limb surgery. At the same time, in future, we will apply for an IRB based on our prototype experimental results and we will conduct the second phase of human trials. The results of this research paper are of great benefit and research value to upper limb orthopedic surgery.

Frame rate improvement in ultrafast coherent plane wave compounding

Coherent plane wave compounding (CPWC), as an ultrafast ultrasound imaging technique, makes a significant breakthrough in frame rate enhancement. However, there exists a compromise between the quality of the final image and the frame rate in CPWC. In this paper, we propose an efficient method to minimize the number of required emissions, and consequently, improve the frame rate, while maintaining the image quality. To this end, we down-sample the angle interval using two specific sampling factors. More precisely, we construct two different subsets, each of which consists of a few numbers of emissions. The optimal values of the angle intervals are achieved based on the beampattern that corresponds to the reference case (that is, the case where all plane waves are used). Finally, in order to keep the image quality comparable with the reference case, we apply some modifications to the image reconstruction procedure. In the proposed algorithm, the Delay-and-Sum beamformed images of two considered subsets are convolved to achieve the final reconstructed image. The obtained results confirm the efficiency of the proposed method in terms of frame rate improvement compared to the reference case. In particular, by using the proposed method, the required emissions in PICMUS data reduce to 16, which is 4.6 times smaller compared to the reference case. Also, the gCNR values of the proposed method and the reference case are obtained as 0.98 and 0.97, respectively, for in-vivo dataset. This demonstrates that the proposed method successfully preserves the quality of the reconstructed image by using much fewer emissions.

HTC-Net: Hashimoto's thyroiditis ultrasound image classification model based on residual network reinforced by channel attention mechanism

Convolutional neural network (CNN) is efficient in extracting and aggregating local features in the spatial dimension of the images. However, obtaining the inapparent texture information of the low-echo area in the ultrasound images is not easy, and it is especially challenging for the early lesion recognition in Hashimoto's thyroiditis (HT) ultrasound images. In this paper, a HT ultrasound image classification model HTC-Net based on residual network reinforced by channel attention mechanism is proposed. HTC-Net strengthens the features of the important channels by reinforced channel attention mechanism through which the high-level semantic information is enchanced and the low-level semantic information is suppressed. Residual network assists HTC-Net focus on the key local areas of the ultrasound images while pay attention to the global semantic information. Furthermore, in order to solve the problem of uneven distribution caused by large amount of difficult-to-classify samples in the data sets, a new feature loss function TanCELoss with weight factor dynamically adjusting is constructed. TanCELoss function can better assist HTC-Net to transform difficult-to-classify samples into easy-to-classify samples gradually, and improve the balancing distribution of the samples. The experiments are implemented based on data sets collected by the Endocrinology Department of four branches from Guangdong Provincial Hospital of Chinese Medicine. Both quantitative testing and visualization results show that HTC-Net obtains STOA performance for early lesions recognition in HT ultrasound images. HTC-Net has great application value especially under the condition of owning only small data samples.

A two-dimensional angular interpolation based on radial basis functions for high frame rate ultrafast imaging

To solve the problem of reduced image quality in plane wave imaging (PWI), coherent plane wave compounding (CPWC) has been introduced, based on a combination of plane wave images from several directions (i.e., with different angles). However, the number of angles needed to reach a reasonable image quality affects the maximum achievable frame rate in CPWC. In this study, we suggest reducing the tradeoff between the image quality and the frame rate in CPWC by employing two-dimensional (2D) interpolation based on radial basis functions. More specifically, we propose constructing a three-dimensional spatio-angular structure to integrate both spatial and angular information into the reconstruction prior to 2D interpolation. The rationale behind our proposal is to reduce the number of transmissions and then apply the 2D interpolation along the angle dimension to reconstruct the missing information corresponding to the angles not selected for CPWC imaging. To evaluate the proposed technique, we applied it to the PWI challenges in the medical ultrasound database. Results show that we can achieve 3× to 4× improvement in frame rate while maintaining acceptable image quality compared to the case of using all the angles.

Extended aperture image reconstruction for plane-wave imaging

B-mode images undergo degradation in the boundary region because of the limited number of elements in the ultrasound probe. Herein, a deep learning-based extended aperture image reconstruction method is proposed to reconstruct a B-mode image with an enhanced boundary region. The proposed network can reconstruct an image using pre-beamformed raw data received from the half-aperture of the probe. To generate a high-quality training target without degradation in the boundary region, the target data were acquired using the full-aperture. Training data were acquired from an experimental study using a tissue-mimicking phantom, vascular phantom, and simulation of random point scatterers. Compared with plane-wave images from delay and sum beamforming, the proposed extended aperture image reconstruction method achieves improvement at the boundary region in terms of the multi-scale structure of similarity and peak signal-to-noise ratio by 8% and 4.10 dB in resolution evaluation phantom, 7% and 3.15 dB in contrast speckle phantom, and 5% and 3 dB in in vivo study of carotid artery imaging. The findings in this study prove the feasibility of a deep learning-based extended aperture image reconstruction method for boundary region improvement.

Ultrasonic imaging using conditional generative adversarial networks

The Full Matrix Capture (FMC) and Total Focusing Method (TFM) combination is often considered the gold standard in ultrasonic nondestructive testing, however it may be impractical due to the amount of time required to gather and process the FMC, particularly for high cadence inspections. This study proposes replacing conventional FMC acquisition and TFM processing with a single zero-degree plane wave (PW) insonification and a conditional Generative Adversarial Network (cGAN) trained to produce TFM-like images. Three models with different cGAN architectures and loss formulations were tested in different scenarios. Their performances were compared with conventional TFM computed from FMC. The proposed cGANs were able to recreate TFM-like images with the same resolution while improving the contrast in more than 94% of the reconstructions in comparison with conventional TFM reconstructions. Indeed, thanks to the use of a bias in the cGANs' training, the contrast was systematically increased through a reduction of the background noise level and the elimination of some artifacts. Finally, the proposed method led to a reduction of the computation time and file size by a factor of 120 and 75, respectively.

A Single-Shot Harmonic Imaging Approach Utilizing Deep Learning for Medical Ultrasound

Tissue harmonic imaging (THI) is an invaluable tool in clinical ultrasound due to its enhanced contrast resolution and reduced reverberation clutter in comparison with fundamental mode imaging. However, harmonic content separation based on high-pass filtering suffers from potential contrast degradation or lower axial resolution due to spectral leakage, whereas nonlinear multipulse harmonic imaging schemes, such as amplitude modulation and pulse inversion, suffer from a reduced frame rate and comparatively higher motion artifacts due to the necessity of at least two pulse echo acquisitions. To address this problem, we propose a deep-learning-based single-shot harmonic imaging technique capable of generating comparable image quality to pulse amplitude modulation methods, yet at a higher frame rate and with fewer motion artifacts. Specifically, an asymmetric convolutional encoder-decoder structure is designed to estimate the combination of the echoes resulting from the half-amplitude transmissions using the echo produced from the full amplitude transmission as input. The echoes were acquired with the checkerboard amplitude modulation technique for training. The model was evaluated across various targets and samples to illustrate generalizability as well as the possibility and impact of transfer learning. Furthermore, for possible interpretability of the network, we investigate if the latent space of the encoder holds information on the nonlinearity parameter of the medium. We demonstrate the ability of the proposed approach to generate harmonic images with a single firing that are comparable to those from a multipulse acquisition.

Ultrasound Signal Processing: From Models to Deep Learning

Medical ultrasound imaging relies heavily on high-quality signal processing to provide reliable and interpretable image reconstructions. Conventionally, reconstruction algorithms have been derived from physical principles. These algorithms rely on assumptions and approximations of the underlying measurement model, limiting image quality in settings where these assumptions break down. Conversely, more sophisticated solutions based on statistical modeling or careful parameter tuning or derived from increased model complexity can be sensitive to different environments. Recently, deep learning-based methods, which are optimized in a data-driven fashion, have gained popularity. These model-agnostic techniques often rely on generic model structures and require vast training data to converge to a robust solution. A relatively new paradigm combines the power of the two: leveraging data-driven deep learning and exploiting domain knowledge. These model-based solutions yield high robustness and require fewer parameters and training data than conventional neural networks. In this work we provide an overview of these techniques from the recent literature and discuss a wide variety of ultrasound applications. We aim to inspire the reader to perform further research in this area and to address the opportunities within the field of ultrasound signal processing. We conclude with a future perspective on model-based deep learning techniques for medical ultrasound.

A progressively dual reconstruction network for plane wave beamforming with both paired and unpaired training data

High-frame-rate plane wave (PW) imaging suffers from unsatisfactory image quality due to the absence of focus in transmission. Although coherent compounding from tens of PWs can improve PW image quality, it in turn results in a decreased frame rate, which is limited for tracking fast moving tissues. To overcome the trade-off between frame rate and image quality, we propose a progressively dual reconstruction network (PDRN) to achieve adaptive beamforming and enhance the image quality via both supervised and transfer learning in the condition of single or a few PWs transmission. Specifically, the proposed model contains a progressive network and a dual network to form a close loop and provide collaborative supervision for model optimization. The progressive network takes the channel delay of each spatial point as input and progressively learns coherent compounding beamformed data with increased numbers of steered PWs step by step. The dual network learns the downsampling process and reconstructs the beamformed data with decreased numbers of steered PWs, which reduces the space of the possible learning functions and improves the model's discriminative ability. In addition, the dual network is leveraged to perform transfer learning for the training data without sufficient steered PWs. The simulated, in vivo, vocal cords (VCs), and public available CUBDL dataset are collected for model evaluation.

Deep reconstruction of high-quality ultrasound images from raw plane-wave data: A simulation and in vivo study

This paper presents a novel beamforming approach based on deep learning to get closer to the ideal Point Spread Function (PSF) in Plane-Wave Imaging (PWI). The proposed approach is designed to reconstruct a high-quality version of Tissue Reflectivity Function (TRF) from echo traces acquired by transducer elements using only a single plane-wave transmission. In this approach, first, a model for the TRF is introduced by setting the imaging PSF as an isotropic (i.e., circularly symmetric) 2D Gaussian kernel convolved with a cosine function. Then, a mapping function between the pre-beamformed Radio-Frequency (RF) channel data and the proposed output is constructed using deep learning. Network architecture contains multi-resolution decomposition and reconstruction using wavelet transform for effective recovery of high-frequency content of the desired output. We exploit step by step training from coarse (mean square error) to fine (ℓ_0.2) loss functions. The proposed method is trained on 1174 simulation ultrasound data with the ground-truth echogenicity map extracted from real photographic images. The performance of the trained network is evaluated on the publicly available simulation and in vivo test data without any further fine-tuning. Simulation test results show an improvement of 37.5% and 65.8% in terms of axial and lateral resolution as compared to Delay-And-Sum (DAS) results, respectively. The contrast is also improved by 33.7% in comparison to DAS. Furthermore, the reconstructed in vivo images confirm that the trained mapping function does not need any fine-tuning in the new domain. Therefore, the proposed approach maintains high resolution, contrast, and framerate simultaneously.

Deep Learning for Ultrasound Image Formation: CUBDL Evaluation Framework and Open Datasets

Deep learning for ultrasound image formation is rapidly garnering research support and attention, quickly rising as the latest frontier in ultrasound image formation, with much promise to balance both image quality and display speed. Despite this promise, one challenge with identifying optimal solutions is the absence of unified evaluation methods and datasets that are not specific to a single research group. This article introduces the largest known international database of ultrasound channel data and describes the associated evaluation methods that were initially developed for the challenge on ultrasound beamforming with deep learning (CUBDL), which was offered as a component of the 2020 IEEE International Ultrasonics Symposium. We summarize the challenge results and present qualitative and quantitative assessments using both the initially closed CUBDL evaluation test dataset (which was crowd-sourced from multiple groups around the world) and additional in vivo breast ultrasound data contributed after the challenge was completed. As an example quantitative assessment, single plane wave images from the CUBDL Task 1 dataset produced a mean generalized contrast-to-noise ratio (gCNR) of 0.67 and a mean lateral resolution of 0.42 mm when formed with delay-and-sum beamforming, compared with a mean gCNR as high as 0.81 and a mean lateral resolution as low as 0.32 mm when formed with networks submitted by the challenge winners. We also describe contributed CUBDL data that may be used for training of future networks. The compiled database includes a total of 576 image acquisition sequences. We additionally introduce a neural-network-based global sound speed estimator implementation that was necessary to fairly evaluate the results obtained with this international database. The integration of CUBDL evaluation methods, evaluation code, network weights from the challenge winners, and all datasets described herein are publicly available (visit https://cubdl.jhu.edu for details).

A Deep Learning-Based Automatic First-Arrival Picking Method for Ultrasound Sound-Speed Tomography

Ultrasound sound-speed tomography (USST) has shown great prospects for breast cancer diagnosis due to its advantages of nonradiation, low cost, 3-D breast images, and quantitative indicators. However, the reconstruction quality of USST is highly dependent on the first-arrival picking of the transmission wave. Traditional first-arrival picking methods have low accuracy and noise robustness. To improve the accuracy and robustness, we introduced a self-attention mechanism into the bidirectional long short-term memory (BLSTM) network and proposed the self-attention BLSTM (SAT-BLSTM) network. The proposed method predicts the probability of the first-arrival time and selects the time with maximum probability. A numerical simulation and prototype experiment were conducted. In the numerical simulation, the proposed SAT-BLSTM showed the best results. For signal-to-noise ratios (SNRs) of 50, 30, and 15 dB, the mean absolute errors (MAEs) were 48, 49, and 76 ns, respectively. The BLSTM had the second-best results, with MAEs of 55, 56, and 85 ns, respectively. The MAEs of the Akaike information criterion (AIC) method were 57, 296, and 489 ns, respectively. In the prototype experiment, the MAEs of the SAT-BLSTM, the BLSTM, and the AIC were 94, 111, and 410 ns, respectively.