Ultrasound Image Enhancement Using Structure Oriented Adversarial Network

An accurate paradigm for denoising degraded ultrasound images based on artificial intelligence systems

Ultrasound images are susceptible to various forms of quality degradation that negatively impact diagnosis. Common degradations include speckle noise, Gaussian noise, salt and pepper noise, and blurring. This research proposes an accurate ultrasound image denoising strategy based on firstly detecting the noise type, then, suitable denoising methods can be applied for each corruption. The technique depends on convolutional neural networks to categorize the type of noise affecting an input ultrasound image. Pre-trained convolutional neural network models including GoogleNet, VGG-19, AlexNet and AlexNet-support vector machine (SVM) are developed and trained to perform this classification. A dataset of 782 numerically generated ultrasound images across different diseases and noise types is utilized for model training and evaluation. Results show AlexNet-SVM achieves the highest accuracy of 99.2% in classifying noise types. The results indicate that, the present technique is considered one of the top-performing models is then applied to real ultrasound images with different noise corruptions to demonstrate efficacy of the proposed detect-then-denoise system. RESEARCH HIGHLIGHTS: Proposes an accurate ultrasound image denoising strategy based on detecting noise type first. Uses pre-trained convolutional neural networks to categorize noise type in input images. Evaluates GoogleNet, VGG-19, AlexNet, and AlexNet-support vector machine (SVM) models on a dataset of 782 synthetic ultrasound images. AlexNet-SVM achieves highest accuracy of 99.2% in classifying noise types. Demonstrates efficacy of the proposed detect-then-denoise system on real ultrasound images.

Quantum inspired approach for denoising with application to medical imaging

Background noise in many fields such as medical imaging poses significant challenges for accurate diagnosis, prompting the development of denoising algorithms. Traditional methodologies, however, often struggle to address the complexities of noisy environments in high dimensional imaging systems. This paper introduces a novel quantum-inspired approach for image denoising, drawing upon principles of quantum and condensed matter physics. Our approach views medical images as amorphous structures akin to those found in condensed matter physics and we propose an algorithm that incorporates the concept of mode resolved localization directly into the denoising process. Notably, our approach eliminates the need for hyperparameter tuning. The proposed method is a standalone algorithm with minimal manual intervention, demonstrating its potential to use quantum-based techniques in classical signal denoising. Through numerical validation, we showcase the effectiveness of our approach in addressing noise-related challenges in imaging and especially medical imaging, underscoring its relevance for possible quantum computing applications.

Breast ultrasound image despeckling using multi-filtering DFrFT and adaptive fast BM3D

Improving the quality of breast ultrasound images is of great significance for clinical diagnosis which can greatly boost the diagnostic accuracy of ultrasonography. However, due to the influence of ultrasound imaging principles and acquisition equipment, the collected ultrasound images naturally contain a large amount of speckle noise, which leads to a decrease in image quality and affects clinical diagnosis. To overcome this problem, we propose an improved denoising algorithm combining multi-filter DFrFT (Discrete Fractional Fourier Transform) and the adaptive fast BM3D (Block Matching and 3D collaborative filtering) method. Firstly, we provide the multi-filtering DFrFT method for preprocessing the original breast ultrasound image so as to remove some speckle noise early in fractional transformation domain. Based on the fractional frequency spectrum characteristics of breast ultrasound images, three types of filters are designed correspondingly in low, medium, and high frequency domains. And by integrating filtered images, the enhanced images are obtained which not only remove some speckle noise in background but also preserve the details of breast lesions. Secondly, for further enhancing the image quality on the basis of multi-filter DFrFT, we propose the adaptive fast BM3D method by introducing the DBSCAN-based super pixel segmentation to block matching process, which utilizes super pixel segmentation labels to provide a reference on how similar it is between target block and retrieval blocks. It reduces the number of blocks to be retrieved and make the matched blocks with more similar features. At last, the local noise parameter estimation is also adopted in the hard threshold filtering process of traditional BM3D algorithm to achieve local adaptive filtering and further improving the denoising effect. The synthetic data and real breast ultrasound data examples show that this combined method can improve the speckle suppression level and keep the fidelity of structure effectively without increasing time cost.

De-Noising of Photoacoustic Microscopy Images by Attentive Generative Adversarial Network

As a hybrid imaging technology, photoacoustic microscopy (PAM) imaging suffers from noise due to the maximum permissible exposure of laser intensity, attenuation of ultrasound in the tissue, and the inherent noise of the transducer. De-noising is an image processing method to reduce noise, and PAM image quality can be recovered. However, previous de-noising techniques usually heavily rely on manually selected parameters, resulting in unsatisfactory and slow de-noising performance for different noisy images, which greatly hinders practical and clinical applications. In this work, we propose a deep learning-based method to remove noise from PAM images without manual selection of settings for different noisy images. An attention enhanced generative adversarial network is used to extract image features and adaptively remove various levels of Gaussian, Poisson, and Rayleigh noise. The proposed method is demonstrated on both synthetic and real datasets, including phantom (leaf veins) and in vivo (mouse ear blood vessels and zebrafish pigment) experiments. In the in vivo experiments using synthetic datasets, our method achieves the improvement of 6.53 dB and 0.26 in peak signal-to-noise ratio and structural similarity metrics, respectively. The results show that compared with previous PAM de-noising methods, our method exhibits good performance in recovering images qualitatively and quantitatively. In addition, the de-noising processing speed of 0.016 s is achieved for an image with 256×256 pixels, which has the potential for real-time applications. Our approach is effective and practical for the de-noising of PAM images.

US-Net: A lightweight network for simultaneous speckle suppression and texture enhancement in ultrasound images

BACKGROUND

Numerous traditional filtering approaches and deep learning-based methods have been proposed to improve the quality of ultrasound (US) image data. However, their results tend to suffer from over-smoothing and loss of texture and fine details. Moreover, they perform poorly on images with different degradation levels and mainly focus on speckle reduction, even though texture and fine detail enhancement are of crucial importance in clinical diagnosis.

METHODS

We propose an end-to-end framework termed US-Net for simultaneous speckle suppression and texture enhancement in US images. The architecture of US-Net is inspired by U-Net, whereby a feature refinement attention block (FRAB) is introduced to enable an effective learning of multi-level and multi-contextual representative features. Specifically, FRAB aims to emphasize high-frequency image information, which helps boost the restoration and preservation of fine-grained and textural details. Furthermore, our proposed US-Net is trained essentially with real US image data, whereby real US images embedded with simulated multi-level speckle noise are used as an auxiliary training set.

RESULTS

Extensive quantitative and qualitative experiments indicate that although trained with only one US image data type, our proposed US-Net is capable of restoring images acquired from different body parts and scanning settings with different degradation levels, while exhibiting favorable performance against state-of-the-art image enhancement approaches. Furthermore, utilizing our proposed US-Net as a pre-processing stage for COVID-19 diagnosis results in a gain of 3.6% in diagnostic accuracy.

CONCLUSIONS

The proposed framework can help improve the accuracy of ultrasound diagnosis.

Medical ultrasound image speckle reduction and resolution enhancement using texture compensated multi-resolution convolution neural network

Ultrasound (US) imaging is a mature technology that has widespread applications especially in the healthcare sector. Despite its widespread use and popularity, it has an inherent disadvantage that ultrasound images are prone to speckle and other kinds of noise. The image quality in the low-cost ultrasound imaging systems is degraded due to the presence of such noise and low resolution of such ultrasound systems. Herein, we propose a method for image enhancement where, the overall quality of the US images is improved by simultaneous enhancement of US image resolution and noise suppression. To avoid over-smoothing and preserving structural/texture information, we devise texture compensation in our proposed method to retain the useful anatomical features. Moreover, we also utilize US image formation physics knowledge to generate augmentation datasets which can improve the training of our proposed method. Our experimental results showcase the performance of the proposed network as well as the effectiveness of the utilization of US physics knowledge to generate augmentation datasets.

CNN-Based Image Reconstruction Method for Ultrafast Ultrasound Imaging

Ultrafast ultrasound (US) revolutionized biomedical imaging with its capability of acquiring full-view frames at over 1 kHz, unlocking breakthrough modalities such as shear-wave elastography and functional US neuroimaging. Yet, it suffers from strong diffraction artifacts, mainly caused by grating lobes, sidelobes, or edge waves. Multiple acquisitions are typically required to obtain a sufficient image quality, at the cost of a reduced frame rate. To answer the increasing demand for high-quality imaging from single unfocused acquisitions, we propose a two-step convolutional neural network (CNN)-based image reconstruction method, compatible with real-time imaging. A low-quality estimate is obtained by means of a backprojection-based operation, akin to conventional delay-and-sum beamforming, from which a high-quality image is restored using a residual CNN with multiscale and multichannel filtering properties, trained specifically to remove the diffraction artifacts inherent to ultrafast US imaging. To account for both the high dynamic range and the oscillating properties of radio frequency US images, we introduce the mean signed logarithmic absolute error (MSLAE) as a training loss function. Experiments were conducted with a linear transducer array, in single plane-wave (PW) imaging. Trainings were performed on a simulated dataset, crafted to contain a wide diversity of structures and echogenicities. Extensive numerical evaluations demonstrate that the proposed approach can reconstruct images from single PWs with a quality similar to that of gold-standard synthetic aperture imaging, on a dynamic range in excess of 60 dB. In vitro and in vivo experiments show that trainings carried out on simulated data perform well in experimental settings.

On the Reliability of CNNs in Clinical Practice: A Computer-Aided Diagnosis System Case Study

Leukocytes classification is essential to assess their number and status since they are the body’s first defence against infection and disease. Automation of the process can reduce the laborious manual process of review and diagnosis by operators and has been the subject of study for at least two decades. Most computer-aided systems exploit convolutional neural networks for classification purposes without any intermediate step to produce an accurate classification. This work explores the current limitations of deep learning-based methods applied to medical blood smear data. In particular, we consider leukocyte analysis oriented towards leukaemia prediction as a case study. In particular, we aim to demonstrate that a single classification step can undoubtedly lead to incorrect predictions or, worse, to correct predictions obtained with wrong indicators provided by the images. By generating new synthetic leukocyte data, it is possible to demonstrate that the inclusion of a fine-grained method, such as detection or segmentation, before classification is essential to allow the network to understand the adequate information on individual white blood cells correctly. The effectiveness of this study is thoroughly analysed and quantified through a series of experiments on a public data set of blood smears taken under a microscope. Experimental results show that residual networks perform statistically better in this scenario, even though they make correct predictions with incorrect information.

Complex Convolutional Neural Networks for Ultrafast Ultrasound Imaging Reconstruction From In-Phase/Quadrature Signal

Ultrafast ultrasound imaging remains an active area of interest in the ultrasound community due to its ultrahigh frame rates. Recently, a wide variety of studies based on deep learning have sought to improve ultrafast ultrasound imaging. Most of these approaches have been performed on radio frequency (RF) signals. However, in- phase/quadrature (I/Q) digital beamformers are now widely used as low-cost strategies. In this work, we used complex convolutional neural networks for reconstruction of ultrasound images from I/Q signals. We recently described a convolutional neural network architecture called ID-Net, which exploited an inception layer designed for reconstruction of RF diverging-wave ultrasound images. In the present study, we derive the complex equivalent of this network, i.e., complex-valued inception for diverging-wave network (CID-Net) that operates on I/Q data. We provide experimental evidence that CID-Net provides the same image quality as that obtained from RF-trained convolutional neural networks, i.e., using only three I/Q images, CID-Net produces high-quality images that can compete with those obtained by coherently compounding 31 RF images. Moreover, we show that CID-Net outperforms the straightforward architecture that consists of processing real and imaginary parts of the I/Q signal separately, which thereby indicates the importance of consistently processing the I/Q signals using a network that exploits the complex nature of such signals.

Deep Convolutional Neural Networks for Displacement Estimation in ARFI Imaging

Ultrasound elasticity imaging in soft tissue with acoustic radiation force requires the estimation of displacements, typically on the order of several microns, from serially acquired raw data A-lines. In this work, we implement a fully convolutional neural network (CNN) for ultrasound displacement estimation. We present a novel method for generating ultrasound training data, in which synthetic 3-D displacement volumes with a combination of randomly seeded ellipsoids are created and used to displace scatterers, from which simulated ultrasonic imaging is performed using Field II. Network performance was tested on these virtual displacement volumes, as well as an experimental ARFI phantom data set and a human in vivo prostate ARFI data set. In the simulated data, the proposed neural network performed comparably to Loupas's algorithm, a conventional phase-based displacement estimation algorithm; the rms error was [Formula: see text] for the CNN and 0.73 [Formula: see text] for Loupas. Similarly, in the phantom data, the contrast-to-noise ratio (CNR) of a stiff inclusion was 2.27 for the CNN-estimated image and 2.21 for the Loupas-estimated image. Applying the trained network to in vivo data enabled the visualization of prostate cancer and prostate anatomy. The proposed training method provided 26 000 training cases, which allowed robust network training. The CNN had a computation time that was comparable to Loupas's algorithm; further refinements to the network architecture may provide an improvement in the computation time. We conclude that deep neural network-based displacement estimation from ultrasonic data is feasible, providing comparable performance with respect to both accuracy and speed compared to current standard time-delay estimation approaches.

Target-Independent Domain Adaptation for WBC Classification Using Generative Latent Search

Automating the classification of camera-obtained microscopic images of White Blood Cells (WBCs) and related cell subtypes has assumed importance since it aids the laborious manual process of review and diagnosis. Several State-Of-The-Art (SOTA) methods developed using Deep Convolutional Neural Networks suffer from the problem of domain shift - severe performance degradation when they are tested on data (target) obtained in a setting different from that of the training (source). The change in the target data might be caused by factors such as differences in camera/microscope types, lenses, lighting-conditions etc. This problem can potentially be solved using Unsupervised Domain Adaptation (UDA) techniques albeit standard algorithms presuppose the existence of a sufficient amount of unlabelled target data which is not always the case with medical images. In this paper, we propose a method for UDA that is devoid of the need for target data. Given a test image from the target data, we obtain its 'closest-clone' from the source data that is used as a proxy in the classifier. We prove the existence of such a clone given that infinite number of data points can be sampled from the source distribution. We propose a method in which a latent-variable generative model based on variational inference is used to simultaneously sample and find the 'closest-clone' from the source distribution through an optimization procedure in the latent space. We demonstrate the efficacy of the proposed method over several SOTA UDA methods for WBC classification on datasets captured using different imaging modalities under multiple settings.

A unified deep network for beamforming and speckle reduction in plane wave imaging: A simulation study

Plane Wave Imaging is a fast imaging method used in ultrasound, which allows a high frame rate, but with compromised image quality when a single wave is used. In this work a learning-based approach was used to obtain improved image quality. The entire process of beamforming and speckle reduction was embedded in a single deep convolutional network, and trained with two types of simulated data. The network architecture was designed based on traditional physical considerations of the ultrasonic image formation pipe. As such, it includes beamforming with spatial matched filters, envelope detection, and a speckle reduction stage done in log-signal representation, with all stages containing trainable parameters. The approach was tested on the publicly available PICMUS datasets, achieving axial and lateral full-width-half-maximum (FWHM) resolution values of 0.22 mm and 0.35 mm respectively, and a Contrast to Noise Ratio (CNR) metric of 16.75 on the experimental datasets.

Carotid artery ultrasound image analysis: A review of the literature

Stroke is one of the prominent causes of death in the recent days. The existence of susceptible plaque in the carotid artery can be used in ascertaining the possibilities of cardiovascular diseases and long-term disabilities. The imaging modality used for early screening of the disease is B-mode ultrasound image of the person in the artery area. The objective of this article is to give a widespread review of the imaging modes and methods used for studying the carotid artery for identifying stroke, atherosclerosis and related cardiovascular diseases. We encompass the review in methods used for artery wall tracking, intima-media, and lumen segmentation which will help in finding the extent of the disease. Due to the characteristics of the imaging modality used, the images have speckle noise which worsens the image quality. Adaptive homomorphic filtering with wavelet and contourlet transforms, Levy Shrink, gamma distribution were used for image denoising. Learning-based neural network approaches for denoising give better edge preservation. Domain knowledge-based segmentation approaches have proved to provide more accurate intima-media thickness measurements. There is a requirement of useful fully automatic segmentation approaches, 3D, 4D systems, and plaque motion analysis. Taking into consideration the image priors like geometry, imaging physics, intensity and temporal data, image analysis has to be performed. Encouragingly more research has focused on content-specific segmentation and classification techniques. With the evaluation of machine learning algorithms, classifying the image as with or without a fat deposit has gained better accuracy and sensitivity. Machine learning-based approaches like self-organizing map, k-nearest neighborhood and support vector machine achieve promising accuracy and sensitivity in classification. The literature reveals that there is more scope in identifying a patient-specific model in a fully automatic manner.

DeSpecNet: a CNN-based method for speckle reduction in retinal optical coherence tomography images

Speckle is a major quality degrading factor in optical coherence tomography (OCT) images. In this work we propose a new deep learning network for speckle reduction in retinal OCT images, termed DeSpecNet. Unlike traditional algorithms, the model can learn from training data instead of manually selecting parameters such as noise level. The proposed deep convolutional neural network (CNN) applies strategies including residual learning, shortcut connection, batch normalization and leaky rectified linear units to achieve good despeckling performance. Application of the proposed method to the OCT images shows great improvement in both visual quality and quantitative indices. The proposed method provides good generalization ability for different types of retinal OCT images. It outperforms state-of-the-art methods in suppressing speckles and revealing subtle features while preserving edges.

Deep Learning in the Biomedical Applications: Recent and Future Status

Deep neural networks represent, nowadays, the most effective machine learning technology in biomedical domain. In this domain, the different areas of interest concern the Omics (study of the genome—genomics—and proteins—transcriptomics, proteomics, and metabolomics), bioimaging (study of biological cell and tissue), medical imaging (study of the human organs by creating visual representations), BBMI (study of the brain and body machine interface) and public and medical health management (PmHM). This paper reviews the major deep learning concepts pertinent to such biomedical applications. Concise overviews are provided for the Omics and the BBMI. We end our analysis with a critical discussion, interpretation and relevant open challenges.