Evaluation and comparison of current biopsy needle localization and tracking methods using 3D ultrasound

Optimization of reconstruction time of ultrasound computed tomography with a piecewise homogeneous region-based refract-ray model

In this article, a novel ultrasound computed tomography (USCT) reconstruction algorithm for breast imaging is proposed. This algorithm is based on an ultrasound propagation model, the refract-ray model (RRM). In this model, the field of imaging is assumed as piecewise homogenous and is divided into several regions. The ultrasound propagation paths are considered polylines that only refract at the borders of the regions. The edge information is provided by B-mode imaging. Both simulations and experiments are implemented to validate the proposed algorithm. Compared with the traditional bent-ray model (BRM), the time of reconstructions using RRM decreases by over 90 %. In simulations, the imaging qualities for RRM and BRM are comparable, in terms of the root mean square error, the Tenengrad value, and the deformation of digital phantom. In the experiments, a cylindrical agar phantom is imaged using a customized imaging system. When imaging using RRM, the estimate of the phantom radius is about 0.1 mm in error, while it is about 0.3 mm in error using BRM. Moreover, the Tenengrad value of the result using RRM is much higher than that using BRM (9.76 compared to 0.79). The results show that the proposed algorithm can better delineate the phantom within a water bath. In future work, further experimental work is required to validate the method for improving imaging quality under breast-mimicking imaging conditions.

Medical instrument detection in ultrasound: a review

Medical instrument detection is essential for computer-assisted interventions, since it facilitates clinicians to find instruments efficiently with a better interpretation, thereby improving clinical outcomes. This article reviews image-based medical instrument detection methods for ultrasound-guided (US-guided) operations. Literature is selected based on an exhaustive search in different sources, including Google Scholar, PubMed, and Scopus. We first discuss the key clinical applications of medical instrument detection in the US, including delivering regional anesthesia, biopsy taking, prostate brachytherapy, and catheterization. Then, we present a comprehensive review of instrument detection methodologies, including non-machine-learning and machine-learning methods. The conventional non-machine-learning methods were extensively studied before the era of machine learning methods. The principal issues and potential research directions for future studies are summarized for the computer-assisted intervention community. In conclusion, although promising results have been obtained by the current (non-) machine learning methods for different clinical applications, thorough clinical validations are still required.

Biopsy Needle Segmentation using Deep Networks on inhomogeneous Ultrasound Images

In minimally invasive interventional surgery, ultrasound imaging is usually used to provide real-time feedback in order to obtain the best diagnostic results or realize treatment plans, so how to accurately obtain the position of the medical biopsy needle is a problem worthy of study. 2D ultrasound simulation images containing the medical biopsy needle are generated, and our images background is from the real breast ultrasound image. Based on the deep learning network, the images containing the medical biopsy needle are used to analyze the effectiveness of different networks for needle localization for the purpose of returning needle positions in non-uniform ultrasound images. The results show that attention U-Net performed best and can accurately reflect the real position of the medical biopsy needle. The IoU and Precision can reach 90.19% and 96.25%, and the Angular Error is 0.40°. Clinical Relevance- Based on the deep network, for 2D ultrasound images containing medical biopsy needle, the localization precision can reach 96.25% and the Angular Error is 0.40°.

The Revisited Frequency-Shift Method for Shear Wave Attenuation Computation and Imaging

Ultrasound (US) shear wave (SW) elastography has been widely studied and implemented on clinical systems to assess the elasticity of living organs. Imaging of SW attenuation reflecting viscous properties of tissues has received less attention. A revisited frequency shift (R-FS) method is proposed to improve the robustness of SW attenuation imaging. Performances are compared with the FS method that we originally proposed and with the two-point frequency shift (2P-FS) and attenuation measuring US SW elastography (AMUSE) methods. In the proposed R-FS method, the shape parameter of the gamma distribution fitting SW spectra is assumed to vary with distance, in contrast to FS. Second, an adaptive random sample consensus (A-RANSAC) line fitting method is used to prevent outlier attenuation values in the presence of noise. Validation was made on ten simulated phantoms with two viscosities (0.5 and 2 Pa [Formula: see text]) and different noise levels (15 to -5 dB), two experimental homogeneous gel phantoms, and six in vivo liver acquisitions on awake ducks (including three normal and three fatty duck livers). According to the conducted experiments, R-FS revealed mean reductions in coefficients of variation (CV) of 62.6% on simulations, 62.5% with phantoms, and 62.3% in vivo compared with FS. Corresponding reductions compared with 2P-FS were 45.4%, 77.1%, and 62.0%, respectively. Reductions in normalized root-mean-square errors for simulations were 63.9% and 48.7% with respect to FS and 2P-FS, respectively.

Multi-Model Adaptation Learning With Possibilistic Clustering Assumption for EEG-Based Emotion Recognition

In machine learning community, graph-based semi-supervised learning (GSSL) approaches have attracted more extensive research due to their elegant mathematical formulation and good performance. However, one of the reasons affecting the performance of the GSSL method is that the training data and test data need to be independently identically distributed (IID); any individual user may show a completely different encephalogram (EEG) data in the same situation. The EEG data may be non-IID. In addition, noise/outlier sensitiveness still exist in GSSL approaches. To these ends, we propose in this paper a novel clustering method based on structure risk minimization model, called multi-model adaptation learning with possibilistic clustering assumption for EEG-based emotion recognition (MA-PCA). It can effectively minimize the influence from the noise/outlier samples based on different EEG-based data distribution in some reproduced kernel Hilbert space. Our main ideas are as follows: (1) reducing the negative impact of noise/outlier patterns through fuzzy entropy regularization, (2) considering the training data and test data are IID and non-IID to obtain a better performance by multi-model adaptation learning, and (3) the algorithm implementation and convergence theorem are also given. A large number of experiments and deep analysis on real DEAP datasets and SEED datasets was carried out. The results show that the MA-PCA method has superior or comparable robustness and generalization performance to EEG-based emotion recognition.

Automatic and accurate needle detection in 2D ultrasound during robot-assisted needle insertion process

PURPOSE

Robot-assisted needle insertion guided by 2D ultrasound (US) can effectively improve the accuracy and success rate of clinical puncture. To this end, automatic and accurate needle-tracking methods are important for monitoring puncture processes, avoiding the needle deviating from the intended path, and reducing the risk of injury to surrounding tissues. This work aims to develop a framework for automatic and accurate detection of an inserted needle in 2D US image during the insertion process.

METHODS

We propose a novel convolutional neural network architecture comprising of a two-channel encoder and single-channel decoder for needle segmentation using needle motion information extracted from two adjacent US image frames. Based on the novel network, we further propose an automatic needle detection framework. According to the prediction result of the previous frame, a region of interest of the needle in the US image was extracted and fed into the proposed network to achieve finer and faster continuous needle localization.

RESULTS

The performance of our method was evaluated based on 1000 pairs of US images extracted from robot-assisted needle insertions on freshly excised bovine and porcine tissues. The needle segmentation network achieved 99.7% accuracy, 86.2% precision, 89.1% recall, and an F₁-score of 0.87. The needle detection framework successfully localized the needle with a mean tip error of 0.45 ± 0.33 mm and a mean orientation error of 0.42° ± 0.34° and achieved a total processing time of 50 ms per image.

CONCLUSION

The proposed framework demonstrated the capability to realize robust, accurate, and real-time needle localization during robot-assisted needle insertion processes. It has a promising application in tracking the needle and ensuring the safety of robotic-assisted automatic puncture during challenging US-guided minimally invasive procedures.

An Automatic Biopsy Needle Detection and Segmentation on Ultrasound Images Using a Convolutional Neural Network

Needle visualization in the ultrasound image is essential to successfully perform the ultrasound-guided core needle biopsy. Automatic needle detection can significantly reduce the procedure time, false-negative rate, and highly improve the diagnosis. In this paper, we present a CNN-based, fully automatic method for detection of core needle in 2D ultrasound images. Adaptive moment estimation optimizer is proposed as CNN architecture. Radon transform is applied to locate the needle. The network's model was trained and tested on the total of 619 2D images from 91 cases of breast cancer. The model has achieved an average weighted intersection over union (the weighted Jaccard Index) of 0.986, F1 Score of 0.768, and angle RMSE of 3.73°. The obtained results exceed the other solutions by at least 0.27 and 7° in case of F1 score and angle RMSE, respectively. Finally, the needle is detected in a single frame averagely in 21.6 ms on a modern PC.

Efficient Medical Instrument Detection in 3D Volumetric Ultrasound Data

Ultrasound-guided procedures have been applied in many clinical therapies, such as cardiac catheterization and regional anesthesia. Medical instrument detection in 3D Ultrasound (US) is highly desired, but the existing approaches are far from real-time performance. Our objective is to investigate an efficient instrument detection method in 3D US for practical clinical use. We propose a novel Multi-dimensional Mixed Network for efficient instrument detection in 3D US, which extracts the discriminating features at 3D full-image level by a 3D encoder, and then applies a specially designed dimension reduction block to reduce the spatial complexity of the feature maps by projecting from 3D space into 2D space. A 2D decoder is adopted to detect the instrument along the specified axes. By projecting the predicted 2D outputs, the instrument is detected or visualized in the 3D volume. Furthermore, to enable the network to better learn the discriminative information, we propose a multi-level loss function to capture both pixel- and image-level differences. We carried out extensive experiments on two datasets for two tasks: (1) catheter detection for cardiac RF-ablation and (2) needle detection for regional anesthesia. Our experiments show that our proposed method achieves a detection error of 2-3 voxels with an efficiency of about 0.12 sec per 3D US volume. The proposed method is 3-8 times faster than the state-of-the-art methods, leading to real-time performance. The results show that our proposed method has significant clinical value for real-time 3D US-guided intervention.

Multi-Needle Detection in 3D Ultrasound Images Using Unsupervised Order-Graph Regularized Sparse Dictionary Learning

Accurate and automatic multi-needle detection in three-dimensional (3D) ultrasound (US) is a key step of treatment planning for US-guided brachytherapy. However, most current studies are concentrated on single-needle detection by only using a small number of images with a needle, regardless of the massive database of US images without needles. In this paper, we propose a workflow for multi-needle detection by considering the images without needles as auxiliary. Concretely, we train position-specific dictionaries on 3D overlapping patches of auxiliary images, where we develop an enhanced sparse dictionary learning method by integrating spatial continuity of 3D US, dubbed order-graph regularized dictionary learning. Using the learned dictionaries, target images are reconstructed to obtain residual pixels which are then clustered in every slice to yield centers. With the obtained centers, regions of interest (ROIs) are constructed via seeking cylinders. Finally, we detect needles by using the random sample consensus algorithm per ROI and then locate the tips by finding the sharp intensity drops along the detected axis for every needle. Extensive experiments were conducted on a phantom dataset and a prostate dataset of 70/21 patients without/with needles. Visualization and quantitative results show the effectiveness of our proposed workflow. Specifically, our method can correctly detect 95% of needles with a tip location error of 1.01 mm on the prostate dataset. This technique provides accurate multi-needle detection for US-guided HDR prostate brachytherapy, facilitating the clinical workflow.

Three-dimensional therapy needle applicator segmentation for ultrasound-guided focal liver ablation

PURPOSE

Minimally invasive procedures, such as microwave ablation, are becoming first-line treatment options for early-stage liver cancer due to lower complication rates and shorter recovery times than conventional surgical techniques. Although these procedures are promising, one reason preventing widespread adoption is inadequate local tumor ablation leading to observations of higher local cancer recurrence compared to conventional procedures. Poor ablation coverage has been associated with two-dimensional (2D) ultrasound (US) guidance of the therapy needle applicators and has stimulated investigation into the use of three-dimensional (3D) US imaging for these procedures. We have developed a supervised 3D US needle applicator segmentation algorithm using a single user input to augment the addition of 3D US to the current focal liver tumor ablation workflow with the goals of identifying and improving needle applicator localization efficiency.

METHODS

The algorithm is initialized by creating a spherical search space of line segments around a manually chosen seed point that is selected by a user on the needle applicator visualized in a 3D US image. The most probable trajectory is chosen by maximizing the count and intensity of threshold voxels along a line segment and is filtered using the Otsu method to determine the tip location. Homogeneous tissue mimicking phantom images containing needle applicators were used to optimize the parameters of the algorithm prior to a four-user investigation on retrospective 3D US images of patients who underwent microwave ablation for liver cancer. Trajectory, axis localization, and tip errors were computed based on comparisons to manual segmentations in 3D US images.

RESULTS

Segmentation of needle applicators in ten phantom 3D US images was optimized to median (Q1, Q3) trajectory, axis, and tip errors of 2.1 (1.1, 3.6)°, 1.3 (0.8, 2.1) mm, and 1.3 (0.7, 2.5) mm, respectively, with a mean ± SD segmentation computation time of 0.246 ± 0.007 s. Use of the segmentation method with a 16 in vivo 3D US patient dataset resulted in median (Q1, Q3) trajectory, axis, and tip errors of 4.5 (2.4, 5.2)°, 1.9 (1.7, 2.1) mm, and 5.1 (2.2, 5.9) mm based on all users.

CONCLUSIONS

Segmentation of needle applicators in 3D US images during minimally invasive liver cancer therapeutic procedures could provide a utility that enables enhanced needle applicator guidance, placement verification, and improved clinical workflow. A semi-automated 3D US needle applicator segmentation algorithm used in vivo demonstrated localization of the visualized trajectory and tip with less than 5° and 5.2 mm errors, respectively, in less than 0.31 s. This offers the ability to assess and adjust needle applicator placements intraoperatively to potentially decrease the observed liver cancer recurrence rates associated with current ablation procedures. Although optimized for deep and oblique angle needle applicator insertions, this proposed workflow has the potential to be altered for a variety of image-guided minimally invasive procedures to improve localization and verification of therapy needle applicators intraoperatively.

Antlion optimizer-ANFIS load frequency control for multi-interconnected plants comprising photovoltaic and wind turbine

This paper proposes optimal load frequency control (LFC) designed by Adaptive Neuro Fuzzy Inference System (ANFIS) trained via antlion optimizer (ALO) for multi-interconnected system comprising renewable energy sources (RESs). Two systems are modeled and investigated; the first one has two plants of grid connected photovoltaic (PV) system with maximum power point tracker (MPPT) and thermal plant while the second comprises four plants of thermal, wind turbine and grid connected PV systems. ALO is employed to get the optimal gains of Proportional-Integral (PI) controller such that the integral time absolute error (ITAE) of frequency and tie line power deviations is minimized. The input and output of the optimized PI controller are used to train the ANFIS-LFC with Gaussian surface membership functions. Different load disturbances are studied and the results are compared with other reported approaches. The obtained results confirmed the accuracy and reliability of the proposed approach in designing LFC for multi-interconnected power systems.

Sliding mode control and sampling rate strategy for Networked control systems with packet disordering via Markov chain prediction

This paper investigates sliding mode control combined with sampling rate control for networked control systems subject to packet disordering via Markov chain prediction. The main objectives of the proposed method are to predict the probability of the occurrence of packet disordering when packet disordering is unknown in the networks, to control sampling rate to restrain heavy packet disordering, and to stabilize the Markovian jump system with variable parameters by sliding mode techniques. Firstly, an argument system with sampling rate and a plant state is established. Then, the Networked control system based on Markov chain probability prediction and statistical analysis of this probability is modeled as a Markovian jump system with two Markov chains. Next, sliding mode controller is designed to stabilize the dynamic Markovian jump system. Finally, experiments are conducted to illustrate the effectiveness and benefits of proposed method.

A hybrid camera- and ultrasound-based approach for needle localization and tracking using a 3D motorized curvilinear ultrasound probe

Three-dimensional (3D) motorized curvilinear ultrasound probes provide an effective, low-cost tool to guide needle interventions, but localizing and tracking the needle in 3D ultrasound volumes is often challenging. In this study, a new method is introduced to localize and track the needle using 3D motorized curvilinear ultrasound probes. In particular, a low-cost camera mounted on the probe is employed to estimate the needle axis. The camera-estimated axis is used to identify a volume of interest (VOI) in the ultrasound volume that enables high needle visibility. This VOI is analyzed using local phase analysis and the random sample consensus algorithm to refine the camera-estimated needle axis. The needle tip is determined by searching the localized needle axis using a probabilistic approach. Dynamic needle tracking in a sequence of 3D ultrasound volumes is enabled by iteratively applying a Kalman filter to estimate the VOI that includes the needle in the successive ultrasound volume and limiting the localization analysis to this VOI. A series of ex vivo animal experiments are conducted to evaluate the accuracy of needle localization and tracking. The results show that the proposed method can localize the needle in individual ultrasound volumes with maximum error rates of 0.7 mm for the needle axis, 1.7° for the needle angle, and 1.2 mm for the needle tip. Moreover, the proposed method can track the needle in a sequence of ultrasound volumes with maximum error rates of 1.0 mm for the needle axis, 2.0° for the needle angle, and 1.7 mm for the needle tip. These results suggest the feasibility of applying the proposed method to localize and track the needle using 3D motorized curvilinear ultrasound probes.

CASPER: computer-aided segmentation of imperceptible motion-a learning-based tracking of an invisible needle in ultrasound

PURPOSE

This paper presents a new micro-motion-based approach to track a needle in ultrasound images captured by a handheld transducer.

METHODS

We propose a novel learning-based framework to track a handheld needle by detecting microscale variations of motion dynamics over time. The current state of the art on using motion analysis for needle detection uses absolute motion and hence work well only when the transducer is static. We have introduced and evaluated novel spatiotemporal and spectral features, obtained from the phase image, in a self-supervised tracking framework to improve the detection accuracy in the subsequent frames using incremental training. Our proposed tracking method involves volumetric feature selection and differential flow analysis to incorporate the neighboring pixels and mitigate the effects of the subtle tremor motion of a handheld transducer. To evaluate the detection accuracy, the method is tested on porcine tissue in-vivo, during the needle insertion in the biceps femoris muscle.

RESULTS

Experimental results show the mean, standard deviation and root-mean-square errors of [Formula: see text], [Formula: see text] and [Formula: see text] in the insertion angle, and 0.82, 1.21, 1.47 mm, in the needle tip, respectively.

CONCLUSIONS

Compared to the appearance-based detection approaches, the proposed method is especially suitable for needles with ultrasonic characteristics that are imperceptible in the static image and to the naked eye.

Ultrasound-Guided Biopsies of Bone Lesions Without Cortical Disruption Using Fusion Imaging and Needle Tracking: Proof of Concept

OBJECTIVE

To assess the technical feasibility and safety of combined fusion imaging and needle tracking under ultrasound guidance to target bone lesions without cortical disruption.

MATERIALS AND METHODS

Between January 2016 and March 2016, seven patients underwent US-guided biopsy of bone lesions without cortical disruption. Targeted bone lesions were measuring more than 1.5 cm with a thin cortex, a trans-osseous pathway not exceeding 2 cm and without any adjacent vulnerable structures. First three procedures were performed in the CT suite to aid the needle tracking where necessary (group 1), the remaining four procedures were performed in the US suite (group 2). In group 1, deviation from the real position of the bone trocar (estimated on CT) was compared to the virtual position (estimated on the fusion CT-US images). In both group, procedure data and histopathological results were collected, and compared to the suspected diagnosis and follow-up.

RESULTS

Mean procedure duration was 44 min. Total number of synchronisation points for combined fusion imaging were 3.3 on average. In group 1, mean deviation between the virtual and real CT coordinates was 5.3 mm on average. All biopsies yielded adequate quality analysable bone sample. Histopathological analysis revealed malignancy in three cases, non-specific inflammation in two cases, and normal bone in two cases. The four benign results were confirmed as true negative results. There were no immediate or post-procedural complications.

CONCLUSION

The use of combined fusion imaging and needle tracking ultrasound guidance to target bone lesions without cortical disruption seems technically feasible, provided the patient and lesion selection is appropriate.