Phantomless Auto-Calibration and Online Calibration Assessment for a Tracked Freehand 2-D Ultrasound Probe

Sensorless End-to-End Freehand 3-D Ultrasound Reconstruction With Physics-Guided Deep Learning

Three-dimensional ultrasound (3-D US) imaging with freehand scanning is utilized in cardiac, obstetric, abdominal, and vascular examinations. While 3-D US using either a "wobbler" or "matrix" transducer suffers from a small field of view and low acquisition rates, freehand scanning offers significant advantages due to its ease of use. However, current 3-D US volumetric reconstruction methods with freehand sweeps are limited by imaging plane shifts along the scanning path, i.e., out-of-plane (OOP) motion. Prior studies have incorporated motion sensors attached to the transducer, which is cumbersome and inconvenient in a clinical setting. Recent work has introduced deep neural networks (DNNs) with 3-D convolutions to estimate the position of imaging planes from a series of input frames. These approaches, however, fall short for estimating OOP motion. The goal of this article is to bridge the gap by designing a novel, physics-inspired DNN for freehand 3-D US reconstruction without motion sensors, aiming to improve the reconstruction quality and, at the same time, to reduce computational resources needed for training and inference. To this end, we present our physics-guided learning-based prediction of pose information (PLPPI) model for 3-D freehand US reconstruction without 3-D convolution. PLPPI yields significantly more accurate reconstructions and offers a major reduction in computation time. It attains a performance increase in the double digits in terms of mean percentage error, with up to 106% speedup and 131% reduction in graphic processing unit (GPU) memory usage, when compared to the latest deep learning methods.

Blind Image Quality Assessment for Pathological Microscopic Image Under Screen and Immersion Scenarios

The high-quality pathological microscopic images are essential for physicians or pathologists to make a correct diagnosis. Image quality assessment (IQA) can quantify the visual distortion degree of images and guide the imaging system to improve image quality, thus raising the quality of pathological microscopic images. Current IQA methods are not ideal for pathological microscopy images due to their specificity. In this paper, we present deep learning-based blind image quality assessment model with saliency block and patch block for pathological microscopic images. The saliency block and patch block can handle the local and global distortions, respectively. To better capture the area of interest of pathologists when viewing pathological images, the saliency block is fine-tuned by eye movement data of pathologists. The patch block can capture lots of global information strongly related to image quality via the interaction between different image patches from different positions. The performance of the developed model is validated by the home-made Pathological Microscopic Image Quality Database under Screen and Immersion Scenarios (PMIQD-SIS) and cross-validated by the five public datasets. The results of ablation experiments demonstrate the contribution of the added blocks. The dataset and the corresponding code are publicly available at: https://github.com/mikugyf/PMIQD-SIS.

Coplanarity Constrained Ultrasound Probe Calibration Based on N-Wire Phantom

OBJECTIVE

N-wire phantom-based ultrasound probe calibration has been used widely in many freehand tracked ultrasound imaging systems. The calibration matrix is obtained by registering the coplanar point cloud in ultrasound space and non-coplanar point cloud in tracking sensor space based on the least squares method. This method is sensitive to outliers and loses the coplanar information of the fiducial points. In this article, we describe a coplanarity-constrained calibration algorithm focusing on these issues.

METHODS

We verified that the out-of-plane error along the oblique wire in the N-wire phantom followed a normal distribution and used it to remove the experimental outliers and fit the plane with the Levenberg-Marquardt algorithm. Then, we projected the points to the plane along the oblique wire. Coplanarity-constrained point cloud registration was used to calculate the transformation matrix.

RESULTS

Compared with the other two commonly used methods, our method had the best calibration precision and achieved 25% and 36% improvement of the mean calibration accuracy than the closed-form solution and in-plane error method respectively at depth 16. Experiments at different depths revealed that our algorithm had better performance in our setup.

CONCLUSION

Our proposed coplanarity-constrained calibration algorithm achieved significant improvement in both precision and accuracy compared with existing algorithms with the same N-wire phantom. It is expected that calibration accuracy will improve when the algorithm is applied to all other N-wire phantom-based calibration procedures.

Automatic spatial calibration of freehand ultrasound probe with a multilayer N-wire phantom

The classic N-wire phantom has been widely used in the calibration of freehand ultrasound probes. One of the main challenges of the phantom is accurately identifying N-fiducials in ultrasound images, especially with multiple N-wire structures. In this study, a method using a multilayer N-wire phantom for the automatic spatial calibration of ultrasound images is proposed. All dots in the ultrasound image are segmented, scored, and classified according to the unique geometric features of the multilayer N-wire phantom. A recognition method for identifying N-fiducials from the dots is proposed for calibrating the spatial transformation of the ultrasound probe. At depths of 9, 11, 13, and 15 cm, the reconstruction error of 50 points is 1.24 ± 0.16, 1.09 ± 0.06, 0.95 ± 0.08, 1.02 ± 0.05 mm, respectively. The reconstruction mockup test shows that the distance accuracy is 1.11 ± 0.82 mm at a depth of 15 cm.

A novel ultrasound probe calibration method for multimodal image guidance of needle placement in cervical cancer brachytherapy

PURPOSE

Interstitial needles placement is a critical component of combined intracavitary/interstitial (IC/IS) brachytherapy (BT). To ensure precise placement of interstitial needles, we proposed a novel ultrasonic (US) probe calibration method to accurately register the US image in the magnetic resonance imaging (MRI) image and provide multimodal image guidance for needle placement.

METHODS

A wire-based calibration phantom combined with the stylus was developed for the calibration of US probe. The calibration phantom helps to quickly align the imaging plane of the US probe with the fiducial points to obtain US images of these points. The coordinates of fiducial points in US images were located automatically by feature extraction algorithms and were further corrected by the proposed correction method. Ingenious structures were designed on both sides of the calibration phantom to accurately obtain the coordinates of the fiducial points relative to the tracking device. Marker validation and pelvic phantom study were performed to evaluate the accuracy of the proposed calibration method.

RESULTS

In the marker validation, the US probe calibration method with corrected transformation achieves a registration accuracy of 0.694 ± 0.014 mm, and the uncorrected one is 0.746 ± 0.018 mm. In the pelvic phantom study, the needle tip difference was 1.096 ± 0.225 mm and trajectory difference was 1.416 ± 0.284 degrees.

CONCLUSION

The proposed US probe calibration method is helpful to achieve more accurate multimodality image guidance for needle placement.

Characterization of top 100 researches on e-waste based on bibliometric analysis

With rapid development of energy, information, and communication technology, e-waste problem has become one of the global issues to be settled urgently. The main features on publication years, journals, countries and institutions, authors, keywords, and content types of the 100 most-cited articles on e-waste had been unfolded in this research. The direction and way forward were illustrated, the trends to date were demonstrated, and the terrain and pathways were evaluated on the research of e-waste. Bibliometric analysis method was applied to analyze various attributes on the 100 most-cited articles which were retrieved from WoSCC on May 25, 2021, by utilizing the software tools Microsoft Excel 2016 and VOS viewer 1.6.9. The publication year and citation number of the 100 articles ranged between 2003 and 2017 and from 83 to 925, respectively. Environmental Science & Technology (n=17) published the maximum articles. Waste Management, Journal of Hazardous Materials, and Environmental Science & Technology were the core journals on e-waste. One hundred twenty-three institutions and 25 countries were involved in publishing the 100 articles. Three hundred seventy authors contributed to the 100 articles in total. A total of 267 keywords occurred in the 100 articles. The keywords "e-waste" and "recycling" held the highest occurrences. The study content of the 100 articles could be classified into four types including the characteristic-and-property type, the environment-and-health type, the management-and-economic type, and the technique-and-processing type. Overall completeness and applicability of the evidence found in this study were verified sufficiently; the potential biases in the review process were also considered. The innovations of the research from the past bibliometric analysis work on e-waste were stated, and the implication for practice and research of this study were explained as well. 2007, 2008, and 2009 were a peak of the researches on e-waste, while the recent years were experiencing a valley. China and its institutions were most influential in this field on e-waste. India was becoming more and more influential on e-waste research in the world. Nigeria was the research center in Africa, and Brazil was the research center in Latin America. Wong Minghung was the most important expert on e-waste. The impact on environment and human being's health was the hot topic of researches on e-waste; the characteristic and property of e-waste were studied not enough. The researches of technique and processing would be the direction and way forward in the study field on e-waste. The characteristic and property on e-waste would need more attention to be researched. The researchers could develop new pathways based on and beyond the four types of content evaluated in this research.

Integrated multi-modality image-guided navigation for neurosurgery: open-source software platform using state-of-the-art clinical hardware

PURPOSE

Image-guided surgery (IGS) is an integral part of modern neuro-oncology surgery. Navigated ultrasound provides the surgeon with reconstructed views of ultrasound data, but no commercial system presently permits its integration with other essential non-imaging-based intraoperative monitoring modalities such as intraoperative neuromonitoring. Such a system would be particularly useful in skull base neurosurgery.

METHODS

We established functional and technical requirements of an integrated multi-modality IGS system tailored for skull base surgery with the ability to incorporate: (1) preoperative MRI data and associated 3D volume reconstructions, (2) real-time intraoperative neurophysiological data and (3) live reconstructed 3D ultrasound. We created an open-source software platform to integrate with readily available commercial hardware. We tested the accuracy of the system's ultrasound navigation and reconstruction using a polyvinyl alcohol phantom model and simulated the use of the complete navigation system in a clinical operating room using a patient-specific phantom model.

RESULTS

Experimental validation of the system's navigated ultrasound component demonstrated accuracy of [Formula: see text] and a frame rate of 25 frames per second. Clinical simulation confirmed that system assembly was straightforward, could be achieved in a clinically acceptable time of [Formula: see text] and performed with a clinically acceptable level of accuracy.

CONCLUSION

We present an integrated open-source research platform for multi-modality IGS. The present prototype system was tailored for neurosurgery and met all minimum design requirements focused on skull base surgery. Future work aims to optimise the system further by addressing the remaining target requirements.

Verification of Guiding Needle Placement by Registered Ultrasound Image During Combined Intracavitary/Interstitial Gynecologic Brachytherapy

Purpose

Our previous research demonstrated that under ideal conditions, rigid registration between MRI images and US images had high accuracy for real-time image guidance. The work presented in this paper focused on the application of the previously established procedures to a new context, including preoperative CT images.

Materials and Methods

We used a template to calibrate the US probe and completed the registration between preoperative CT images and US images. Marker experiments on the accuracy of real-time needle trajectories in CT images were performed using micro electromagnetic sensors. Pelvic phantom experiments were carried out to test the registration accuracy between CT and US images, in addition to registration accuracy between US images and real-time needle trajectories (real-time space model).

Results

The US probe calibration error in CT images was 0.879 ± 0.149 mm. The difference of registration between US images and CT images was 0.935 ± 0.166 mm in the axial plane (n = 30) and 0.916 ± 0.143 mm in the sagittal plane (n =12). The difference of registration between US images and the needle’s real-time trajectories was 0.951 ± 0.202 mm.

Conclusion

Under ideal conditions, rigid registration between CT images and US images had high accuracy for real-time image guidance.

The Essential Role of Open Data and Software for the Future of Ultrasound-Based Neuronavigation

With the recent developments in machine learning and modern graphics processing units (GPUs), there is a marked shift in the way intra-operative ultrasound (iUS) images can be processed and presented during surgery. Real-time processing of images to highlight important anatomical structures combined with in-situ display, has the potential to greatly facilitate the acquisition and interpretation of iUS images when guiding an operation. In order to take full advantage of the recent advances in machine learning, large amounts of high-quality annotated training data are necessary to develop and validate the algorithms. To ensure efficient collection of a sufficient number of patient images and external validity of the models, training data should be collected at several centers by different neurosurgeons, and stored in a standard format directly compatible with the most commonly used machine learning toolkits and libraries. In this paper, we argue that such effort to collect and organize large-scale multi-center datasets should be based on common open source software and databases. We first describe the development of existing open-source ultrasound based neuronavigation systems and how these systems have contributed to enhanced neurosurgical guidance over the last 15 years. We review the impact of the large number of projects worldwide that have benefited from the publicly available datasets “Brain Images of Tumors for Evaluation” (BITE) and “Retrospective evaluation of Cerebral Tumors” (RESECT) that include MR and US data from brain tumor cases. We also describe the need for continuous data collection and how this effort can be organized through the use of a well-adapted and user-friendly open-source software platform that integrates both continually improved guidance and automated data collection functionalities.

Verification of needle guidance accuracy in pelvic phantom using registered ultrasound and MRI images for intracavitary/interstitial gynecologic brachytherapy

Purpose

In combined intracavitary/interstitial (IC/IS) gynecologic brachytherapy, trackers attached to interstitial needles of localize real-time needle trajectories, and intraoperative ultrasound (US) images provide updated anatomy information during needle insertions. To achieve an effective visualization and image guidance, real-time needle trajectories and US images can be unified in preoperative magnetic resonance imaging (MRI) image space together. This study evaluates the rigid registration accuracy between US images and MRI images as well as the registration accuracy between US images and real-time needle trajectories in a pelvic phantom.

Material and methods

A method for US probe calibration and accomplished rigid registration between MRI images and US images was proposed. An IC/IS applicator was designed. Micro electromagnetic sensor to track and localize real-time needle trajectories in 3D MRI image space was used. Marker validation to test the accuracy of US probe calibration and pelvic phantom validation to test the registration accuracy between US images and MRI images was conducted as well as and pelvic phantom study to verify the registration accuracy between real-time needle trajectories and needle trajectories in registered US images.

Results

US probe calibration accuracy was 0.80 ±0.23 mm (n = 60). Registration accuracy between US images and MRI images were 1.01 ±0.22 mm in the axial plane (n = 60) and 1.14 ±0.20 mm in the sagittal plane (n = 24). Registration accuracy between real-time needle trajectories and needle trajectories in registered US images were 1.25 ±0.31 mm (n = 40) and 1.61 ±0.28 degrees (n = 5), respectively.

Conclusions

In this study, we showed that under ideal conditions, rigid registration between MRI images and US images obtained high accuracy for real-time image guidance. Additionally, registered US images provided accurate image guidance during visual needle insertion in IC/IS gynecologic brachytherapy to achieve a combination of effective visualization and image guidance.

Efficient image based method using water-filled balloons for improving probe spatial calibration in 3D freehand ultrasonography

The ultrasound (US) probe spatial calibration is a key prerequisite for enabling the use of the 3D freehand US technique. Several methods have been proposed for achieving an accurate and precise calibration, although these methods still require specialised equipment. This equipment is often not available in research or clinical facilities. Therefore, the present investigation aimed to propose an efficient US probe calibration method that is accessible in terms of cost, easy to apply and capable of achieving results suitable for clinical applications. The data acquisition was carried out by performing two perpendicular US sweeps over water filled balloon phantoms. The data analysis was carried out by computing the similarity measures between 2D images from the first sweep and the corresponding images of the 3D reconstruction of the second sweep. These measures were maximized by using the Nelder-Mead algorithm, to find the optimal solution for the calibration parameters. The calibration results were evaluated in terms of accuracy and precision by comparing known phantom geometries with those extracted from the US images. The accuracy and the precision after applying the calibration method were improved. By using the parameters obtained from the plane phantom method as initialization of the calibration parameters, the accuracy and the precision in the best scenario was 0.4 mm and 1.5 mm, respectively. These results were in line with the methods requiring specialised equipment. However, the applied method was unable to consistently produce this level of accuracy and precision. The calibration parameters were also tested in a musculoskeletal application, revealing sufficient matching of the relevant anatomical features when multiple US sweeps are combined in a 3D reconstruction. To improve the current results and increase the reproducibility of this research, the developed software is made available.

Deformable multimodal registration for navigation in beating-heart cardiac surgery

PURPOSE

Minimally invasive beating-heart surgery is currently performed using endoscopes and without navigation. Registration of intraoperative ultrasound to a preoperative cardiac CT scan is a valuable step toward image-guided navigation.

METHODS

The registration was achieved by first extracting a representative point set from each ultrasound image in the sequence using a deformable registration. A template shape representing the cardiac chambers was deformed through a hierarchy of affine transformations to match each ultrasound image using a generalized expectation maximization algorithm. These extracted point sets were matched to the CT by exhaustively searching over a large number of precomputed slices of 3D geometry. The result is a similarity transformation mapping the intraoperative ultrasound to preoperative CT.

RESULTS

Complete data sets were acquired for four patients. Transesophageal echocardiography ultrasound sequences were deformably registered to a model of oriented points with a mean error of 2.3 mm. Ultrasound and CT scans were registered to a mean of 3 mm, which is comparable to the error of 2.8 mm expected by merging ultrasound registration with uncertainty of cardiac CT.

CONCLUSION

The proposed algorithm registered 3D CT with dynamic 2D intraoperative imaging. The algorithm aligned the images in both space and time, needing neither dynamic CT imaging nor intraoperative electrocardiograms. The accuracy was sufficient for navigation in thoracoscopically guided beating-heart surgery.

3D freehand ultrasound without external tracking using deep learning

This work aims at creating 3D freehand ultrasound reconstructions from 2D probes with image-based tracking, therefore not requiring expensive or cumbersome external tracking hardware. Existing model-based approaches such as speckle decorrelation only partially capture the underlying complexity of ultrasound image formation, thus producing reconstruction accuracies incompatible with current clinical requirements. Here, we introduce an alternative approach that relies on a statistical analysis rather than physical models, and use a convolutional neural network (CNN) to directly estimate the motion of successive ultrasound frames in an end-to-end fashion. We demonstrate how this technique is related to prior approaches, and derive how to further improve its predictive capabilities by incorporating additional information such as data from inertial measurement units (IMU). This novel method is thoroughly evaluated and analyzed on a dataset of 800 in vivo ultrasound sweeps, yielding unprecedentedly accurate reconstructions with a median normalized drift of 5.2%. Even on long sweeps exceeding 20 cm with complex trajectories, this allows to obtain length measurements with median errors of 3.4%, hence paving the way toward translation into clinical routine.