Robot-assisted augmented reality surgical navigation based on optical tracking for mandibular reconstruction surgery

PURPOSE

This work proposes a robot-assisted augmented reality (AR) surgical navigation system for mandibular reconstruction. The system accurately superimposes the preoperative osteotomy plan of the mandible and fibula into a real scene. It assists the doctor in osteotomy quickly and safely under the guidance of the robotic arm.

METHODS

The proposed system mainly consists of two modules: the AR guidance module of the mandible and fibula and the robot navigation module. In the AR guidance module, we propose an AR calibration method based on the spatial registration of the image tracking marker to superimpose the virtual models of the mandible and fibula into the real scene. In the robot navigation module, the posture of the robotic arm is first calibrated under the tracking of the optical tracking system. The robotic arm can then be positioned at the planned osteotomy after the registration of the computed tomography image and the patient position. The combined guidance of AR and robotic arm can enhance the safety and precision of the surgery.

RESULTS

The effectiveness of the proposed system was quantitatively assessed on cadavers. In the AR guidance module, osteotomies of the mandible and fibula achieved mean errors of 1.61 ± 0.62 and 1.08 ± 0.28 mm, respectively. The mean reconstruction error of the mandible was 1.36 ± 0.22 mm. In the AR-robot guidance module, the mean osteotomy errors of the mandible and fibula were 1.47 ± 0.46 and 0.98 ± 0.24 mm, respectively. The mean reconstruction error of the mandible was 1.20 ± 0.36 mm.

CONCLUSIONS

The cadaveric experiments of 12 fibulas and six mandibles demonstrate the proposed system's effectiveness and potential clinical value in reconstructing the mandibular defect with a free fibular flap.

A spatial registration method based on 2D-3D registration for an augmented reality spinal surgery navigation system

BACKGROUND

In order to provide accurate and reliable image guidance for augmented reality (AR) spinal surgery navigation, a spatial registration method has been proposed.

METHODS

In the AR spinal surgery navigation system, grayscale-based 2D/3D registration technology has been used to register preoperative computed tomography images with intraoperative X-ray images to complete the spatial registration, and then the fusion of virtual image and real spine has been realised.

RESULTS

In the image registration experiment, the success rate of spine model registration was 90%. In the spinal model verification experiment, the surface registration error of the spinal model ranged from 0.361 to 0.612 mm, and the total average surface registration error was 0.501 mm.

CONCLUSION

The spatial registration method based on 2D/3D registration technology can be used in AR spinal surgery navigation systems and is highly accurate and minimally invasive.

Relationship between Eccentricity and Volume Determined by Spectral Algorithms Applied to Spatially Registered Bi-Parametric MRI and Prostate Tumor Aggressiveness: A Pilot Study

(1) Background: Non-invasive prostate cancer assessments using multi-parametric MRI are essential to the reliable detection of lesions and proper management of patients. While current guidelines call for the administration of Gadolinium-containing intravenous contrast injections, eliminating such injections would simplify scanning and reduce patient risk and costs. However, augmented image analysis is necessary to extract important diagnostic information from MRIs. Purpose: This study aims to extend previous work on the signal to clutter ratio and test whether prostate tumor eccentricity and volume are indicators of tumor aggressiveness using bi-parametric (BP)-MRI. (2) Methods: This study retrospectively processed 42 consecutive prostate cancer patients from the PI-CAI data collection. BP-MRIs (apparent diffusion coefficient, high b-value, and T2 images) were resized, translated, cropped, and stitched to form spatially registered BP-MRIs. The International Society of Urological Pathology (ISUP) grade was used to judge cases of prostate cancer as either clinically significant prostate cancer (CsPCa) (ISUP ≥ 2) or clinically insignificant prostate cancer (CiPCa) (ISUP < 2). The Adaptive Cosine Estimator (ACE) algorithm was applied to the BP-MRIs, followed by thresholding, and then eccentricity and volume computations, from the labeled and blobbed detection maps. Then, univariate and multivariate linear regression fittings of eccentricity and volume were applied to the ISUP grade. The fits were quantitatively evaluated by computing correlation coefficients (R) and p-values. Area under the curve (AUC) and receiver operator characteristic (ROC) curve scores were used to assess the logistic fitting to CsPCa/CiPCa. (3) Results: Modest correlation coefficients (R) (>0.35) and AUC scores (0.70) for the linear and/or logistic fits from the processed prostate tumor eccentricity and volume computations for the spatially registered BP-MRIs exceeded fits using the parameters of prostate serum antigen, prostate volume, and patient age (R~0.17). (4) Conclusions: This is the first study that applied spectral approaches to BP-MRIs to generate tumor eccentricity and volume metrics to assess tumor aggressiveness. This study found significant values of R and AUC (albeit below those from multi-parametric MRI) to fit and relate the metrics to the ISUP grade and CsPCA/CiPCA, respectively.

Low-cost, smartphone-based instant three-dimensional registration system for infant functional near-infrared spectroscopy applications

Significance

To effectively apply functional near-infrared spectroscopy (fNIRS)/diffuse optical tomography (DOT) devices, a three-dimensional (3D) model of the position of each optode on a subject's scalp and the positions of that subject's cranial landmarks are critical. Obtaining this information accurately in infants, who rarely stop moving, is an ongoing challenge.

Aim

We propose a smartphone-based registration system that can potentially achieve a full-head 3D scan of a 6-month-old infant instantly.

Approach

The proposed system is remotely controlled by a custom-designed Bluetooth controller. The scanned images can either be manually or automatically aligned to generate a 3D head surface model.

Results

A full-head 3D scan of a 6-month-old infant can be achieved within 2 s via this system. In testing on a realistic but static infant head model, the average Euclidean error of optode position using this device was 1.8 mm.

Conclusions

This low-cost 3D registration system therefore has the potential to permit accurate and near-instant fNIRS/DOT spatial registration.

A New Spatial Registration Algorithm of Aerial Moving Platform to Sea Target Tracking

Spatial registration is the primary challenge affecting target tracking accuracy, especially for the aerial moving platform and sea target tracking. In this environment, it is important to account for both the errors in sensor observations and the variations in platform attitude. In order to solve the problem of complex types of errors in the tracking of sea targets by aerial moving platforms, a new spatial registration algorithm is proposed. Through separating and analyzing observation data, the influence of sensor observation error and attitude error on observation data is obtained, and a systematic error consistency matrix is established. Based on observation information from multiple platforms, accurate tracking of sea targets can be accomplished without estimating systematic error. In order to verify the effectiveness of the algorithm, we carried out simulation experiments and practical experiments on the lake, which showed that the new algorithm was more efficient than traditional algorithms.

A Method for Spatially Registered Microprofilometry Combining Intensity-Height Datasets from Interferometric Sensors

A recognized problem in profilometry applied to artworks is the spatial referencing of the surface topography at micrometer scale due to the lack of references in the height data with respect to the "visually readable" surface. We demonstrate a novel workflow for spatially referenced microprofilometry based on conoscopic holography sensors for scanning in situ heterogeneous artworks. The method combines the raw intensity signal collected by the single-point sensor and the (interferometric) height dataset, which are mutually registered. This dual dataset provides a surface topography registered to the artwork features up to the precision that is given by the acquisition scanning system (mainly, scan step and laser spot). The advantages are: (1) the raw signal map provides additional information about materials texture, e.g., color changes or artist marks, for spatial registration and data fusion tasks; (2) and microtexture information can be reliably processed for precision diagnostic tasks, e.g., surface metrology in specific sub-domains and multi-temporal monitoring. Proof of concept is given with exemplary applications: book heritage, 3D artifacts, surface treatments. The potential of the method is clear for both quantitative surface metrology and qualitative inspection of the morphology, and it is expected to open future applications for microprofilometry in heritage science.

A neuroscientist's guide to using murine brain atlases for efficient analysis and transparent reporting

Brain atlases are widely used in neuroscience as resources for conducting experimental studies, and for integrating, analyzing, and reporting data from animal models. A variety of atlases are available, and it may be challenging to find the optimal atlas for a given purpose and to perform efficient atlas-based data analyses. Comparing findings reported using different atlases is also not trivial, and represents a barrier to reproducible science. With this perspective article, we provide a guide to how mouse and rat brain atlases can be used for analyzing and reporting data in accordance with the FAIR principles that advocate for data to be findable, accessible, interoperable, and re-usable. We first introduce how atlases can be interpreted and used for navigating to brain locations, before discussing how they can be used for different analytic purposes, including spatial registration and data visualization. We provide guidance on how neuroscientists can compare data mapped to different atlases and ensure transparent reporting of findings. Finally, we summarize key considerations when choosing an atlas and give an outlook on the relevance of increased uptake of atlas-based tools and workflows for FAIR data sharing.

Combining Spatial Registration With Clutter Filtering for Power-Doppler Imaging in Peripheral Perfusion Applications

Power-Doppler ultrasonic (PD-US) imaging is sensitive to echoes from blood cell motion in the microvasculature but generally nonspecific because of difficulties with filtering nonblood-echo sources. We are studying the potential for using PD-US imaging for routine assessments of peripheral blood perfusion without contrast media. The strategy developed is based on an experimentally verified computational model of tissue perfusion that simulates typical in vivo conditions. The model considers directed and diffuse blood perfusion states in a field of moving clutter and noise. A spatial registration method is applied to minimize tissue motion prior to clutter and noise filtering. The results show that in-plane clutter motion is effectively minimized. While out-of-plane motion remains a strong source of clutter-filter leakage, those registration errors are readily minimized by straightforward modification of scanning techniques and spatial averaging.

Quantifying Differences Between Affine and Nonlinear Spatial Normalization of FP-CIT Spect Images

Spatial normalization helps us to compare quantitatively two or more input brain scans. Although using an affine normalization approach preserves the anatomical structures, the neuroimaging field is more common to find works that make use of nonlinear transformations. The main reason is that they facilitate a voxel-wise comparison, not only when studying functional images but also when comparing MRI scans given that they fit better to a reference template. However, the amount of bias introduced by the nonlinear transformations can potentially alter the final outcome of a diagnosis especially when studying functional scans for neurological disorders like Parkinson's Disease. In this context, we have tried to quantify the bias introduced by the affine and the nonlinear spatial registration of FP-CIT SPECT volumes of healthy control subjects and patients with PD. For that purpose, we calculated the deformation fields of each participant and applied these deformation fields to a 3D-grid. As the space between the edges of small cubes comprising the grid change, we can quantify which parts from the brain have been enlarged, compressed or just remain the same. When the nonlinear approach is applied, scans from PD patients show a region near their striatum very similar in shape to that of healthy subjects. This artificially increases the interclass separation between patients with PD and healthy subjects as the local intensity is decreased in the latter region, and leads machine learning systems to biased results due to the artificial information introduced by these deformations.

A robust automated surface-matching registration method for neuronavigation

PURPOSE

The surface-matching registration method in the current neuronavigation completes the coarse registration mainly by manually selecting anatomical landmarks, which increases the registration time, makes the automatic registration impossible and sometimes results in mismatch. It may be more practical to use a fast, accurate, and automatic spatial registration method for the patient-to-image registration.

METHODS

A coarse-to-fine spatial registration method to automatically register the patient space to the image space without placing any markers on the head of the patient was proposed. Three-dimensional (3D) keypoints were extracted by 3D Harris corner detector from the point clouds in the patient and image spaces, and used as input to the 4-points congruent sets (4PCS) algorithm which automatically registered the keypoints in the patient space with the keypoints in the image space without any assumptions about initial alignment. Coarsely aligned point clouds in the patient and image space were then fine-registered with a variant of the iterative closest point (ICP) algorithm. Two experiments were designed based on one phantom and five patients to validate the efficiency and effectiveness of the proposed method.

RESULTS

Keypoints were extracted within 7.0 s with a minimum threshold 0.001. In the phantom experiment, the mean target registration error (TRE) of 15 targets on the surface of the elastic phantom in the five experiments was 1.17 ± 0.04 mm, and the average registration time was 17.4 s. In the clinical experiments, the mean TRE of the targets on the first, second, third, fourth, and fifth patient's head surface were 1.70 ± 0.32 mm, 1.83 ± 0.38 mm, 1.64 ± 0.3 mm, 1.67 ± 0.35 mm, and 1.72 ± 0.31 mm, respectively, and the average registration time was 21.4 s. Compared with the method only based on the 4PCS and ICP algorithm and the current clinical method, the proposed method has obvious speed advantage while ensuring the registration accuracy.

CONCLUSIONS

The proposed method greatly improves the registration speed while guaranteeing the equivalent or higher registration accuracy, and avoids a tedious manual process for the coarse registration.

Comparison of cartilage and bone morphological models of the ankle joint derived from different medical imaging technologies

Background

Accurate geometrical models of bones and cartilage are necessary in biomechanical modelling of human joints, and in planning and designing of joint replacements. Image-based subject-specific model development requires image segmentation, spatial filtering and 3-dimensional rendering. This is usually based on computed tomography (CT) for bone models, on magnetic resonance imaging (MRI) for cartilage models. This process has been reported extensively in the past, but no studies have ever compared the accuracy and quality of these models when obtained also by merging different imaging modalities. The scope of the present work is to provide this comparative analysis in order to identify optimal imaging modality and registration techniques for producing 3-dimensional bone and cartilage models of the ankle joint.

Methods

One cadaveric leg was instrumented with multimodal markers and scanned using five different imaging modalities: a standard, a dual-energy and a cone-beam CT (CBCT) device, and a 1.5 and 3.0 Tesla MRI devices. Bone, cartilage, and combined bone and cartilage models were produced from each of these imaging modalities, and registered in space according to matching model surfaces or to corresponding marker centres. To assess the quality in overall model reconstruction, distance map analyses were performed and the difference between model surfaces obtained from the different imaging modalities and registration techniques was measured.

Results

The registration between models worked better with model surface matching than corresponding marker positions, particularly with MRI. The best bone models were obtained with the CBCT. Models with cartilage were defined better with the 3.0 Tesla than the 1.5 Tesla. For the combined bone and cartilage models, the colour maps and the numerical results from distance map analysis (DMA) showed that the smallest distances and the largest homogeneity were obtained from the CBCT and the 3.0 T MRI via model surface registration.

Conclusions

These observations are important in producing accurate bone and cartilage models from medical imaging and relevant for applications such as designing of custom-made ankle replacements or, more in general, of implants for total as well as focal joint replacements.

Improved Spatial Registration and Target Tracking Method for Sensors on Multiple Missiles

Inspired by the problem that the current spatial registration methods are unsuitable for three-dimensional (3-D) sensor on high-dynamic platform, this paper focuses on the estimation for the registration errors of cooperative missiles and motion states of maneuvering target. There are two types of errors being discussed: sensor measurement biases and attitude biases. Firstly, an improved Kalman Filter on Earth-Centered Earth-Fixed (ECEF-KF) coordinate algorithm is proposed to estimate the deviations mentioned above, from which the outcomes are furtherly compensated to the error terms. Secondly, the Pseudo Linear Kalman Filter (PLKF) and the nonlinear scheme the Unscented Kalman Filter (UKF) with modified inputs are employed for target tracking. The convergence of filtering results are monitored by a position-judgement logic, and a low-pass first order filter is selectively introduced before compensation to inhibit the jitter of estimations. In the simulation, the ECEF-KF enhancement is proven to improve the accuracy and robustness of the space alignment, while the conditional-compensation-based PLKF method is demonstrated to be the optimal performance in target tracking.

A novel registration method for image-guided neurosurgery system based on stereo vision

This study presents a novel spatial registration method of Image-guided neurosurgery system (IGNS) based on stereo-vision. Images of the patient's head are captured by a video camera, which is calibrated and tracked by an optical tracking system. Then, a set of sparse facial data points are reconstructed from them by stereo vision in the patient space. Surface matching method is utilized to register the reconstructed sparse points and the facial surface reconstructed from preoperative images of the patient. Simulation experiments verified the feasibility of the proposed method. The proposed method it is a new low-cost and easy-to-use spatial registration method for IGNS, with good prospects for clinical application.

Methods for Acquisition of Quantitative Data from Confocal Images of Gene Expression in situ

In this review, we summarize original methods for the extraction of quantitative information from confocal images of gene-expression patterns. These methods include image segmentation, the extraction of quantitative numerical data on gene expression, and the removal of background signal and spatial registration. Finally, it is possible to construct a spatiotemporal atlas of gene expression from individual images recorded at each developmental stage. Initially all methods were developed to extract quantitative numerical information from confocal images of segmentation gene expression in Drosophila melanogaster. The application of these methods to Drosophila images makes it possible to reveal new mechanisms in the formation of segmentation gene expression domains, as well as to construct a quantitative atlas of segmentation gene expression. Most image processing procedures can be easily adapted to process a wide range of biological images.

Pipeline for acquisition of quantitative data on segmentation gene expression from confocal images

We describe a data pipeline developed to extract the quantitative data on segmentation gene expression from confocal images of gene expression patterns in Drosophila. The pipeline consists of five steps: image segmentation, background removal, temporal characterization of an embryo, data registration and data averaging. This pipeline was successfully applied to obtain quantitative gene expression data at cellular resolution in space and at the 6.5-minute resolution in time, as well as to construct a spatiotemporal atlas of segmentation gene expression. Each data pipeline step can be easily adapted to process a wide range of images of gene expression patterns.