High-accuracy patient-to-image registration for the facilitation of image-guided robotic microsurgery on the head

Registration of preoperative temporal bone CT-scan to otoendoscopic video for augmented-reality based on convolutional neural networks

PURPOSE

Patient-to-image registration is a preliminary step required in surgical navigation based on preoperative images. Human intervention and fiducial markers hamper this task as they are time-consuming and introduce potential errors. We aimed to develop a fully automatic 2D registration system for augmented reality in ear surgery.

METHODS

CT-scans and corresponding oto-endoscopic videos were collected from 41 patients (58 ears) undergoing ear examination (vestibular schwannoma before surgery, profound hearing loss requiring cochlear implant, suspicion of perilymphatic fistula, contralateral ears in cases of unilateral chronic otitis media). Two to four images were selected from each case. For the training phase, data from patients (75% of the dataset) and 11 cadaveric specimens were used. Tympanic membranes and malleus handles were contoured on both video images and CT-scans by expert surgeons. The algorithm used a U-Net network for detecting the contours of the tympanic membrane and the malleus on both preoperative CT-scans and endoscopic video frames. Then, contours were processed and registered through an iterative closest point algorithm. Validation was performed on 4 cases and testing on 6 cases. Registration error was measured by overlaying both images and measuring the average and Hausdorff distances.

RESULTS

The proposed registration method yielded a precision compatible with ear surgery with a 2D mean overlay error of 0.65 ± 0.60 mm for the incus and 0.48 ± 0.32 mm for the round window. The average Hausdorff distance for these 2 targets was 0.98 ± 0.60 mm and 0.78 ± 0.34 mm respectively. An outlier case with higher errors (2.3 mm and 1.5 mm average Hausdorff distance for incus and round window respectively) was observed in relation to a high discrepancy between the projection angle of the reconstructed CT-scan and the video image. The maximum duration for the overall process was 18 s.

CONCLUSIONS

A fully automatic 2D registration method based on a convolutional neural network and applied to ear surgery was developed. The method did not rely on any external fiducial markers nor human intervention for landmark recognition. The method was fast and its precision was compatible with ear surgery.

Image-to-Patient Registration in Computer-Assisted Surgery of Head and Neck: State-of-the-Art, Perspectives, and Challenges

Today, image-guided systems play a significant role in improving the outcome of diagnostic and therapeutic interventions. They provide crucial anatomical information during the procedure to decrease the size and the extent of the approach, to reduce intraoperative complications, and to increase accuracy, repeatability, and safety. Image-to-patient registration is the first step in image-guided procedures. It establishes a correspondence between the patient's preoperative imaging and the intraoperative data. When it comes to the head-and-neck region, the presence of many sensitive structures such as the central nervous system or the neurosensory organs requires a millimetric precision. This review allows evaluating the characteristics and the performances of different registration methods in the head-and-neck region used in the operation room from the perspectives of accuracy, invasiveness, and processing times. Our work led to the conclusion that invasive marker-based methods are still considered as the gold standard of image-to-patient registration. The surface-based methods are recommended for faster procedures and applied on the surface tissues especially around the eyes. In the near future, computer vision technology is expected to enhance these systems by reducing human errors and cognitive load in the operating room.

An incremental registration method for endoscopic sinus and skull base surgery navigation: From phantom study to clinical trials

PURPOSE

Surface-based image-to-patient registration in current surgical navigation is mainly achieved by a 3D scanner, which has several limitations in clinical practice such as uncontrollable scanning range, complicated operation, and even high failure rate. An accurate, robust, and easy-to-perform image-to-patient registration method is urgently required.

METHODS

An incremental point cloud registration method was proposed for surface-based image-to-patient registration. The point cloud in image space was extracted from the computed tomography (CT) image, and a template matching method was applied to remove the redundant points. The corresponding point cloud in patient space was incrementally collected by an optically tracked pointer, while the nearest point distance (NPD) constraint was applied to ensure the uniformity of the collected points. A coarse-to-fine registration method under the constraints of coverage ratio (CR) and outliers ratio (OR) was then proposed to obtain the optimal rigid transformation from image to patient space. The proposed method was integrated in the recently developed endoscopic navigation system, and phantom study and clinical trials were conducted to evaluate the performance of the proposed method.

RESULTS

The results of the phantom study revealed that the proposed constraints greatly improved the accuracy and robustness of registration. The comparative experimental results revealed that the proposed registration method significantly outperform the scanner-based method, and achieved comparable accuracy to the fiducial-based method. In the clinical trials, the average registration duration was 1.24 ± 0.43 min, the target registration error (TRE) of 294 marker points (59 patients) was 1.25 ± 0.40 mm, and the lower 97.5% confidence limit of the success rate of positioning marker points exceeds the expected value (97.56% vs. 95.00%), revealed that the accuracy of the proposed method significantly met the clinical requirements (TRE ⩽ 2 mm, p < 0.05).

CONCLUSIONS

The proposed method has both the advantages of high accuracy and convenience, which were absent in the scanner-based method and the fiducial-based method. Our findings will help improve the quality of endoscopic sinus and skull base surgery.

Fusion of Technology in Cochlear Implantation Surgery: Investigation of Fluoroscopically Assisted Robotic Electrode Insertion

The HEARO cochlear implantation surgery aims to replace the conventional wide mastoidectomy approach with a minimally invasive direct cochlear access. The main advantage of the HEARO access would be that the trajectory accommodates the optimal and individualized insertion parameters such as type of cochlear access and trajectory angles into the cochlea. To investigate the quality of electrode insertion with the HEARO procedure, the insertion process was inspected under fluoroscopy in 16 human cadaver temporal bones. Prior to the insertion, the robotic middle and inner ear access were performed through the HEARO procedures. The status of the insertion was analyzed on the post-operative image with Siemens Artis Pheno (Siemens AG, Munich, Germany). The completion of the full HEARO procedure, including the robotic inner ear access and fluoroscopy electrode insertion, was possible in all 16 cases. It was possible to insert the electrode in all 16 cases through the drilled tunnel. However, one case in which the full cochlea was not visible on the post-operative image for analysis was excluded. The post-operative analysis of the electrode insertion showed an average insertion angle of 507°, which is equivalent to 1.4 turns of the cochlea, and minimal and maximal insertion angles were recorded as 373° (1 cochlear turn) and 645° (1.8 cochlear turn), respectively. The fluoroscopy inspection indicated no sign of complications during the insertion.

Homography-based robust pose compensation and fusion imaging for augmented reality based endoscopic navigation system

BACKGROUND

Augmented reality (AR) based fusion imaging in endoscopic surgeries rely on the quality of image-to-patient registration and camera calibration, and these two offline steps are usually performed independently to get the target transformation separately. The optimal solution can be obtained under independent conditions but may not be globally optimal. All residual errors will be accumulated and eventually lead to inaccurate AR fusion.

METHODS

After a careful analysis of the principle of AR imaging, a robust online calibration framework was proposed for an endoscopic camera to enable accurate AR fusion. A 2D checkerboard-based homography estimation algorithm was proposed to estimate the local pose of the endoscopic camera, and the least square method was used to calculate the compensation matrix in combination with the optical tracking system.

RESULTS

In comparison with conventional methods, the proposed compensation method improved the performance of AR fusion, which reduced physical error by up to 82%, reduced pixel error by up to 83%, and improved target coverage by up to 6%. Experimental results of simulating mechanical noise revealed that the proposed compensation method effectively corrected the fusion errors caused by the rotation of the endoscopic tube without recalibrating the camera. Furthermore, the simulation results revealed the robustness of the proposed compensation method to noises.

CONCLUSIONS

Overall, the experiment results proved the effectiveness of the proposed compensation method and online calibration framework, and revealed a considerable potential in clinical practice.

Automatic registration and precise tumour localization method for robot-assisted puncture procedure under inconsistent breath-holding conditions

BACKGROUND

During percutaneous puncture procedure, breath holding is subjectively controlled by patients, and it is difficult to ensure consistent tumour position between the preoperative CT scanning phase and the intraoperative puncture phase. In addition, the manual registration process is time-consuming and has low accuracy.

METHODS

We have proposed an automatic registration method using optical markers and a tumour breath-holding position estimation model based on the support vector regression algorithm. A robot system and a tumour respiratory motion simulation platform are built to perform puncture tests under different breath-holding states.

RESULTS

The experimental results show that automatic registration has higher accuracy than manual registration, and with the tumour breath-holding position estimation model, the targeting accuracy of puncture under inconsistent breath-holding conditions is greatly improved.

CONCLUSIONS

The proposed automatic registration and tumour breath-holding position estimation model can improve the accuracy and efficiency of puncture under inconsistent breath-holding conditions.

Calibrating 3D Scanner in the Coordinate System of Optical Tracker for Image-To-Patient Registration

Three-dimensional scanners have been widely applied in image-guided surgery (IGS) given its potential to solve the image-to-patient registration problem. How to perform a reliable calibration between a 3D scanner and an external tracker is especially important for these applications. This study proposes a novel method for calibrating the extrinsic parameters of a 3D scanner in the coordinate system of an optical tracker. We bound an optical marker to a 3D scanner and designed a specified 3D benchmark for calibration. We then proposed a two-step calibration method based on the pointset registration technique and nonlinear optimization algorithm to obtain the extrinsic matrix of the 3D scanner. We applied repeat scan registration error (RSRE) as the cost function in the optimization process. Subsequently, we evaluated the performance of the proposed method on a recaptured verification dataset through RSRE and Chamfer distance (CD). In comparison with the calibration method based on 2D checkerboard, the proposed method achieved a lower RSRE (1.73 mm vs. 2.10, 1.94, and 1.83 mm) and CD (2.83 mm vs. 3.98, 3.46, and 3.17 mm). We also constructed a surgical navigation system to further explore the application of the tracked 3D scanner in image-to-patient registration. We conducted a phantom study to verify the accuracy of the proposed method and analyze the relationship between the calibration accuracy and the target registration error (TRE). The proposed scanner-based image-to-patient registration method was also compared with the fiducial-based method, and TRE and operation time (OT) were used to evaluate the registration results. The proposed registration method achieved an improved registration efficiency (50.72 ± 6.04 vs. 212.97 ± 15.91 s in the head phantom study). Although the TRE of the proposed registration method met the clinical requirements, its accuracy was lower than that of the fiducial-based registration method (1.79 ± 0.17 mm vs. 0.92 ± 0.16 mm in the head phantom study). We summarized and analyzed the limitations of the scanner-based image-to-patient registration method and discussed its possible development.

Automated fiducial marker detection and localization in volumetric computed tomography images: a three-step hybrid approach with deep learning

Purpose: Automating fiducial detection and localization in the patient’s pre-operative images can lead to better registration accuracy, reduced human errors, and shorter intervention time. Most current approaches are optimized for a single marker type, mainly spherical adhesive markers. A fully automated algorithm is proposed and evaluated for screw and spherical titanium fiducials, typically used in high-accurate frameless surgical navigation.

Approach: The algorithm builds on previous approaches with morphological functions and pose estimation algorithms. A 3D convolutional neural network (CNN) is proposed for the fiducial classification task and evaluated for both traditional closed-set and emerging open-set classifiers. A proposed digital ground-truth experiment, with cone-beam computed tomography (CBCT) imaging software, is performed to determine the localization accuracy of the algorithm. The localized fiducial positions in the CBCT images by the presented algorithm were compared to the actual known positions in the virtual phantom models. The difference represents the fiducial localization error (FLE).

Results: A total of 241 screws, 151 spherical fiducials, and 1550 other structures are identified with the best true positive rate 95.9% for screw and 99.3% for spherical fiducials at 8.7% and 3.4% false positive rate, respectively. The best achieved FLE mean and its standard deviation for a screw and spherical marker are 58 (14) and 14 (6)  μm, respectively.

Conclusions: Accurate marker detection and localization were achieved, with spherical fiducials being superior to screws. Large marker volume and smaller voxel size yield significantly smaller FLEs. Attenuating noise by mesh smoothing has a minor effect on FLE. Future work will focus on expanding the CNN for image segmentation.

Evolution and Stagnation of Image Guidance for Surgery in the Lateral Skull: A Systematic Review 1989-2020

Objective: Despite three decades of pre-clinical and clinical research into image guidance solutions as a more accurate and less invasive alternative for instrument and anatomy localization, translation into routine clinical practice for surgery in the lateral skull has not yet happened. The aim of this review is to identify challenges that need to be solved in order to provide image guidance solutions that are safe and beneficial for use during lateral skull surgery and to synthesize factors that facilitate the development of such solutions. Methods: Literature search was conducted via PubMed using terms relating to image guidance and the lateral skull. Data extraction included the following variables: image guidance error, imaging resolution, image guidance system, tracking technology, registration method, study endpoints, clinical target application, and publication year. A subsequent search of FDA 510(k) database for identified image guidance systems and extraction of the year of approval, intended use, and indications for use was performed. The study objectives and endpoints were subdivided in three time phases and summarized. Furthermore, it was analyzed which factors correlated with the image guidance error. Factor values for which an error ≤0.5 mm (μ_error + 3σ_error) was measured in more than one study were identified and inspected for time trends. Results: A descriptive statistics-based summary of study objectives and findings separated in three time intervals is provided. The literature provides qualitative and quantitative evidence that image guidance systems must provide an accuracy ≤0.5 mm (μ_error + 3σ_error) for their safe and beneficial application during surgery in the lateral skull. Spatial tracking accuracy and precision and medical image resolution both correlate with the image guidance accuracy, and all of them improved over the years. Tracking technology with accuracy ≤0.05 mm, computed tomography imaging with slice thickness ≤0.2 mm, and registration based on bone-anchored titanium fiducials are components that provide a sufficient setting for the development of sufficiently accurate image guidance. Conclusion: Image guidance systems must reliably provide an accuracy ≤0.5 mm (μ_error + 3σ_error) for their safe and beneficial use during surgery in the lateral skull. Advances in tracking and imaging technology contribute to the improvement of accuracy, eventually enabling the development and wide-scale adoption of image guidance solutions that can be used safely and beneficially during lateral skull surgery.

Stereotactic Neuro-Navigation Phantom Designs: A Systematic Review

Diverse stereotactic neuro-navigation systems are used daily in neurosurgery and novel systems are continuously being developed. Prior to clinical implementation of new surgical tools, methods or instruments, in vitro experiments on phantoms should be conducted. A stereotactic neuro-navigation phantom denotes a rigid or deformable structure resembling the cranium with the intracranial area. The use of phantoms is essential for the testing of complete procedures and their workflows, as well as for the final validation of the application accuracy. The aim of this study is to provide a systematic review of stereotactic neuro-navigation phantom designs, to identify their most relevant features, and to identify methodologies for measuring the target point error, the entry point error, and the angular error (α). The literature on phantom designs used for evaluating the accuracy of stereotactic neuro-navigation systems, i.e., robotic navigation systems, stereotactic frames, frameless navigation systems, and aiming devices, was searched. Eligible articles among the articles written in English in the period 2000-2020 were identified through the electronic databases PubMed, IEEE, Web of Science, and Scopus. The majority of phantom designs presented in those articles provide a suitable methodology for measuring the target point error, while there is a lack of objective measurements of the entry point error and angular error. We identified the need for a universal phantom design, which would be compatible with most common imaging techniques (e.g., computed tomography and magnetic resonance imaging) and suitable for simultaneous measurement of the target point, entry point, and angular errors.

Prospective Validation of Facial Nerve Monitoring to Prevent Nerve Damage During Robotic Drilling

Facial nerve damage has a detrimental effect on a patient's life, therefore safety mechanisms to ensure its preservation are essential during lateral skull base surgery. During robotic cochlear implantation a trajectory passing the facial nerve at <0.5 mm is needed. Recently a stimulation probe and nerve monitoring approach were developed and introduced clinically, however for patient safety no trajectory was drilled closer than 0.4 mm. Here we assess the performance of the nerve monitoring system at closer distances. In a sheep model eight trajectories were drilled to test the setup followed by 12 trajectories during which the ENT surgeon relied solely on the nerve monitoring system and aborted the robotic drilling process if intraoperative nerve monitoring alerted of a distance <0.1 mm. Microcomputed tomography images and histopathology showed prospective use of the technology prevented facial nerve damage. Facial nerve monitoring integrated in a robotic system supports the surgeon's ability to proactively avoid damage to the facial nerve during robotic drilling in the mastoid.

Robotic Stereotaxy in Cranial Neurosurgery: A Qualitative Systematic Review

BACKGROUND

Modern-day stereotactic techniques have evolved to tackle the neurosurgical challenge of accurately and reproducibly accessing specific brain targets. Neurosurgical advances have been made in synergy with sophisticated technological developments and engineering innovations such as automated robotic platforms. Robotic systems offer a unique combination of dexterity, durability, indefatigability, and precision.

OBJECTIVE

To perform a systematic review of robotic integration for cranial stereotactic guidance in neurosurgery. Specifically, we comprehensively analyze the strengths and weaknesses of a spectrum of robotic technologies, past and present, including details pertaining to each system's kinematic specifications and targeting accuracy profiles.

METHODS

Eligible articles on human clinical applications of cranial robotic-guided stereotactic systems between 1985 and 2017 were extracted from several electronic databases, with a focus on stereotactic biopsy procedures, stereoelectroencephalography, and deep brain stimulation electrode insertion.

RESULTS

Cranial robotic stereotactic systems feature serial or parallel architectures with 4 to 7 degrees of freedom, and frame-based or frameless registration. Indications for robotic assistance are diversifying, and include stereotactic biopsy, deep brain stimulation and stereoelectroencephalography electrode placement, ventriculostomy, and ablation procedures. Complication rates are low, and mainly consist of hemorrhage. Newer systems benefit from increasing targeting accuracy, intraoperative imaging ability, improved safety profiles, and reduced operating times.

CONCLUSION

We highlight emerging future directions pertaining to the integration of robotic technologies into future neurosurgical procedures. Notably, a trend toward miniaturization, cost-effectiveness, frameless registration, and increasing safety and accuracy characterize successful stereotactic robotic technologies.

Robotic middle ear access for cochlear implantation: First in man

To demonstrate the feasibility of robotic middle ear access in a clinical setting, nine adult patients with severe-to-profound hearing loss indicated for cochlear implantation were included in this clinical trial. A keyhole access tunnel to the tympanic cavity and targeting the round window was planned based on preoperatively acquired computed tomography image data and robotically drilled to the level of the facial recess. Intraoperative imaging was performed to confirm sufficient distance of the drilling trajectory to relevant anatomy. Robotic drilling continued toward the round window. The cochlear access was manually created by the surgeon. Electrode arrays were inserted through the keyhole tunnel under microscopic supervision via a tympanomeatal flap. All patients were successfully implanted with a cochlear implant. In 9 of 9 patients the robotic drilling was planned and performed to the level of the facial recess. In 3 patients, the procedure was reverted to a conventional approach for safety reasons. No change in facial nerve function compared to baseline measurements was observed. Robotic keyhole access for cochlear implantation is feasible. Further improvements to workflow complexity, duration of surgery, and usability including safety assessments are required to enable wider adoption of the procedure.

A "eye-in-body" integrated surgery robot system for stereotactic surgery

PURPOSE

Current stereotactic surgical robots system relies on cumbersome operations such as calibration, tracking and registration to establish the accurate intraoperative coordinate transformation chain, which makes the system not easy to use. To overcome this problem, a novel stereotactic surgical robot system has been proposed and validated.

METHODS

First, a hand-eye integrated scheme is proposed to avoid the intraoperative calibration between robot arm and motion tracking system. Second, a special reference-tool-based patient registration and tracking method is developed to avoid intraoperative registration. Third, a model-free visual servo method is used to reduce the accuracy requirement of hand-eye relationship and robot kinematic model. Finally, a prototype of the system is constructed and performance tests and a pedicle screw drilling experiment are performed.

RESULTS

The results show that the proposed system has acceptable accuracy. The target positioning error in the plane was - 0.68 ± 0.52 mm and 0.06 ± 0.41 mm. The orientation error was 0.43 ± 0.25°. The pedicle screw drilling experiment shows that the system can complete accurate stereotactic surgery.

CONCLUSIONS

The stereotactic surgical robot system described in this paper can perform stereotactic surgery without the intraoperative hand-eye calibration and nor manual registration and can achieve an acceptable position and orientation accuracy while tolerating the errors in the hand-eye coordinate transformation error and the robot kinematics model error.

Automatic Registration for Navigation at the Anterior and Lateral Skull Base

OBJECTIVE

Navigation systems create a connection between imaging data and intraoperative situs, allowing the surgeon to consistently determine the location of instruments and patient anatomy during the surgical procedure. The best results regarding the target registration error (measurement uncertainty) are normally demonstrated using fiducials. This study aimed at investigating a new registration strategy for an electromagnetic navigation device.

METHODS

For evaluation of an electromagnetic navigation system and comparison of registration with screw markers and automatic registration, we are calculating the target registration error in the region of the paranasal sinuses/anterior and lateral skull base with the use of an electromagnetic navigation system and intraoperative digital volume tomography (cone-beam computed tomography). We carried out 10 registrations on a head model (total n = 150 measurements) and 10 registrations on 4 temporal bone specimens (total n = 160 measurements).

RESULTS

All in all, the automatic registration was easy to perform. For the models that were used, a significant difference between an automatic registration and the registration on fiducials was evident for just a limited number of screws. Furthermore, the observed differences varied in terms of the preferential registration procedure.

CONCLUSION

The automatic registration strategy seems to be an alternative to the established methods in artificial and cadaver models of intraoperative scenarios. Using intraoperative imaging, there is an option to resort to this kind of registration as needed.

Neuromonitoring During Robotic Cochlear Implantation: Initial Clinical Experience

During robotic cochlear implantation a drill trajectory often passes at submillimeter distances from the facial nerve due to close lying critical anatomy of the temporal bone. Additional intraoperative safety mechanisms are thus required to ensure preservation of this vital structure in case of unexpected navigation system error. Electromyography based nerve monitoring is widely used to aid surgeons in localizing vital nerve structures at risk of injury during surgery. However, state of the art neuromonitoring systems, are unable to discriminate facial nerve proximity within submillimeter ranges. Previous work demonstrated the feasibility of utilizing combinations of monopolar and bipolar stimulation threshold measurements to discretize facial nerve proximity with greater sensitivity and specificity, enabling discrimination between safe (> 0.4 mm) and unsafe (< 0.1 mm) trajectories during robotic cochlear implantation (in vivo animal model). Herein, initial clinical validation of the determined stimulation protocol and nerve proximity analysis integrated into an image guided system for safety measurement is presented. Stimulation thresholds and corresponding nerve proximity values previously determined from an animal model have been validated in a first-in-man clinical trial of robotic cochlear implantation. Measurements performed automatically at preoperatively defined distances from the facial nerve were used to determine safety of the drill trajectory intraoperatively. The presented system and automated analysis correctly determined sufficient safety distance margins (> 0.4 mm) to the facial nerve in all cases.

Electrical Impedance to Assess Facial Nerve Proximity During Robotic Cochlear Implantation

Reported studies pertaining to needle guidance suggest that tissue impedance available from neuromonitoring systems can be used to discriminate nerve tissue proximity. In this pilot study, the existence of a relationship between intraoperative electrical impedance and tissue density, estimated from computer tomography (CT) images, is evaluated in the mastoid bone of in vivo sheep. In five subjects, nine trajectories were drilled using an image-guided surgical robot. Per trajectory, five measurement points near the facial nerve were accessed and electrical impedance was measured (≤1 KHz) using a multipolar electrode probe. Micro-CT was used postoperatively to measure the distances from the drilled trajectories to the facial nerve. Tissue density was determined from coregistered preoperative CT images and, following sensitivity field modeling of the measuring tip, tissue resistivity was calculated. The relationship between impedance and density was determined for 29 trajectories passing or intersecting the facial nerve. A monotonic decrease in impedance magnitude was observed in all trajectories with a drill axis intersecting the facial nerve. Mean tissue densities intersecting with the facial nerve (971-1161 HU) were different (p <0.01) from those along safe trajectories passing the nerve (1194-1449 HU). However, mean resistivity values of trajectories intersecting the facial nerve (14-24 Ωm) were similar to those of safe passing trajectories (17-23 Ωm). The determined relationship between tissue density and electrical impedance during neuromonitoring of the facial nerve suggests that impedance spectroscopy may be used to increase the accuracy of tissue discrimination, and ultimately improve nerve safety distance assessment in the future.

Robotic cochlear implantation: surgical procedure and first clinical experience

CONCLUSION

A system for robotic cochlear implantation (rCI) has been developed and a corresponding surgical workflow has been described. The clinical feasibility was demonstrated through the conduction of a safe and effective rCI procedure.

OBJECTIVES

To define a clinical workflow for rCI and demonstrate its feasibility, safety, and effectiveness within a clinical setting.

METHOD

A clinical workflow for use of a previously described image guided surgical robot system for rCI was developed. Based on pre-operative images, a safe drilling tunnel targeting the round window was planned and drilled by the robotic system. Intra-operatively the drill path was assessed using imaging and sensor-based data to confirm the proximity of the facial nerve. Electrode array insertion was manually achieved under microscope visualization. Electrode array placement, structure preservation, and the accuracy of the drilling and of the safety mechanisms were assessed on post-operative CT images.

RESULTS

Robotic drilling was conducted with an accuracy of 0.2 mm and safety mechanisms predicted proximity of the nerves to within 0.1 mm. The approach resulted in a minimal mastoidectomy and minimal incisions. Manual electrode array insertion was successfully performed through the robotically drilled tunnel. The procedure was performed without complications, and all surrounding structures were preserved.

Instrument flight to the inner ear

Surgical robot systems can work beyond the limits of human perception, dexterity and scale making them inherently suitable for use in microsurgical procedures. However, despite extensive research, image-guided robotics applications for microsurgery have seen limited introduction into clinical care to date. Among others, challenges are geometric scale and haptic resolution at which the surgeon cannot sufficiently control a device outside the range of human faculties. Mechanisms are required to ascertain redundant control on process variables that ensure safety of the device, much like instrument-flight in avionics. Cochlear implantation surgery is a microsurgical procedure, in which specific tasks are at sub-millimetric scale and exceed reliable visuo-tactile feedback. Cochlear implantation is subject to intra- and inter-operative variations, leading to potentially inconsistent clinical and audiological outcomes for patients. The concept of robotic cochlear implantation aims to increase consistency of surgical outcomes such as preservation of residual hearing and reduce invasiveness of the procedure. We report successful image-guided, robotic CI in human. The robotic treatment model encompasses: computer-assisted surgery planning, precision stereotactic image-guidance, in-situ assessment of tissue properties and multipolar neuromonitoring (NM), all based on in vitro, in vivo and pilot data. The model is expandable to integrate additional robotic functionalities such as cochlear access and electrode insertion. Our results demonstrate the feasibility and possibilities of using robotic technology for microsurgery on the lateral skull base. It has the potential for benefit in other microsurgical domains for which there is no task-oriented, robotic technology available at present.

Ensemble image registration by a spatially constrained clustering approach

In this article, a novel spatially constrained clustering approach is proposed for ensemble image registration. We use a spatially constrained Gaussian mixture model, which is based on a joint Gaussian mixture model and Markov random field, to model the joint intensity scatter plot of the unregistered images. The spatially constrained Gaussian mixture model has the capability of performing the correlation among neighboring observations. A cost function of reducing the dispersion in the joint intensity scatter plot is proposed using the spatially constrained Gaussian mixture model to simultaneously register a group of images. We derive an expectation maximization algorithm for the proposed model. Computer simulations demonstrate the effectiveness of the proposed method.

Registration using 3D-printed rigid templates outperforms manually scanned surface matching in image-guided temporal bone surgery

PURPOSE

Image-guided surgery (IGS) for otological procedures requires minimal invasiveness and a high degree of accuracy. We have recently developed a noninvasive registration method, the Surface Template-Assisted Marker Positioning (STAMP) method, which uses a rigid template of the surface of the temporal bone. However, the STAMP method is not applicable when the bony surface is not exposed, such as in endoscopic surgery. Thus, we extended our research to apply the STAMP method onto the skin and tested its feasibility in this study.

METHODS

We designed a phantom made of a rigid box and soft material for the study. The target registration error (TRE) was measured at preset measuring points in the phantom. We modified the STAMP method to be applicable for use on the skin around the ears (S-STAMP). The same phantom was also registered using the conventional, manually scanned surface matching method. We compared the TRE after the different registration methods.

RESULTS

The TRE after the S-STAMP registration method was significantly smaller than that of the conventional surface matching method at all error measurement points in the phantom. However, the TRE after the S-STAMP registration method was significantly larger than that of paired point registration using invasive fiducial markers.

CONCLUSIONS

The S-STAMP method using a rigid template on the soft surface yields a significantly smaller TRE than that of conventional, manually scanned surface matching registration. This strategy provides an alternative option to improve the accuracy of IGS without loading patients with additional invasive procedures.

Surface matching for high-accuracy registration of the lateral skull base

PURPOSE

The accuracy achievable when utilizing image guidance depends to a large extent on the accuracy with which the patient can be registered to preoperative image data. This work proposes a method for the registration of the temporal bone based on surface matching and investigates the achievable accuracy of the technique.

METHODS

Fourteen human temporal bones were utilized for evaluation; incisions were made, fiducial screws were implanted to act as a ground truth, and imaging was performed. The positions of the fiducials and surface of the mastoid were extracted from image data and reference positions defined at the round window and the mastoid surface. The surface of the bone was then digitized using a tracked pointer within the region exposed by the incisions and the physical and image point clouds registered, with the result compared to the fiducial-based registration.

RESULTS

Results of one case were excluded due to a problem with the ground truth registration. In the remaining cases an accuracy of [Formula: see text] and [Formula: see text] mm was observed relative to the ground truth at the surface of the mastoid and round window, respectively.

CONCLUSIONS

A technique for the registration of the temporal bone was proposed, based on surface matching after exposure of the mastoid surface, and evaluated on human temporal bone specimens. The results reveal that high-accuracy patient-to-image registration is possible without the use of fiducial screws.

Strategy for accurate liver intervention by an optical tracking system

Image-guided navigation for radiofrequency ablation of liver tumors requires the accurate guidance of needle insertion into a tumor target. The main challenge of image-guided navigation for radiofrequency ablation of liver tumors is the occurrence of liver deformations caused by respiratory motion. This study reports a strategy of real-time automatic registration to track custom fiducial markers glued onto the surface of a patient's abdomen to find the respiratory phase, in which the static preoperative CT is performed. Custom fiducial markers are designed. Real-time automatic registration method consists of the automatic localization of custom fiducial markers in the patient and image spaces. The fiducial registration error is calculated in real time and indicates if the current respiratory phase corresponds to the phase of the static preoperative CT. To demonstrate the feasibility of the proposed strategy, a liver simulator is constructed and two volunteers are involved in the preliminary experiments. An ex-vivo porcine liver model is employed to further verify the strategy for liver intervention. Experimental results demonstrate that real-time automatic registration method is rapid, accurate, and feasible for capturing the respiratory phase from which the static preoperative CT anatomical model is generated by tracking the movement of the skin-adhered custom fiducial markers.

Precision of image-based registration for intraoperative navigation in the presence of metal artifacts: Application to corrective osteotomy surgery

Navigation for corrective osteotomy surgery requires patient-to-image registration. When registration is based on intraoperative 3-D cone-beam CT (CBCT) imaging, metal landmarks may be used that deteriorate image quality. This study investigates whether metal artifacts influence the precision of image-to-patient registration, either with or without intermediate user intervention during the registration procedure, in an application for corrective osteotomy of the distal radius. A series of 3-D CBCT scans is made of a cadaver arm with and without metal landmarks. Metal artifact reduction (MAR) based on inpainting techniques is used to improve 3-D CBCT images hampered by metal artifacts. This provides three sets of images (with metal, with MAR, and without metal), which enable investigating the differences in precision of intraoperative registration. Gray-level based point-to-image registration showed a better correlation coefficient if intraoperative images with MAR are used, indicating a better image similarity. The precision of registration without intermediate user intervention during the registration procedure, expressed as the residual angulation and displacement error after repetitive registration was very low and showed no improvement when MAR was used. By adding intermediate user intervention to the registration procedure however, precision was very high but was not affected by the presence of metal artifacts in the specific application.

Feasibility of using EMG for early detection of the facial nerve during robotic direct cochlear access

HYPOTHESIS

Facial nerve monitoring can be used synchronous with a high-precision robotic tool as a functional warning to prevent of a collision of the drill bit with the facial nerve during direct cochlear access (DCA).

BACKGROUND

Minimally invasive direct cochlear access (DCA) aims to eliminate the need for a mastoidectomy by drilling a small tunnel through the facial recess to the cochlea with the aid of stereotactic tool guidance. Because the procedure is performed in a blind manner, structures such as the facial nerve are at risk. Neuromonitoring is a commonly used tool to help surgeons identify the facial nerve (FN) during routine surgical procedures in the mastoid. Recently, neuromonitoring technology was integrated into a commercially available drill system enabling real-time monitoring of the FN. The objective of this study was to determine if this drilling system could be used to warn of an impending collision with the FN during robot-assisted DCA.

MATERIALS AND METHODS

The sheep was chosen as a suitable model for this study because of its similarity to the human ear anatomy. The same surgical workflow applicable to human patients was performed in the animal model. Bone screws, serving as reference fiducials, were placed in the skull near the ear canal. The sheep head was imaged using a computed tomographic scanner and segmentation of FN, mastoid, and other relevant structures as well as planning of drilling trajectories was carried out using a dedicated software tool. During the actual procedure, a surgical drill system was connected to a nerve monitor and guided by a custom built robot system. As the planned trajectories were drilled, stimulation and EMG response signals were recorded. A postoperative analysis was achieved after each surgery to determine the actual drilled positions.

RESULTS

Using the calibrated pose synchronized with the EMG signals, the precise relationship between distance to FN and EMG with 3 different stimulation intensities could be determined for 11 different tunnels drilled in 3 different subjects.

CONCLUSION

From the results, it was determined that the current implementation of the neuromonitoring system lacks sensitivity and repeatability necessary to be used as a warning device in robotic DCA. We hypothesize that this is primarily because of the stimulation pattern achieved using a noninsulated drill as a stimulating probe. Further work is necessary to determine whether specific changes to the design can improve the sensitivity and specificity.

Mechatronic feasibility of minimally invasive, atraumatic cochleostomy

Robotic assistance in the context of lateral skull base surgery, particularly during cochlear implantation procedures, has been the subject of considerable research over the last decade. The use of robotics during these procedures has the potential to provide significant benefits to the patient by reducing invasiveness when gaining access to the cochlea, as well as reducing intracochlear trauma when performing a cochleostomy. Presented herein is preliminary work on the combination of two robotic systems for reducing invasiveness and trauma in cochlear implantation procedures. A robotic system for minimally invasive inner ear access was combined with a smart drilling tool for robust and safe cochleostomy; evaluation was completed on a single human cadaver specimen. Access to the middle ear was successfully achieved through the facial recess without damage to surrounding anatomical structures; cochleostomy was completed at the planned position with the endosteum remaining intact after drilling as confirmed by microscope evaluation.

Semiautomatic cochleostomy target and insertion trajectory planning for minimally invasive cochlear implantation

A major component of minimally invasive cochlear implantation is atraumatic scala tympani (ST) placement of the electrode array. This work reports on a semiautomatic planning paradigm that uses anatomical landmarks and cochlear surface models for cochleostomy target and insertion trajectory computation. The method was validated in a human whole head cadaver model (n = 10 ears). Cochleostomy targets were generated from an automated script and used for consecutive planning of a direct cochlear access (DCA) drill trajectory from the mastoid surface to the inner ear. An image-guided robotic system was used to perform both, DCA and cochleostomy drilling. Nine of 10 implanted specimens showed complete ST placement. One case of scala vestibuli insertion occurred due to a registration/drilling error of 0.79 mm. The presented approach indicates that a safe cochleostomy target and insertion trajectory can be planned using conventional clinical imaging modalities, which lack sufficient resolution to identify the basilar membrane.