Augmented Reality (AR) for Surgical Robotic and Autonomous Systems: State of the Art, Challenges, and Solutions

Despite the substantial progress achieved in the development and integration of augmented reality (AR) in surgical robotic and autonomous systems (RAS), the center of focus in most devices remains on improving end-effector dexterity and precision, as well as improved access to minimally invasive surgeries. This paper aims to provide a systematic review of different types of state-of-the-art surgical robotic platforms while identifying areas for technological improvement. We associate specific control features, such as haptic feedback, sensory stimuli, and human-robot collaboration, with AR technology to perform complex surgical interventions for increased user perception of the augmented world. Current researchers in the field have, for long, faced innumerable issues with low accuracy in tool placement around complex trajectories, pose estimation, and difficulty in depth perception during two-dimensional medical imaging. A number of robots described in this review, such as Novarad and SpineAssist, are analyzed in terms of their hardware features, computer vision systems (such as deep learning algorithms), and the clinical relevance of the literature. We attempt to outline the shortcomings in current optimization algorithms for surgical robots (such as YOLO and LTSM) whilst providing mitigating solutions to internal tool-to-organ collision detection and image reconstruction. The accuracy of results in robot end-effector collisions and reduced occlusion remain promising within the scope of our research, validating the propositions made for the surgical clearance of ever-expanding AR technology in the future.

Optical and Electromagnetic Tracking Systems for Biomedical Applications: A Critical Review on Potentialities and Limitations

Optical and electromagnetic tracking systems represent the two main technologies integrated into commercially-available surgical navigators for computer-assisted image-guided surgery so far. Optical Tracking Systems (OTSs) work within the optical spectrum to track the position and orientation, i.e., pose of target surgical instruments. OTSs are characterized by high accuracy and robustness to environmental conditions. The main limitation of OTSs is the need of a direct line-of-sight between the optical markers and the camera sensor, rigidly fixed into the operating theatre. Electromagnetic Tracking Systems (EMTSs) use electromagnetic field generator to detect the pose of electromagnetic sensors. EMTSs do not require such a direct line-of-sight, however the presence of metal or ferromagnetic sources in the operating workspace can significantly affect the measurement accuracy. The aim of the proposed review is to provide a complete and detailed overview of optical and electromagnetic tracking systems, including working principles, source of error and validation protocols. Moreover, commercial and research-oriented solutions, as well as clinical applications, are described for both technologies. Finally, a critical comparative analysis of the state of the art which highlights the potentialities and the limitations of each tracking system for a medical use is provided.

Fiducial-based fusion of 3D dental models with magnetic resonance imaging

PURPOSE

Magnetic resonance imaging (MRI) is widely used in study of maxillofacial structures. While MRI is the modality of choice for soft tissues, it fails to capture hard tissues such as bone and teeth. Virtual dental models, acquired by optical 3D scanners, are becoming more accessible for dental practice and are starting to replace the conventional dental impressions. The goal of this research is to fuse the high-resolution 3D dental models with MRI to enhance the value of imaging for applications where detailed analysis of maxillofacial structures are needed such as patient examination, surgical planning, and modeling.

METHODS

A subject-specific dental attachment was digitally designed and 3D printed based on the subject's face width and dental anatomy. The attachment contained 19 semi-ellipsoidal concavities in predetermined positions where oil-based ellipsoidal fiducial markers were later placed. The MRI was acquired while the subject bit on the dental attachment. The spatial position of the center of mass of each fiducial in the resultant MR Image was calculated by averaging its voxels' spatial coordinates. The rigid transformation to fuse dental models to MRI was calculated based on the least squares mapping of corresponding fiducials and solved via singular-value decomposition.

RESULTS

The target registration error (TRE) of the proposed fusion process, calculated in a leave-one-fiducial-out fashion, was estimated at 0.49 mm. The results suggest that 6-9 fiducials suffice to achieve a TRE of equal to half the MRI voxel size.

CONCLUSION

Ellipsoidal oil-based fiducials produce distinguishable intensities in MRI and can be used as registration fiducials. The achieved accuracy of the proposed approach is sufficient to leverage the merged 3D dental models with the MRI data for a finer analysis of the maxillofacial structures where complete geometry models are needed.

A surgical navigation system for aortic vascular surgery: a practical approach

In aortic vascular surgery, a navigation system must represent the anatomical map of individual patient in order to detect the important artery. To provide a proper fit for positions along the dorsoventral axis, the spinous process was added to a currently used anatomical point set consisting of four anterior body landmarks. In addition, we attempted to reduce the registration error by compensating for alignment errors resulting from variations in tissue thickness at each landmark. The alignment values were examined using a human phantom consisting of a skeleton model with subcutaneous tissue in the semilateral position. Using this method, a phantom simulation and five clinical trials were performed. Target errors were evaluated at the orifice of the intercostal artery. In the phantom simulation, the error at the target point was 4.1 ± 2.7 mm. However, for one patient undergoing thoracoabdominal aortic aneurysm replacement surgery, the target error was 8.0 mm using the proposed method.

Official measurement protocol and accuracy results for an optical surgical navigation system (NPU)

Image-guided surgical navigation is on the rise in many different areas of modern medicine and is already an established standard in some disciplines like ear nose and throat (ENT) or maxillofacial surgery. When evaluating surgical navigation systems the absolute accuracy of the device is of major concern to the surgeon. The following work presents two different ways of measuring the accuracy of surgical navigation systems using the example of the KARL STORZ Navigation Panel Unit (NPU). According to these protocols the FDA approval of the NPU navigation system was prepared. In a first series of experiments the accuracy under realistic surgical conditions is evaluated with a phantom of a human head, which is manufactured in rapid-prototyping processes. In another series of experiments a custom registration board is used, which provides means to evaluate the accuracy under optimal conditions and also allows further measurements regarding the registration error, that are not possible with the phantom. In the experiments an accuracy of 1.44 mm ± 0.18 mm was measured in the surgical setup and 0.63 mm ± 0.07 mm under ideal conditions.

Design and accuracy evaluation of a new navigated drill system for computer assisted ENT-surgery

In this article a new navigated drill system for computer assisted ear, nose and throat (ENT) surgery is presented. The navigated drill and the microscope probe are part of a surgical navigation system for ENT-surgery. In particular, the accuracy of the new navigated drill is compared to an existing navigated drill experimentally under conditions close to the surgical workflow. For the technical accuracy experiment, the new navigated drill in combination with the new microscope probe and a particular navigated measurement board have been integrated, together with the current navigated drill, in a navigation system by a special navigation software with measuring function, based on a standard ENT navigation software. The developed navigated measurement board provided the implementation of reproducible experiments and the direct accuracy comparison of the two navigated instruments under the same conditions. Thereby, N = 15 accuracy experiments are performed with both navigated drill systems with three possible tracker positions. The distance between the planned and the touched points were calculated and compared. The average distances from the planned points to the touched points with the new navigated drill is in the left tracker position 1.10 mm, in the middle tracker position 1.14 mm and in the right tracker position 1.59 mm. In comparison to the existing drill, the new navigated drill, measured with each tracker position, is 0.62 mm more accurate.