A simple sensor calibration technique for estimating the 3D pose of endoscopic instruments

An EM-Tracked Approach for Calibrating the 3D Pose of Flexible Endoscopes

Flexible endoscopes are ideal instruments for visualizing and diagnosing the inner surfaces of organs via a minimally invasive incision. Calibrating a flexible endoscope is a troublesome yet inevitable process in image-based tools tracking. Aiming to simplify the calibration process, we propose an electromagnetic (EM)-tracked calibration approach that does not require any predefined poses of the EM sensor. A three-stage calibration protocol was presented in an extensor. First, the orientation of the endoscope tube was derived by conducting a circular rotation of the endoscope around its axis utilizing a pair of tightly bearing stands. Second, the 3D position of the endoscope tip was acquired by having the tip come into contact with a flat plane. Third, the pose model of the bending section was derived and transformed into the local coordinate system of the EM sensor attached to the endoscope handle. To assess the accuracy of the proposed calibration approach, two experiments were designed and performed. Experimental results indicate accuracies of 0.09 ± 0.06 deg and 0.03 ± 0.19 deg in the estimation of the endoscope tube orientation and 0.52 ± 0.29, 0.33 ± 0.11, and 0.29 ± 0.17 mm in the x, y, and z estimations of the endoscope tip position, respectively. The proposed approach is accurate and easy to operate, does not require the employment of custom calibration markers, and can be used not only in surgical training systems but also in the endoscopic-based tools tracking.

ArthroNavi framework: stereo endoscope-guided instrument localization for arthroscopic minimally invasive surgeries

Significance

As an example of a minimally invasive arthroscopic surgical procedure, arthroscopic osteochondral autograft transplantation (OAT) is a common option for repairing focal cartilage defects in the knee joints. Arthroscopic OAT offers considerable benefits to patients, such as less post-operative pain and shorter hospital stays. However, performing OAT arthroscopically is an extremely demanding task because the osteochondral graft harvester must remain perpendicular to the cartilage surface to avoid differences in angulation.

Aim

We present a practical ArthroNavi framework for instrument pose localization by combining a self-developed stereo endoscopy with electromagnetic computation, which equips surgeons with surgical navigation assistance that eases the operational constraints of arthroscopic OAT surgery.

Approach

A prototype of a stereo endoscope specifically fit for a texture-less scene is introduced extensively. Then, the proposed framework employs the semi-global matching algorithm integrating the matching cubes method for real-time processing of the 3D point cloud. To address issues regarding initialization and occlusion, a displaying method based on patient tracking coordinates is proposed for intra-operative robust navigation. A geometrical constraint method that utilizes the 3D point cloud is used to compute a pose for the instrument. Finally, a hemisphere tabulation method is presented for pose accuracy evaluation.

Results

Experimental results show that our endoscope achieves 3D shape measurement with an accuracy of < 730    μ m . The mean error of pose localization is 15.4 deg (range of 10.3 deg to 21.3 deg; standard deviation of 3.08 deg) in our ArthroNavi method, which is within the same order of magnitude as that achieved by experienced surgeons using a freehand technique.

Conclusions

The effectiveness of the proposed ArthroNavi has been validated on a phantom femur. The potential contribution of this framework may provide a new computer-aided option for arthroscopic OAT surgery.

Hemisphere Tabulation Method: An Ingenious Approach for Pose Evaluation of Instruments Using the Electromagnetic-Based Stereo Imaging Method

Drilling of a bone surface often occurs in clinical orthopaedic surgery. The position and orientation of the instrument are the most important factors in this process. Theoretically, some mechanical components may assist in orienting an instrument to certain bone shapes, such as the knee joint and caput femoris. However, the mechanical assisting component does not seem to work in some confined spaces where the bone shape is a free-form surface. In this paper, we propose an ingenious hemisphere tabulation method (HTM) for assessing the pose accuracy of an instrument. The acquisition and assessment of HTM is conducted based on an electromagnetic-based stereo imaging method using a custom-made optical measurement unit, and the operation steps of HTM are described in detail. Experimental results based on 50 tests show that the HTM can identify ideal poses and the evaluated pose of an instrument location on a hemisphere model. The mean error of pose localisation is 7.24 deg, with a range of 1.35 to 15.84 and a standard of 3.66 deg, which is more accurate than our previous method.

Local Style Preservation in Improved GAN-Driven Synthetic Image Generation for Endoscopic Tool Segmentation

Accurate semantic image segmentation from medical imaging can enable intelligent vision-based assistance in robot-assisted minimally invasive surgery. The human body and surgical procedures are highly dynamic. While machine-vision presents a promising approach, sufficiently large training image sets for robust performance are either costly or unavailable. This work examines three novel generative adversarial network (GAN) methods of providing usable synthetic tool images using only surgical background images and a few real tool images. The best of these three novel approaches generates realistic tool textures while preserving local background content by incorporating both a style preservation and a content loss component into the proposed multi-level loss function. The approach is quantitatively evaluated, and results suggest that the synthetically generated training tool images enhance UNet tool segmentation performance. More specifically, with a random set of 100 cadaver and live endoscopic images from the University of Washington Sinus Dataset, the UNet trained with synthetically generated images using the presented method resulted in 35.7% and 30.6% improvement over using purely real images in mean Dice coefficient and Intersection over Union scores, respectively. This study is promising towards the use of more widely available and routine screening endoscopy to preoperatively generate synthetic training tool images for intraoperative UNet tool segmentation.