An EM-Tracked Approach for Calibrating the 3D Pose of Flexible Endoscopes

Flexible endoscopes are ideal instruments for visualizing and diagnosing the inner surfaces of organs via a minimally invasive incision. Calibrating a flexible endoscope is a troublesome yet inevitable process in image-based tools tracking. Aiming to simplify the calibration process, we propose an electromagnetic (EM)-tracked calibration approach that does not require any predefined poses of the EM sensor. A three-stage calibration protocol was presented in an extensor. First, the orientation of the endoscope tube was derived by conducting a circular rotation of the endoscope around its axis utilizing a pair of tightly bearing stands. Second, the 3D position of the endoscope tip was acquired by having the tip come into contact with a flat plane. Third, the pose model of the bending section was derived and transformed into the local coordinate system of the EM sensor attached to the endoscope handle. To assess the accuracy of the proposed calibration approach, two experiments were designed and performed. Experimental results indicate accuracies of 0.09 ± 0.06 deg and 0.03 ± 0.19 deg in the estimation of the endoscope tube orientation and 0.52 ± 0.29, 0.33 ± 0.11, and 0.29 ± 0.17 mm in the x, y, and z estimations of the endoscope tip position, respectively. The proposed approach is accurate and easy to operate, does not require the employment of custom calibration markers, and can be used not only in surgical training systems but also in the endoscopic-based tools tracking.

Homography-based robust pose compensation and fusion imaging for augmented reality based endoscopic navigation system

BACKGROUND

Augmented reality (AR) based fusion imaging in endoscopic surgeries rely on the quality of image-to-patient registration and camera calibration, and these two offline steps are usually performed independently to get the target transformation separately. The optimal solution can be obtained under independent conditions but may not be globally optimal. All residual errors will be accumulated and eventually lead to inaccurate AR fusion.

METHODS

After a careful analysis of the principle of AR imaging, a robust online calibration framework was proposed for an endoscopic camera to enable accurate AR fusion. A 2D checkerboard-based homography estimation algorithm was proposed to estimate the local pose of the endoscopic camera, and the least square method was used to calculate the compensation matrix in combination with the optical tracking system.

RESULTS

In comparison with conventional methods, the proposed compensation method improved the performance of AR fusion, which reduced physical error by up to 82%, reduced pixel error by up to 83%, and improved target coverage by up to 6%. Experimental results of simulating mechanical noise revealed that the proposed compensation method effectively corrected the fusion errors caused by the rotation of the endoscopic tube without recalibrating the camera. Furthermore, the simulation results revealed the robustness of the proposed compensation method to noises.

CONCLUSIONS

Overall, the experiment results proved the effectiveness of the proposed compensation method and online calibration framework, and revealed a considerable potential in clinical practice.