Transtympanic endoscopy for diagnosis of middle ear pathology

Registration of preoperative temporal bone CT-scan to otoendoscopic video for augmented-reality based on convolutional neural networks

PURPOSE

Patient-to-image registration is a preliminary step required in surgical navigation based on preoperative images. Human intervention and fiducial markers hamper this task as they are time-consuming and introduce potential errors. We aimed to develop a fully automatic 2D registration system for augmented reality in ear surgery.

METHODS

CT-scans and corresponding oto-endoscopic videos were collected from 41 patients (58 ears) undergoing ear examination (vestibular schwannoma before surgery, profound hearing loss requiring cochlear implant, suspicion of perilymphatic fistula, contralateral ears in cases of unilateral chronic otitis media). Two to four images were selected from each case. For the training phase, data from patients (75% of the dataset) and 11 cadaveric specimens were used. Tympanic membranes and malleus handles were contoured on both video images and CT-scans by expert surgeons. The algorithm used a U-Net network for detecting the contours of the tympanic membrane and the malleus on both preoperative CT-scans and endoscopic video frames. Then, contours were processed and registered through an iterative closest point algorithm. Validation was performed on 4 cases and testing on 6 cases. Registration error was measured by overlaying both images and measuring the average and Hausdorff distances.

RESULTS

The proposed registration method yielded a precision compatible with ear surgery with a 2D mean overlay error of 0.65 ± 0.60 mm for the incus and 0.48 ± 0.32 mm for the round window. The average Hausdorff distance for these 2 targets was 0.98 ± 0.60 mm and 0.78 ± 0.34 mm respectively. An outlier case with higher errors (2.3 mm and 1.5 mm average Hausdorff distance for incus and round window respectively) was observed in relation to a high discrepancy between the projection angle of the reconstructed CT-scan and the video image. The maximum duration for the overall process was 18 s.

CONCLUSIONS

A fully automatic 2D registration method based on a convolutional neural network and applied to ear surgery was developed. The method did not rely on any external fiducial markers nor human intervention for landmark recognition. The method was fast and its precision was compatible with ear surgery.

Minimally invasive ossiculoplasty via an endoscopic transtympanic approach

INTRODUCTION

The aim of this study was to evaluate the feasibility of ossiculoplasty via a minimally invasive endoscopic transtympanic approach (ETTA).

CASE SERIES

We investigated the exposure of target structures (incus and stapes) on 4 human temporal bones by placing an endoscope into the middle ear cleft through the 4 tympanic quadrants. Then, on 3 additional specimens, we performed an incudostapedial disjunction and repaired it with a drop of hydroxyapatite cement via ETTA. We measured the size of tympanic perforation, and the acoustic transfer function of the middle ear (125-8000Hz) before and after repair by placing an insert in the external auditory canal and recording the acoustic signal in the utricle by a microphone. The acoustic signal gain was estimated in dB μV. Exposure was similar in all four quadrants but ergonomics was better with a posteroinferior myringotomy. Ossicular chain repair was conducted successfully in all cases and the acoustic transfer function of the middle ear was significantly improved. Residual tympanic perforation (n=3) was 2±0.3mm in diameter.

CONCLUSION

ETTA to reconstruct incudostapedial joint with bone cement was feasible and effective. It opens perspectives for robot-based procedures guided by augmented reality.

Vision-Based Augmented Reality System for Middle Ear Surgery: Evaluation in Operating Room Environment

HYPOTHESIS

Augmented reality (AR) solely based on image features is achievable in operating room conditions and its precision is compatible with otological surgery.

BACKGROUND

The objective of this work was to evaluate the performance of a vision-based AR system for middle ear surgery in the operating room conditions.

METHODS

Nine adult patients undergoing ossicular procedures were included in this prospective study. AR was obtained by combining real-time video from the operating microscope with the virtual image obtained from the preoperative computed tomography (CT)-scan. Initial registration between the video and the virtual CT image was achieved using manual selection of six points on the tympanic sulcus. Patient-microscope movements during the procedure were tracked using image-feature matching algorithm. The microscope was randomly moved at an approximated speed of 5 mm/s in the three axes of space and rotation for 180 seconds. The accuracy of the system was assessed by calculating the distance between each fiducial point selected on the video image and its corresponding point on the scanner.

RESULTS

AR could be obtained for at least 3 minutes in seven out of nine patients. The overlay fiducial and target registration errors were 0.38 ± 0.23 mm (n = 7) and 0.36 ± 0.15 mm (n = 5) respectively, with a drift error of 1.2 ± 0.5 μm/s. The system was stable throughout the procedure and achieved a refresh rate of 12 fps. Moderate bleeding and introduction of surgical instruments did not compromise the performance of the system.

CONCLUSION

The AR system yielded sub-millimetric accuracy and remained stable throughout the experimental study despite patient-microscope movements and field of view obtrusions.

Video-based augmented reality combining CT-scan and instrument position data to microscope view in middle ear surgery

The aim of the study was to develop and assess the performance of a video-based augmented reality system, combining preoperative computed tomography (CT) and real-time microscopic video, as the first crucial step to keyhole middle ear procedures through a tympanic membrane puncture. Six different artificial human temporal bones were included in this prospective study. Six stainless steel fiducial markers were glued on the periphery of the eardrum, and a high-resolution CT-scan of the temporal bone was obtained. Virtual endoscopy of the middle ear based on this CT-scan was conducted on Osirix software. Virtual endoscopy image was registered to the microscope-based video of the intact tympanic membrane based on fiducial markers and a homography transformation was applied during microscope movements. These movements were tracked using Speeded-Up Robust Features (SURF) method. Simultaneously, a micro-surgical instrument was identified and tracked using a Kalman filter. The 3D position of the instrument was extracted by solving a three-point perspective framework. For evaluation, the instrument was introduced through the tympanic membrane and ink droplets were injected on three middle ear structures. An average initial registration accuracy of 0.21 ± 0.10 mm (n = 3) was achieved with a slow propagation error during tracking (0.04 ± 0.07 mm). The estimated surgical instrument tip position error was 0.33 ± 0.22 mm. The target structures’ localization accuracy was 0.52 ± 0.15 mm. The submillimetric accuracy of our system without tracker is compatible with ear surgery.