Real-Time Retinal Vessel Mapping and Localization for Intraocular Surgery

EyeSAM: graph-based localization and mapping of retinal vasculature during intraocular microsurgery

PURPOSE

Robot-assisted intraocular microsurgery can improve performance by aiding the surgeon in operating on delicate micron-scale anatomical structures of the eye. In order to account for the eyeball motion that is typical in intraocular surgery, there is a need for fast and accurate algorithms that map the retinal vasculature and localize the retina with respect to the microscope.

METHODS

This work extends our previous work by a graph-based SLAM formulation using a sparse incremental smoothing and mapping (iSAM) algorithm.

RESULTS

The resulting technique, "EyeSAM," performs SLAM for intraoperative vitreoretinal surgical use while avoiding spurious duplication of structures as with the previous simpler technique. The technique also yields reduction in average pixel error in the camera motion estimation.

CONCLUSIONS

This work provides techniques to improve intraoperative tracking of retinal vasculature by handling loop closures and achieving increased robustness to quick shaky motions and drift due to uncertainties in the motion estimation.

Autonomous Positioning of Eye Surgical Robot Using the Tool Shadow and Kalman Filtering

Vitreoretinal surgery is one of the most difficult surgical operations, even for experienced surgeons. Thus, a master-slave eye surgical robot has been developed to assist the surgeon in safely performing vitreoretinal surgeries; however, in the master-slave control, the robotic positioning accuracy depends on the surgeon's coordination skills. This paper proposes a new method of autonomous robotic positioning using the shadow of the surgical instrument. First, the microscope image is segmented into three regions-namely, a micropipette, its shadow, and the eye ground-using a Gaussian mixture model (GMM). The tips of the micropipette and its shadow are then extracted from the contour lines of the segmented regions. The micropipette is then autonomously moved down to the simulated eye ground until the distance between the tips of micropipette and its shadow in the microscopic image reaches a predefined threshold. To handle possible occlusions, the tip of the shadow is estimated using a Kalman filter. Experiments to evaluate the robotic positioning accuracy in the vertical direction were performed. The results show that the autonomous positioning using the Kalman filter enhanced the accuracy of robotic positioning.

Toward Monocular Camera-Guided Retinal Vein Cannulation with an Actively Stabilized Handheld Robot

In this paper we describe work towards retinal vessel cannulation using an actively stabilized handheld robot, guided by monocular vision. We employ a previously developed monocular camera based surface reconstruction method using automated laser beam scanning over the retina. We use the reconstructed plane to find a coordinate transform between the 2D image plane coordinate system and the global 3D frame. Within a hemispherical region around the target, we use motion scaling for higher precision. The contribution of this work is the homography matrix estimation using monocular vision and application of the previously developed laser surface reconstruction to Micron guided vein cannulation. Experiments are conducted in a wet eye phantom to show the higher accuracy of the surface reconstruction as compared to standard stereo reconstruction. Further, experiments to show the increased surgical accuracy due to motion scaling are also carried out.

EyeSLAM: Real-time simultaneous localization and mapping of retinal vessels during intraocular microsurgery

BACKGROUND

Fast and accurate mapping and localization of the retinal vasculature is critical to increasing the effectiveness and clinical utility of robot-assisted intraocular microsurgery such as laser photocoagulation and retinal vessel cannulation.

METHODS

The proposed EyeSLAM algorithm delivers 30 Hz real-time simultaneous localization and mapping of the human retina and vasculature during intraocular surgery, combining fast vessel detection with 2D scan-matching techniques to build and localize a probabilistic map of the vasculature.

RESULTS

In the harsh imaging environment of retinal surgery with high magnification, quick shaky motions, textureless retina background, variable lighting and tool occlusion, EyeSLAM can map 75% of the vessels within two seconds of initialization and localize the retina in real time with a root mean squared (RMS) error of under 5.0 pixels (translation) and 1° (rotation).

CONCLUSIONS

EyeSLAM robustly provides retinal maps and registration that enable intelligent surgical micromanipulators to aid surgeons in simulated retinal vessel tracing and photocoagulation tasks.

Comparative Evaluation of Handheld Robot-Aided Intraocular Laser Surgery

This paper presents robot-aided intraocular laser surgery using a handheld robot known as Micron. The micromanipulator incorporated in Micron enables visual servoing of a laser probe, while maintaining a constant distance of the tool tip from the retinal surface. The comparative study was conducted with various control methods for evaluation of robot-aided intraocular laser surgery.

Handheld-automated microsurgical instrumentation for intraocular laser surgery

BACKGROUND AND OBJECTIVE

Laser photocoagulation is a mainstay or adjuvant treatment for a variety of common retinal diseases. Automated laser photocoagulation during intraocular surgery has not yet been established. The authors introduce an automated laser photocoagulation system for intraocular surgery, based on a novel handheld instrument. The goals of the system are to enhance accuracy and efficiency and improve safety.

MATERIALS AND METHODS

Triple-ring patterns are introduced as a typical arrangement for the treatment of proliferative retinopathy and registered to a preoperative fundus image. In total, 32 target locations are specified along the circumferences of three rings having diameters of 1, 2, and 3 mm, with a burn spacing of 600 μm. Given the initial system calibration, the retinal surface is reconstructed using stereo vision, and the targets specified on the preoperative image are registered with the control system. During automated operation, the laser probe attached to the manipulator of the active handheld instrument is deflected as needed via visual servoing in order to correct the error between the aiming beam and a specified target, regardless of any erroneous handle motion by the surgeon. A constant distance of the laser probe from the retinal surface is maintained in order to yield consistent size of burns and ensure safety during operation. Real-time tracking of anatomical features enables compensation for any movement of the eye. A graphical overlay system within operating microscope provides the surgeon with guidance cues for automated operation. Two retinal surgeons performed automated and manual trials in an artificial model of the eye, with each trial repeated three times. For the automated trials, various targeting thresholds (50-200 μm) were used to automatically trigger laser firing. In manual operation, fixed repetition rates were used, with frequencies of 1.0-2.5 Hz. The power of the 532 nm laser was set at 3.0 W with a duration of 20 ms. After completion of each trial, the speed of operation and placement error of burns were measured. The performance of the automated laser photocoagulation was compared with manual operation, using interpolated data for equivalent firing rates from 1.0 to 1.75 Hz.

RESULTS

In automated trials, average error increased from 45 ± 27 to 60 ± 37 μm as the targeting threshold varied from 50 to 200 μm, while average firing rate significantly increased from 0.69 to 1.71 Hz. The average error in the manual trials increased from 102 ± 67 to 174 ± 98 μm as firing rate increased from 1.0 to 2.5 Hz. Compared to the manual trials, the average error in the automated trials was reduced by 53.0-56.4%, resulting in statistically significant differences (P ≤ 10(-20) ) for all equivalent frequencies (1.0-1.75 Hz). The depth of the laser tip in the automated trials was consistently maintained within 18 ± 2 μm root-mean-square (RMS) of its initial position, whereas it significantly varied in the manual trials, yielding an error of 296 ± 30 μm RMS. At high firing rates in manual trials, such as at 2.5 Hz, laser photocoagulation is marginally attained, yielding failed burns of 30% over the entire pattern, whereas no failed burns are found in automated trials. Relatively regular burn sizes are attained in the automated trials by the depth servoing of the laser tip, while burn sizes in the manual trials vary considerably. Automated avoidance of blood vessels was also successfully demonstrated, utilizing the retina-tracking feature to identify avoidance zones.

CONCLUSION

Automated intraocular laser surgery can improve the accuracy of photocoagulation while ensuring safety during operation. This paper provides an initial demonstration of the technique under reasonably realistic laboratory conditions; development of a clinically applicable system requires further work.

Manipulator Design and Operation for a Six-Degree-of-Freedom Handheld Tremor-Canceling Microsurgical Instrument

This paper presents the design and actuation of a six-degree-of-freedom (6-DOF) manipulator for a handheld instrument, known as "Micron," which performs active tremor compensation during microsurgery. The design incorporates a Gough-Stewart platform based on piezoelectric linear motor, with a specified minimum workspace of a cylinder 4 mm long and 4 mm in diameter at the end-effector. Given the stall force of the motors and the loading typically encountered in vitreoretinal microsurgery, the dimensions of the manipulator are optimized to tolerate a transverse load of 0.2 N on a remote center of motion near the midpoint of the tool shaft. The optimization yields a base diameter of 23 mm and a height of 37 mm. The fully handheld instrument includes a custom-built optical tracking system for control feedback, and an ergonomic housing to serve as a handle. The manipulation performance was investigated in both clamped and handheld conditions. In positioning experiments with varying side loads, the manipulator tolerates side load up to 0.25 N while tracking a sinusoidal target trajectory with less than 20 μm error. Physiological hand tremor is reduced by about 90% in a pointing task, and error less than 25 μm is achieved in handheld circle-tracing.

Toward Automated Intraocular Laser Surgery Using a Handheld Micromanipulator

This paper presents a technique for automated intraocular laser surgery using a handheld micromanipulator known as Micron. The novel handheld manipulator enables the automated scanning of a laser probe within a cylinder of 4 mm long and 4 mm in diameter. For the automation, the surface of the retina is reconstructed using a stereomicroscope, and then preplanned targets are placed on the surface. The laser probe is precisely located on the target via visual servoing of the aiming beam, while maintaining a specific distance above the surface. In addition, the system is capable of tracking the surface of the eye in order to compensate for any eye movement introduced during the operation. We compared the performance of the automated scanning using various control thresholds, in order to find the most effective threshold in terms of accuracy and speed. Given the selected threshold, we conducted the handheld operation above a fixed target surface. The average error and execution time are reduced by 63.6% and 28.5%, respectively, compared to the unaided trials. Finally, the automated laser photocoagulation was demonstrated also in an eye phantom, including compensation for the eye movement.