Possibilities and limitations of current stereo-endoscopy

3D presentation in surgery: a review of technology and adverse effects

A systematic review was undertaken to assess the technology used to create stereovision for human perception. Adverse effects associated with artificial stereoscopic technology were reviewed with an emphasis on the impact of surgical performance in the operating room. MEDLINE/PubMed library databases were used to identify literature published up to Aug 2017. In the past 60 years, four major types of technologies have been used for reconstructing stereo images: anaglyph, polarization, active shutter, and autostereoscopy. As none of them can perfectly duplicate our natural stereoperception, user exposure to this artificial environment for a period of time can lead to a series of psychophysiological responses including nausea, dizziness, and others. The exact mechanism underlying these symptoms is not clear. Neurophysiologic evidences suggest that the visuo-vestibular pathway plays a vital role in coupling unnatural visual inputs to autonomic neural responses. When stereoscopic technology was used in surgical environments, controversial results were reported. Although recent advances in stereoscopy are promising, no definitive evidence has yet been presented to support that stereoscopes can enhance surgical performance in image-guided surgery. Stereoscopic technology has been rapidly introduced to healthcare. Adverse effects to human operators caused by immature technology seem inevitable. The impact on surgeons working with this visualization system needs to be explored and its safety and feasibility need to be addressed.

Image-based 3D surface approximation of the bladder using structure-from-motion for enhanced cystoscopy based on phantom data

Bladder cancer is likely to recur after resection. For this reason, bladder cancer survivors often undergo follow-up cystoscopy for years after treatment to look for bladder cancer recurrence. 3D modeling of the bladder could provide more reliable cystoscopic documentation by giving an overall picture of the organ and tumor positions. However, 3D reconstruction of the urinary bladder based on endoscopic images is challenging. This is due to the small field of view of the endoscope, considerable image distortion, and occlusion by urea, blood or particles. In this paper, we will demonstrate a method for the conversion of uncalibrated, monocular, endoscopic videos of the bladder into a 3D model using structure-from-motion (SfM). First of all, frames are extracted from video sequences. Distortions are then corrected in a calibration procedure. Finally, the 3D reconstruction algorithm generates a sparse surface approximation of the bladder lining based on the corrected frames. This method was tested using an endoscopic video of a phantom that mimics the rich structure of the bladder. The reconstructed 3D model covered a large part of the object, with an average reprojection error of 1.15 pixels and a relative accuracy of 99.4%.

Fiber bundle endomicroscopy with multi-illumination for three-dimensional reflectance image reconstruction

Bundled fiber optics allow in vivo imaging at deep sites in a body. The intrinsic optical contrast detects detailed structures in blood vessels and organs. We developed a bundled-fiber-coupled endomicroscope, enabling stereoscopic three-dimensional (3-D) reflectance imaging with a multipositional illumination scheme. Two illumination sites were attached to obtain reflectance images with left and right illumination. Depth was estimated by the horizontal disparity between the two images under alternative illuminations and was calibrated by the targets with known depths. This depth reconstruction was applied to an animal model to obtain the 3-D structure of blood vessels of the cerebral cortex (Cereb cortex) and preputial gland (Pre gla). The 3-D endomicroscope could be instrumental to microlevel reflectance imaging, improving the precision in subjective depth perception, spatial orientation, and identification of anatomical structures.

Modeling the convergence accommodation of stereo vision for binocular endoscopy

BACKGROUND

The stereo laparoscope is an important tool for achieving depth perception in robot-assisted minimally invasive surgery (MIS).

METHODS

A dynamic convergence accommodation algorithm is proposed to improve the viewing experience and achieve accurate depth perception. Based on the principle of the human vision system, a positional kinematic model of the binocular view system is established. The imaging plane pair is rectified to ensure that the two rectified virtual optical axes intersect at the fixation target to provide immersive depth perception.

RESULTS

Stereo disparity was simulated with the roll and pitch movements of the binocular system. The chessboard test and the endoscopic peg transfer task were performed, and the results demonstrated the improved disparity distribution and robustness of the proposed convergence accommodation method with respect to the position of the fixation target.

CONCLUSIONS

This method offers a new solution for effective depth perception with the stereo laparoscopes used in robot-assisted MIS.

Evolution of stereoscopic imaging in surgery and recent advances

In the late 1980s the first laparoscopic cholecystectomies were performed prompting a sudden rise in technological innovations as the benefits and feasibility of minimal access surgery became recognised. Monocular laparoscopes provided only two-dimensional (2D) viewing with reduced depth perception and contributed to an extended learning curve. Attention turned to producing a usable three-dimensional (3D) endoscopic view for surgeons; utilising different technologies for image capture and image projection. These evolving visual systems have been assessed in various research environments with conflicting outcomes of success and usability, and no overall consensus to their benefit. This review article aims to provide an explanation of the different types of technologies, summarise the published literature evaluating 3D vs 2D laparoscopy, to explain the conflicting outcomes, and discuss the current consensus view.

Depth Perception of Surgeons in Minimally Invasive Surgery

Minimally invasive surgery (MIS) poses visual challenges to the surgeons. In MIS, binocular disparity is not freely available for surgeons, who are required to mentally rebuild the 3-dimensional (3D) patient anatomy from a limited number of monoscopic visual cues. The insufficient depth cues from the MIS environment could cause surgeons to misjudge spatial depth, which could lead to performance errors thus jeopardizing patient safety. In this article, we will first discuss the natural human depth perception by exploring the main depth cues available for surgeons in open procedures. Subsequently, we will reveal what depth cues are lost in MIS and how surgeons compensate for the incomplete depth presentation. Next, we will further expand our knowledge by exploring some of the available solutions for improving depth presentation to surgeons. Here we will review the innovative approaches (multiple 2D camera assembly, shadow introduction) and devices (3D monitors, head-mounted devices, and auto-stereoscopic monitors) for 3D image presentation from the past few years.

3D Laparoscopy - Help or Hype; Initial Experience of A Tertiary Health Centre

INTRODUCTION

To evaluate the advantages of 3D laparoscopy and compare its significance with conventional 2D laparoscopy during various operative procedures.

METHODS

During present study, two groups were formed. Group A included patients who were operated using 3D laparoscopic imaging and Group B consisted of operated patients by 2D laparoscopy. Operative performance of both the groups was compared in terms of operative time and quality of imaging.

RESULTS

Operative time interval for various procedures was significantly less in Group A as compared to Group B. Also, imaging quality was far superior with use of 3D imaging system especially in terms of depth perception.

CONCLUSION

Advantages of 3D laparoscopy are well appreciated during operative procedures as previously documented by other studies in training models.

Stereoscopic visualization of laparoscope image using depth information from 3D model

Laparoscopic surgery is indispensable from the current surgical procedures. It uses an endoscope system of camera and light source, and surgical instruments which pass through the small incisions on the abdomen of the patients undergoing laparoscopic surgery. Conventional laparoscope (endoscope) systems produce 2D colored video images which do not provide surgeons an actual depth perception of the scene. In this work, the problem was formulated as synthesizing a stereo image of the monocular (conventional) laparoscope image by incorporating into them the depth information from a 3D CT model. Various algorithms of the computer vision including the algorithms for the feature detection, matching and tracking in the video frames, and for the reconstruction of 3D shape from shading in the 2D laparoscope image were combined for making the system. The current method was applied to the laparoscope video at the rate of up to 5 frames per second to visualize its stereo video. A correlation was investigated between the depth maps calculated with our method with those from the shape from shading algorithm. The correlation coefficients between the depth maps were within the range of 0.70-0.95 (P<0.05). A t-test was used for the statistical analysis.

Three-dimensional representation software as image enhancement tool in small-bowel capsule endoscopy: a feasibility study

BACKGROUND

Three-dimensional imaging in capsule endoscopy is not currently feasible due to hardware limitations. However, software algorithms that enable three-dimensional reconstruction in capsule endoscopy are available.

METHODS

Feasibility study. A phantom was designed to test the accuracy of three-dimensional reconstruction. Thereafter, 192 small-bowel capsule endoscopy images (of vascular: 50; inflammatory: 73; protruding structures: 69) were reviewed with the aid of a purpose-built three-dimensional reconstruction software. Seven endoscopists rated visualisation improved or non-improved. Subgroup analyses performed for diagnostic category, diagnosis, image surface morphology and colour and SBCE equipment used (PillCam(®) vs. MiroCam(®)).

RESULTS

Overall, phantom experiments showed that the three-dimensional reconstruction software was accurate at 90% of red, 70% of yellow and 45% of white phantom models. Enhanced visualisation for 56% of vascular, 23% of inflammatory and <10% of protruding structures was noted (P=0.007, 0.172 and 0.008, respectively). Furthermore, three-dimensional software application enhanced 53.7% of red, 21.8% of white, 17.3% of red and white, and 9.2% of images of lesions with colour similar to that of the surrounding mucosa, P<0.0001.

CONCLUSIONS

Application of a three-dimensional reconstruction software in capsule endoscopy leads to image enhancement for a significant proportion of vascular, but less so for inflammatory and protruding lesions. Until optics technology allows hardware-enabled three-dimensional reconstruction, it seems a plausible alternative.

Viability of topical endoscopic imaging system for vitreous surgery in rabbit eyes

Vitreous surgery was performed in four rabbit eyes to investigate the viability of a topical endoscopic imaging system. This technique was evaluated during core vitrectomy, artificial posterior vitreous detachment, peripheral vitrectomy, production of intentional retinal breaks, and endophotocoagulation. To observe the posterior fundus during the surgeries, the endoscope was fixed in place. In addition, by tilting the endoscope toward the ciliary body, it was possible to clearly observe the trocar ports. The endoscope was easy to use and obtained good resolution. In addition, all procedures could be performed accurately when using the technique. The current study demonstrated that the topical endoscopic imaging system is a convenient method for obtaining both wide-field viewing and panfocal imaging during vitrectomy. This method may prove to be a new observational tool that can be employed during intraocular surgeries in the clinical setting.

Three-dimensional reconstruction of the digestive wall in capsule endoscopy videos using elastic video interpolation

Wireless capsule endoscopy is a revolutionary technology that allows physicians to examine the digestive tract of a human body in the minimum invasive way. Physicians can detect diseases such as blood-based abnormalities, polyps, ulcers, and Crohn's disease. Although this technology is really a marvel of our modern times, currently it suffers from two serious drawbacks: 1) frame rate is low (3 frames/s) and 2) no 3-D representation of the objects is captured from the camera of the capsule. In this paper we offer solutions (methodologies) that deal with each of the above issues improving the current technology without forcing hardware upgrades. These methodologies work synergistically to create smooth and visually friendly interpolated images from consecutive frames, while preserving the structure of the observed objects. They also extract and represent the texture of the surface of the digestive tract in 3-D. Thus the purpose of our methodology is not to reduce the time that the gastroenterologists need to spend to examine the video. On the contrary, the purpose is to enhance the video and therefore improve the viewing of the digestive tract leading to a more qualitative and efficient examination. The proposed work introduces 3-D capsule endoscopy textured results that have been welcomed by Digestive Specialists, Inc., Dayton, OH. Finally, illustrative results are given at the end of the paper.

Evaluation of three laparoscopic modalities: robotics versus three-dimensional vision laparoscopy versus standard laparoscopy

PURPOSE

Standard laparoscopy has undergone many recent advances with the advent of three-dimensional visual systems and robotic surgical systems. In evaluating the usefulness of these new systems, it is difficult to objectively measure their advantages in the operating room. Therefore, we designed a trial using three different laparoscopic modalities to evaluate the strengths and weaknesses of each modality.

MATERIALS AND METHODS

Twenty-seven subjects were entered into the study. Three different laparoscopic modalities were tested. These included standard laparoscopy with two-dimensional cameras, the 3Di Endosite visual system, and the daVinci Robotic Surgical System. A standard laparoscopic trainer was utilized and testing consisted of three different tasks: peg transfer, ring manipulation, and cannulation.

RESULTS

Of the 27 subjects, 16 (60%) reported some degree of laparoscopic experience. The number of pegs transferred with standard laparoscopy and the Endosite 3Di system was significantly greater than with the robot. The number of errors committed during the peg transfer test and the amount of time required was significantly lower with the Endosite 3Di system compared to the robot. Subjects completed the ring manipulation task significantly faster with the robot, but the number of errors committed was no different among the three modalities. Subjects were able to complete the cannulation task with their dominant hand significantly faster with the robot compared to the Endosite 3Di system or standard laparoscopy, and committed fewer errors using the robot compared to standard laparoscopy.

CONCLUSIONS

This study showed improved performance using three-dimensional optics on some tasks, but not a significant improvement in overall results. Three-dimensional vision does appear beneficial during performance of some complex tasks. The wrist-like action of the robot improved performance on some tasks, while the lack of tactile feedback likely was a source of errors on other tasks.

Perceptual distortions produce multidimensional stress profiles in novice users of an endoscopic surgery simulator

OBJECTIVES

We determine the impact of perceptual-motor distortions on multidimensional stress dynamics in novice users of an endoscopic/laparoscopic surgery simulator during performance of a peg-transfer task.

BACKGROUND

Surgeons find the endoscopic/laparoscopic surgery procedure to be more mentally stressful than open surgery. This investigation was designed to identify specific stress dimensions associated with these procedures and to determine the contributions to that stress made by loss of depth information resulting from image-guided views of the surgical field and by disruption of eye-hand mapping. Because stress reactions might depend upon familiarity with these procedures, the study focused upon novice participants.

METHOD

An endoscopic box-simulator featured in surgical training was used in conjunction with the Dundee Stress State Questionnaire, a well-validated multidimensional stress state instrument. A control group (no perceptual distortions) viewed the simulated "surgical field" directly. Two other groups viewed the surgical field through TV images in which spatial rotation of the images was absent or in which the images were rotated 90 degrees from the actual line of sight.

RESULTS

Performance efficiency in the simulator varied inversely with the degree of perceptual-motor distortion. Reactions reflecting increased task coping were observed in all groups. These were accompanied in the image groups by negative reactions involving decreases in hedonic tone and control and confidence and an increase in tense arousal.

CONCLUSIONS

Perceptual-motor distortions are sources of complex task-induced stress profiles in novices using an endoscopic surgery simulator.

APPLICATION

Procedures to reduce stress in endoscopic/laparoscopic surgery trainees may benefit from knowledge regarding specific stress dimensions involved.

Three-dimensional laparoscope based on the manipulation of polarized light by a cellophane half-wave plate

A three-dimensional laparoscope that can capture three-dimensional images during surgery is reported. The principle is solely based on the manipulation of polarized light by a cellophane half-wave plate rather than computer processing; hence there is no time delay (it operates in real time) and lesions are viewed in true color, which is important for diagnostics. Three-dimensional images are obtained with a single laparoscope. A unique feature of this three-dimensional laparoscope is that it includes a virtual ruler to measure distances without physically touching the affected areas. The structure is simple, sturdy, lightweight, and its diameter is no bigger than a standard 10 mm diam laparoscope.