Surgical microscope integrated MHz SS-OCT with live volumetric visualization

Context-Aware Real-Time Semantic View Expansion of Intraoperative 4D OCT

Four-dimensional microscope-integrated optical coherence tomography enables volumetric imaging of tissue structures and tool-tissue interactions in ophthalmic surgery at interactive update rates. This enables surgeons to undertake particular surgical steps under four-dimensional optical coherence tomography (4D OCT) guidance. However, current 4D OCT systems are limited by their field of view and signal quality. Both are attributable to the emphasis on high volume acquisition rates, which is critical for smooth visual perception by the surgeon. Existing 3D volume mosaicing methods are developed in the context of diagnostic imaging and do not take dynamic surgical interactions and real-time processing into account. In this paper, we propose a novel volume mosaicing and visualization methodology that not only aims at leveraging the temporal information to overcome some of the current limitations and imaging artifacts of 4D OCT, but also is aware of the surgical context and dynamic instrument motion implicitly during registration and explicitly for visualization. We propose a rapid 4-degrees of freedom volume registration, integrating an innovative approach for volume mosaicing that takes temporal recency and semantic information into account for enhanced surgical visualization. Our experiments on 4D OCT datasets demonstrate high registration accuracy and illustrate the benefits for visualization by reducing imaging artifacts and dynamically expanding the surgical view.

Intraoperative adaptive eye model based on instrument-integrated OCT for robot-assisted vitreoretinal surgery

PURPOSE

Pars plana vitrectomy (PPV) is the most common surgical procedure performed by retinal specialists, highlighting the need for model-based assistance and automation in surgical treatment. An intraoperative retinal model provides precise anatomical information relative to the surgical instrument, enhancing surgical precision and safety.

METHODS

This work focuses on the intraoperative parametrization of retinal shape using 1D instrument-integrated optical coherence tomography distance measurements combined with a surgical robot. Our approach accommodates variability in eye geometries by transitioning from an initial spherical model to an ellipsoidal representation, improving accuracy as more data is collected through sensor motion.

RESULTS

We demonstrate that ellipsoid fitting outperforms sphere fitting for regular eye shapes, achieving a mean absolute error of less than 40  μ m in simulation and below 200  μ m on 3D printed models and ex vivo porcine eyes. The model reliably transitions from a spherical to an ellipsoidal representation across all six tested eye shapes when specific criteria are satisfied.

CONCLUSION

The adaptive eye model developed in this work meets the accuracy requirements for clinical application in PPV within the central retina. Additionally, the global model effectively extrapolates beyond the scanned area to encompass the retinal periphery.This capability enhances PPV procedures, particularly through virtual boundary assistance and improved surgical navigation, ultimately contributing to safer surgical outcomes.

From tissue to sound: A new paradigm for medical sonic interaction design

Medical imaging maps tissue characteristics into image intensity values, enhancing human perception. However, comprehending this data, especially in high-stakes scenarios such as surgery, is prone to errors. Additionally, current multimodal methods do not fully leverage this valuable data in their design. We introduce "From Tissue to Sound," a new paradigm for medical sonic interaction design. This paradigm establishes a comprehensive framework for mapping tissue characteristics to auditory displays, providing dynamic and intuitive access to medical images that complement visual data, thereby enhancing multimodal perception. "From Tissue to Sound" provides an advanced and adaptable framework for the interactive sonification of multimodal medical imaging data. This framework employs a physics-based sound model composed of a network of multiple oscillators, whose mechanical properties-such as friction and stiffness-are defined by tissue characteristics extracted from imaging data. This approach enables the representation of anatomical structures and the creation of unique acoustic profiles in response to excitations of the sound model. This method allows users to explore data at a fundamental level, identifying tissue characteristics ranging from rigid to soft, dense to sparse, and structured to scattered. It facilitates intuitive discovery of both general and detailed patterns with minimal preprocessing. Unlike conventional methods that transform low-dimensional data into global sound features through a parametric approach, this method utilizes model-based unsupervised mapping between data and an anatomical sound model, enabling high-dimensional data processing. The versatility of this method is demonstrated through feasibility experiments confirming the generation of perceptually discernible acoustic signals. Furthermore, we present a novel application developed based on this framework for retinal surgery. This new paradigm opens up possibilities for designing multisensory applications for multimodal imaging data. It also facilitates the creation of interactive sonification models with various auditory causality approaches, enhancing both directness and richness.

Investigating the impact of different time delays between OCT signal and k-clock signal on the structural and vascular imaging in SS-OCT

High-quality swept-source optical coherence tomography (SS-OCT) imaging systems require precise synchronization between the OCT signal and the k-clock signal. However, in practical applications, an uncertain time delay between these signals can cause inaccurate k-space sampling, leading to degraded imaging resolution. This study first simulates the axial resolution degradation curve caused by varying time delays and experimentally validates the results. Additionally, the effects of different time delays on both OCT structural and blood flow images are systematically investigated through experiments. To address this issue, a numerical calibration method is implemented to compensate for the nonlinear phase component. This approach involves acquiring two reflection signals at different depths, unwrapping the phase, performing high-order polynomial fitting, and removing nonlinear phase components induced by time delay, which effectively corrects the resolution degradation. Experiments conducted on semi-transparent white tape, blood flow phantom, and human nailfold demonstrate that the proposed correction algorithm significantly improves the axial resolution of both structural and blood flow images. The findings indicate that our investigation and the developed calibration method are instrumental in reconstructing high-resolution SS-OCT images, which are essential for accurate diagnosis and effective treatment monitoring in clinical applications.

Robotics and optical coherence tomography: current works and future perspectives [Invited]

Optical coherence tomography (OCT) is an interferometric technique for micron-level imaging in biological and non-biological contexts. As a non-invasive, non-ionizing, and video-rate imaging modality, OCT is widely used in biomedical and clinical applications, especially ophthalmology, where it functions in many roles, including tissue mapping, disease diagnosis, and intrasurgical visualization. In recent years, the rapid growth of medical robotics has led to new applications for OCT, primarily for 3D free-space scanning, volumetric perception, and novel optical designs for specialized medical applications. This review paper surveys these recent developments at the intersection of OCT and robotics and organizes them by degree of integration and application, with a focus on biomedical and clinical topics. We conclude with perspectives on how these recent innovations may lead to further advances in imaging and medical technology.

New Directions for Ophthalmic OCT - Handhelds, Surgery, and Robotics

The introduction of optical coherence tomography (OCT) in the 1990s revolutionized diagnostic ophthalmic imaging. Initially, OCT's role was primarily in the adult ambulatory ophthalmic clinics. Subsequent advances in handheld form factors, integration into surgical microscopes, and robotic assistance have expanded OCT's utility and impact outside of its initial environment in the adult outpatient ophthalmic clinic. In this review, we cover the use of OCT in the neonatal intensive care unit (NICU) environment with a handheld OCT, recent developments in intraoperative OCT for data visualization and measurements, and recent work and demonstration of robotically aligned OCT systems outside of eye clinics. Of note, advances in these areas are a legacy of our colleague, the late Joseph Izatt. OCT has been an important innovation for ocular diagnostics, and these advances have helped it continue to extend in new directions.

Vitreoschisis-induced vitreous cortex remnants in proliferative vitreoretinopathy: A comprehensive review from basic research to clinical practice

Proliferative vitreoretinopathy (PVR) significantly impacts the prognosis of rhegmatogenous retinal detachment (RRD), one of the most critical and increasing causes of vision loss in the Western world. Despite advancements in surgical instruments and techniques, the failure rate due to PVR remains substantial, necessitating additional surgeries and often leading to unsatisfactory visual outcomes. This comprehensive review explores the role of vitreoschisis-induced vitreous cortex remnants (VCR) as a critical, previously under-recognised factor contributing to PVR. Vitreoschisis, a phenomenon where the inner lamellae of the posterior vitreous cortex detach while the outermost layers remain attached to the retina, creates VCR that may contain hyalocytes and serve as scaffolds for fibrocellular proliferation. These remnants are difficult to visualise without triamcinolone acetonide (TA) staining, leading to their frequent lack of recognition in clinical practice. Moreover, removing VCR can be challenging and time-consuming, often requiring meticulous surgical techniques to avoid retinal damage and ensure complete elimination. This review consolidates insights from basic research and clinical practice, emphasising the importance of complete vitreous removal and effective VCR detection and removal to mitigate PVR risks. It highlights the histopathological and clinical evidence supporting the hypothesis that VCR, containing hyalocytes, play a pivotal role in preretinal membrane formation. The review also discusses epidemiological data, surgical management strategies and potential future directions, including improved visualisation techniques and the development of new surgical tools and methods. This review aims to improve surgical outcomes and reduce the frequency and burden of RRD-related complications by addressing VCR as a critical factor in PVR.

Microscope integrated MHz optical coherence tomography system for neurosurgery: development and clinical in-vivo imaging

Neurosurgical interventions on the brain are impeded by the requirement to keep damages to healthy tissue at a minimum. A new contrast channel enhancing the visual separation of malign tissue should be created. A commercially available surgical microscope was modified with adaptation optics adapting the MHz speed optical coherence tomography (OCT) imaging system developed in our group. This required the design of a scanner optics and beam delivery system overcoming constraints posed by the mechanical and optical parameters of the microscope. High quality volumetric OCT C-scans with dense sample spacing can be acquired in-vivo as part of surgical procedures within seconds and are immediately available for post-processing.

Real-time mapping of photo-sono therapy induced cavitation using Doppler optical coherence tomography

Photo-sono therapy (PST) is an innovative anti-vascular approach based on cavitation-induced spallation. Currently, passive cavitation detection (PCD) is the prevalent technique for cavitation monitoring during treatment. However, the limitations of PCD are the lack of spatial information of bubbles and the difficulty of integration with the PST system. To address this, we proposed a new, to the best of our knowledge, cavitation mapping method that integrates Doppler optical coherence tomography (OCT) with PST to visualize bubble dynamics in real time. The feasibility of the proposed system has been confirmed through experiments on vascular-mimicking phantoms and in vivo rabbit ear vessels, and the results are compared to high-speed camera observations and PCD data. The findings demonstrate that Doppler OCT effectively maps cavitation in real time and holds promise for guiding PST treatments and other cavitation-related clinical applications.

Wide Dynamic Range Digital Aberration Measurement and Fast Anterior-Segment OCT Imaging †

Ocular aberrometry with a wide dynamic range for assessing vision performance and anterior segment imaging that provides anatomical details of the eye are both essential for vision research and clinical applications. Defocus error is a major limitation of digital wavefront aberrometry (DWA), as the blurring of the detected point spread function (PSF) significantly reduces the signal-to-noise ratio (SNR) beyond the ±3 D range. With the aid of Badal-like precompensation of defocus, the dynamic defocus range of the captured aberrated PSFs can be effectively extended. We demonstrate a dual-modality MHz VCSEL-based swept-source OCT (SS-OCT) system with easy switching between DWA and OCT imaging modes. The system is capable of measuring aberrations with defocus dynamic range of 20 D as well as providing fast anatomical imaging of the anterior segment at an A-scan rate of 1.6 MHz.

Exploring the Needle Tip Interaction Force with Retinal Tissue Deformation in Vitreoretinal Surgery

Recent advancements in age-related macular degeneration treatments necessitate precision delivery into the subretinal space, emphasizing minimally invasive procedures targeting the retinal pigment epithelium (RPE)-Bruch's membrane complex without causing trauma. Even for skilled surgeons, the inherent hand tremors during manual surgery can jeopardize the safety of these critical interventions. This has fostered the evolution of robotic systems designed to prevent such tremors. These robots are enhanced by FBG sensors, which sense the small force interactions between the surgical instruments and retinal tissue. To enable the community to design algorithms taking advantage of such force feedback data, this paper focuses on the need to provide a specialized dataset, integrating optical coherence tomography (OCT) imaging together with the aforementioned force data. We introduce a unique dataset, integrating force sensing data synchronized with OCT B-scan images, derived from a sophisticated setup involving robotic assistance and OCT integrated microscopes. Furthermore, we present a neural network model for image-based force estimation to demonstrate the dataset's applicability.

Visualization of Cataract Surgery Steps With 4D Microscope-Integrated Swept-Source Optical Coherence Tomography in Ex Vivo Porcine Eyes

Purpose

To investigate the visualization capabilities of high-speed swept-source optical coherence tomography (SS-OCT) in cataract surgery.

Methods

Cataract surgery was simulated in wet labs with ex vivo porcine eyes. Each phase of the surgery was visualized with a novel surgical microscope-integrated SS-OCT with a variable imaging speed of over 1 million A-scans per second. It was designed to provide four-dimensional (4D) live-volumetric videos, live B-scans, and volume capture scans.

Results

Four-dimensional videos, B-scans, and volume capture scans of corneal incision, ophthalmic viscosurgical device injection, capsulorrhexis, phacoemulsification, intraocular lens (IOL) injection, and position of unfolded IOL in the capsular bag were recorded. The flexibility of the SS-OCT system allowed us to tailor the scanning parameters to meet the specific demands of dynamic surgical steps and static pauses. The entire length of the eye was recorded in a single scan, and unfolding of the IOL was visualized dynamically.

Conclusions

The presented novel visualization method for fast ophthalmic surgical microscope-integrated intraoperative OCT imaging in cataract surgery allowed the visualization of all major steps of the procedure by achieving large imaging depths covering the entire eye and high acquisition speeds enabling live volumetric 4D-OCT imaging. This promising technology may become an integral part of routine and advanced robotic-assisted cataract surgery in the future.

Translational Relevance

We demonstrate the visualization capabilities of a cutting edge swept-source OCT system integrated into an ophthalmic surgical microscope during cataract surgery.

Virtual Hall sensor triggered multi-MHz endoscopic OCT imaging for stable real-time visualization

Circumferential scanning in endoscopic imaging is crucial across various disciplines, and optical coherence tomography (OCT) is often the preferred choice due to its high-speed, high-resolution, and micron-scale imaging capabilities. Moreover, real-time and high-speed 3D endoscopy is a pivotal technology for medical screening and precise surgical guidance, among other applications. However, challenges such as image jitter and non-uniform rotational distortion (NURD) are persistent obstacles that hinder real-time visualization during high-speed OCT procedures. To address this issue, we developed an innovative, low-cost endoscope that employs a brushless DC motor for scanning, and a sensorless technique for triggering and synchronizing OCT imaging with the scanning motor. This sensorless approach uses the motor's electrical feedback (back electromotive force, BEMF) as a virtual Hall sensor to initiate OCT image acquisition and synchronize it with a Fourier Domain Mode-Locked (FDML)-based Megahertz OCT system. Notably, the implementation of BEMF-triggered OCT has led to a substantial reduction in image jitter and NURD (<4 mrad), thereby opening up a new window for real-time visualization capabilities. This approach suggests potential benefits across various applications, aiming to provide a more accurate, deployable, and cost-effective solution. Subsequent studies can explore the adaptability of this system to specific clinical scenarios and its performance under practical endoscopic conditions.

Five degrees-of-freedom mechanical arm with remote center of motion (RCM) device for volumetric optical coherence tomography (OCT) retinal imaging

Handheld optical coherence tomography (HH-OCT) is gaining popularity for diagnosing retinal diseases in neonates (e.g. retinopathy of prematurity). Diagnosis accuracy is degraded by hand tremor and patient motion when using commercially available handheld retinal OCT probes. This work presents a low-cost arm designed to address ergonomic challenges of holding a commercial OCT probe and alleviating hand tremor. Experiments with a phantom eye show enhanced geometric uniformity and volumetric accuracy when obtaining OCT scans with our device compared to handheld imaging approaches. An in-vivo porcine volumetric image was also obtained with the mechanical arm demonstrating clinical deployability.

Visualization of surgical maneuvers using intraoperative real-time volumetric optical coherence tomography

Ophthalmic microsurgery is traditionally performed using stereomicroscopes and requires visualization and manipulation of sub-millimeter tissue structures with limited contrast. Optical coherence tomography (OCT) is a non-invasive imaging modality that can provide high-resolution, depth-resolved cross sections, and has become a valuable tool in clinical practice in ophthalmology. While there has been substantial progress in both research and commercialization efforts to bring OCT imaging into live surgery, its use is still somewhat limited due to factors such as low imaging speed, limited scan configurations, and suboptimal data visualization. In this paper we describe, to the best of our knowledge, the translation of the fastest swept-source intraoperative OCT system with real-time volumetric imaging with stereoscopic data visualization provided via a heads-up display into the operating room. Results from a sampling of human anterior segment and retinal surgeries chosen from 93 human surgeries using the system are shown and the benefits that this mode of intrasurgical OCT imaging provides are discussed.

Methods for real-time feature-guided image fusion of intrasurgical volumetric optical coherence tomography with digital microscopy

4D-microscope-integrated optical coherence tomography (4D-MIOCT) is an emergent multimodal imaging technology in which live volumetric OCT (4D-OCT) is implemented in tandem with standard stereo color microscopy. 4D-OCT provides ophthalmic surgeons with many useful visual cues not available in standard microscopy; however it is challenging for the surgeon to effectively integrate cues from simultaneous-but-separate imaging in real-time. In this work, we demonstrate progress towards solving this challenge via the fusion of data from each modality guided by segmented 3D features. In this way, a more readily interpretable visualization that combines and registers important cues from both modalities is presented to the surgeon.

Geometrically accurate real-time volumetric visualization of the middle ear using optical coherence tomography

We introduce a novel system for geometrically accurate, continuous, live, volumetric middle ear optical coherence tomography imaging over a 10.9mm×30∘×30∘ field of view (FOV) from a handheld imaging probe. The system employs a discretized spiral scanning (DC-SC) pattern to rapidly collect volumetric data and applies real-time scan conversion and lateral angular distortion correction to reduce geometric inaccuracies to below the system's lateral resolution over 92% of the FOV. We validate the geometric accuracy of the resulting images through comparison with co-registered micro-computed tomography (micro-CT) volumes of a phantom target and a cadaveric middle ear. The system's real-time volumetric imaging capabilities are assessed by imaging the ear of a healthy subject while performing dynamic pressurization of the middle ear in a Valsalva maneuver.

Live 4D-OCT denoising with self-supervised deep learning

By providing three-dimensional visualization of tissues and instruments at high resolution, live volumetric optical coherence tomography (4D-OCT) has the potential to revolutionize ophthalmic surgery. However, the necessary imaging speed is accompanied by increased noise levels. A high data rate and the requirement for minimal latency impose major limitations for real-time noise reduction. In this work, we propose a low complexity neural network for denoising, directly incorporated into the image reconstruction pipeline of a microscope-integrated 4D-OCT prototype with an A-scan rate of 1.2 MHz. For this purpose, we trained a blind-spot network on unpaired OCT images using a self-supervised learning approach. With an optimized U-Net, only a few milliseconds of additional latency were introduced. Simultaneously, these architectural adaptations improved the numerical denoising performance compared to the basic setup, outperforming non-local filtering algorithms. Layers and edges of anatomical structures in B-scans were better preserved than with Gaussian filtering despite comparable processing time. By comparing scenes with and without denoising employed, we show that neural networks can be used to improve visual appearance of volumetric renderings in real time. Enhancing the rendering quality is an important step for the clinical acceptance and translation of 4D-OCT as an intra-surgical guidance tool.