Real-time intraoperative 4D full-range FD-OCT based on the dual graphics processing units architecture for microsurgery guidance

Higher-order regression three-dimensional motion-compensation method for real-time optical coherence tomography volumetric imaging of the cornea

SIGNIFICANCE

Optical coherence tomography (OCT) allows high-resolution volumetric three-dimensional (3D) imaging of biological tissues in vivo. However, 3D-image acquisition can be time-consuming and often suffers from motion artifacts due to involuntary and physiological movements of the tissue, limiting the reproducibility of quantitative measurements.

AIM

To achieve real-time 3D motion compensation for corneal tissue with high accuracy.

APPROACH

We propose an OCT system for volumetric imaging of the cornea, capable of compensating both axial and lateral motion with micron-scale accuracy and millisecond-scale time consumption based on higher-order regression. Specifically, the system first scans three reference B-mode images along the C-axis before acquiring a standard C-mode image. The difference between the reference and volumetric images is compared using a surface-detection algorithm and higher-order polynomials to deduce 3D motion and remove motion-related artifacts.

RESULTS

System parameters are optimized, and performance is evaluated using both phantom and corneal (ex vivo) samples. An overall motion-artifact error of <4.61 microns and processing time of about 3.40 ms for each B-scan was achieved.

CONCLUSIONS

Higher-order regression achieved effective and real-time compensation of 3D motion artifacts during corneal imaging. The approach can be expanded to 3D imaging of other ocular tissues. Implementing such motion-compensation strategies has the potential to improve the reliability of objective and quantitative information that can be extracted from volumetric OCT measurements.

A Mouse Model of Endometriosis with Nanoparticle Labeling for In Vivo Photoacoustic Imaging

Endometriosis is a condition of the female reproductive tract characterized by endometrium-like tissue growing outside the uterus. Though it is a common cause of pelvic pain and infertility, there is currently no reliable noninvasive method to diagnose the presence of endometriosis without surgery, and the pathophysiological mechanisms that lead to the occurrence of symptoms require further inquiry. Due to patient heterogeneity and delayed diagnosis, animal models are commonly used to study the development of endometriosis, but these are costly due to the large number of animals needed to test various treatments and experimental conditions at multiple endpoints. Here, we describe a method for synthesis of multimodal imaging gold-fluorescein isothiocyanate (FITC) nanoparticles with preclinical application via induction of nanoparticle-labeled endometriosis-like lesions in mice. Labeling donor endometrial tissue fragments with gold-FITC nanoparticles prior to induction of endometriosis in recipients enables in vivo detection of the gold-labeled lesions with photoacoustic imaging. The same imaging method can be used to visualize embryos noninvasively in pregnant mice. Furthermore, the conjugated FITC dye on the gold nanoparticles allows easy isolation of labeled lesion tissue under a fluorescence dissection microscope. After dissection, the presence of gold-FITC nanoparticles and endometrium-like histology of lesions can be verified through fluorescence imaging, gold enhancement, and immunostaining. This method for in vivo imaging of endometriosis-like lesions and fluorescence-guided dissection will permit new experimental possibilities for the longitudinal study of endometriosis development and progression as well as endometriosis-related infertility.

Automated instrument-tracking for 4D video-rate imaging of ophthalmic surgical maneuvers

Intraoperative image-guidance provides enhanced feedback that facilitates surgical decision-making in a wide variety of medical fields and is especially useful when haptic feedback is limited. In these cases, automated instrument-tracking and localization are essential to guide surgical maneuvers and prevent damage to underlying tissue. However, instrument-tracking is challenging and often confounded by variations in the surgical environment, resulting in a trade-off between accuracy and speed. Ophthalmic microsurgery presents additional challenges due to the nonrigid relationship between instrument motion and instrument deformation inside the eye, image field distortion, image artifacts, and bulk motion due to patient movement and physiological tremor. We present an automated instrument-tracking method by leveraging multimodal imaging and deep-learning to dynamically detect surgical instrument positions and re-center imaging fields for 4D video-rate visualization of ophthalmic surgical maneuvers. We are able to achieve resolution-limited tracking accuracy at varying instrument orientations as well as at extreme instrument speeds and image defocus beyond typical use cases. As proof-of-concept, we perform automated instrument-tracking and 4D imaging of a mock surgical task. Here, we apply our methods for specific applications in ophthalmic microsurgery, but the proposed technologies are broadly applicable for intraoperative image-guidance with high speed and accuracy.

CNN-based CP-OCT sensor integrated with a subretinal injector for retinal boundary tracking and injection guidance

SIGNIFICANCE

Subretinal injection is an effective way of delivering transplant genes and cells to treat many degenerative retinal diseases. However, the technique requires high-dexterity and microscale precision of experienced surgeons, who have to overcome the physiological hand tremor and limited visualization of the subretinal space.

AIM

To automatically guide the axial motion of microsurgical tools (i.e., a subretinal injector) with microscale precision in real time using a fiber-optic common-path swept-source optical coherence tomography distal sensor.

APPROACH

We propose, implement, and study real-time retinal boundary tracking of A-scan optical coherence tomography (OCT) images using a convolutional neural network (CNN) for automatic depth targeting of a selected retinal boundary for accurate subretinal injection guidance. A simplified 1D U-net is used for the retinal layer segmentation on A-scan OCT images. A Kalman filter, combining retinal boundary position measurement by CNN and velocity measurement by cross correlation between consecutive A-scan images, is applied to optimally estimate the retinal boundary position. Unwanted axial motions of the surgical tools are compensated by a piezoelectric linear motor based on the retinal boundary tracking.

RESULTS

CNN-based segmentation on A-scan OCT images achieves the mean unsigned error (MUE) of ∼3  pixels (8.1  μm) using an ex vivo bovine retina model. GPU parallel computing allows real-time inference (∼2  ms) and thus real-time retinal boundary tracking. Involuntary tremors, which include low-frequency draft in hundreds of micrometers and physiological tremors in tens of micrometers, are compensated effectively. The standard deviations of photoreceptor (PR) and choroid (CH) boundary positions get as low as 10.8  μm when the depth targeting is activated.

CONCLUSIONS

A CNN-based common-path OCT distal sensor successfully tracks retinal boundaries, especially the PR/CH boundary for subretinal injection, and automatically guides the tooltip's axial position in real time. The microscale depth targeting accuracy of our system shows its promising possibility for clinical application.

Analysis and evaluation of BC-mode OCT image visualization for microsurgery guidance

Optical coherence tomography (OCT) has been gaining acceptance in image-guided microsurgery as a noninvasive imaging technique. However, when using B-mode OCT imaging, it is difficult to continuously keep the surgical tool in the imaging field, and the image of the tissue beneath the tool is corrupted by shadow effects. The alternative using C-mode OCT imaging is either too slow in imaging speed when operating in a high-resolution mode, or provides a poor image resolution in a high-speed mode, with the sweep rate less than one million hertz. Moreover, the 3-dimensional rendering of C-mode OCT image makes it difficult to visualize the tissue structure and track the surgical tool beneath the tissue surface. To solve these problems, we propose a BC-mode OCT image visualization method. This method uses a sparse C-scanning scheme, which provides a set of high-resolution B-mode OCT images at sparsely spaced cross sections. The final BC-mode OCT image is obtained by averaging the image set, with inter frame variance processing to enhance the signal of the surgical tool and tissue layers. The performance of BC-mode OCT images, such as image resolution, signal to noise ratio (SNR), imaging speed, and surgical tool tracking accuracy, is analyzed theoretically and verified experimentally. The feasibility of the proposed method is evaluated by guiding the insertion of a 30-gauge needle into the cornea of an ex-vivo human eye freehand. The results show that this provides better visualization of both the surgical tool and the tissue structure than the conventional B- or C- mode OCT image.

Microscope-Integrated Intraoperative Ultrahigh-Speed Swept-Source Optical Coherence Tomography for Widefield Retinal and Anterior Segment Imaging

BACKGROUND AND OBJECTIVE

To demonstrate the feasibility of retinal and anterior segment intraoperative widefield imaging using an ultrahigh-speed, swept-source optical coherence tomography (SS-OCT) surgical microscope attachment.

PATIENTS AND METHODS

A prototype post-objective SS-OCT using a 1,050-nm wavelength, 400 kHz A-scan rate, vertical cavity surface-emitting laser (VCSEL) light source was integrated to a commercial ophthalmic surgical microscope after the objective. Each widefield OCT data set was acquired in 3 seconds (1,000 × 1,000 A-scans, 12 × 12 mm² for retina and 10 × 10 mm² for anterior segment).

RESULTS

Intraoperative SS-OCT was performed in 20 eyes of 20 patients. In six of seven membrane peels and five of seven rhegmatogenous retinal detachment repair surgeries, widefield retinal imaging enabled evaluation pre- and postoperatively. In all seven cataract cases, anterior imaging evaluated the integrity of the posterior lens capsule.

CONCLUSIONS

Ultrahigh-speed SS-OCT enables widefield intraoperative viewing in the posterior and anterior eye. Widefield imaging visualizes ocular structures and pathology without requiring OCT realignment. [Ophthalmic Surg Lasers Imaging Retina. 2018;49:94-102.].

Depth-Based, Motion-Stabilized Colorization of Microscope-Integrated Optical Coherence Tomography Volumes for Microscope-Independent Microsurgery

Purpose

We develop and assess the impact of depth-based, motion-stabilized colorization (color) of microscope-integrated optical coherence tomography (MIOCT) volumes on microsurgical performance and ability to interpret surgical volumes.

Methods

Color was applied in real-time as gradients indicating axial position and stabilized based on calculated center of mass. In a test comparing colorization versus grayscale visualizations of prerecorded intraoperative volumes from human surgery, ophthalmologists (N = 7) were asked to identify retinal membranes, the presence of an instrument, its contact with tissue, and associated deformation of the retina. In a separate controlled trial, trainees (N = 15) performed microsurgical skills without conventional optical visualization and compared colorized versus grayscale MIOCT visualization on a stereoptic screen. Skills included thickness identification, instrument placement, and object manipulation, and were assessed based on time, performance metrics, and confidence.

Results

In intraoperative volume testing, colorization improved ability to differentiate membrane from retina (P < 0.01), correctly identify instrument contact with membrane (P = 0.03), and retinal deformation (P = 0.01). In model microsurgical skills testing, trainees working with colorized volumes were faster (P < 0.01) and more correct (P < 0.01) in assessments of thickness for recessed and elevated objects, were less likely to inadvertently contact a surface when approaching with an instrument (P < 0.01), and uniformly more confident (P < 0.01 for each) in conducting each skill.

Conclusions

Depth-based colorization enables effective identification of retinal membranes and tissue deformation. In microsurgical skill testing, it improves user efficiency, and confidence in microscope-independent, OCT-guided model surgical maneuvers.

Translational Relevance

Novel depth-based colorization and stabilization technology improves the use of intraoperative MIOCT.

Constant linear velocity spiral scanning for near video rate 4D OCT ophthalmic and surgical imaging with isotropic transverse sampling

Ultrahigh speed optical coherence tomography (OCT) systems with >100 kHz A-scan rates can generate volumes rapidly with minimal motion artifacts and are well suited for 4D imaging (volumes through time) applications such as intra-operative imaging. In such systems, high OCT data acquisition efficiency (defined as the fraction of usable A-scans generated during the total acquisition time) is desired to maximize the volumetric frame rate and sampling pitch. However, current methods for beam scanning using non-resonant and resonant mirror scanners can result in severe scan distortion and transverse oversampling as well as require acquisition dead times, which limit the acquisition efficiency and performance of ultrahigh speed 4D OCT. We introduce constant linear velocity spiral scanning (CLV-SC) as a novel beam scanning method to maximize the data acquisition efficiency of ultrahigh speed 4D OCT systems. We demonstrate that CLV-SC does not require acquisition dead times and achieves more uniform transverse sampling compared to raster scanning. To assess its clinical utility, we implement CLV-SC with a 400 kHz OCT system and image the anterior eye and retina of healthy adults at up to 10 volumes per second with isotropic transverse sampling, allowing B-scans with equal sampling pitch to be extracted from arbitrary locations within a single volume. The feasibility of CLV-SC for intra-operative imaging is also demonstrated using a 800 kHz OCT system to image simulated retinal surgery at 15 volumes per second with isotropic transverse sampling, resulting in high quality volume renders that enable clear visualization of surgical instruments and manipulation of tissue.

Real-time corneal segmentation and 3D needle tracking in intrasurgical OCT

Ophthalmic procedures demand precise surgical instrument control in depth, yet standard operating microscopes supply limited depth perception. Current commercial microscope-integrated optical coherence tomography partially meets this need with manually-positioned cross-sectional images that offer qualitative estimates of depth. In this work, we present methods for automatic quantitative depth measurement using real-time, two-surface corneal segmentation and needle tracking in OCT volumes. We then demonstrate these methods for guidance of ex vivo deep anterior lamellar keratoplasty (DALK) needle insertions. Surgeons using the output of these methods improved their ability to reach a target depth, and decreased their incidence of corneal perforations, both with statistical significance. We believe these methods could increase the success rate of DALK and thereby improve patient outcomes.

Optical Coherence Tomography for Retinal Surgery: Perioperative Analysis to Real-Time Four-Dimensional Image-Guided Surgery

Magnification of the surgical field using the operating microscope facilitated profound innovations in retinal surgery in the 1970s, such as pars plana vitrectomy. Although surgical instrumentation and illumination techniques are continually developing, the operating microscope for vitreoretinal procedures has remained essentially unchanged and currently limits the surgeon's depth perception and assessment of subtle microanatomy. Optical coherence tomography (OCT) has revolutionized clinical management of retinal pathology, and its introduction into the operating suite may have a similar impact on surgical visualization and treatment. In this article, we review the evolution of OCT for retinal surgery, from perioperative analysis to live volumetric (four-dimensional, 4D) image-guided surgery. We begin by briefly addressing the benefits and limitations of the operating microscope, the progression of OCT technology, and OCT applications in clinical/perioperative retinal imaging. Next, we review intraoperative OCT (iOCT) applications using handheld probes during surgical pauses, two-dimensional (2D) microscope-integrated OCT (MIOCT) of live surgery, and volumetric MIOCT of live surgery. The iOCT discussion focuses on technological advancements, applications during human retinal surgery, translational difficulties and limitations, and future directions.

Review of intraoperative optical coherence tomography: technology and applications [Invited]

During microsurgery, en face imaging of the surgical field through the operating microscope limits the surgeon's depth perception and visualization of instruments and sub-surface anatomy. Surgical procedures outside microsurgery, such as breast tumor resections, may also benefit from visualization of the sub-surface tissue structures. The widespread clinical adoption of optical coherence tomography (OCT) in ophthalmology and its growing prominence in other fields, such as cancer imaging, has motivated the development of intraoperative OCT for real-time tomographic visualization of surgical interventions. This article reviews key technological developments in intraoperative OCT and their applications in human surgery. We focus on handheld OCT probes, microscope-integrated OCT systems, and OCT-guided laser treatment platforms designed for intraoperative use. Moreover, we discuss intraoperative OCT adjuncts and processing techniques currently under development to optimize the surgical feedback derivable from OCT data. Lastly, we survey salient clinical studies of intraoperative OCT for human surgery.

Feasibility study: protein denaturation and coagulation monitoring with speckle variance optical coherence tomography

We performed the feasibility study using speckle variance optical coherence tomography (SvOCT) to monitor the thermally induced protein denaturation and coagulation process as a function of temperature and depth. SvOCT provided the depth-resolved image of protein denaturation and coagulation with microscale resolution. This study was conducted using egg white. During the heating process, as the temperature increased, increases in the speckle variance signal was observed as the egg white proteins coagulated. Additionally, by calculating the cross-correlation coefficient in specific areas, denaturized egg white conditions were successfully estimated. These results indicate that SvOCT could be used to monitor the denaturation process of various proteins.

High-resolution, dual-depth spectral-domain optical coherence tomography with interlaced detection for whole-eye imaging

Dual-depth spectral-domain optical coherence tomography (SD-OCT) enables high-resolution in vivo whole-eye imaging. Two orthogonally polarized beams from a source are focused simultaneously on two axial positions of the anterior segment and the retina. For the detector arm, a 1×2 ultrafast optical switch sequentially delivers two spectral interference signals to a single spectrometer, which extends the in-air axial depth range up to 9.44 mm. An off-pivot complex conjugate removal technique doubles the depth range for all anterior segment imaging. The graphics-processing-unit-based parallel signal processing algorithm supports fast two- and three-dimensional image displays. The obtained high-resolution anterior and retinal images are measured biometrically. The dual-depth SD-OCT system has an axial resolution of ∼6.4  μm in air, and the sensitivity is 91.79 dB at 150 μm from the zero-delay line.

Live volumetric (4D) visualization and guidance of in vivo human ophthalmic surgery with intraoperative optical coherence tomography

Minimally-invasive microsurgery has resulted in improved outcomes for patients. However, operating through a microscope limits depth perception and fixes the visual perspective, which result in a steep learning curve to achieve microsurgical proficiency. We introduce a surgical imaging system employing four-dimensional (live volumetric imaging through time) microscope-integrated optical coherence tomography (4D MIOCT) capable of imaging at up to 10 volumes per second to visualize human microsurgery. A custom stereoscopic heads-up display provides real-time interactive volumetric feedback to the surgeon. We report that 4D MIOCT enhanced suturing accuracy and control of instrument positioning in mock surgical trials involving 17 ophthalmic surgeons. Additionally, 4D MIOCT imaging was performed in 48 human eye surgeries and was demonstrated to successfully visualize the pathology of interest in concordance with preoperative diagnosis in 93% of retinal surgeries and the surgical site of interest in 100% of anterior segment surgeries. In vivo 4D MIOCT imaging revealed sub-surface pathologic structures and instrument-induced lesions that were invisible through the operating microscope during standard surgical maneuvers. In select cases, 4D MIOCT guidance was necessary to resolve such lesions and prevent post-operative complications. Our novel surgical visualization platform achieves surgeon-interactive 4D visualization of live surgery which could expand the surgeon’s capabilities.

Use of a new intra-ocular spectral domain optical coherence tomography in vitreoretinal surgery

PURPOSE

To describe the use of a novel intra-ocular side-scanning probe enabling the acquisition of spectral-domain optical coherence tomography (SD-OCT) images during surgery in a series of patients with complex forms of retinal detachment.

METHODS

A 23-gauge, side-scanning SD-OCT probe (C7 System; LightLab Imaging, Inc/St Jude Medical, St. Paul, MN, USA) in a 20-gauge catheter, was used to acquire the intra-operative OCT images in seven patients with vitreoretinal diseases. Twenty-five gauge pars plana vitrectomy (PPV) was performed in every patient in a standard fashion. After enlarging the temporal sclerotomy to a 20-gauge port, all the patients were scanned with intra-ocular side-scanning SD-OCT, during different steps of the surgery based on surgeon needs. Scans were recorded real time and directly evaluated on a screen during surgery. Optical coherence tomography (OCT) scans were judged beneficial when they would recognize structures otherwise not seen on biomicroscopy.

RESULTS

The intra-ocular SD-OCT has been helpful in acquiring extra information during vitreoretinal surgery such as the detection of the presence of otherwise invisible membranes (epiretinal membrane, subretinal membrane), the location of small tears and the identification of the retinal plane under suboptimal conditions for visualization.

CONCLUSION

The use of an intra-ocular SD-OCT can expand upon visual cues during surgery, helping in the decision-making process and allowing additional deliberate surgical manoeuvres aimed at improving surgical outcomes.

Master slave en-face OCT/SLO

Master Slave optical coherence tomography (MS-OCT) is an OCT method that does not require resampling of data and can be used to deliver en-face images from several depths simultaneously. As the MS-OCT method requires important computational resources, the number of multiple depth en-face images that can be produced in real-time is limited. Here, we demonstrate progress in taking advantage of the parallel processing feature of the MS-OCT technology. Harnessing the capabilities of graphics processing units (GPU)s, information from 384 depth positions is acquired in one raster with real time display of up to 40 en-face OCT images. These exhibit comparable resolution and sensitivity to the images produced using the conventional Fourier domain based method. The GPU facilitates versatile real time selection of parameters, such as the depth positions of the 40 images out of the set of 384 depth locations, as well as their axial resolution. In each updated displayed frame, in parallel with the 40 en-face OCT images, a scanning laser ophthalmoscopy (SLO) lookalike image is presented together with two B-scan OCT images oriented along rectangular directions. The thickness of the SLO lookalike image is dynamically determined by the choice of number of en-face OCT images displayed in the frame and the choice of differential axial distance between them.

On the possibility of producing true real-time retinal cross-sectional images using a graphics processing unit enhanced master-slave optical coherence tomography system

In a previous report, we demonstrated master-slave optical coherence tomography (MS-OCT), an OCT method that does not need resampling of data and can be used to deliver en face images from several depths simultaneously. In a separate report, we have also demonstrated MS-OCT's capability of producing cross-sectional images of a quality similar to those provided by the traditional Fourier domain (FD) OCT technique, but at a much slower rate. Here, we demonstrate that by taking advantage of the parallel processing capabilities offered by the MS-OCT method, cross-sectional OCT images of the human retina can be produced in real time. We analyze the conditions that ensure a true real-time B-scan imaging operation and demonstrate in vivo real-time images from human fovea and the optic nerve, with resolution and sensitivity comparable to those produced using the traditional FD-based method, however, without the need of data resampling.

Accurate real-time depth control for CP-SSOCT distal sensor based handheld microsurgery tools

This paper presents a novel intuitive targeting and tracking scheme that utilizes a common-path swept source optical coherence tomography (CP-SSOCT) distal sensor integrated handheld microsurgical tool. To achieve micron-order precision control, a reliable and accurate OCT distal sensing method is required; simultaneously, a prediction algorithm is necessary to compensate for the system delay associated with the computational, mechanical and electronic latencies. Due to the multi-layered structure of retina, it is necessary to develop effective surface detection methods rather than simple peak detection. To achieve this, a shifted cross-correlation method is applied for surface detection in order to increase robustness and accuracy in distal sensing. A predictor based on Kalman filter was implemented for more precise motion compensation. The performance was first evaluated using an established dry phantom consisting of stacked cellophane tape. This was followed by evaluation in an ex-vivo bovine retina model to assess system accuracy and precision. The results demonstrate highly accurate depth targeting with less than 5 μm RMSE depth locking.

High-performance multi-megahertz optical coherence tomography based on amplified optical time-stretch

As the key prerequisite of high-speed volumetric structural and functional tissue imaging in real-time, scaling the A-scan rate beyond MHz has been one of the major pursuits in the development of optical coherence tomography (OCT). Along with a handful of techniques enabling multi-MHz, amplified optical time-stretch OCT (AOT-OCT) has recently been demonstrated as a viable alternative for ultrafast swept-source OCT well above MHz without the need for the mechanical wavelength-tuning mechanism. In this paper, we report a new generation of AOT-OCT demonstrating superior performance to its older generation and all other time-stretch-based OCT modalities in terms of shot-to-shot stability, sensitivity (~90dB), roll-off performance (>4 mm/dB) and A-scan rate (11.5 MHz). Such performance is mainly attributed to the combined contribution from the stable operation of the broadband and compact mode-locked fiber laser as well as the optical amplification in-line with the time-stretch process. The system allows us, for the first time, to deliver volumetric time-stretch-based OCT of biological tissues with the single-shot A-scan rate beyond 10 MHz. Comparing with the existing high-speed OCT systems, the inertia-free AOT-OCT shows promises to realize high-performance 3D OCT imaging at video rate.

High definition live 3D-OCT in vivo: design and evaluation of a 4D OCT engine with 1 GVoxel/s

We present a 1300 nm OCT system for volumetric real-time live OCT acquisition and visualization at 1 billion volume elements per second. All technological challenges and problems associated with such high scanning speed are discussed in detail as well as the solutions. In one configuration, the system acquires, processes and visualizes 26 volumes per second where each volume consists of 320 x 320 depth scans and each depth scan has 400 usable pixels. This is the fastest real-time OCT to date in terms of voxel rate. A 51 Hz volume rate is realized with half the frame number. In both configurations the speed can be sustained indefinitely. The OCT system uses a 1310 nm Fourier domain mode locked (FDML) laser operated at 3.2 MHz sweep rate. Data acquisition is performed with two dedicated digitizer cards, each running at 2.5 GS/s, hosted in a single desktop computer. Live real-time data processing and visualization are realized with custom developed software on an NVidia GTX 690 dual graphics processing unit (GPU) card. To evaluate potential future applications of such a system, we present volumetric videos captured at 26 and 51 Hz of planktonic crustaceans and skin.

In vivo imaging of functional microvasculature within tissue beds of oral and nasal cavities by swept-source optical coherence tomography with a forward/side-viewing probe

We report three-dimensional (3D) imaging of microcirculation within human cavity tissues in vivo using a high-speed swept-source optical coherence tomography (SS-OCT) at 1300 nm with a modified probe interface. Volumetric structural OCT images of the inner tissues of oral and nasal cavities are acquired with a field of view of 2 mm × 2 mm. Two types of disposable and detachable probe attachments are devised and applied to the port of the imaging probe of OCT system, enabling forward and side imaging scans for selective and easy access to specific cavity tissue sites. Blood perfusion is mapped with OCT-based microangiography from 3D structural OCT images, in which a novel vessel extraction algorithm is used to decouple dynamic light scattering signals, due to moving blood cells, from the background scattering signals due to static tissue elements. Characteristic tissue anatomy and microvessel architectures of various cavity tissue regions of a healthy human volunteer are identified with the 3D OCT images and the corresponding 3D vascular perfusion maps at a level approaching capillary resolution. The initial finding suggests that the proposed method may be engineered into a promising tool for evaluating and monitoring tissue microcirculation and its alteration within a wide-range of cavity tissues in the patients with various pathological conditions.

Parallelized multi-graphics processing unit framework for high-speed Gabor-domain optical coherence microscopy

Gabor-domain optical coherence microscopy (GD-OCM) is a volumetric high-resolution technique capable of acquiring three-dimensional (3-D) skin images with histological resolution. Real-time image processing is needed to enable GD-OCM imaging in a clinical setting. We present a parallelized and scalable multi-graphics processing unit (GPU) computing framework for real-time GD-OCM image processing. A parallelized control mechanism was developed to individually assign computation tasks to each of the GPUs. For each GPU, the optimal number of amplitude-scans (A-scans) to be processed in parallel was selected to maximize GPU memory usage and core throughput. We investigated five computing architectures for computational speed-up in processing 1000×1000 A-scans. The proposed parallelized multi-GPU computing framework enables processing at a computational speed faster than the GD-OCM image acquisition, thereby facilitating high-speed GD-OCM imaging in a clinical setting. Using two parallelized GPUs, the image processing of a 1×1×0.6  mm3 skin sample was performed in about 13 s, and the performance was benchmarked at 6.5 s with four GPUs. This work thus demonstrates that 3-D GD-OCM data may be displayed in real-time to the examiner using parallelized GPU processing.

Imaging the eye fundus with real-time en-face spectral domain optical coherence tomography

Real-time display of processed en-face spectral domain optical coherence tomography (SD-OCT) images is important for diagnosis. However, due to many steps of data processing requirements, such as Fast Fourier transformation (FFT), data re-sampling, spectral shaping, apodization, zero padding, followed by software cut of the 3D volume acquired to produce an en-face slice, conventional high-speed SD-OCT cannot render an en-face OCT image in real time. Recently we demonstrated a Master/Slave (MS)-OCT method that is highly parallelizable, as it provides reflectivity values of points at depth within an A-scan in parallel. This allows direct production of en-face images. In addition, the MS-OCT method does not require data linearization, which further simplifies the processing. The computation in our previous paper was however time consuming. In this paper we present an optimized algorithm that can be used to provide en-face MS-OCT images much quicker. Using such an algorithm we demonstrate around 10 times faster production of sets of en-face OCT images than previously obtained as well as simultaneous real-time display of up to 4 en-face OCT images of 200 × 200 pixels(2) from the fovea and the optic nerve of a volunteer. We also demonstrate 3D and B-scan OCT images obtained from sets of MS-OCT C-scans, i.e. with no FFT and no intermediate step of generation of A-scans.

Extravascular optical coherence tomography: evaluation of carotid atherosclerosis and pravastatin therapy

BACKGROUND AND PURPOSE

Extravascular optical coherence tomography (OCT), as a noninvasive imaging methodology with micrometer resolution, was evaluated in a murine model of carotid atherosclerosis by way of assessing the efficacy of pravastatin therapy.

METHODS

An OCT device was engineered for extravascular plaque imaging. Wild-type mice and apolipoprotein E-deficient (ApoE(-/-)) mice were randomized to 3 treatment groups: (1) wild-type on a diet of standard rodent chow (n=13); (2) ApoE(-/-) on a high-fat, atherosclerotic diet (HFD; n=13); and (3) ApoE(-/-) on a HFD given daily pravastatin (n=13). Mice were anesthetized and the left common carotid was surgically exposed. Three-dimensional (3D; 2 spatial dimensions+time) and 4D (3 spatial dimensions+time) OCT images of the vessel lumen patency were evaluated. After perfusion, in situ OCT imaging was performed for statistical comparison with the in vivo results and final histology.

RESULTS

Intraoperative OCT imaging positively identified carotid plaque in 100% of ApoE(-/-) mice on HFD. ApoE(-/-) mice on HFD had a significantly decreased lumen patency when compared with that in wild-type mice (P<0.001). Pravastatin therapy was found to increase lumen patency significantly in ApoE(-/-) mice on HFD (P<0.01; compared with ApoE(-/-) on HFD). The findings were confirmed with OCT imaging after perfusion and histology.

CONCLUSIONS

OCT imaging offers the potential for real-time, detailed vessel lumen evaluation, potentially improving surgical accuracy and outcomes during cerebrovascular neurosurgical procedures. Pravastatin significantly increases vessel lumen patency in the ApoE(-/-) mouse on HFD.

Stimulated penetrating keratoplasty using real-time virtual intraoperative surgical optical coherence tomography

An intraoperative surgical microscope is an essential tool in a neuro- or ophthalmological surgical environment. Yet, it has an inherent limitation to classify subsurface information because it only provides the surface images. To compensate for and assist in this problem, combining the surgical microscope with optical coherence tomography (OCT) has been adapted. We developed a real-time virtual intraoperative surgical OCT (VISOCT) system by adapting a spectral-domain OCT scanner with a commercial surgical microscope. Thanks to our custom-made beam splitting and image display subsystems, the OCT images and microscopic images are simultaneously visualized through an ocular lens or the eyepiece of the microscope. This improvement helps surgeons to focus on the operation without distraction to view OCT images on another separate display. Moreover, displaying the OCT live images on the eyepiece helps surgeon's depth perception during the surgeries. Finally, we successfully processed stimulated penetrating keratoplasty in live rabbits. We believe that these technical achievements are crucial to enhance the usability of the VISOCT system in a real surgical operating condition.

Real-time acquisition and display of flow contrast using speckle variance optical coherence tomography in a graphics processing unit

In this report, we describe a graphics processing unit (GPU)-accelerated processing platform for real-time acquisition and display of flow contrast images with Fourier domain optical coherence tomography (FDOCT) in mouse and human eyes in vivo. Motion contrast from blood flow is processed using the speckle variance OCT (svOCT) technique, which relies on the acquisition of multiple B-scan frames at the same location and tracking the change of the speckle pattern. Real-time mouse and human retinal imaging using two different custom-built OCT systems with processing and display performed on GPU are presented with an in-depth analysis of performance metrics. The display output included structural OCT data, en face projections of the intensity data, and the svOCT en face projections of retinal microvasculature; these results compare projections with and without speckle variance in the different retinal layers to reveal significant contrast improvements. As a demonstration, videos of real-time svOCT for in vivo human and mouse retinal imaging are included in our results. The capability of performing real-time svOCT imaging of the retinal vasculature may be a useful tool in a clinical environment for monitoring disease-related pathological changes in the microcirculation such as diabetic retinopathy.

Graphics processing unit accelerated intensity-based optical coherence tomography angiography using differential frames with real-time motion correction

We demonstrate intensity-based optical coherence tomography (OCT) angiography using the squared difference of two sequential frames with bulk-tissue-motion (BTM) correction. This motion correction was performed by minimization of the sum of the pixel values using axial- and lateral-pixel-shifted structural OCT images. We extract the BTM-corrected image from a total of 25 calculated OCT angiographic images. Image processing was accelerated by a graphics processing unit (GPU) with many stream processors to optimize the parallel processing procedure. The GPU processing rate was faster than that of a line scan camera (46.9 kHz). Our OCT system provides the means of displaying structural OCT images and BTM-corrected OCT angiographic images in real time.

Development of real-time dual-display handheld and bench-top hybrid-mode SD-OCTs

Development of a dual-display handheld optical coherence tomography (OCT) system for retina and optic-nerve-head diagnosis beyond the volunteer motion constraints is reported. The developed system is portable and easily movable, containing the compact portable OCT system that includes the handheld probe and computer. Eye posterior chambers were diagnosed using the handheld probe, and the probe could be fixed to the bench-top cradle depending on the volunteers' physical condition. The images obtained using this handheld probe were displayed in real time on the computer monitor and on a small secondary built-in monitor; the displayed images were saved using the handheld probe's built-in button. Large-scale signal-processing procedures such as k-domain linearization, fast Fourier transform (FFT), and log-scaling signal processing can be rapidly applied using graphics-processing-unit (GPU) accelerated processing rather than central-processing-unit (CPU) processing. The Labview-based system resolution is 1,024 × 512 pixels, and the frame rate is 56 frames/s, useful for real-time display. The 3D images of the posterior chambers including the retina, optic-nerve head, blood vessels, and optic nerve were composed using real-time displayed images with 500 × 500 × 500 pixel resolution. A handheld and bench-top hybrid mode with a dual-display handheld OCT was developed to overcome the drawbacks of the conventional method.

Medical image processing on the GPU - past, present and future

Graphics processing units (GPUs) are used today in a wide range of applications, mainly because they can dramatically accelerate parallel computing, are affordable and energy efficient. In the field of medical imaging, GPUs are in some cases crucial for enabling practical use of computationally demanding algorithms. This review presents the past and present work on GPU accelerated medical image processing, and is meant to serve as an overview and introduction to existing GPU implementations. The review covers GPU acceleration of basic image processing operations (filtering, interpolation, histogram estimation and distance transforms), the most commonly used algorithms in medical imaging (image registration, image segmentation and image denoising) and algorithms that are specific to individual modalities (CT, PET, SPECT, MRI, fMRI, DTI, ultrasound, optical imaging and microscopy). The review ends by highlighting some future possibilities and challenges.

Speckle-constrained variational methods for image restoration in optical coherence tomography

A number of despeckling methods for optical coherence tomography (OCT) have been proposed. In these digital filtering techniques, speckle noise is often simplified as additive white Gaussian noise due to the logarithmic compression for the signal. The approximation is not completely consistent with the characteristic of OCT speckle noise, and cannot be reasonably extended to deconvolution algorithms. This paper presents a deconvolution model that combines the variational regularization term with the statistical characteristic constraints of data corrupted by OCT speckle noise. In the data fidelity term, speckle noise is modeled as signal dependent, and the point spread function of OCT systems is included. The regularization functional introduces a priori information on the original images, and a regularization term based on block matching 3D modeling is used to construct the variational model in the paper. Finally, the method is applied to the restoration of actual OCT raw data of human skin. The numerical results demonstrate that the proposed deconvolution algorithm can simultaneously enhance regions of images containing detail and remove OCT speckle noise.

Motion-compensated hand-held common-path Fourier-domain optical coherence tomography probe for image-guided intervention

A motion-compensated, hand-held, common-path, Fourier-domain optical coherence tomography imaging probe has been developed for image-guided intervention during microsurgery. A hand-held prototype instrument was achieved by integrating an imaging fiber probe inside a stainless steel needle and attached to the ceramic shaft of a piezoelectric motor housed in an aluminum handle. The fiber probe obtains A-scan images. The distance information was extracted from the A-scans to track the sample surface distance and a fixed distance was maintained by a feedback motor control which effectively compensated hand tremor and target movements in the axial direction. Real-time data acquisition, processing, motion compensation, and image visualization and saving were implemented on a custom CPU-GPU hybrid architecture. We performed 10× zero padding to the raw spectrum to obtain 0.16 µm position accuracy with a compensation rate of 460 Hz. The root-mean-square error of hand-held distance variation from target position was measured to be 2.93 µm. We used a cross-correlation maximization-based shift correction algorithm for topology correction. To validate the system, we performed free-hand OCT M-scan imaging using various samples.

Spectral domain optical coherence tomography of multi-MHz A-scan rates at 1310 nm range and real-time 4D-display up to 41 volumes/second

An ultrafast frequency domain optical coherence tomography system was developed at A-scan rates between 2.5 and 10 MHz, a B-scan rate of 4 or 8 kHz, and volume-rates between 12 and 41 volumes/second. In the case of the worst duty ratio of 10%, the averaged A-scan rate was 1 MHz. Two optical demultiplexers at a center wavelength of 1310 nm were used for linear-k spectral dispersion and simultaneous differential signal detection at 320 wavelengths. The depth-range, sensitivity, sensitivity roll-off by 6 dB, and axial resolution were 4 mm, 97 dB, 6 mm, and 23 μm, respectively. Using FPGAs for FFT and a GPU for volume rendering, a real-time 4D display was demonstrated at a rate up to 41 volumes/second for an image size of 256 (axial) × 128 × 128 (lateral) voxels.

Real-time 3D and 4D Fourier domain Doppler optical coherence tomography based on dual graphics processing units

We present real-time 3D (2D cross-sectional image plus time) and 4D (3D volume plus time) phase-resolved Doppler OCT (PRDOCT) imaging based on configuration of dual graphics processing units (GPU). A GPU-accelerated phase-resolving processing algorithm was developed and implemented. We combined a structural image intensity-based thresholding mask and average window method to improve the signal-to-noise ratio of the Doppler phase image. A 2D simultaneous display of the structure and Doppler flow images was presented at a frame rate of 70 fps with an image size of 1000 × 1024 (X × Z) pixels. A 3D volume rendering of tissue structure and flow images-each with a size of 512 × 512 pixels-was presented 64.9 milliseconds after every volume scanning cycle with a volume size of 500 × 256 × 512 (X × Y × Z) voxels, with an acquisition time window of only 3.7 seconds. To the best of our knowledge, this is the first time that an online, simultaneous structure and Doppler flow volume visualization has been achieved. Maximum system processing speed was measured to be 249,000 A-scans per second with each A-scan size of 2048 pixels.

Real-time three-dimensional Fourier-domain optical coherence tomography video image guided microsurgeries

The authors describe the development of an ultrafast three-dimensional (3D) optical coherence tomography (OCT) imaging system that provides real-time intraoperative video images of the surgical site to assist surgeons during microsurgical procedures. This system is based on a full-range complex conjugate free Fourier-domain OCT (FD-OCT). The system was built in a CPU-GPU heterogeneous computing architecture capable of video OCT image processing. The system displays at a maximum speed of 10 volume/s for an image volume size of 160 × 80 × 1024(X × Y × Z) pixels. We have used this system to visualize and guide two prototypical microsurgical maneuvers: microvascular anastomosis of the rat femoral artery and ultramicrovascular isolation of the retinal arterioles of the bovine retina. Our preliminary experiments using 3D-OCT-guided microvascular anastomosis showed optimal visualization of the rat femoral artery (diameter<0.8 mm), instruments, and suture material. Real-time intraoperative guidance helped facilitate precise suture placement due to optimized views of the vessel wall during anastomosis. Using the bovine retina as a model system, we have performed "ultra microvascular" feasibility studies by guiding handheld surgical micro-instruments to isolate retinal arterioles (diameter ~0.1 mm). Isolation of the microvessels was confirmed by successfully passing a suture beneath the vessel in the 3D imaging environment.

Real-time speckle variance swept-source optical coherence tomography using a graphics processing unit

Advances in swept source laser technology continues to increase the imaging speed of swept-source optical coherence tomography (SS-OCT) systems. These fast imaging speeds are ideal for microvascular detection schemes, such as speckle variance (SV), where interframe motion can cause severe imaging artifacts and loss of vascular contrast. However, full utilization of the laser scan speed has been hindered by the computationally intensive signal processing required by SS-OCT and SV calculations. Using a commercial graphics processing unit that has been optimized for parallel data processing, we report a complete high-speed SS-OCT platform capable of real-time data acquisition, processing, display, and saving at 108,000 lines per second. Subpixel image registration of structural images was performed in real-time prior to SV calculations in order to reduce decorrelation from stationary structures induced by the bulk tissue motion. The viability of the system was successfully demonstrated in a high bulk tissue motion scenario of human fingernail root imaging where SV images (512 × 512 pixels, n = 4) were displayed at 54 frames per second.

Ultra-fast displaying Spectral Domain Optical Doppler Tomography system using a Graphics Processing Unit

We demonstrate an ultrafast displaying Spectral Domain Optical Doppler Tomography system using Graphics Processing Unit (GPU) computing. The calculation of FFT and the Doppler frequency shift is accelerated by the GPU. Our system can display processed OCT and ODT images simultaneously in real time at 120 fps for 1,024 pixels × 512 lateral A-scans. The computing time for the Doppler information was dependent on the size of the moving average window, but with a window size of 32 pixels the ODT computation time is only 8.3 ms, which is comparable to the data acquisition time. Also the phase noise decreases significantly with the window size. Since the performance of a real-time display for OCT/ODT is very important for clinical applications that need immediate diagnosis for screening or biopsy. Intraoperative surgery can take much benefit from the real-time display flow rate information from the technology. Moreover, the GPU is an attractive tool for clinical and commercial systems for functional OCT features as well.

Common approach for compensation of axial motion artifacts in swept-source OCT and dispersion in Fourier-domain OCT

Swept-source optical coherence tomography (SS-OCT) is sensitive to sample motion during the wavelength sweep, which leads to image blurring and image artifacts. In line-field and full-field SS-OCT parallelization is achieved by using a line or area detector, respectively. Thus, approximately 1000 lines or images at different wavenumbers are acquired. The sweep duration is identically with the acquisition time of a complete B-scan or volume, rendering parallel SS-OCT more sensitive to motion artifacts than scanning OCT. The effect of axial motion on the measured spectra is similar to the effect of non-balanced group velocity dispersion (GVD) in the interferometer arms. It causes the apparent optical path lengths in the sample arm to vary with the wavenumber. Here we propose the cross-correlation of sub-bandwidth reconstructions (CCSBR) as a new algorithm that is capable of detecting and correcting the artifacts induced by axial motion in line-field or full-field SS-OCT as well as GVD mismatch in any Fourier-domain OCT (FD-OCT) setup. By cross-correlating images which were reconstructed from a limited spectral range of the interference signal, a phase error is determined which is used to correct the spectral modulation prior to the calculation of the A-scans. Performance of the algorithm is demonstrated on in vivo full-field SS-OCT images of skin and scanning FD-OCT of skin and retina.

Selection of convolution kernel in non-uniform fast Fourier transform for Fourier domain optical coherence tomography

Gridding based non-uniform fast Fourier transform (NUFFT) has recently been shown as an efficient method of processing non-linearly sampled data from Fourier-domain optical coherence tomography (FD-OCT). This method requires selecting design parameters, such as kernel function type, oversampling ratio and kernel width, to balance between computational complexity and accuracy. The Kaiser-Bessel (KB) and Gaussian kernels have been used independently on the NUFFT algorithm for FD-OCT. This paper compares the reconstruction error and speed for the optimization of these design parameters and justifies particular kernel choice for FD-OCT applications. It is found that for on-the-fly computation of the kernel function, the simpler Gaussian function offers a better accuracy-speed tradeoff. The KB kernel, however, is a better choice in the pre-computed kernel mode of NUFFT, in which the processing speed is no longer dependent on the kernel function type. Finally, the algorithm is used to reconstruct in-vivo images of a human finger at a camera limited 50k A-line/s.

Common-path low-coherence interferometry fiber-optic sensor guided microincision

We propose and demonstrate a common-path low-coherence interferometry (CP-LCI) fiber-optic sensor guided precise microincision. The method tracks the target surface and compensates the tool-to-surface relative motion with better than ± 5 μm resolution using a precision micromotor connected to the tool tip. A single-fiber distance probe integrated microdissector was used to perform an accurate 100 μm incision into the surface of an Intralipid phantom. The CP-LCI guided incision quality in terms of depth was evaluated afterwards using three-dimensional Fourier-domain optical coherence tomography imaging, which showed significant improvement of incision accuracy compared to free-hand-only operations.

High-speed camera with real time processing for frequency domain imaging

We describe a high-speed camera system for frequency domain imaging suitable for applications such as in vivo diffuse optical imaging and fluorescence lifetime imaging. 14-bit images are acquired at 2 gigapixels per second and analyzed with real-time pipeline processing using field programmable gate arrays (FPGAs). Performance of the camera system has been tested both for RF-modulated laser imaging in combination with a gain-modulated image intensifier and a simpler system based upon an LED light source. System amplitude and phase noise are measured and compared against theoretical expressions in the shot noise limit presented for different frequency domain configurations. We show the camera itself is capable of shot noise limited performance for amplitude and phase in as little as 3 ms, and when used in combination with the intensifier the noise levels are nearly shot noise limited. The best phase noise in a single pixel is 0.04 degrees for a 1 s integration time.