Improving spatial resolution of a fiber bundle optical biopsy system

Real-timing processing of fiber bundle endomicroscopy images in Python using PyFibreBundle

Fiber imaging bundles allow the transfer of optical images from place-to-place along narrow and flexible conduits. Traditionally used extensively in medical endoscopy, bundles are now finding new applications in endoscopic microscopy and other emerging techniques. PyFibreBundle is an open-source Python package for fast processing of images acquired through imaging bundles. This includes detection and removal of the fiber core pattern by filtering or interpolation, and application of background and flat-field corrections. It also allows images to be stitched together to create mosaics and resolution to be improved by combining multiple shifted images. This paper describes the technical implementation of PyFibreBundle and provides example results from three endomicroscopy imaging systems: color transmission, monochrome transmission, and confocal fluorescence. This allows various processing options to be compared quantitatively and qualitatively, and benchmarking demonstrates that PyFibreBundle can achieve state-of-the-art performance in an open-source package. The paper demonstrates core removal by interpolation and mosaicing at over 100 fps, real-time multi-frame resolution enhancement and the first demonstration of real-time endomicroscopy image processing, including core removal, on a Raspberry Pi single board computer. This demonstrates that PyFibreBundle is potentially a valuable tool for the development of low-cost, high-performance fiber bundle imaging systems.

Improvement in optical fiber bundle-based imaging using synchronized fiber motion

Image quality in fiber bundle-based imaging systems is inherently limited by the size and spacing of the individual fiber cores. The fiber bundle limits the achievable spatial resolution and superimposes a fixed pattern of signal variability across the image. To overcome these limitations, piezoelectric tubes were used to synchronously dither the fiber bundle on both ends. Experimental results using the dithering approach with a commercial fiber bundle showed a substantial decrease in fixed pattern noise and an increase in spatial resolution.

Image computing for fibre-bundle endomicroscopy: A review

Endomicroscopy is an emerging imaging modality, that facilitates the acquisition of in vivo, in situ optical biopsies, assisting diagnostic and potentially therapeutic interventions. While there is a diverse and constantly expanding range of commercial and experimental optical biopsy platforms available, fibre-bundle endomicroscopy is currently the most widely used platform and is approved for clinical use in a range of clinical indications. Miniaturised, flexible fibre-bundles, guided through the working channel of endoscopes, needles and catheters, enable high-resolution imaging across a variety of organ systems. Yet, the nature of image acquisition though a fibre-bundle gives rise to several inherent characteristics and limitations necessitating novel and effective image pre- and post-processing algorithms, ranging from image formation, enhancement and mosaicing to pathology detection and quantification. This paper introduces the underlying technology and most prevalent clinical applications of fibre-bundle endomicroscopy, and provides a comprehensive, up-to-date, review of relevant image reconstruction, analysis and understanding/inference methodologies. Furthermore, current limitations as well as future challenges and opportunities in fibre-bundle endomicroscopy computing are identified and discussed.

Miniature probe for dual-modality photoacoustic microscopy and white-light microscopy for image guidance: A prototype toward an endoscope

In this study, a novel photoacoustic microscopy (PAM) probe integrating white-light microscopy (WLM) modality that provides guidance for PAM imaging and complementary information is implemented. One single core of an imaging fiber bundle is employed to deliver a pulsed laser for photoacoustic excitation for PAM mode, which provides high resolution with deep penetration. Meanwhile, for WLM mode, the imaging fiber bundle is used to transmit two-dimensional superficial images. Lateral resolution of 7.2 μm for PAM is achieved. Since miniature components are used, the probe diameter is only 1.7 mm. Imaging of phantom and animals in vivo is conducted to show the imaging capability of the probe. The probe has several advantages by introducing the WLM mode, such as being able to conveniently identify regions of interest and align the focus for PAM mode. The prototype of an endoscope shows potential to facilitate clinical photoacoustic endoscopic applications.

Computational endoscopy-a framework for improving spatial resolution in fiber bundle imaging

This Letter presents a framework for computational imaging (CI) in fiber-bundle-based endoscopy systems. Multiple observations are acquired of objects spatially modulated with different random binary masks. Sparse-recovery algorithms then reconstruct images with more resolved pixels than individual fibers in the bundle. Object details lying within the diameter of single fibers are resolved, allowing images with 41,663 resolvable points to be generated through a bundle with 2,420 fibers. Computational fiber bundle imaging of micro- and macro-scale objects is demonstrated using fluorescent standards and biological tissues, including in vivo imaging of a human fingertip. In each case, CI recovers details that conventional endoscopy does not provide.

Estimation of tissue oxygen saturation from RGB images and sparse hyperspectral signals based on conditional generative adversarial network

PURPOSE

Intra-operative measurement of tissue oxygen saturation ([Formula: see text]) is important in detection of ischaemia, monitoring perfusion and identifying disease. Hyperspectral imaging (HSI) measures the optical reflectance spectrum of the tissue and uses this information to quantify its composition, including [Formula: see text]. However, real-time monitoring is difficult due to capture rate and data processing time.

METHODS

An endoscopic system based on a multi-fibre probe was previously developed to sparsely capture HSI data (sHSI). These were combined with RGB images, via a deep neural network, to generate high-resolution hypercubes and calculate [Formula: see text]. To improve accuracy and processing speed, we propose a dual-input conditional generative adversarial network, Dual2StO2, to directly estimate [Formula: see text] by fusing features from both RGB and sHSI.

RESULTS

Validation experiments were carried out on in vivo porcine bowel data, where the ground truth [Formula: see text] was generated from the HSI camera. Performance was also compared to our previous super-spectral-resolution network, SSRNet in terms of mean [Formula: see text] prediction accuracy and structural similarity metrics. Dual2StO2 was also tested using simulated probe data with varying fibre number.

CONCLUSIONS

[Formula: see text] estimation by Dual2StO2 is visually closer to ground truth in general structure and achieves higher prediction accuracy and faster processing speed than SSRNet. Simulations showed that results improved when a greater number of fibres are used in the probe. Future work will include refinement of the network architecture, hardware optimization based on simulation results, and evaluation of the technique in clinical applications beyond [Formula: see text] estimation.

High-resolution endomicroscopy with a spectrally encoded miniature objective

Fiber bundle endomicroscopy techniques have been used for numerous minimally invasive imaging applications. However, these techniques may provide limited spatial sampling due to the limited number of imaging cores inside the fiber bundle. Here, we present a custom-fabricated miniature objective that can be coupled to a fiber bundle and can overcome the fiber bundle's sampling threshold by utilizing the spectral encoding concept. The objective has an NA of 0.3 and an outer diameter of 2.4 mm, and can yield a maximum spatial resolution of 2 μm. The objective has been validated against a USAF resolution target and ex vivo tissue samples, and as a result yielded images with higher resolution and more details after the spectral encoding concept was employed.

Fiber bundle shifting endomicroscopy for high-resolution imaging

Flexible endomicroscopes commonly use coherent fiber bundles with high core densities to facilitate high-resolution in vivo imaging during endoscopic and minimally-invasive procedures. However, under-sampling due to the inter-core spacing limits the spatial resolution, making it difficult to resolve smaller cellular features. Here, we report a compact and rapid piezoelectric transducer (PZT) based bundle-shifting endomicroscopy system in which a super-resolution (SR) image is restored from multiple pixelation-limited images by computational means. A miniaturized PZT tube actuates the fiber bundle behind a GRIN micro-lens and a Delaunay triangulation based algorithm reconstructs an enhanced SR image. To enable real-time cellular-level imaging, imaging is performed using a line-scan confocal laser endomicroscope system with a raw frame rate of 120 fps, delivering up to 2 times spatial resolution improvement for a field of view of 350 µm at a net frame rate of 30 fps. The resolution enhancement is confirmed using resolution phantoms and ex vivo fluorescence endomicroscopy imaging of human breast specimens is demonstrated.

Resolution enhancement for fiber bundle imaging using maximum a posteriori estimation

We propose a new framework to jointly improve spatial resolution and remove fixed structural patterns for coherent fiber bundle imaging systems, based on inverting a principled forward model. The forward model maps a high-resolution representation to multiple images modeling random probe motions. We then apply a point spread function to simulate low-resolution figure bundle image capture. Our forward model also uses a smoothing prior. We compute a maximum a posteriori (MAP) estimate of the high-resolution image from one or more low-resolution images using conjugate gradient descent. Unique aspects of our approach include (1) supporting a variety of possible applicable transformations; (2) applying principled forward modeling and MAP estimation to this domain. We test our method on data synthesized from the USAF target, data captured from a transmissive USAF target, and data from lens tissue. In the case of the USAF target and 16 low-resolution captures, spatial resolution is enhanced by a factor of 2.8.

Random transverse motion-induced spatial compounding for fiber bundle imaging

We propose and demonstrate a novel fiber bundle imaging based on spatial compounding induced by random transverse motion to remove the pixelation effect, to improve resolution, and to increase image quality. The experimental results using a USAF target and pyramidal neuron cell showed that 20-frame compounding improved image quality (contrast-to-noise ratio by >9  dB, global SNR by >6  dB, equivalent number of looks by >1.8  times, and 1/β by >1.5  times), resolution by better than 2 μm, and completely eliminated pixelation artifact.

Numerical characterization of an ultra-high NA coherent fiber bundle part II: point spread function analysis

Straightforward numerical integration of the Rayleigh-Sommerfeld diffraction integral (R-SDI) remains computationally challenging, even with today's computational resources. As such, approximating the R-SDI to decrease the computation time while maintaining a good accuracy is still a topic of interest. In this paper, we apply an approximation for the R-SDI that is to be used to propagate the field exiting a Coherent Fiber Bundle (CFB) with ultra-high numerical aperture (0.928) of which we presented the design and modal properties in previous work. Since our CFB has single-mode cores with a diameter (550 nm) smaller than the wavelength (850 nm) for which the CFB was designed, we approximate the highly divergent fundamental modes of the cores with real Dirac delta functions. We find that with this approximation we can strongly reduce the computation time of the R-SDI while maintaining a good agreement with the results of the full R-SDI. Using this approximation, we first determine the Point Spread Function (PSF) for an 'ideal' output field exiting the CFB (identical amplitudes for cores on a perfect hexagonal lattice with the phase of each core determined by the appropriate spherical and tilted plane wave front). Next, we analyze the PSF when amplitude or phase noise is superposed onto this 'ideal' field. We find that even in the presence of these types of noise, the effect on the central peak of PSF is limited. From these types of noise, phase noise is found to have the biggest impact on the PSF.

Snapshot spectrally encoded fluorescence imaging through a fiber bundle

Fiber optic endomicroscopy is a valuable tool for clinical diagnostics and animal studies because it can capture images of tissue in vivo with subcellular resolution. Current configurations for endomicroscopes have either limited spatial resolution or require a scanning mechanism at the distal end of the fiber, which can slow imaging speed and increase the probe size. We present a novel configuration that provides high contrast 350 × 350 pixel images at 7.2 frames per second, without the need for mechanical scanning at the proximal or distal end of the fiber. The proof-of-concept benchtop system is tested in fluorescence mode and can resolve 1.5 µm features of a high resolution 1951 USAF target.