Two-color widefield fluorescence microendoscopy enables multiplexed molecular imaging in the alveolar space of human lung tissue

Photonic neural probe enabled microendoscopes for light-sheet light-field computational fluorescence brain imaging

Significance

Light-sheet fluorescence microscopy is widely used for high-speed, high-contrast, volumetric imaging. Application of this technique to in vivo brain imaging in non-transparent organisms has been limited by the geometric constraints of conventional light-sheet microscopes, which require orthogonal fluorescence excitation and collection objectives. We have recently demonstrated implantable photonic neural probes that emit addressable light sheets at depth in brain tissue, miniaturizing the excitation optics. Here, we propose a microendoscope consisting of a light-sheet neural probe packaged together with miniaturized fluorescence collection optics based on an image fiber bundle for lensless, light-field, computational fluorescence imaging.

Aim

Foundry-fabricated, silicon-based, light-sheet neural probes can be packaged together with commercially available image fiber bundles to form microendoscopes for light-sheet light-field fluorescence imaging at depth in brain tissue.

Approach

Prototype microendoscopes were developed using light-sheet neural probes with five addressable sheets and image fiber bundles. Fluorescence imaging with the microendoscopes was tested with fluorescent beads suspended in agarose and fixed mouse brain tissue.

Results

Volumetric light-sheet light-field fluorescence imaging was demonstrated using the microendoscopes. Increased imaging depth and enhanced reconstruction accuracy were observed relative to epi-illumination light-field imaging using only a fiber bundle.

Conclusions

Our work offers a solution toward volumetric fluorescence imaging of brain tissue with a compact size and high contrast. The proof-of-concept demonstrations herein illustrate the operating principles and methods of the imaging approach, providing a foundation for future investigations of photonic neural probe enabled microendoscopes for deep-brain fluorescence imaging in vivo.

Unsupervised full-color cellular image reconstruction through disordered optical fiber

Recent years have witnessed the tremendous development of fusing fiber-optic imaging with supervised deep learning to enable high-quality imaging of hard-to-reach areas. Nevertheless, the supervised deep learning method imposes strict constraints on fiber-optic imaging systems, where the input objects and the fiber outputs have to be collected in pairs. To unleash the full potential of fiber-optic imaging, unsupervised image reconstruction is in demand. Unfortunately, neither optical fiber bundles nor multimode fibers can achieve a point-to-point transmission of the object with a high sampling density, as is a prerequisite for unsupervised image reconstruction. The recently proposed disordered fibers offer a new solution based on the transverse Anderson localization. Here, we demonstrate unsupervised full-color imaging with a cellular resolution through a meter-long disordered fiber in both transmission and reflection modes. The unsupervised image reconstruction consists of two stages. In the first stage, we perform a pixel-wise standardization on the fiber outputs using the statistics of the objects. In the second stage, we recover the fine details of the reconstructions through a generative adversarial network. Unsupervised image reconstruction does not need paired images, enabling a much more flexible calibration under various conditions. Our new solution achieves full-color high-fidelity cell imaging within a working distance of at least 4 mm by only collecting the fiber outputs after an initial calibration. High imaging robustness is also demonstrated when the disordered fiber is bent with a central angle of 60°. Moreover, the cross-domain generality on unseen objects is shown to be enhanced with a diversified object set.

Fiber Bundle Image Reconstruction Using Convolutional Neural Networks and Bundle Rotation in Endomicroscopy

Fiber-bundle endomicroscopy has several recognized drawbacks, the most prominent being the honeycomb effect. We developed a multi-frame super-resolution algorithm exploiting bundle rotation to extract features and reconstruct underlying tissue. Simulated data was used with rotated fiber-bundle masks to create multi-frame stacks to train the model. Super-resolved images are numerically analyzed, which demonstrates that the algorithm can restore images with high quality. The mean structural similarity index measurement (SSIM) improved by a factor of 1.97 compared with linear interpolation. The model was trained using images taken from a single prostate slide, 1343 images were used for training, 336 for validation, and 420 for testing. The model had no prior information about the test images, adding to the robustness of the system. Image reconstruction was completed in 0.03 s for 256 × 256 images indicating future real-time performance is within reach. The combination of fiber bundle rotation and multi-frame image enhancement through machine learning has not been utilized before in an experimental setting but could provide a much-needed improvement to image resolution in practice.

Quantitative phase imaging through an ultra-thin lensless fiber endoscope

Quantitative phase imaging (QPI) is a label-free technique providing both morphology and quantitative biophysical information in biomedicine. However, applying such a powerful technique to in vivo pathological diagnosis remains challenging. Multi-core fiber bundles (MCFs) enable ultra-thin probes for in vivo imaging, but current MCF imaging techniques are limited to amplitude imaging modalities. We demonstrate a computational lensless microendoscope that uses an ultra-thin bare MCF to perform quantitative phase imaging with microscale lateral resolution and nanoscale axial sensitivity of the optical path length. The incident complex light field at the measurement side is precisely reconstructed from the far-field speckle pattern at the detection side, enabling digital refocusing in a multi-layer sample without any mechanical movement. The accuracy of the quantitative phase reconstruction is validated by imaging the phase target and hydrogel beads through the MCF. With the proposed imaging modality, three-dimensional imaging of human cancer cells is achieved through the ultra-thin fiber endoscope, promising widespread clinical applications.

High frame rate video mosaicking microendoscope to image large regions of intact tissue with subcellular resolution

High-resolution microendoscopy (HRME) is a low-cost strategy to acquire images of intact tissue with subcellular resolution at frame rates ranging from 11 to 18 fps. Current HRME imaging strategies are limited by the small microendoscope field of view (∼0.5 mm2); multiple images must be acquired and reliably registered to assess large regions of clinical interest. Image mosaics have been assembled from co-registered frames of video acquired as a microendoscope is slowly moved across the tissue surface, but the slow frame rate of previous HRME systems made this approach impractical for acquiring quality mosaicked images from large regions of interest. Here, we present a novel video mosaicking microendoscope incorporating a high frame rate CMOS sensor and optical probe holder to enable high-speed, high quality interrogation of large tissue regions of interest. Microendoscopy videos acquired at >90 fps are assembled into an image mosaic. We assessed registration accuracy and image sharpness across the mosaic for images acquired with a handheld probe over a range of translational speeds. This high frame rate video mosaicking microendoscope enables in vivo probe translation at >15 millimeters per second while preserving high image quality and accurate mosaicking, increasing the size of the region of interest that can be interrogated at high resolution from 0.5 mm2 to >30 mm2. Real-time deployment of this high-frame rate system is demonstrated in vivo and source code made publicly available.

Molecular Supracence Resolving Eight Colors in 300-nm Width: Unprecedented Spectral Resolution

Monitoring multiple molecular probes simultaneously establishes their correlations and reveals the holistic mechanism. Current fluorescence imaging, however, is limited to about four colors because of typically circa 100-nm spectral width. Herein, we show that molecular supracence imparts superior spectral resolution, resolving eight colors in 300-nm width, about 37.5-nm per color. A recently discovered light-molecule interacting phenomenon, supracence only measures molecular emission above its excitation energy due to entanglement between atomic quantum system and electronic quantum system. As such, supracence takes advantage of sharp spectral edge of a single pathway and excitation specificity to produce narrow bands, whereas fluorescence has to deal with multiple pathways spilling out low-energy long tail, that causes poor resolution. Thus, supracence enables myriad innovative molecular spectroscopy and microscopic imaging with profound impact broadly.

Image computing for fibre-bundle endomicroscopy: A review

Endomicroscopy is an emerging imaging modality, that facilitates the acquisition of in vivo, in situ optical biopsies, assisting diagnostic and potentially therapeutic interventions. While there is a diverse and constantly expanding range of commercial and experimental optical biopsy platforms available, fibre-bundle endomicroscopy is currently the most widely used platform and is approved for clinical use in a range of clinical indications. Miniaturised, flexible fibre-bundles, guided through the working channel of endoscopes, needles and catheters, enable high-resolution imaging across a variety of organ systems. Yet, the nature of image acquisition though a fibre-bundle gives rise to several inherent characteristics and limitations necessitating novel and effective image pre- and post-processing algorithms, ranging from image formation, enhancement and mosaicing to pathology detection and quantification. This paper introduces the underlying technology and most prevalent clinical applications of fibre-bundle endomicroscopy, and provides a comprehensive, up-to-date, review of relevant image reconstruction, analysis and understanding/inference methodologies. Furthermore, current limitations as well as future challenges and opportunities in fibre-bundle endomicroscopy computing are identified and discussed.

Demonstrating the Use of Optical Fibres in Biomedical Sensing: A Collaborative Approach for Engagement and Education

This paper demonstrates how research at the intersection of physics, engineering, biology and medicine can be presented in an interactive and educational way to a non-scientific audience. Interdisciplinary research with a focus on prevalent diseases provides a relatable context that can be used to engage with the public. Respiratory diseases are significant contributors to avoidable morbidity and mortality and have a growing social and economic impact. With the aim of improving lung disease understanding, new techniques in fibre-based optical endomicroscopy have been recently developed. Here, we present a novel engagement activity that resembles a bench-to-bedside pathway. The activity comprises an inexpensive educational tool (<$70) adapted from a clinical optical endomicroscopy system and tutorials that cover state-of-the-art research. The activity was co-created by high school science teachers and researchers in a collaborative way that can be implemented into any engagement development process.

Optical Molecular Imaging of Inflammatory Cells in Interventional Medicine-An Emerging Strategy

The optical molecular imaging of inflammation is an emerging strategy for interventional medicine and diagnostics. The host's inflammatory response and adaptation to acute and chronic diseases provides unique signatures that have the potential to guide interventions. Thus, there are emerging a suite of molecular imaging and sensing approaches for a variety of targets in this area. This review will focus on two key cellular orchestrators that dominate this area, neutrophils and macrophages, with recent developments in molecular probes and approaches discussed.

In Vivo Endomicroscopy of Lung Injury and Repair in ARDS: Potential Added Value to Current Imaging

BACKGROUND

Standard clinical imaging of the acute respiratory distress syndrome (ARDS) lung lacks resolution and offers limited possibilities in the exploration of the structure-function relationship, and therefore cannot provide an early and clear discrimination of patients with unexpected diagnosis and unrepair profile. The current gold standard is open lung biopsy (OLB). However, despite being able to reveal precise information about the tissue collected, OLB cannot provide real-time information on treatment response and is accompanied with a complication risk rate up to 25%, making longitudinal monitoring a dangerous endeavor. Intravital probe-based confocal laser endomicroscopy (pCLE) is a developing and innovative high-resolution imaging technology. pCLE offers the possibility to leverage multiple and specific imaging probes to enable multiplex screening of several proteases and pathogenic microorganisms, simultaneously and longitudinally, in the lung. This bedside method will ultimately enable physicians to rapidly, noninvasively, and accurately diagnose degrading lung and/or fibrosis without the need of OLBs.

OBJECTIVES AND METHODS

To extend the information provided by standard imaging of the ARDS lung with a bedside, high-resolution, miniaturized pCLE through the detailed molecular imaging of a carefully selected region-of-interest (ROI). To validate and quantify real-time imaging to validate pCLE against OLB.

RESULTS

Developments in lung pCLE using fluorescent affinity- or activity-based probes at both preclinical and clinical (first-in-man) stages are ongoing-the results are promising, revealing correlations with OLBs in problematic ARDS.

CONCLUSION

It can be envisaged that safe, high-resolution, noninvasive pCLE with activatable fluorescence probes will provide a "virtual optical biopsy" and will provide decisive information in selected ARDS patients at the bedside.

Bayesian bacterial detection using irregularly sampled optical endomicroscopy images

Pneumonia is a major cause of morbidity and mortality of patients in intensive care. Rapid determination of the presence and gram status of the pathogenic bacteria in the distal lung may enable a more tailored treatment regime. Optical Endomicroscopy (OEM) is an emerging medical imaging platform with preclinical and clinical utility. Pulmonary OEM via multi-core fibre bundles has the potential to provide in vivo, in situ, fluorescent molecular signatures of the causes of infection and inflammation. This paper presents a Bayesian approach for bacterial detection in OEM images. The model considered assumes that the observed pixel fluorescence is a linear combination of the actual intensity value associated with tissues or background, corrupted by additive Gaussian noise and potentially by an additional sparse outlier term modelling anomalies (bacteria). The bacteria detection problem is formulated in a Bayesian framework and prior distributions are assigned to the unknown model parameters. A Markov chain Monte Carlo algorithm based on a partially collapsed Gibbs sampler is used to sample the posterior distribution of the unknown parameters. The proposed algorithm is first validated by simulations conducted using synthetic datasets for which good performance is obtained. Analysis is then conducted using two ex vivo lung datasets in which fluorescently labelled bacteria are present in the distal lung. A good correlation between bacteria counts identified by a trained clinician and those of the proposed method, which detects most of the manually annotated regions, is observed.

Enhanced avidity from a multivalent fluorescent antimicrobial peptide enables pathogen detection in a human lung model

Rapid in situ detection of pathogens coupled with high resolution imaging in the distal human lung has the potential to provide new insights and diagnostic utility in patients in whom pneumonia is suspected. We have previously described an antimicrobial peptide (AMP) Ubiquicidin (fragment UBI29-41) labelled with an environmentally sensitive fluorophore that optically detected bacteria in vitro but not ex vivo. Here, we describe further chemical development of this compound and demonstrate that altering the secondary structure of the AMP to generate a tri-branched dendrimeric scaffold provides enhanced signal in vitro and ex vivo and consequently allows the rapid detection of pathogens in situ in an explanted human lung. This compound (NBD-UBIdend) demonstrates bacterial labelling specificity for a broad panel of pathogenic bacteria and Aspergillus fumigatus. NBD-UBIdend demonstrated high signal-to-noise fluorescence amplification upon target engagement, did not label host mammalian cells and was non-toxic and chemically robust within the inflamed biological environment. Intrapulmonary delivery of NBD-UBIdend, coupled with optical endomicroscopy demonstrated real-time, in situ detection of bacteria in explanted whole human Cystic Fibrosis lungs.

Fibre-based spectral ratio endomicroscopy for contrast enhancement of bacterial imaging and pulmonary autofluorescence

Fibre-based optical endomicroscopy (OEM) permits high resolution fluorescence microscopy in endoscopically accessible tissues. Fibred OEM has the potential to visualise pathologies targeted with fluorescent imaging probes and provide an in vivo in situ molecular pathology platform to augment disease understanding, diagnosis and stratification. Here we present an inexpensive widefield ratiometric fibred OEM system capable of enhancing the contrast between similar spectra of pathologically relevant fluorescent signals without the burden of complex spectral unmixing. As an exemplar, we demonstrate the potential of the platform to detect fluorescently labelled Gram-negative bacteria in the challenging environment of highly autofluorescent lung tissue in whole ex vivo human lungs.

Optical fiber bundles: Ultra-slim light field imaging probes

Optical fiber bundle microendoscopes are widely used for visualizing hard-to-reach areas of the human body. These ultrathin devices often forgo tunable focusing optics because of size constraints and are therefore limited to two-dimensional (2D) imaging modalities. Ideally, microendoscopes would record 3D information for accurate clinical and biological interpretation, without bulky optomechanical parts. Here, we demonstrate that the optical fiber bundles commonly used in microendoscopy are inherently sensitive to depth information. We use the mode structure within fiber bundle cores to extract the spatio-angular description of captured light rays-the light field-enabling digital refocusing, stereo visualization, and surface and depth mapping of microscopic scenes at the distal fiber tip. Our work opens a route for minimally invasive clinical microendoscopy using standard bare fiber bundle probes. Unlike coherent 3D multimode fiber imaging techniques, our incoherent approach is single shot and resilient to fiber bending, making it attractive for clinical adoption.

High-speed dual color fluorescence lifetime endomicroscopy for highly-multiplexed pulmonary diagnostic applications and detection of labeled bacteria

We present a dual-color laser scanning endomicroscope capable of fluorescence lifetime endomicroscopy at one frame per second (FPS). The scanning system uses a coherent imaging fiber with 30,000 cores. High-speed lifetime imaging is achieved by distributing the signal over an array of 1024 parallel single-photon avalanche diode detectors (SPADs), minimizing detection dead-time maximizing the number of photons detected per excitation pulse without photon pile-up to achieve the high frame rate. This also enables dual color fluorescence imaging by temporally shifting the dual excitation lasers, with respect to each other, to separate the two spectrally distinct fluorescent decays in time. Combining the temporal encoding, to provide spectral separation, with lifetime measurements we show a one FPS, multi-channel endomicroscopy platform for clinical applications and diagnosis. We demonstrate the potential of the system by imaging SmartProbe labeled bacteria in ex vivo samples of human lung using lifetime to differentiate bacterial fluorescence from the strong background lung autofluorescence which was used to provide structural information.

Development of an alveolar transbronchial catheter for concurrent fiber optics based imaging and fluid delivery

Optical molecular imaging is an emerging field and high resolution optical imaging of the distal lung parenchyma has been made possible with the advent of clinically approved fiber based imaging modalities. However, currently, there is no single method of allowing the simultaneous imaging and delivery of targeted molecular imaging agents. The objective of this research is to create a catheterized device capable of fulfilling this need. We describe the rationale, development, and validation in ex vivo ovine lung to near clinical readiness of a triple lumen bronchoscopy catheter that allows concurrent imaging and fluid delivery, with the aim of clinical use to deliver multiple fluorescent compounds to image alveolar pathology. Using this device, we were able to produce high-quality images of bacterial infiltrates in ex-vivo ovine lung within 60 seconds of instilling a single microdose of (<100 mcgs) imaging agent. This has many advantages for future clinical usage over the current state of the art.

Low-cost high sensitivity pulsed endomicroscopy to visualize tricolor optical signatures

A highly sensitive, modular three-color fluorescence endomicroscopy imaging platform spanning the visible to near-infrared (NIR) range is demonstrated. Light-emitting diodes (LEDs) were sequentially pulsed along with the camera acquisition to provide up to 20 frames per second (fps) three-color imaging performance or 60 fps single color imaging. The system was characterized for bacterial and cellular molecular imaging in ex vivo human lung tissue and for bacterial and indocyanine green imaging in ex vivo perfused sheep lungs. A practical method to reduce background tissue autofluorescence is also proposed. The platform was clinically translated into six patients with pulmonary disease to delineate healthy, cancerous, and fibrotic tissue autofluorescent structures. The instrument is the most broadband clinical endomicroscopy system developed to date (covering visible to the NIR, 500 to 900 nm) and demonstrates significant potential for future clinical utility due to its low cost and modular capability to suit a wide variety of molecular imaging applications.

Optical Screening of Novel Bacteria-specific Probes on Ex Vivo Human Lung Tissue by Confocal Laser Endomicroscopy

Improving the speed and accuracy of bacterial detection is important for patient stratification and to ensure the appropriate use of antimicrobials. To achieve this goal, the development of diagnostic techniques to recognize bacterial presence in real-time at the point-of-care is required. Optical imaging for direct identification of bacteria within the host is an attractive approach. Several attempts at chemical probe design and validation have been investigated, however none have yet been successfully translated into the clinic. Here we describe a method for ex vivo validation of bacteria-specific probes for identification of bacteria within the distal lung, imaged by fibered confocal fluorescence microscopy (FCFM). Our model used ex vivo human lung tissue and a clinically approved confocal laser endomicroscopy (CLE) platform to screen novel bacteria-specific imaging compounds, closely mimicking imaging conditions expected to be encountered with patients. Therefore, screening compounds by this technique provides confidence of potential clinical tractability.

Ballistic and snake photon imaging for locating optical endomicroscopy fibres

We demonstrate determination of the location of the distal-end of a fibre-optic device deep in tissue through the imaging of ballistic and snake photons using a time resolved single-photon detector array. The fibre was imaged with centimetre resolution, within clinically relevant settings and models. This technique can overcome the limitations imposed by tissue scattering in optically determining the in vivo location of fibre-optic medical instruments.

Characterization and modelling of inter-core coupling in coherent fiber bundles

Recent developments in optical endomicroscopy (OEM) and associated fluorescent SmartProbes present a need for sensitive imaging with high detection performance. Inter-core coupling within coherent fiber bundles is a well recognized limitation, affecting the technology's imaging capabilities. Fiber cross coupling has been studied both experimentally and within a theoretical framework (coupled mode theory), providing (i) insights on the factors affecting cross talk, and (ii) recommendations for optimal fiber bundle design. However, due to physical limitations, such as the tradeoff between cross coupling and core density, cross coupling can be suppressed yet not eliminated through optimal fiber design. This study introduces a novel approach for measuring, analyzing and quantifying cross coupling within coherent fiber bundles, in a format that can be integrated into a linear model, which in turn can enable computational compensation of the associated blurring introduced to OEM images.

Low index contrast imaging fibers

We present high-resolution imaging fibers made from low-cost commercially available fiber preforms manufactured for the telecommunications industry. Our fabrication method involves multi-stacking arrays of different sized cores in order to suppress core-to-core crosstalk whilst building up a large array of cores. One of the fibers, based on a square array of cores, has comparable imaging performance to commercial imaging fibers but without the need for exceptionally high refractive index contrasts, and will enable the development of economically viable single-use disposable imaging fibers.

Ultrabright fluorescent silica particles with a large number of complex spectra excited with a single wavelength for multiplex applications

We report on a novel approach to synthesize ultrabright fluorescent silica particles capable of producing a large number of complex spectra. The spectra can be excited using a single wavelength which is paramount in quantitative fluorescence imaging, flow cytometry and sensing applications. The approach employs the physical encapsulation of organic fluorescent molecules inside a nanoporous silica matrix with no dye leakage. As was recently demonstrated, such an encapsulation allowed for the encapsulation of very high concentrations of organic dyes without quenching their fluorescent efficiency. As a result, dye molecules are distanced within ∼5 nm from each other; it theoretically allows for efficient exchange of excitation energy via Förster resonance energy transfer (FRET). Here we present the first experimental demonstration of the encapsulation of fluorescent dyes in the FRET sequence. Attaining a FRET sequence of up to five different dyes is presented. The number of distinguishable spectra can be further increased by using different relative concentrations of encapsulated dyes. Combining these approaches allows for creating a large number of ultrabright fluorescent particles with substantially different fluorescence spectra. We also demonstrate the utilization of these particles for potential multiplexing applications. Though fluorescence spectra of the obtained multiplex probes are typically overlapping, they can be distinguished by using standard linear decomposition algorithms.