Hyperspectral light sheet microscopy

Imaging 3D cell cultures with optical microscopy

Three-dimensional (3D) cell cultures have gained popularity in recent years due to their ability to represent complex tissues or organs more faithfully than conventional two-dimensional (2D) cell culture. This article reviews the application of both 2D and 3D microscopy approaches for monitoring and studying 3D cell cultures. We first summarize the most popular optical microscopy methods that have been used with 3D cell cultures. We then discuss the general advantages and disadvantages of various microscopy techniques for several broad categories of investigation involving 3D cell cultures. Finally, we provide perspectives on key areas of technical need in which there are clear opportunities for innovation. Our goal is to guide microscope engineers and biomedical end users toward optimal imaging methods for specific investigational scenarios and to identify use cases in which additional innovations in high-resolution imaging could be helpful.

Light sheet illumination in single-molecule localization microscopy for imaging of cellular architectures and molecular dynamics

Single-molecule localization microscopy has revealed cellular architectures and molecular dynamics beyond the diffraction limit of light. However, imaging thick samples presents challenges from increased fluorescence background. Light sheet illumination, which utilizes a plane of light for optical sectioning, is effective in reducing fluorescence background, photobleaching, and photodamage. Here, we present the principles of single-molecule localization microscopy and light sheet illumination, followed by an introduction to light sheet microscopy geometries and their imaging applications. Finally, we discuss light sheet illumination approaches for high- and super-resolution imaging of biological structures and dynamics.

Hyperspectral oblique plane microscopy enables spontaneous, label-free imaging of biological dynamic processes in live animals

Spontaneous Raman imaging has emerged as powerful label-free technique for investigating the molecular composition of medicines and biological specimens. Although Raman imaging can facilitate understanding of complex biological phenomena in vivo, current imaging modalities are limited in speed and sample compatibility. Here, we introduce a single-objective line-scanning light-sheet microscope, named [Formula: see text]-OPM, which records Raman images on a timescale of minutes to seconds. To demonstrate its function, we use [Formula: see text]-OPM to map and identify microplastic particles based on their Raman spectral characteristics. In live zebrafish embryos, we show that [Formula: see text]-OPM can capture wound dynamics at five-minute intervals, revealing rapid changes in cellular and extracellular matrix composition in the wounded region. Finally, we use [Formula: see text]-OPM to synchronize and average 36,800 individual frames to obtain hyperspectral videos of a zebrafish embryo's beating heart at an effective 28 frames per second, recording compositional changes throughout the cardiac cycle.

Rational partitioning of spectral feature space for effective clustering of massive spectral image data

We have successfully proposed and demonstrated a clustering method that overcomes the "needle-in-a-haystack problem" (finding minuscule important regions from massive spectral image data sets). The needle-in-a-haystack problem is of central importance in the characterization of materials since in bulk materials, the properties of a very tiny region often dominate the entire function. To solve this problem, we propose that rational partitioning of the spectral feature space in which spectra are distributed, or defining of the decision boundaries for clustering, can be performed by focusing on the discrimination limit defined by the measurement noise and partitioning the space at intervals of this limit. We verified the proposed method, applied it to actual measurement data, and succeeded in detecting tiny (~ 0.5%) important regions that were difficult for human researchers and other machine learning methods to detect in discovering unknown phases. The ability to detect these crucial regions helps in understanding materials and designing more functional materials.

Alkyne-tagged imidazolium-based membrane cholesterol analogs for Raman imaging applications

Cholesterol is an important lipid playing a crucial role in mediating essential cellular processes as well as maintaining the basic structural integrity of biological membranes. Given its vast biological importance, there is an unabated need for sophisticated strategies to investigate cholesterol-mediated biological processes. Raman-tagged sterol analogs offer the advantage of being visualizable without the need for a bulky dye that potentially affects natural membrane integration and cellular interactions as it is the case for many conventionally used fluorescent analogs. Herein, we report a series of alkyne-tagged imidazolium-based cholesterol analogs (CHIMs) with large Raman scattering cross-sections that readily integrate into HEK cells and primary monocyte-derived macrophages and allow (multiplexed) cellular Raman imaging. We envision Raman-tagged CHIM analogs to be a powerful platform for the investigation of cholesterol-mediated cellular processes complementary to other established methods, such as the use of fluorescent analogs.

Multiplexed volumetric CLEM enabled by scFvs provides insights into the cytology of cerebellar cortex

Mapping neuronal networks is a central focus in neuroscience. While volume electron microscopy (vEM) can reveal the fine structure of neuronal networks (connectomics), it does not provide molecular information to identify cell types or functions. We developed an approach that uses fluorescent single-chain variable fragments (scFvs) to perform multiplexed detergent-free immunolabeling and volumetric-correlated-light-and-electron-microscopy on the same sample. We generated eight fluorescent scFvs targeting brain markers. Six fluorescent probes were imaged in the cerebellum of a female mouse, using confocal microscopy with spectral unmixing, followed by vEM of the same sample. The results provide excellent ultrastructure superimposed with multiple fluorescence channels. Using this approach, we documented a poorly described cell type, two types of mossy fiber terminals, and the subcellular localization of one type of ion channel. Because scFvs can be derived from existing monoclonal antibodies, hundreds of such probes can be generated to enable molecular overlays for connectomic studies.

Deep 3D histology powered by tissue clearing, omics and AI

To comprehensively understand tissue and organism physiology and pathophysiology, it is essential to create complete three-dimensional (3D) cellular maps. These maps require structural data, such as the 3D configuration and positioning of tissues and cells, and molecular data on the constitution of each cell, spanning from the DNA sequence to protein expression. While single-cell transcriptomics is illuminating the cellular and molecular diversity across species and tissues, the 3D spatial context of these molecular data is often overlooked. Here, I discuss emerging 3D tissue histology techniques that add the missing third spatial dimension to biomedical research. Through innovations in tissue-clearing chemistry, labeling and volumetric imaging that enhance 3D reconstructions and their synergy with molecular techniques, these technologies will provide detailed blueprints of entire organs or organisms at the cellular level. Machine learning, especially deep learning, will be essential for extracting meaningful insights from the vast data. Further development of integrated structural, molecular and computational methods will unlock the full potential of next-generation 3D histology.

Linear spectral unmixing analysis in single-molecule FRET spectroscopy for fluorophores with large spectral overlap

Fluorescence resonance energy transfer (FRET) is a highly useful tool to investigate biomolecular interactions and dynamics in single-molecule spectroscopy and nanoscopy. However, the use of spectrally overlapping dye pairs results in various artifact signals that prevent accurate determination of FRET values. In this paper, an algorithmic method of spectral unmixing was devised to extract FRET values of spectrally overlapping dye pairs at the single molecule level. Application of this method allows the determination of both the donor-acceptor composition and the FRET efficiency of the samples labelled with spectrally overlapping dye pairs.

Multispectral imaging for characterizing autofluorescent tissues

Selective Plane Illumination Microscopy (SPIM) has become an emerging technology since its first application for 3D in-vivo imaging of the development of a living organism. An extensive number of works have been published, improving both the speed of acquisition and the resolution of the systems. Furthermore, multispectral imaging allows the effective separation of overlapping signals associated with different fluorophores from the spectrum over the whole field-of-view of the analyzed sample. To eliminate the need of using fluorescent dyes, this technique can also be applied to autofluorescence imaging. However, the effective separation of the overlapped spectra in autofluorescence imaging necessitates the use of mathematical tools. In this work, we explore the application of a method based on Principal Component Analysis (PCA) that enables tissue characterization upon spectral autofluorescence data without the use of fluorophores. Thus, enabling the separation of different tissue types in fixed and living samples with no need of staining techniques. Two procedures are described for acquiring spectral data, including a single excitation based method and a multi-excitation scanning approach. In both cases, we demonstrate the effective separation of various tissue types based on their unique autofluorescence spectra.

Deep learning-enhanced snapshot hyperspectral confocal microscopy imaging system

Laser-scanning confocal hyperspectral microscopy is a powerful technique to identify the different sample constituents and their spatial distribution in three-dimensional (3D). However, it suffers from low imaging speed because of the mechanical scanning methods. To overcome this challenge, we propose a snapshot hyperspectral confocal microscopy imaging system (SHCMS). It combined coded illumination microscopy based on a digital micromirror device (DMD) with a snapshot hyperspectral confocal neural network (SHCNet) to realize single-shot confocal hyperspectral imaging. With SHCMS, high-contrast 160-bands confocal hyperspectral images of potato tuber autofluorescence can be collected by only single-shot, which is almost 5 times improvement in the number of spectral channels than previously reported methods. Moreover, our approach can efficiently record hyperspectral volumetric imaging due to the optical sectioning capability. This fast high-resolution hyperspectral imaging method may pave the way for real-time highly multiplexed biological imaging.

Planar 16-band metasurface-enhanced spectral filter for integrated image sensing

We study theoretically and demonstrate experimentally a 16-band narrow band wavelength selective filter in the near-infrared range. The combination of a pair of distributed Bragg reflectors with a sub-wavelength grating metasurface embedded in the intra-cavity provides a narrow response which can be tuned by adjusting the geometry of the sub-wavelength grating metasurface. The key advantage of this approach is its ease of fabrication, where the spectral response is tuned by merely changing the grating period, resulting in a perfectly planar geometry that can be easily integrated with a broad variety of photodetectors, thus enabling attractive applications such as bio-imaging, time-of-flight sensors and LiDAR. The experimental results are supported by numerical simulations and effective medium theory that unveil the mechanisms that lead to the optical response of the device. It is also shown how the polarization dependence of the structure can be used to determine very accurately the polarization of incoming light.

Multiplexed volumetric CLEM enabled by antibody derivatives provides new insights into the cytology of the mouse cerebellar cortex

Mapping neuronal networks that underlie behavior has become a central focus in neuroscience. While serial section electron microscopy (ssEM) can reveal the fine structure of neuronal networks (connectomics), it does not provide the molecular information that helps identify cell types or their functional properties. Volumetric correlated light and electron microscopy (vCLEM) combines ssEM and volumetric fluorescence microscopy to incorporate molecular labeling into ssEM datasets. We developed an approach that uses small fluorescent single-chain variable fragment (scFv) immuno-probes to perform multiplexed detergent-free immuno-labeling and ssEM on the same samples. We generated eight such fluorescent scFvs that targeted useful markers for brain studies (green fluorescent protein, glial fibrillary acidic protein, calbindin, parvalbumin, voltage-gated potassium channel subfamily A member 2, vesicular glutamate transporter 1, postsynaptic density protein 95, and neuropeptide Y). To test the vCLEM approach, six different fluorescent probes were imaged in a sample of the cortex of a cerebellar lobule (Crus 1), using confocal microscopy with spectral unmixing, followed by ssEM imaging of the same sample. The results show excellent ultrastructure with superimposition of the multiple fluorescence channels. Using this approach we could document a poorly described cell type in the cerebellum, two types of mossy fiber terminals, and the subcellular localization of one type of ion channel. Because scFvs can be derived from existing monoclonal antibodies, hundreds of such probes can be generated to enable molecular overlays for connectomic studies.

Multiplexed volumetric CLEM enabled by antibody derivatives provides new insights into the cytology of the mouse cerebellar cortex

Mapping neuronal networks that underlie behavior has become a central focus in neuroscience. While serial section electron microscopy (ssEM) can reveal the fine structure of neuronal networks (connectomics), it does not provide the molecular information that helps identify cell types or their functional properties. Volumetric correlated light and electron microscopy (vCLEM) combines ssEM and volumetric fluorescence microscopy to incorporate molecular labeling into ssEM datasets. We developed an approach that uses small fluorescent single-chain variable fragment (scFv) immuno-probes to perform multiplexed detergent-free immuno-labeling and ssEM on the same samples. We generated eight such fluorescent scFvs that targeted useful markers for brain studies (green fluorescent protein, glial fibrillary acidic protein, calbindin, parvalbumin, voltage-gated potassium channel subfamily A member 2, vesicular glutamate transporter 1, postsynaptic density protein 95, and neuropeptide Y). To test the vCLEM approach, six different fluorescent probes were imaged in a sample of the cortex of a cerebellar lobule (Crus 1), using confocal microscopy with spectral unmixing, followed by ssEM imaging of the same sample. The results show excellent ultrastructure with superimposition of the multiple fluorescence channels. Using this approach we could document a poorly described cell type in the cerebellum, two types of mossy fiber terminals, and the subcellular localization of one type of ion channel. Because scFvs can be derived from existing monoclonal antibodies, hundreds of such probes can be generated to enable molecular overlays for connectomic studies.

A single-shot hyperspectral phasor camera for fast, multi-color fluorescence microscopy

Hyperspectral fluorescence imaging improves multiplexed observations of biological samples by utilizing multiple color channels across the spectral range to compensate for spectral overlap between labels. Typically, spectral resolution comes at a cost of decreased detection efficiency, which both hampers imaging speed and increases photo-toxicity to the samples. Here, we present a high-speed, high-efficiency snapshot spectral acquisition method, based on optical compression of the fluorescence spectra via Fourier transform, that overcomes the challenges of discrete spectral sampling: single-shot hyperspectral phasor camera (SHy-Cam). SHy-Cam captures fluorescence spatial and spectral information in a single exposure with a standard scientific CMOS camera, with photon efficiency of over 80%, easily and with acquisition rates exceeding 30 datasets per second, making it a powerful tool for multi-color in vivo imaging. Its simple design, using readily available optical components, and its easy integration provide a low-cost solution for multi-color fluorescence imaging with increased efficiency and speed.

In Situ Visualization of Membrane Fouling Evolution during Ultrafiltration Using Label-Free Hyperspectral Light Sheet Fluorescence Imaging

Profound understanding of fouling behaviors and underlying mechanisms is fundamentally important for fouling control in membrane-based environmental applications. Therefore, it entails novel noninvasive analytical approaches for in situ characterizing the formation and development of membrane fouling processes. This work presents a characterization approach based on hyperspectral light sheet fluorescence microscopy (HSPEC-LSFM), which is capable of discriminating various foulants and providing their 2-dimensional/3-dimensional spatial distributions on/in membranes in a label-free manner. A fast, highly sensitive and noninvasive imaging platform was established by developing a HSPEC-LSFM system and further extending it to incorporate a laboratory-scale pressure-driven membrane filtration system. Hyperspectral data sets with a spectral resolution of ∼1.1 nm and spatial resolution of ∼3 μm as well as the temporal resolution of ∼8 s/plane were obtained, and the fouling formation and development process of foulants onto membrane surfaces, within the pores and on the pore walls were clearly observed during the ultrafiltration of protein and humic substances solutions. Pore blocking/constriction at short times while cake growth/concentration polarization at longer times was found to have coupled effects for the flux decline in these filtration tests, and yet the contribution of each effect as well as the transition of the governing mechanisms was found distinct. These results demonstrate in situ label-free characterization of membrane fouling evolution with the recognition of foulant species during filtration and provide new insights into membrane fouling. This work offers a powerful tool to investigate dynamic processes for a wide range of membrane-based explorations.

HyU: Hybrid Unmixing for longitudinal in vivo imaging of low signal-to-noise fluorescence

The expansion of fluorescence bioimaging toward more complex systems and geometries requires analytical tools capable of spanning widely varying timescales and length scales, cleanly separating multiple fluorescent labels and distinguishing these labels from background autofluorescence. Here we meet these challenging objectives for multispectral fluorescence microscopy, combining hyperspectral phasors and linear unmixing to create Hybrid Unmixing (HyU). HyU is efficient and robust, capable of quantitative signal separation even at low illumination levels. In dynamic imaging of developing zebrafish embryos and in mouse tissue, HyU was able to cleanly and efficiently unmix multiple fluorescent labels, even in demanding volumetric timelapse imaging settings. HyU permits high dynamic range imaging, allowing simultaneous imaging of bright exogenous labels and dim endogenous labels. This enables coincident studies of tagged components, cellular behaviors and cellular metabolism within the same specimen, providing more accurate insights into the orchestrated complexity of biological systems.

Tunable image projection spectrometry

We present tunable image projection spectrometry (TIPS), a Fourier-domain line-scan spectral imager with a tunable compression ratio. Compared to state-of-the-art spatial-domain pushbroom hyperspectral cameras, TIPS requires much fewer measurements and provides a higher light throughput. Using a rotating Dove prism and a cylindrical field lens, TIPS scans an input scene in the Fourier domain and captures a subset of multi-angled one-dimensional (1D) en face projections of the input scene, allowing a tailored data compression ratio for a given scene. We demonstrate the spectral imaging capability of TIPS with a hematoxylin and eosin (H&E) stained pathology slide. Moreover, we showed the spectral information obtained can be further converted to depths when combining TIPS with a low-coherence full-field spectral-domain interferometer.

Multispectral confocal 3D imaging of intact healthy and tumor tissue using mLSR-3D

Revealing the 3D composition of intact tissue specimens is essential for understanding cell and organ biology in health and disease. State-of-the-art 3D microscopy techniques aim to capture tissue volumes on an ever-increasing scale, while also retaining sufficient resolution for single-cell analysis. Furthermore, spatial profiling through multi-marker imaging is fast developing, providing more context and better distinction between cell types. Following these lines of technological advance, we here present a protocol based on FUnGI (fructose, urea and glycerol clearing solution for imaging) optical clearing of tissue before multispectral large-scale single-cell resolution 3D (mLSR-3D) imaging, which implements 'on-the-fly' linear unmixing of up to eight fluorophores during a single acquisition. Our protocol removes the need for repetitive illumination, thereby allowing larger volumes to be scanned with better image quality in less time, also reducing photo-bleaching and file size. To aid in the design of multiplex antibody panels, we provide a fast and manageable intensity equalization assay with automated analysis to design a combination of markers with balanced intensities suitable for mLSR-3D. We demonstrate effective mLSR-3D imaging of various tissues, including patient-derived organoids and xenografted tumors, and, furthermore, describe an optimized workflow for mLSR-3D imaging of formalin-fixed paraffin-embedded samples. Finally, we provide essential steps for 3D image data processing, including shading correction that does not require pre-acquired shading references and 3D inhomogeneity correction to correct fluorescence artefacts often afflicting 3D datasets. Together, this provides a one-week protocol for eight-fluorescent-marker 3D visualization and exploration of intact tissue of various origins at single-cell resolution.

Super-multiplexing excitation spectral microscopy with multiple fluorescence bands

Fluorescence microscopy, with high molecular specificity and selectivity, is a valuable tool for studying complex biological systems and processes. However, the ability to distinguish a large number of distinct subcellular structures in a single sample is impeded by the broad spectra of molecular fluorescence. We have recently shown that excitation spectral microscopy provides a powerful means to unmix up to six fluorophores in a single fluorescence band. Here, by working with multiple fluorescence bands, we extend this approach to the simultaneous imaging of up to ten targets, with the potential for further expansions. By covering the excitation/emission bandwidth across the full visible range, an ultra-broad 24-wavelength excitation scheme is established through frame-synchronized scanning of the excitation wavelength from a white lamp via an acousto-optic tunable filter (AOTF), so that full-frame excitation-spectral images are obtained every 24 camera frames, offering superior spectral information and multiplexing capability. With numerical simulations, we validate the concurrent imaging of 10 fluorophores spanning the visible range to achieve exceptionally low (∼0.5%) crosstalks. For cell imaging experiments, we demonstrate unambiguous identification of up to eight different intracellular structures labeled by common fluorophores of substantial spectral overlap with minimal color crosstalks. We thus showcase an easy-to-implement, cost-effective microscopy system for visualizing complex cellular components with more colors and lower crosstalks.

Pterygium and Ocular Surface Squamous Neoplasia: Optical Biopsy Using a Novel Autofluorescence Multispectral Imaging Technique

In this study, differentiation of pterygium vs. ocular surface squamous neoplasia based on multispectral autofluorescence imaging technique was investigated. Fifty (N = 50) patients with histopathological diagnosis of pterygium (PTG) and/or ocular surface squamous neoplasia (OSSN) were recruited. Fixed unstained biopsy specimens were imaged by multispectral microscopy. Tissue autofluorescence images were obtained with a custom-built fluorescent microscope with 59 spectral channels, each with specific excitation and emission wavelength ranges, suitable for the most abundant tissue fluorophores such as elastin, flavins, porphyrin, and lipofuscin. Images were analyzed using a new classification framework called fused-classification, designed to minimize interpatient variability, as an established support vector machine learning method. Normal, PTG, and OSSN regions were automatically detected and delineated, with accuracy evaluated against expert assessment by a specialist in OSSN pathology. Signals from spectral channels yielding signals from elastin, flavins, porphyrin, and lipofuscin were significantly different between regions classified as normal, PTG, and OSSN (p < 0.01). Differential diagnosis of PTG/OSSN and normal tissue had accuracy, sensitivity, and specificity of 88 ± 6%, 84 ± 10% and 91 ± 6%, respectively. Our automated diagnostic method generated maps of the reasonably well circumscribed normal/PTG and OSSN interface. PTG and OSSN margins identified by our automated analysis were in close agreement with the margins found in the H&E sections. Such a map can be rapidly generated on a real time basis and potentially used for intraoperative assessment.

Computational hyperspectral light-sheet microscopy

We describe a computational light-sheet microscope designed for hyperspectral acquisition at high spectral resolution. The fluorescence light emitted from the full field-of-view is focused along the entrance slit of an imaging spectrometer using a cylindrical lens. To acquire the spatial dimension orthogonal to the slit of the spectrometer, we propose to illuminate the specimen with a sequence of structured light patterns and to solve the image reconstruction problem. Beam shaping is obtained simply using a digital micromirror device in conjunction with a traditional selective plane illumination microscopy setup. We demonstrate the feasibility of this method and report the first results in vivo in hydra specimens labeled using two fluorophores.

Innovations in ex vivo Light Sheet Fluorescence Microscopy

Light Sheet Fluorescence Microscopy (LSFM) has revolutionized how optical imaging of biological specimens can be performed as this technique allows to produce 3D fluorescence images of entire samples with a high spatiotemporal resolution. In this manuscript, we aim to provide readers with an overview of the field of LSFM on ex vivo samples. Recent advances in LSFM architectures have made the technique widely accessible and have improved its acquisition speed and resolution, among other features. These developments are strongly supported by quantitative analysis of the huge image volumes produced thanks to the boost in computational capacities, the advent of Deep Learning techniques, and by the combination of LSFM with other imaging modalities. Namely, LSFM allows for the characterization of biological structures, disease manifestations and drug effectivity studies. This information can ultimately serve to develop novel diagnostic procedures, treatments and even to model the organs physiology in healthy and pathological conditions.

Full spectrum fluorescence lifetime imaging with 0.5 nm spectral and 50 ps temporal resolution

The use of optical techniques to interrogate wide ranging samples from semiconductors to biological tissue for rapid analysis and diagnostics has gained wide adoption over the past decades. The desire to collect ever more spatially, spectrally and temporally detailed optical signatures for sample characterization has specifically driven a sharp rise in new optical microscopy technologies. Here we present a high-speed optical scanning microscope capable of capturing time resolved images across 512 spectral and 32 time channels in a single acquisition with the potential for ~0.2 frames per second (256 × 256 image pixels). Each pixel in the resulting images contains a detailed data cube for the study of diverse time resolved light driven phenomena. This is enabled by integration of system control electronics and on-chip processing which overcomes the challenges presented by high data volume and low imaging speed, often bottlenecks in previous systems.

High data volumes from multidimensional imaging techniques can lead to slow collection and processing times. Here, the authors implement multispectral fluorescence lifetime imaging microscopy (FLIM) that uses time-correlated photon counting technology to reach simultaneously high imaging rates combined with high spectral and temporal resolution.

Snapshot multicolor fluorescence imaging using double multiplexing of excitation and emission on a single detector

Fluorescence-based multispectral imaging of rapidly moving or dynamic samples requires both fast two-dimensional data acquisition as well as sufficient spectral sensitivity for species separation. As the number of fluorophores in the experiment increases, meeting both these requirements becomes technically challenging. Although several solutions for fast imaging of multiple fluorophores exist, they all have one main restriction; they rely solely on spectrally resolving either the excitation- or the emission characteristics of the fluorophores. This inability directly limits how many fluorophores existing methods can simultaneously distinguish. Here we present a snapshot multispectral imaging approach that not only senses the excitation and emission characteristics of the probed fluorophores but also all cross term combinations of excitation and emission. To the best of the authors’ knowledge, this is the only snapshot multispectral imaging method that has this ability, allowing us to even sense and differentiate between light of equal wavelengths emitted from the same fluorescing species but where the signal components stem from different excitation sources. The current implementation of the technique allows us to simultaneously gather 24 different spectral images on a single detector, from which we demonstrate the ability to visualize and distinguish up to nine fluorophores within the visible wavelength range.

Out-of-Phase Imaging after Optical Modulation (OPIOM) for Multiplexed Fluorescence Imaging Under Adverse Optical Conditions

Fluorescence imaging has become a powerful tool for observations in biology. Yet it has also encountered limitations to overcome optical interferences of ambient light, autofluorescence, and spectrally interfering fluorophores. In this account, we first examine the current approaches which address these limitations. Then we more specifically report on Out-of-Phase Imaging after Optical Modulation (OPIOM), which has proved attractive for highly selective multiplexed fluorescence imaging even under adverse optical conditions. After exposing the OPIOM principle, we detail the protocols for successful OPIOM implementation.

Phasor-based hyperspectral snapshot microscopy allows fast imaging of live, three-dimensional tissues for biomedical applications

Hyperspectral imaging is highly sought after in many fields including mineralogy and geology, environment and agriculture, astronomy and, importantly, biomedical imaging and biological fluorescence. We developed ultrafast phasor-based hyperspectral snapshot microscopy based on sine/cosine interference filters for biomedical imaging not feasible with conventional hyperspectral detection methods. Current approaches rely on slow spatial or spectral scanning limiting their application in living biological tissues, while faster snapshot methods such as image mapping spectrometry and multispectral interferometry are limited in spatial and/or spectral resolution, are computationally demanding, and imaging devices are very expensive to manufacture. Leveraging light sheet microscopy, phasor-based hyperspectral snapshot microscopy improved imaging speed 10-100 fold which, combined with minimal light exposure and high detection efficiency, enabled hyperspectral metabolic imaging of live, three-dimensional mouse tissues not feasible with other methods. As a fit-free method that does not require any a priori information often unavailable in complex and evolving biological systems, the rule of linear combinations of the phasor could spectrally resolve subtle differences between cell types in the developing zebrafish retina and spectrally separate and track multiple organelles in 3D cultured cells over time. The sine/cosine snapshot method is adaptable to any microscope or imaging device thus making hyperspectral imaging and fit-free analysis based on linear combinations broadly available to researchers and the public.

Hyperspectral Three-Dimensional Fluorescence Imaging Using Snapshot Optical Tomography

Hyperspectral three-dimensional (3D) imaging can provide both 3D structural and functional information of a specimen. The imaging throughput is typically very low due to the requirement of scanning mechanisms for different depths and wavelengths. Here we demonstrate hyperspectral 3D imaging using Snapshot projection optical tomography (SPOT) and Fourier-transform spectroscopy (FTS). SPOT allows us to instantaneously acquire the projection images corresponding to different viewing angles, while FTS allows us to perform hyperspectral imaging at high spectral resolution. Using fluorescent beads and sunflower pollens, we demonstrate the imaging performance of the developed system.

IDCube Lite: Free Interactive Discovery Cube software for multi- and hyperspectral applications

Multi- and hyperspectral imaging modalities encompass a growing number of spectral techniques that find many applications in geospatial, biomedical, machine vision and other fields. The rapidly increasing number of applications requires convenient easy-to-navigate software that can be used by new and experienced users to analyse data, and develop, apply and deploy novel algorithms. Herein, we present our platform, IDCube Lite, an Interactive Discovery Cube that performs essential operations in hyperspectral data analysis to realise the full potential of spectral imaging. The strength of the software lies in its interactive features that enable the users to optimise parameters and obtain visual input for the user in a way not previously accessible with other software packages. The entire software can be operated without any prior programming skills allowing interactive sessions of raw and processed data. IDCube Lite, a free version of the software described in the paper, has many benefits compared to existing packages and offers structural flexibility to discover new, hidden features that allow users to integrate novel computational methods.

Fluorophore unmixing based on bleaching and recovery kinetics using MCR-ALS

Fluorescence microscopy is a key technology in the life sciences, though its performance is constrained by the number of labels that can be recorded. We propose to use the kinetics of fluorophore photodestruction and subsequent fluorescence recovery to distinguish multiple spectrally-overlapping emitters in fixed cells, thus enhancing the information that can be obtained from a single measurement. We show that the data can be directly processed using multivariate curve resolution - alternating least squares (MCR-ALS) to deliver distinct images for each fluorophore in their local environment, and apply this methodology to membrane imaging using DiBAC₄(3) and concanavalin A - Alexa Fluor 488 as the fluorophores. We find that the DiBAC₄(3) displays two distinct degradation/recovery kinetics that correspond to two different label distributions, allowing us to simultaneously distinguish three different fluorescence distributions from two spectrally overlapping fluorophores. We expect that our approach will scale to other dynamically-binding dyes, leading to similarly increased multiplexing capability.

A versatile deep learning architecture for classification and label-free prediction of hyperspectral images

Hyperspectral imaging is a technique that provides rich chemical or compositional information not regularly available to traditional imaging modalities such as intensity imaging or color imaging based on the reflection, transmission, or emission of light. Analysis of hyperspectral imaging often relies on machine learning methods to extract information. Here, we present a new flexible architecture, the U-within-U-Net, that can perform classification, segmentation, and prediction of orthogonal imaging modalities on a variety of hyperspectral imaging techniques. Specifically, we demonstrate feature segmentation and classification on the Indian Pines hyperspectral dataset and simultaneous location prediction of multiple drugs in mass spectrometry imaging of rat liver tissue. We further demonstrate label-free fluorescence image prediction from hyperspectral stimulated Raman scattering microscopy images. The applicability of the U-within-U-Net architecture on diverse datasets with widely varying input and output dimensions and data sources suggest that it has great potential in advancing the use of hyperspectral imaging across many different application areas ranging from remote sensing, to medical imaging, to microscopy.

DMD-based hyperspectral microscopy with flexible multiline parallel scanning

As one of the most common hyperspectral microscopy (HSM) techniques, line-scanning HSM is currently utilized in many fields. However, its scanning efficiency is still considered to be inadequate since many biological and chemical processes occur too rapidly to be captured. Accordingly, in this work, a digital micromirror device (DMD) based on microelectromechanical systems (MEMS) is utilized to demonstrate a flexible multiline scanning HSM system. To the best of our knowledge, this is the first line-scanning HSM system in which the number of scanning lines N can be tuned by simply changing the DMD's parallel scanning units according to diverse applications. This brilliant strategy of effortless adjustability relies only on on-chip scanning methods and totally exploits the benefits of parallelization, aiming to achieve nearly an N-time improvement in the detection efficiency and an N-time decrease in the scanning time and data volume compared with the single-line method under the same operating conditions. To validate this, we selected a few samples of different spectral wavebands to perform reflection imaging, transmission imaging, and fluorescence imaging with varying numbers of scanning lines. The results show the great potential of our DMD-based HSM system for the rapid development of cellular biology, material analysis, and so on. In addition, its on-chip scanning process eliminates the inherent microscopic architecture, making the whole system compact, lightweight, portable, and not subject to site constraints.

High compression deep learning based single-pixel hyperspectral macroscopic fluorescence lifetime imaging in vivo

Single pixel imaging frameworks facilitate the acquisition of high-dimensional optical data in biological applications with photon starved conditions. However, they are still limited to slow acquisition times and low pixel resolution. Herein, we propose a convolutional neural network for fluorescence lifetime imaging with compressed sensing at high compression (NetFLICS-CR), which enables in vivo applications at enhanced resolution, acquisition and processing speeds, without the need for experimental training datasets. NetFLICS-CR produces intensity and lifetime reconstructions at 128 × 128 pixel resolution over 16 spectral channels while using only up to 1% of the required measurements, therefore reducing acquisition times from ∼2.5 hours at 50% compression to ∼3 minutes at 99% compression. Its potential is demonstrated in silico, in vitro and for mice in vivo through the monitoring of receptor-ligand interactions in liver and bladder and further imaging of intracellular delivery of the clinical drug Trastuzumab to HER2-positive breast tumor xenografts. The data acquisition time and resolution improvement through NetFLICS-CR, facilitate the translation of single pixel macroscopic flurorescence lifetime imaging (SP-MFLI) for in vivo monitoring of lifetime properties and drug uptake.

Route to Intelligent Imaging Reconstruction via Terahertz Nonlinear Ghost Imaging

Terahertz (THz) imaging is a rapidly emerging field, thanks to many potential applications in diagnostics, manufacturing, medicine and material characterisation. However, the relatively coarse resolution stemming from the large wavelength limits the deployment of THz imaging in micro- and nano-technologies, keeping its potential benefits out-of-reach in many practical scenarios and devices. In this context, single-pixel techniques are a promising alternative to imaging arrays, in particular when targeting subwavelength resolutions. In this work, we discuss the key advantages and practical challenges in the implementation of time-resolved nonlinear ghost imaging (TIMING), an imaging technique combining nonlinear THz generation with time-resolved time-domain spectroscopy detection. We numerically demonstrate the high-resolution reconstruction of semi-transparent samples, and we show how the Walsh-Hadamard reconstruction scheme can be optimised to significantly reduce the reconstruction time. We also discuss how, in sharp contrast with traditional intensity-based ghost imaging, the field detection at the heart of TIMING enables high-fidelity image reconstruction via low numerical-aperture detection. Even more striking-and to the best of our knowledge, an issue never tackled before-the general concept of "resolution" of the imaging system as the "smallest feature discernible" appears to be not well suited to describing the fidelity limits of nonlinear ghost-imaging systems. Our results suggest that the drop in reconstruction accuracy stemming from non-ideal detection conditions is complex and not driven by the attenuation of high-frequency spatial components (i.e., blurring) as in standard imaging. On the technological side, we further show how achieving efficient optical-to-terahertz conversion in extremely short propagation lengths is crucial regarding imaging performance, and we propose low-bandgap semiconductors as a practical framework to obtain THz emission from quasi-2D structures, i.e., structure in which the interaction occurs on a deeply subwavelength scale. Our results establish a comprehensive theoretical and experimental framework for the development of a new generation of terahertz hyperspectral imaging devices.

Light-Sheet Microscopy for Surface Topography Measurements and Quantitative Analysis

A novel light-sheet microscopy (LSM) system that uses the laser triangulation method to quantitatively calculate and analyze the surface topography of opaque samples is discussed. A spatial resolution of at least 10 μm in z-direction, 10 μm in x-direction and 25 μm in y-direction with a large field-of-view (FOV) is achieved. A set of sample measurements that verify the system′s functionality in various applications are presented. The system has a simple mechanical structure, such that the spatial resolution is easily improved by replacement of the objective, and a linear calibration formula, which enables convenient system calibration. As implemented, the system has strong potential for, e.g., industrial sample line inspections, however, since the method utilizes reflected/scattered light, it also has the potential for three-dimensional analysis of translucent and layered structures.

A versatile, multi-laser twin-microscope system for light-sheet imaging

Light-sheet microscopy offers faster imaging and reduced phototoxicity in comparison to conventional point-scanning microscopy, making it a preferred technique for imaging biological dynamics for durations of hours or days. Such extended imaging sessions pose a challenge, as it reduces the number of specimens that can be imaged in a given day. Here, we present a versatile light-sheet imaging instrument that combines two independently controlled microscope-twins, built so that they can share an ultrafast near-infrared laser and a bank of continuous-wave visible lasers, increasing the throughput and decreasing the cost. To permit a wide variety of specimens to be imaged, each microscope-twin provides flexible imaging parameters, including (i) operation in one-photon and/or two-photon excitation modes, (ii) delivery of one to three light-sheets via a trio of orthogonal excitation arms, (iii) sub-micron to micron imaging resolution, (iv) multicolor compatibility, and (v) upright (with provision for inverted) detection geometry. We offer a detailed description of the twin-microscope design to aid instrument builders who wish to construct and use similar systems. We demonstrate the instrument's versatility for biological investigation by performing fast imaging of the beating heart in an intact zebrafish embryo, deep imaging of thick patient-derived tumor organoids, and gentle whole-brain imaging of neural activity in behaving larval zebrafish.

A background correction method to compensate illumination variation in hyperspectral imaging

Hyperspectral imaging (HSI) can measure both spatial (morphological) and spectral (biochemical) information from biological tissues. While HSI appears promising for biomedical applications, interpretation of hyperspectral images can be challenging when data is acquired in complex biological environments. Variations in surface topology or optical power distribution at the sample, encountered for example during endoscopy, can lead to errors in post-processing of the HSI data, compromising disease diagnostic capabilities. Here, we propose a background correction method to compensate for such variations, which estimates the optical properties of illumination at the target based on the normalised spectral profile of the light source and the measured HSI intensity values at a fixed wavelength where the absorption characteristics of the sample are relatively low (in this case, 800 nm). We demonstrate the feasibility of the proposed method by imaging blood samples, tissue-mimicking phantoms, and ex vivo chicken tissue. Moreover, using synthetic HSI data composed from experimentally measured spectra, we show the proposed method would improve statistical analysis of HSI data. The proposed method could help the implementation of HSI techniques in practical clinical applications, where controlling the illumination pattern and power is difficult.

Dynamic contrast with reversibly photoswitchable fluorescent labels for imaging living cells

Interrogating living cells requires sensitive imaging of a large number of components in real time. The state-of-the-art of multiplexed imaging is usually limited to a few components. This review reports on the promise and the challenges of dynamic contrast to overcome this limitation.

Pre-processing visualization of hyperspectral fluorescent data with Spectrally Encoded Enhanced Representations

Hyperspectral fluorescence imaging is gaining popularity for it enables multiplexing of spatio-temporal dynamics across scales for molecules, cells and tissues with multiple fluorescent labels. This is made possible by adding the dimension of wavelength to the dataset. The resulting datasets are high in information density and often require lengthy analyses to separate the overlapping fluorescent spectra. Understanding and visualizing these large multi-dimensional datasets during acquisition and pre-processing can be challenging. Here we present Spectrally Encoded Enhanced Representations (SEER), an approach for improved and computationally efficient simultaneous color visualization of multiple spectral components of hyperspectral fluorescence images. Exploiting the mathematical properties of the phasor method, we transform the wavelength space into information-rich color maps for RGB display visualization. We present multiple biological fluorescent samples and highlight SEER’s enhancement of specific and subtle spectral differences, providing a fast, intuitive and mathematical way to interpret hyperspectral images during collection, pre-processing and analysis.

Spectral phasor analysis allows unmixing fluorescence microscopy images, but it requires user involvement and has a limited number of labels that can be analyzed and displayed. Here the authors present a semi-automated solution to visualise multiple spectral components of hyperspectral fluorescence images, simultaneously.

Snapshot hyperspectral light field imaging using image mapping spectrometry

In this Letter, we present a snapshot hyperspectral light field imaging system using a single camera. By integrating an unfocused light field camera with a snapshot hyperspectral imager, the image mapping spectrometer, we captured a five-dimensional (5D) ($x,y,u,v,\lambda $x,y,u,v,λ) ($x,y,$x,y, spatial coordinates; $u,v,$u,v, emittance angles; $\lambda ,$λ, wavelength) datacube in a single camera exposure. The corresponding volumetric image ($x,y,z$x,y,z) at each wavelength is then computed through a scale-depth space transform. We demonstrated the snapshot advantage of our system by imaging the spectral-volumetric scenes in real time.

Optical imaging spectroscopy for plant research: more than a colorful picture

Optical imaging is a routine and indispensable tool in plant research. Here we review different emerging spectrally resolved optical imaging approaches and the wealth of information they can be used to obtain pertaining to the underlying chemistry, structure and mechanics of plants.

Imaging, Visualization, and Computation in Developmental Biology

Embryonic development is highly complex and dynamic, requiring the coordination of numerous molecular and cellular events at precise times and places. Advances in imaging technology have made it possible to follow developmental processes at cellular, tissue, and organ levels over time as they take place in the intact embryo. Parallel innovations of in vivo probes permit imaging to report on molecular, physiological, and anatomical events of embryogenesis, but the resulting multidimensional data sets pose significant challenges for extracting knowledge. In this review, we discuss recent and emerging advances in imaging technologies, in vivo labeling, and data processing that offer the greatest potential for jointly deciphering the intricate cellular dynamics and the underlying molecular mechanisms. Our discussion of the emerging area of “image-omics” highlights both the challenges of data analysis and the promise of more fully embracing computation and data science for rapidly advancing our understanding of biology.

DMD-based hyperspectral imaging system with tunable spatial and spectral resolution

Pushbroom hyperspectral imaging (HSI) has been used in many areas from air to land. However, its inherent operational drawback of the bulky slit leads to a limited field of view (FOV) and high energy consumption. Accordingly, a new and versatile HSI system is proposed by employing a smart digital micromirror device (DMD) to replace the mechanical scanning component. Moreover, tunable spatial and spectral resolution is implemented through adjusting the on-chip scanning linewidth and adopting the pixel fusion method, respectively. Meanwhile, three scanning modes including rough scanning, fine scanning, and regional scanning are achieved. These multiple choices increase the system's flexibility, universality, and intelligence, which is attractive for practically different applications, especially for military and remote sensing fields in need of a large FOV, and medical and food fields in need of tunable resolution for various samples.

A clinically translatable hyperspectral endoscopy (HySE) system for imaging the gastrointestinal tract

Hyperspectral imaging (HSI) enables visualisation of morphological and biochemical information, which could improve disease diagnostic accuracy. Unfortunately, the wide range of image distortions that arise during flexible endoscopy in the clinic have made integration of HSI challenging. To address this challenge, we demonstrate a hyperspectral endoscope (HySE) that simultaneously records intrinsically co-registered hyperspectral and standard-of-care white light images, which allows image distortions to be compensated computationally and an accurate hyperspectral data cube to be reconstructed as the endoscope moves in the lumen. Evaluation of HySE performance shows excellent spatial, spectral and temporal resolution and high colour fidelity. Application of HySE enables: quantification of blood oxygenation levels in tissue mimicking phantoms; differentiation of spectral profiles from normal and pathological ex vivo human tissues; and recording of hyperspectral data under freehand motion within an intact ex vivo pig oesophagus model. HySE therefore shows potential for enabling HSI in clinical endoscopy.

Hyperspectral scanning laser optical tomography

In order to study physical relationships within tissue volumes or even organism-level systems, the spatial distribution of multiple fluorescent markers needs to be resolved efficiently in three dimensions. Here, rather than acquiring discrete spectral images sequentially using multiple emission filters, a hyperspectral scanning laser optical tomography system is developed to obtain hyperspectral volumetric data sets with 2-nm spectral resolution of optically transparent mesoscopic (millimeter-centimeter) specimens. This is achieved by acquiring a series of point-scanning hyperspectral extended depth of field images at different angles and subsequently tomographically reconstructing the 3D intensity distribution for each wavelength. This technique is demonstrated to provide robust measurements via the comparison of spectral and intensity profiles of fluorescent bead phantoms. Due to its enhanced spectral resolving ability, this technique is also demonstrated to resolve largely overlapping fluorophores, as demonstrated by the 3D fluorescence hyperspectral reconstruction of a dual-labeled mouse thymus gland sample and the ability to distinguish tumorous and normal tissues of an unlabeled mouse intestine sample.

Novel automated non invasive detection of ocular surface squamous neoplasia using multispectral autofluorescence imaging

PURPOSE

Diagnosing Ocular surface squamous neoplasia (OSSN) using newly designed multispectral imaging technique.

METHODS

Eighteen patients with histopathological diagnosis of Ocular Surface Squamous Neoplasia (OSSN) were recruited. Their previously collected biopsy specimens of OSSN were reprocessed without staining to obtain auto fluorescence multispectral microscopy images. This technique involved a custom-built spectral imaging system with 38 spectral channels. Inter and intra-patient frameworks were deployed to automatically detect and delineate OSSN using machine learning methods. Different machine learning methods were evaluated, with K nearest neighbor and Support Vector Machine chosen as preferred classifiers for intra- and inter-patient frameworks, respectively. The performance of the technique was evaluated against a pathological assessment.

RESULTS

Quantitative analysis of the spectral images provided a strong multispectral signature of a relative difference between neoplastic and normal tissue both within each patient (at p < 0.0005) and between patients (at p < 0.001). Our fully automated diagnostic method based on machine learning produces maps of the relatively well circumscribed neoplastic-non neoplastic interface. Such maps can be rapidly generated in quasi-real time and used for intraoperative assessment. Generally, OSSN could be detected using multispectral analysis in all patients investigated here. The cancer margins detected by multispectral analysis were in close and reasonable agreement with the margins observed in the H&E sections in intra- and inter-patient classification, respectively.

CONCLUSIONS

This study shows the feasibility of using multispectral auto-fluorescence imaging to detect and find the boundary of human OSSN. Fully automated analysis of multispectral images based on machine learning methods provides a promising diagnostic tool for OSSN which can be translated to future clinical applications.

Imaging Flies by Fluorescence Microscopy: Principles, Technologies, and Applications

The development of fluorescent labels and powerful imaging technologies in the last two decades has revolutionized the field of fluorescence microscopy, which is now widely used in diverse scientific fields from biology to biomedical and materials science. Fluorescence microscopy has also become a standard technique in research laboratories working on Drosophila melanogaster as a model organism. Here, we review the principles of fluorescence microscopy technologies from wide-field to Super-resolution microscopy and its application in the Drosophila research field.

Multicolor fluorescent imaging by space-constrained computational spectral imaging

Spectral imaging is a powerful technique used to simultaneously study multiple fluorophore labels with overlapping emissions. Here, we present a computational spectral imaging method, which uses sample spatial fluorescence information as a reconstruction constraint. Our method addresses both the under-sampling issue of compressive spectral imaging and the low throughput issue of scanning spectral imaging. With simulated and experimental data, we have demonstrated the reconstruction precision of our method in two and three-color imaging. We have experimentally validated this method for differentiating cellular structures labeled with two red-colored fluorescent proteins, tdTomato and mCherry, which have highly overlapping emission spectra. Our method has the advantage of totally free wavelength choice and can also be combined with conventional filter-based sequential multi-color imaging to further improve multiplexing capability.

Multicolor fluorescence microscopy using static light sheets and a single-channel detection

We present a multicolor fluorescence microscope system, under a selective plane illumination microscopy (SPIM) configuration, using three continuous wave-lasers and a single-channel-detection camera. The laser intensities are modulated with three time-delayed pulse trains that operate synchronously at one third of the camera frame rate, allowing a sequential excitation and an image acquisition of up to three different biomarkers. The feasibility of this imaging acquisition mode is demonstrated by acquiring single-plane multicolor images of living hyphae of Neurospora crassa. This allows visualizing simultaneously the localization and dynamics of different cellular components involved in apical growth in living hyphae. The configuration presented represents a noncommercial, cost-effective alternative microscopy system for the rapid and simultaneous acquisition of multifluorescent images and can be potentially useful for three-dimensional imaging of large biological samples.

High-Throughput Assessment of Mechanistic Toxicity of Chemicals in Miniaturized 3D Cell Culture

High-content imaging (HCI) assays on two-dimensional (2D) cell cultures often do not represent in vivo characteristics accurately, thus reducing the predictability of drug toxicity/efficacy in vivo. On the other hand, conventional 3D cell cultures are relatively low throughput and possess difficulty in cell imaging. To address these limitations, a miniaturized 3D cell culture has been developed on a micropillar/microwell chip platform with human cells encapsulated in biomimetic hydrogels. Model compounds are used to validate human cell microarrays for high-throughput assessment of mechanistic toxicity. Main mechanisms of toxicity of compounds can be investigated by analyzing multiple parameters such as DNA damage, mitochondrial impairment, intracellular glutathione level, and cell membrane integrity. IC₅₀ values of these parameters can be determined and compared for the compounds to investigate the main mechanism of toxicity. This paper describes miniaturized HCI assays on 3D-cultured cell microarrays for high-throughput assessment of mechanistic profiles of compound-induced toxicity. © 2018 by John Wiley & Sons, Inc.