Multi-view image fusion improves resolution in three-dimensional microscopy

Expansion and Light-Sheet Microscopy for Nanoscale 3D Imaging

Expansion Microscopy (ExM) and Light-Sheet Fluorescence Microscopy (LSFM) are forefront imaging techniques that enable high-resolution visualization of biological specimens. ExM enhances nanoscale investigation using conventional fluorescence microscopes, while LSFM offers rapid, minimally invasive imaging over large volumes. This review explores the joint advancements of ExM and LSFM, focusing on the excellent performance of the integrated modality obtained from the combination of the two, which is refer to as ExLSFM. In doing so, the chemical processes required for ExM, the tailored optical setup of LSFM for examining expanded samples, and the adjustments in sample preparation for accurate data collection are emphasized. It is delve into various specimen types studied using this integrated method and assess its potential for future applications. The goal of this literature review is to enrich the comprehension of ExM and LSFM, encouraging their wider use and ongoing development, looking forward to the upcoming challenges, and anticipating innovations in these imaging techniques.

Tetrahedral serial multiview microscopy and image fusion for improved resolution and extent in stained zebrafish embryos

BACKGROUND

Spatial mapping on the single-cell level over the whole organism can uncover roles of molecular players involved in vertebrate development. Custom microscopes have been developed that use multiple objectives to view a sample from multiple views at the same time. Such multiview imaging approaches can improve resolution and uniformity of image quality as well as allow whole embryos to be imaged (Swoger et al., Opt Express, 2007;15(13):8029). However, multiview imaging is highly restricted to specialized equipment requiring multiple objectives or sample rotation with automated hardware.

RESULTS

Our approach uses a standard single-objective confocal microscope to perform serial multiview imaging. Multiple views are imaged sequentially by mounting the fixed sample in an agarose tetrahedron that is manually rotated in between imaging each face. Computational image fusion allows for a joint 3D image to be created from multiple tiled Z-stacks acquired from different angles. The resulting fused image has improved resolution and imaging extent.

CONCLUSION

With this technique, multiview imaging can be performed on a variety of common single-objective microscopes to allow for whole-embryo, high-resolution imaging.

Photoacoustic Tomography with Temporal Encoding Reconstruction (PATTERN) for cross-modal individual analysis of the whole brain

Cross-modal analysis of the same whole brain is an ideal strategy to uncover brain function and dysfunction. However, it remains challenging due to the slow speed and destructiveness of traditional whole-brain optical imaging techniques. Here we develop a new platform, termed Photoacoustic Tomography with Temporal Encoding Reconstruction (PATTERN), for non-destructive, high-speed, 3D imaging of ex vivo rodent, ferret, and non-human primate brains. Using an optimally designed image acquisition scheme and an accompanying machine-learning algorithm, PATTERN extracts signals of genetically-encoded probes from photobleaching-based temporal modulation and enables reliable visualization of neural projection in the whole central nervous system with 3D isotropic resolution. Without structural and biological perturbation to the sample, PATTERN can be combined with other whole-brain imaging modalities to acquire the whole-brain image with both high resolution and morphological fidelity. Furthermore, cross-modal transcriptome analysis of an individual brain is achieved by PATTERN imaging. Together, PATTERN provides a compatible and versatile strategy for brain-wide cross-modal analysis at the individual level.

Open-top multisample dual-view light-sheet microscope for live imaging of large multicellular systems

Multicellular systems grow over the course of weeks from single cells to tissues or even full organisms, making live imaging challenging. To bridge spatiotemporal scales, we present an open-top dual-view and dual-illumination light-sheet microscope dedicated to live imaging of large specimens at single-cell resolution. The configuration of objectives together with a customizable multiwell mounting system combines dual view with high-throughput multiposition imaging. We use this microscope to image a wide variety of samples and highlight its capabilities to gain quantitative single-cell information in large specimens such as mature intestinal organoids and gastruloids.

Whole-brain Optical Imaging: A Powerful Tool for Precise Brain Mapping at the Mesoscopic Level

The mammalian brain is a highly complex network that consists of millions to billions of densely-interconnected neurons. Precise dissection of neural circuits at the mesoscopic level can provide important structural information for understanding the brain. Optical approaches can achieve submicron lateral resolution and achieve "optical sectioning" by a variety of means, which has the natural advantage of allowing the observation of neural circuits at the mesoscopic level. Automated whole-brain optical imaging methods based on tissue clearing or histological sectioning surpass the limitation of optical imaging depth in biological tissues and can provide delicate structural information in a large volume of tissues. Combined with various fluorescent labeling techniques, whole-brain optical imaging methods have shown great potential in the brain-wide quantitative profiling of cells, circuits, and blood vessels. In this review, we summarize the principles and implementations of various whole-brain optical imaging methods and provide some concepts regarding their future development.

Blackberry Fruit Classification in Underexposed Images Combining Deep Learning and Image Fusion Methods

Berry production is increasing worldwide each year; however, high production leads to labor shortages and an increase in wasted fruit during harvest seasons. This problem opened new research opportunities in computer vision as one main challenge to address is the uncontrolled light conditions in greenhouses and open fields. The high light variations between zones can lead to underexposure of the regions of interest, making it difficult to classify between vegetation, ripe, and unripe blackberries due to their black color. Therefore, the aim of this work is to automate the process of classifying the ripeness stages of blackberries in normal and low-light conditions by exploring the use of image fusion methods to improve the quality of the input image before the inference process. The proposed algorithm adds information from three sources: visible, an improved version of the visible, and a sensor that captures images in the near-infrared spectra, obtaining a mean F1 score of 0.909±0.074 and 0.962±0.028 in underexposed images, without and with model fine-tuning, respectively, which in some cases is an increase of up to 12% in the classification rates. Furthermore, the analysis of the fusion metrics showed that the method could be used in outdoor images to enhance their quality; the weighted fusion helps to improve only underexposed vegetation, improving the contrast of objects in the image without significant changes in saturation and colorfulness.

Multifocal optical projection microscopy enables label-free 3D measurement of cardiomyocyte cluster contractility

Human induced pluripotent stem cell (hiPSC)-derived cardiomyocyte (CM) models have become an attractive tool for in vitro cardiac disease modeling and drug studies. These models are moving towards more complex three-dimensional microphysiological organ-on-chip systems. Label-free imaging-based techniques capable of quantifying contractility in 3D are needed, as traditional two-dimensional methods are ill-suited for 3D applications. Here, we developed multifocal (MF) optical projection microscopy (OPM) by integrating an electrically tunable lens to our in-house built optical projection tomography setup for extended depth of field brightfield imaging in CM clusters. We quantified cluster biomechanics by implementing our previously developed optical flow-based CM video analysis for MF-OPM. To demonstrate, we acquired and analyzed multiangle and multifocal projection videos of beating hiPSC-CM clusters in 3D hydrogel. We further quantified cluster contractility response to temperature and adrenaline and observed changes to beating rate and relaxation. Challenges emerge from light penetration and overlaying textures in larger clusters. However, our findings indicate that MF-OPM is suitable for contractility studies of 3D clusters. Thus, for the first time, MF-OPM is used in CM studies and hiPSC-CM 3D cluster contraction is quantified in multiple orientations and imaging planes.

Light-sheets and smart microscopy, an exciting future is dawning

Light-sheet fluorescence microscopy has transformed our ability to visualize and quantitatively measure biological processes rapidly and over long time periods. In this review, we discuss current and future developments in light-sheet fluorescence microscopy that we expect to further expand its capabilities. This includes smart and adaptive imaging schemes to overcome traditional imaging trade-offs, i.e., spatiotemporal resolution, field of view and sample health. In smart microscopy, a microscope will autonomously decide where, when, what and how to image. We further assess how image restoration techniques provide avenues to overcome these tradeoffs and how "open top" light-sheet microscopes may enable multi-modal imaging with high throughput. As such, we predict that light-sheet microscopy will fulfill an important role in biomedical and clinical imaging in the future.

Active mesh and neural network pipeline for cell aggregate segmentation

Segmenting cells within cellular aggregates in 3D is a growing challenge in cell biology due to improvements in capacity and accuracy of microscopy techniques. Here, we describe a pipeline to segment images of cell aggregates in 3D. The pipeline combines neural network segmentations with active meshes. We apply our segmentation method to cultured mouse mammary gland organoids imaged over 24 h with oblique plane microscopy, a high-throughput light-sheet fluorescence microscopy technique. We show that our method can also be applied to images of mouse embryonic stem cells imaged with a spinning disc microscope. We segment individual cells based on nuclei and cell membrane fluorescent markers, and track cells over time. We describe metrics to quantify the quality of the automated segmentation. Our segmentation pipeline involves a Fiji plugin that implements active mesh deformation and allows a user to create training data, automatically obtain segmentation meshes from original image data or neural network prediction, and manually curate segmentation data to identify and correct mistakes. Our active meshes-based approach facilitates segmentation postprocessing, correction, and integration with neural network prediction.

Light sheet imaging and interactive analysis of the cardiac structure in neonatal mice

Light-sheet microscopy (LSM) enables us to strengthen the understanding of cardiac development, injury, and regeneration in mammalian models. This emerging technique decouples laser illumination and fluorescence detection to investigate cardiac micro-structure and function with a high spatial resolution while minimizing photodamage and maximizing penetration depth. To unravel the potential of volumetric imaging in cardiac development and repair, we sought to integrate our in-house LSM, Adipo-Clear, and virtual reality (VR) with neonatal mouse hearts. We demonstrate the use of Adipo-Clear to render mouse hearts transparent, the development of our in-house LSM to capture the myocardial architecture within the intact heart, and the integration of VR to explore, measure, and assess regions of interests in an interactive manner. Collectively, we have established an innovative and holistic strategy for image acquisition and interpretation, providing an entry point to assess myocardial micro-architecture throughout the entire mammalian heart for the understanding of cardiac morphogenesis.

Time-lapse and cleared imaging of mouse embryonic lung explants to study three-dimensional cell morphology and topology dynamics

Here, we present a protocol for collecting high-spatiotemporal-resolution datasets of undisturbed mouse embryonic epithelial rudiments using light-sheet fluorescence microscopy. We describe steps for rudiment dissection, clearing, and embedding for cleared and live imaging. We then detail procedures for light-sheet imaging followed by image processing and morphometric analysis. We provide protocol variations for imaging both growing and optically cleared lung explants to encourage the quantitative exploration of three-dimensional cell shapes, cell organization, and complex cell-cell dynamics. For complete details on the use and execution of this protocol, please refer to Gómez et al. (2021).¹.

Optothermal rotation of micro-/nano-objects

Due to its contactless and fuel-free operation, optical rotation of micro-/nano-objects provides tremendous opportunities for cellular biology, three-dimensional (3D) imaging, and micro/nanorobotics. However, complex optics, extremely high operational power, and the applicability to limited objects restrict the broader use of optical rotation techniques. This Feature Article focuses on a rapidly emerging class of optical rotation techniques, termed optothermal rotation. Based on light-mediated thermal phenomena, optothermal rotation techniques overcome the bottlenecks of conventional optical rotation by enabling versatile rotary control of arbitrary objects with simpler optics using lower powers. We start with the fundamental thermal phenomena and concepts: thermophoresis, thermoelectricity, thermo-electrokinetics, thermo-osmosis, thermal convection, thermo-capillarity, and photophoresis. Then, we highlight various optothermal rotation techniques, categorizing them based on their rotation modes (i.e., in-plane and out-of-plane rotation) and the thermal phenomena involved. Next, we explore the potential applications of these optothermal manipulation techniques in areas such as single-cell mechanics, 3D bio-imaging, and micro/nanomotors. We conclude the Feature Article with our insights on the operating guidelines, existing challenges, and future directions of optothermal rotation.

Practical considerations for quantitative light sheet fluorescence microscopy

Fluorescence microscopy has evolved from a purely observational tool to a platform for quantitative, hypothesis-driven research. As such, the demand for faster and less phototoxic imaging modalities has spurred a rapid growth in light sheet fluorescence microscopy (LSFM). By restricting the excitation to a thin plane, LSFM reduces the overall light dose to a specimen while simultaneously improving image contrast. However, the defining characteristics of light sheet microscopes subsequently warrant unique considerations in their use for quantitative experiments. In this Perspective, we outline many of the pitfalls in LSFM that can compromise analysis and confound interpretation. Moreover, we offer guidance in addressing these caveats when possible. In doing so, we hope to provide a useful resource for life scientists seeking to adopt LSFM to quantitatively address complex biological hypotheses.

Isotropic imaging across spatial scales with axially swept light-sheet microscopy

Light-sheet fluorescence microscopy is a rapidly growing technique that has gained tremendous popularity in the life sciences owing to its high-spatiotemporal resolution and gentle, non-phototoxic illumination. In this protocol, we provide detailed directions for the assembly and operation of a versatile light-sheet fluorescence microscopy variant, referred to as axially swept light-sheet microscopy (ASLM), that delivers an unparalleled combination of field of view, optical resolution and optical sectioning. To democratize ASLM, we provide an overview of its working principle and applications to biological imaging, as well as pragmatic tips for the assembly, alignment and control of its optical systems. Furthermore, we provide detailed part lists and schematics for several variants of ASLM that together can resolve molecular detail in chemically expanded samples, subcellular organization in living cells or the anatomical composition of chemically cleared intact organisms. We also provide software for instrument control and discuss how users can tune imaging parameters to accommodate diverse sample types. Thus, this protocol will serve not only as a guide for both introductory and advanced users adopting ASLM, but as a useful resource for any individual interested in deploying custom imaging technology. We expect that building an ASLM will take ~1-2 months, depending on the experience of the instrument builder and the version of the instrument.

Axial scanning of dual focus to improve light sheet microscopy

Axially swept light sheet microscopy (ASLM) is an emerging technique that enables isotropic, subcellular resolution imaging with high optical sectioning capability over a large field-of-view (FOV). Due to its versatility across a broad range of immersion media, it has been utilized to image specimens that may range from live cells to intact chemically cleared organs. However, because of its design, the performance of ASLM-based microscopes is impeded by a low detection signal and the maximum achievable frame-rate for full FOV imaging. Here we present a new optical concept that pushes the limits of ASLM further by scanning two staggered light sheets and simultaneously synchronizing the rolling shutter of a scientific camera. For a particular peak-illumination-intensity, this idea can make ASLMs image twice as fast without compromising the detection signal. Alternately, for a particular frame rate our method doubles the detection signal without requiring to double the peak-illumination-power, thereby offering a gentler illumination scheme compared to tradition single-focus ASLM. We demonstrate the performance of our instrument by imaging fluorescent beads and a PEGASOS cleared-tissue mouse brain.

Blind deconvolution in autocorrelation inversion for multiview light-sheet microscopy

Combining the information coming from multiview acquisitions is a problem of great interest in light-sheet microscopy. Aligning the views and increasing the resolution of their fusion can be challenging, especially if the setup is not fully calibrated. Here, we tackle these issues by proposing a new reconstruction method based on autocorrelation inversion that avoids alignment procedures. On top of this, we add a blind deconvolution step to improve the resolution of the final reconstruction. Our method permits us to achieve inherently aligned, highly resolved reconstructions while, at the same time, estimating the unknown point-spread function of the system. RESEARCH HIGHLIGHTS: We tackle the problem of multiview light-sheet deconvolution with a blind approach of autocorrelation inversion Our method recovers the object and PSF, requires no alignment and calibration, and enhances the reconstruction of the specimen.

Navigating the Light-Sheet Image Analysis Software Landscape: Concepts for Driving Cohesion From Data Acquisition to Analysis

From the combined perspective of biologists, microscope instrumentation developers, imaging core facility scientists, and high performance computing experts, we discuss the challenges faced when selecting imaging and analysis tools in the field of light-sheet microscopy. Our goal is to provide a contextual framework of basic computing concepts that cell and developmental biologists can refer to when mapping the peculiarities of different light-sheet data to specific existing computing environments and image analysis pipelines. We provide our perspective on efficient processes for tool selection and review current hardware and software commonly used in light-sheet image analysis, as well as discuss what ideal tools for the future may look like.

Beyond multi view deconvolution for inherently aligned fluorescence tomography

In multi-view fluorescence microscopy, each angular acquisition needs to be aligned with care to obtain an optimal volumetric reconstruction. Here, instead, we propose a neat protocol based on auto-correlation inversion, that leads directly to the formation of inherently aligned tomographies. Our method generates sharp reconstructions, with the same accuracy reachable after sub-pixel alignment but with improved point-spread-function. The procedure can be performed simultaneously with deconvolution further increasing the reconstruction resolution.

Removing striping artifacts in light-sheet fluorescence microscopy: a review

In recent years, light-sheet fluorescence microscopy (LSFM) has found a broad application for imaging of diverse biological samples, ranging from sub-cellular structures to whole animals, both in-vivo and ex-vivo, owing to its many advantages relative to point-scanning methods. By providing the selective illumination of sample single planes, LSFM achieves an intrinsic optical sectioning and direct 2D image acquisition, with low out-of-focus fluorescence background, sample photo-damage and photo-bleaching. On the other hand, such an illumination scheme is prone to light absorption or scattering effects, which lead to uneven illumination and striping artifacts in the images, oriented along the light sheet propagation direction. Several methods have been developed to address this issue, ranging from fully optical solutions to entirely digital post-processing approaches. In this work, we present them, outlining their advantages, performance and limitations.

Evaluation of Multi-Stream Fusion for Multi-View Image Set Comparison

We consider the problem of image set comparison, i.e., to determine whether two image sets show the same unique object (approximately) from the same viewpoints. Our proposition is to solve it by a multi-stream fusion of several image recognition paths. Immediate applications of this method can be found in fraud detection, deduplication procedure, or visual searching. The contribution of this paper is a novel distance measure for similarity of image sets and the experimental evaluation of several streams for the considered problem of same-car image set recognition. To determine a similarity score of image sets (this score expresses the certainty level that both sets represent the same object visible from the same set of views), we adapted a measure commonly applied in blind signal separation (BSS) evaluation. This measure is independent of the number of images in a set and the order of views in it. Separate streams for object classification (where a class represents either a car type or a car model-and-view) and object-to-object similarity evaluation (based on object features obtained alternatively by the convolutional neural network (CNN) or image keypoint descriptors) were designed. A late fusion by a fully-connected neural network (NN) completes the solution. The implementation is of modular structure—for semantic segmentation we use a Mask-RCNN (Mask regions with CNN features) with ResNet 101 as a backbone network; image feature extraction is either based on the DeepRanking neural network or classic keypoint descriptors (e.g., scale-invariant feature transform (SIFT)) and object classification is performed by two Inception V3 deep networks trained for car type-and-view and car model-and-view classification (4 views, 9 car types, and 197 car models are considered). Experiments conducted on the Stanford Cars dataset led to selection of the best system configuration that overperforms a base approach, allowing for a 67.7% GAR (genuine acceptance rate) at 3% FAR (false acceptance rate).

Wide field light-sheet microscopy with lens-axicon controlled two-photon Bessel beam illumination

Two-photon excitation can lower phototoxicity and improve penetration depth, but its narrow excitation range restricts its applications in light-sheet microscopy. Here, we propose simple illumination optics, a lens-axicon triplet composed of an axicon and two convex lenses, to generate longer extent Bessel beams. This unit can stretch the beam full width at half maximum of 600-1000 μm with less than a 4-μm waist when using a 10× illumination lens. A two-photon excitation digital scanned light-sheet microscope possessing this range of field of view and ~2-3-μm axial resolution is constructed and used to analyze the cellular dynamics over the whole body of medaka fish. We demonstrate long-term time-lapse observations over several days and high-speed recording with ~3 mm3 volume per 4 s of the embryos. Our system is minimal and suppresses laser power loss, which can broaden applications of two-photon excitation in light-sheet microscopy.

Multiview deconvolution approximation multiphoton microscopy of tissues and zebrafish larvae

Imaging in three dimensions is necessary for thick tissues and small organisms. This is possible with tomographic optical microscopy techniques such as confocal, multiphoton and light sheet microscopy. All these techniques suffer from anisotropic resolution and limited penetration depth. In the past, Multiview microscopy-imaging the sample from different angles followed by 3D image reconstruction-was developed to address this issue for light sheet microscopy based on fluorescence signal. In this study we applied this methodology to accomplish Multiview imaging with multiphoton microscopy based on fluorescence and additionally second harmonic signal from myosin and collagen. It was shown that isotropic resolution was achieved, the entirety of the sample was visualized, and interference artifacts were suppressed allowing clear visualization of collagen fibrils and myofibrils. This method can be applied to any scanning microscopy technique without microscope modifications. It can be used for imaging tissue and whole mount small organisms such as heart tissue, and zebrafish larva in 3D, label-free or stained, with at least threefold axial resolution improvement which can be significant for the accurate quantification of small 3D structures.

Dual-view light-sheet imaging through a tilted glass interface using a deformable mirror

Light-sheet microscopy has become indispensable for imaging developing organisms, and imaging from multiple directions (views) is essential to improve its spatial resolution. We combine multi-view light-sheet microscopy with microfluidics using adaptive optics (deformable mirror) which corrects aberrations introduced by the 45^o-tilted glass coverslip. The optimal shape of the deformable mirror is computed by an iterative algorithm that optimizes the point-spread function in two orthogonal views. Simultaneous correction in two optical arms is achieved via a knife-edge mirror that splits the excitation path and combines the detection paths. Our design allows multi-view light-sheet microscopy with microfluidic devices for precisely controlled experiments and high-content screening.

Recurrent neural network-based volumetric fluorescence microscopy

Volumetric imaging of samples using fluorescence microscopy plays an important role in various fields including physical, medical and life sciences. Here we report a deep learning-based volumetric image inference framework that uses 2D images that are sparsely captured by a standard wide-field fluorescence microscope at arbitrary axial positions within the sample volume. Through a recurrent convolutional neural network, which we term as Recurrent-MZ, 2D fluorescence information from a few axial planes within the sample is explicitly incorporated to digitally reconstruct the sample volume over an extended depth-of-field. Using experiments on C. elegans and nanobead samples, Recurrent-MZ is demonstrated to significantly increase the depth-of-field of a 63×/1.4NA objective lens, also providing a 30-fold reduction in the number of axial scans required to image the same sample volume. We further illustrated the generalization of this recurrent network for 3D imaging by showing its resilience to varying imaging conditions, including e.g., different sequences of input images, covering various axial permutations and unknown axial positioning errors. We also demonstrated wide-field to confocal cross-modality image transformations using Recurrent-MZ framework and performed 3D image reconstruction of a sample using a few wide-field 2D fluorescence images as input, matching confocal microscopy images of the same sample volume. Recurrent-MZ demonstrates the first application of recurrent neural networks in microscopic image reconstruction and provides a flexible and rapid volumetric imaging framework, overcoming the limitations of current 3D scanning microscopy tools.

FRC-QE: a robust and comparable 3D microscopy image quality metric for cleared organoids

SUMMARY

Here, we propose Fourier ring correlation-based quality estimation (FRC-QE) as a new metric for automated image quality estimation in 3D fluorescence microscopy acquisitions of cleared organoids that yields comparable measurements across experimental replicates, clearing protocols and works for different microscopy modalities.

AVAILABILITY AND IMPLEMENTATION

FRC-QE is written in ImgLib2/Java and provided as an easy-to-use and macro-scriptable plugin for Fiji. Code, documentation, sample images and further information can be found under https://github.com/PreibischLab/FRC-QE.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

Harnessing non-destructive 3D pathology

High-throughput methods for slide-free three-dimensional (3D) pathological analyses of whole biopsies and surgical specimens offer the promise of modernizing traditional histology workflows and delivering improvements in diagnostic performance. Advanced optical methods now enable the interrogation of orders of magnitude more tissue than previously possible, where volumetric imaging allows for enhanced quantitative analyses of cell distributions and tissue structures that are prognostic and predictive. Non-destructive imaging processes can simplify laboratory workflows, potentially reducing costs, and can ensure that samples are available for subsequent molecular assays. However, the large size of the feature-rich datasets that they generate poses challenges for data management and computer-aided analysis. In this Perspective, we provide an overview of the imaging technologies that enable 3D pathology, and the computational tools-machine learning, in particular-for image processing and interpretation. We also discuss the integration of various other diagnostic modalities with 3D pathology, along with the challenges and opportunities for clinical adoption and regulatory approval.

Image Fusion Techniques: A Survey

The necessity of image fusion is growing in recently in image processing applications due to the tremendous amount of acquisition systems. Fusion of images is defined as an alignment of noteworthy Information from diverse sensors using various mathematical models to generate a single compound image. The fusion of images is used for integrating the complementary multi-temporal, multi-view and multi-sensor Information into a single image with improved image quality and by keeping the integrity of important features. It is considered as a vital pre-processing phase for several applications such as robot vision, aerial, satellite imaging, medical imaging, and a robot or vehicle guidance. In this paper, various state-of-art image fusion methods of diverse levels with their pros and cons, various spatial and transform based method with quality metrics and their applications in different domains have been discussed. Finally, this review has concluded various future directions for different applications of image fusion.

Dual-view oblique plane microscopy (dOPM)

We present a new folded dual-view oblique plane microscopy (OPM) technique termed dOPM that enables two orthogonal views of the sample to be obtained by translating a pair of tilted mirrors in refocussing space. Using a water immersion 40× 1.15 NA primary objective, deconvolved image volumes of 200 nm beads were measured to have full width at half maxima (FWHM) of 0.35 ± 0.04 µm and 0.39 ± 0.02 µm laterally and 0.81 ± 0.07 µm axially. The measured z-sectioning value was 1.33 ± 0.45 µm using light-sheet FWHM in the frames of the two views of 4.99 ± 0.58 µm and 4.89 ± 0.63 µm. To qualitatively demonstrate that the system can reduce shadow artefacts while providing a more isotropic resolution, a multi-cellular spheroid approximately 100 µm in diameter was imaged.

Deep-learning super-resolution light-sheet add-on microscopy (Deep-SLAM) for easy isotropic volumetric imaging of large biological specimens

Isotropic 3D histological imaging of large biological specimens is highly desired but remains highly challenging to current fluorescence microscopy technique. Here we present a new method, termed deep-learning super-resolution light-sheet add-on microscopy (Deep-SLAM), to enable fast, isotropic light-sheet fluorescence imaging on a conventional wide-field microscope. After integrating a minimized add-on device that transforms an inverted microscope into a 3D light-sheet microscope, we further integrate a deep neural network (DNN) procedure to quickly restore the ambiguous z-reconstructed planes that suffer from still insufficient axial resolution of light-sheet illumination, thereby achieving isotropic 3D imaging of thick biological specimens at single-cell resolution. We apply this easy and cost-effective Deep-SLAM approach to the anatomical imaging of single neurons in a meso-scale mouse brain, demonstrating its potential for readily converting commonly-used commercialized 2D microscopes to high-throughput 3D imaging, which is previously exclusive for high-end microscopy implementations.

Light-sheet fluorescence imaging charts the gastrula origin of vascular endothelial cells in early zebrafish embryos

It remains challenging to construct a complete cell lineage map of the origin of vascular endothelial cells in any vertebrate embryo. Here, we report the application of in toto light-sheet fluorescence imaging of embryos to trace the origin of vascular endothelial cells (ECs) at single-cell resolution in zebrafish. We first adapted a previously reported method to embryo mounting and light-sheet imaging, created an alignment, fusion, and extraction all-in-one software (AFEIO) for processing big data, and performed quantitative analysis of cell lineage relationships using commercially available Imaris software. Our data revealed that vascular ECs originated from broad regions of the gastrula along the dorsal–ventral and anterior–posterior axes, of which the dorsal–anterior cells contributed to cerebral ECs, the dorsal–lateral cells to anterior trunk ECs, and the ventral–lateral cells to posterior trunk and tail ECs. Therefore, this work, to our knowledge, charts the first comprehensive map of the gastrula origin of vascular ECs in zebrafish, and has potential applications for studying the origin of any embryonic organs in zebrafish and other model organisms.

Performance tradeoffs for single- and dual-objective open-top light-sheet microscope designs: a simulation-based analysis

Light-sheet microscopy (LSM) has emerged as a powerful tool for high-speed volumetric imaging of live model organisms and large optically cleared specimens. When designing cleared-tissue LSM systems with certain desired imaging specifications (e.g. resolution, contrast, and working distance), various design parameters must be taken into consideration. In order to elucidate some of the key design tradeoffs for LSM systems, we present a diffraction-based analysis of single- and dual-objective LSM configurations using simulations of LSM point spread functions. We assume Gaussian illumination is utilized. Specifically, we analyze the effects of the illumination and collection numerical aperture (NA), as well as their crossing angle, on spatial resolution and contrast. Assuming an open-top light-sheet (OTLS) architecture, we constrain these parameters based on fundamental geometric considerations as well as those imposed by currently available microscope objectives. In addition to revealing the performance tradeoffs of various single- and dual-objective LSM configurations, our analysis showcases the potential advantages of a novel, non-orthogonal dual-objective (NODO) architecture, especially for moderate-resolution imaging applications (collection NA of 0.5 to 0.8).

Three-dimensional bright-field microscopy with isotropic resolution based on multi-view acquisition and image fusion reconstruction

Optical Projection Tomography (OPT) is a powerful three-dimensional imaging technique used for the observation of millimeter-scaled biological samples, compatible with bright-field and fluorescence contrast. OPT is affected by spatially variant artifacts caused by the fact that light diffraction is not taken into account by the straight-light propagation models used for reconstruction. These artifacts hinder high-resolution imaging with OPT. In this work we show that, by using a multiview imaging approach, a 3D reconstruction of the bright-field contrast can be obtained without the diffraction artifacts typical of OPT, drastically reducing the amount of acquired data, compared to previously reported approaches. The method, purely based on bright-field contrast of the unstained sample, provides a comprehensive picture of the sample anatomy, as demonstrated in vivo on Arabidopsis thaliana and zebrafish embryos. Furthermore, this bright-field reconstruction can be implemented on practically any multi-view light-sheet fluorescence microscope without complex hardware modifications or calibrations, complementing the fluorescence information with tissue anatomy.

Efficient and cost-effective 3D cellular imaging by sub-voxel-resolving light-sheet add-on microscopy

Light-sheet fluorescence microscopy (LSFM) allows volumetric live imaging at high-speed and with low photo-toxicity. Various LSFM modalities are commercially available, but their size and cost limit their access by the research community. A new method, termed sub-voxel-resolving (SVR) light-sheet add-on microscopy (SLAM), is presented to enable fast, resolution-enhanced light-sheet fluorescence imaging from a conventional wide-field microscope. This method contains two components: a miniature add-on device to regular wide-field microscopes, which contains a horizontal laser light-sheet illumination path to confine fluorophore excitation at the vicinity of the focal plane for optical sectioning; an off-axis scanning strategy and a SVR algorithm that utilizes sub-voxel spatial shifts to reconstruct the image volume that results in a twofold increase in resolution. SLAM method has been applied to observe the muscle activity change of crawling C. elegans, the heartbeat of developing zebrafish embryo, and the neural anatomy of cleared mouse brains, at high spatiotemporal resolution. It provides an efficient and cost-effective solution to convert the vast number of in-service microscopes for fast 3D live imaging with voxel-super-resolved capability.

Whole-Brain Profiling of Cells and Circuits in Mammals by Tissue Clearing and Light-Sheet Microscopy

Tissue clearing and light-sheet microscopy have a 100-year-plus history, yet these fields have been combined only recently to facilitate novel experiments and measurements in neuroscience. Since tissue-clearing methods were first combined with modernized light-sheet microscopy a decade ago, the performance of both technologies has rapidly improved, broadening their applications. Here, we review the state of the art of tissue-clearing methods and light-sheet microscopy and discuss applications of these techniques in profiling cells and circuits in mice. We examine outstanding challenges and future opportunities for expanding these techniques to achieve brain-wide profiling of cells and circuits in primates and humans. Such integration will help provide a systems-level understanding of the physiology and pathology of our central nervous system.

High spatiotemporal resolution and low photo-toxicity fluorescence imaging in live cells and in vivo

Taking advantage of high contrast and molecular specificity, fluorescence microscopy has played a critical role in the visualization of subcellular structures and function, enabling unprecedented exploration from cell biology to neuroscience in living animals. To record and quantitatively analyse complex and dynamic biological processes in real time, fluorescence microscopes must be capable of rapid, targeted access deep within samples at high spatial resolutions, using techniques including super-resolution fluorescence microscopy, light sheet fluorescence microscopy, and multiple photon microscopy. In recent years, tremendous breakthroughs have improved the performance of these fluorescence microscopies in spatial resolution, imaging speed, and penetration. Here, we will review recent advancements of these microscopies in terms of the trade-off among spatial resolution, sampling speed and penetration depth and provide a view of their possible applications.

Tissue clearing and its applications in neuroscience

State-of-the-art tissue-clearing methods provide subcellular-level optical access to intact tissues from individual organs and even to some entire mammals. When combined with light-sheet microscopy and automated approaches to image analysis, existing tissue-clearing methods can speed up and may reduce the cost of conventional histology by several orders of magnitude. In addition, tissue-clearing chemistry allows whole-organ antibody labelling, which can be applied even to thick human tissues. By combining the most powerful labelling, clearing, imaging and data-analysis tools, scientists are extracting structural and functional cellular and subcellular information on complex mammalian bodies and large human specimens at an accelerated pace. The rapid generation of terabyte-scale imaging data furthermore creates a high demand for efficient computational approaches that tackle challenges in large-scale data analysis and management. In this Review, we discuss how tissue-clearing methods could provide an unbiased, system-level view of mammalian bodies and human specimens and discuss future opportunities for the use of these methods in human neuroscience.

Fast, 3D Isotropic Imaging of Whole Mouse Brain Using Multiangle-Resolved Subvoxel SPIM

The recent integration of light-sheet microscopy and tissue-clearing has facilitated an important alternative to conventional histological imaging approaches. However, the in toto cellular mapping of neural circuits throughout an intact mouse brain remains highly challenging, requiring complicated mechanical stitching, and suffering from anisotropic resolution insufficient for high-quality reconstruction in 3D. Here, the use of a multiangle-resolved subvoxel selective plane illumination microscope (Mars-SPIM) is proposed to achieve high-throughput imaging of whole mouse brain at isotropic cellular resolution. This light-sheet imaging technique can computationally improve the spatial resolution over six times under a large field of view, eliminating the use of slow tile stitching. Furthermore, it can recover complete structural information of the sample from images subject to thick-tissue scattering/attenuation. With Mars-SPIM, a digital atlas of a cleared whole mouse brain (≈7 mm × 9.5 mm × 5 mm) can readily be obtained with an isotropic resolution of ≈2 µm (1 µm voxel) and a short acquisition time of 30 min. It provides an efficient way to implement system-level cellular analysis, such as the mapping of different neuron populations and tracing of long-distance neural projections over the entire brain. Mars-SPIM is thus well suited for high-throughput cell-profiling phenotyping of brain and other mammalian organs.

Multiview microscopy of single cells through microstructure-based indirect optical manipulation

Fluorescent observation of cells generally suffers from the limited axial resolution due to the elongated point spread function of the microscope optics. Consequently, three-dimensional imaging results in axial resolution that is several times worse than the transversal. The optical solutions to this problem usually require complicated optics and extreme spatial stability. A straightforward way to eliminate anisotropic resolution is to fuse images recorded from multiple viewing directions achieved mostly by the mechanical rotation of the entire sample. In the presented approach, multiview imaging of single cells is implemented by rotating them around an axis perpendicular to the optical axis by means of holographic optical tweezers. For this, the cells are indirectly trapped and manipulated with special microtools made with two-photon polymerization. The cell is firmly attached to the microtool and is precisely manipulated with 6 degrees of freedom. The total control over the cells' position allows for its multiview fluorescence imaging from arbitrarily selected directions. The image stacks obtained this way are combined into one 3D image array with a multiview image processing pipeline resulting in isotropic optical resolution that approaches the lateral diffraction limit. The presented tool and manipulation scheme can be readily applied in various microscope platforms.

Image quality guided smart rotation improves coverage in microscopy

Fluorescence microscopy is an essential tool for biological discoveries. There is a constant demand for better spatial resolution across a larger field of view. Although strides have been made to improve the theoretical resolution and speed of the optical instruments, in mesoscopic samples, image quality is still largely limited by the optical properties of the sample. In Selective Plane Illumination Microscopy (SPIM), the achievable optical performance is hampered by optical degradations encountered in both the illumination and detection. Multi-view imaging, either through sample rotation or additional optical paths, is a popular strategy to improve sample coverage. In this work, we introduce a smart rotation workflow that utilizes on-the-fly image analysis to identify the optimal light sheet imaging orientations. The smart rotation workflow outperforms the conventional approach without additional hardware and achieves a better sample coverage using the same number of angles or less and thereby reduces data volume and phototoxicity.

Tiling light sheet selective plane illumination microscopy using discontinuous light sheets

Tiling light sheet selective plane illumination microscopy (TLS-SPIM) improves the 3D imaging ability of SPIM by using real-time optimized tiling light sheets. However, the imaging speed decreases and the raw image size increases due to the tiling process and additional camera exposures. The decreased imaging speed and the increased raw data could cause significant problems when TLS-SPIM is used to image large specimens at high spatial resolutions. Here, we present a novel method to solve the problem. Discontinuous light sheets created by scanning coaxial beam arrays synchronized with the detection camera rolling shutter are used in TLS-SPIM for 3D imaging. It improves the imaging efficiency of TLS-SPIM by reducing the number of tiles required per image plane without influencing the spatial resolution. We investigate the method via numerical simulations and experiments. We demonstrate the imaging ability of the TLS-SPIM using discontinuous light sheets and show the improved imaging efficiency by imaging optically cleared mouse brain.

Light-Sheet Microscopy and Its Potential for Understanding Developmental Processes

The ability to visualize and quantitatively measure dynamic biological processes in vivo and at high spatiotemporal resolution is of fundamental importance to experimental investigations in developmental biology. Light-sheet microscopy is particularly well suited to providing such data, since it offers exceptionally high imaging speed and good spatial resolution while minimizing light-induced damage to the specimen. We review core principles and recent advances in light-sheet microscopy, with a focus on concepts and implementations relevant for applications in developmental biology. We discuss how light-sheet microcopy has helped advance our understanding of developmental processes from single-molecule to whole-organism studies, assess the potential for synergies with other state-of-the-art technologies, and introduce methods for computational image and data analysis. Finally, we explore the future trajectory of light-sheet microscopy, discuss key efforts to disseminate new light-sheet technology, and identify exciting opportunities for further advances.

Light sheet fluorescence microscopy versus confocal microscopy: in quest of a suitable tool to assess drug and nanomedicine penetration into multicellular tumor spheroids

We recently constructed a multicellular spheroid model of pancreatic tumor based on a triple co-culture of cancer cells, fibroblasts and endothelial cells and characterized by the presence of fibronectin, an important component of the tumor extracellular matrix. By combining cancer cells and stromal components, this model recreates in vitro the three-dimensional (3D) architecture of solid tumors. In this study, we used these hetero-type spheroids as a tool to assess the penetration of doxorubicin (used as a model drug) through the whole tumor mass either in a free form or loaded into polymer nanoparticles (NPs), and we investigated whether microscopy images, acquired by Confocal Laser Scanning Microscopy (CLSM) and Light Sheet Fluorescence Microscopy (LSFM), would be best to provide reliable information on this process. Results clearly demonstrated that CLSM was not suitable to accurately monitor the diffusion of small molecules such as the doxorubicin. Indeed, it only allowed to scan a layer of 100 µm depth and no information on deeper layers could be available because of a progressive loss of the fluorescence signal. On the contrary, a complete 3D tomography of the hetero-type multicellular tumor spheroids (MCTS) was obtained by LSFM and multi-view image fusion which revealed that the fluorescent molecule was able to reach the core of spheroids as large as 1 mm in diameter. However, no doxorubicin-loaded polymer nanoparticles were detected in the spheroids, highlighting the challenge of nanomedicine delivery through biological barriers. Overall, the combination of hetero-type MCTS and LSFM allowed to carry out a highly informative microscopic assessment and represents a suitable approach to precisely follow up the drug penetration in tumors. Accordingly, it could provide useful support in the preclinical investigation and optimization of nanoscale systems for drug delivery to solid tumors.

Gridrec-MS: an algorithm for multi-slice tomography

Advances in imaging systems and modeling allow for depth information to be retrieved from projections via virtual sectioning of the imaged object. Here we introduce a regridding method that explicitly and directly incorporates this information into a general and non-iterative tomographic reconstruction algorithm. The method is applicable to any imaging scheme that provides depth-resolved projections. Additionally, we show, via numerical simulations, that with this method the required number of projections for adequate angular sampling can be reduced.

High-resolution ultramicroscopy of the developing and adult nervous system in optically cleared Drosophila melanogaster

The fruit fly, Drosophila melanogaster, is an important experimental model to address central questions in neuroscience at an organismic level. However, imaging of neural circuits in intact fruit flies is limited due to structural properties of the cuticle. Here we present a novel approach combining tissue clearing, ultramicroscopy, and data analysis that enables the visualisation of neuronal networks with single-cell resolution from the larval stage up to the adult Drosophila. FlyClear, the signal preserving clearing technique we developed, stabilises tissue integrity and fluorescence signal intensity for over a month and efficiently removes the overall pigmentation. An aspheric ultramicroscope set-up utilising an improved light-sheet generator allows us to visualise long-range connections of peripheral sensory and central neurons in the visual and olfactory system. High-resolution 3D reconstructions with isotropic resolution from entire GFP-expressing flies are obtained by applying image fusion from orthogonal directions. This methodological integration of novel chemical, optical, and computational techniques allows a major advance in the analysis of global neural circuit organisation.

Dual-view acoustic-resolution photoacoustic microscopy with enhanced resolution isotropy

Acoustic-resolution photoacoustic microscopy (AR-PAM) can provide penetration depths beyond the optical diffusion limit, but its lateral resolution is typically much worse than its axial resolution, resulting in a low-resolution isotropy. We herein present a dual-view AR-PAM (DV-AR-PAM) system that exploits two orthogonally arranged focused ultrasonic transducers. Taking advantage of their complementary acoustic detecting profiles, DV-AR-PAM allows the fusion of the dual-view signals, resulting in an enhancement in the resolution isotropy from 29% to 95% within one imaging plane, as demonstrated by simulation results on densely packed particles and experimental results on a single carbon fiber. The application of this method in vivo revealed distinct microvasculature structures in mouse skin that were previously indistinguishable with single-view detection, demonstrating the potential of DV-AR-PAM for vascular and neurological studies.

Multiscale imaging of plant development by light-sheet fluorescence microscopy

Light-sheet fluorescence microscopy (LSFM) methods collectively represent the major breakthrough in developmental bio-imaging of living multicellular organisms. They are becoming a mainstream approach through the development of both commercial and custom-made LSFM platforms that are adjusted to diverse biological applications. Based on high-speed acquisition rates under conditions of low light exposure and minimal photo-damage of the biological sample, these methods provide ideal means for long-term and in-depth data acquisition during organ imaging at single-cell resolution. The introduction of LSFM methods into biology extended our understanding of pattern formation and developmental progress of multicellular organisms from embryogenesis to adult body. Moreover, LSFM imaging allowed the dynamic visualization of biological processes under almost natural conditions. Here, we review the most important, recent biological applications of LSFM methods in developmental studies of established and emerging plant model species, together with up-to-date methods of data editing and evaluation for modelling of complex biological processes. Recent applications in animal models push LSFM into the forefront of current bio-imaging approaches. Since LSFM is now the single most effective method for fast imaging of multicellular organisms, allowing quantitative analyses of their long-term development, its broader use in plant developmental biology will likely bring new insights.

Multiscale light-sheet for rapid imaging of cardiopulmonary system

The ability to image tissue morphogenesis in real-time and in 3-dimensions (3-D) remains an optical challenge. The advent of light-sheet fluorescence microscopy (LSFM) has advanced developmental biology and tissue regeneration research. In this review, we introduce a LSFM system in which the illumination lens reshapes a thin light-sheet to rapidly scan across a sample of interest while the detection lens orthogonally collects the imaging data. This multiscale strategy provides deep-tissue penetration, high-spatiotemporal resolution, and minimal photobleaching and phototoxicity, allowing in vivo visualization of a variety of tissues and processes, ranging from developing hearts in live zebrafish embryos to ex vivo interrogation of the microarchitecture of optically cleared neonatal hearts. Here, we highlight multiple applications of LSFM and discuss several studies that have allowed better characterization of developmental and pathological processes in multiple models and tissues. These findings demonstrate the capacity of multiscale light-sheet imaging to uncover cardiovascular developmental and regenerative phenomena.