Three-dimensional imaging by deconvolution microscopy

Imaging 3D cell cultures with optical microscopy

Three-dimensional (3D) cell cultures have gained popularity in recent years due to their ability to represent complex tissues or organs more faithfully than conventional two-dimensional (2D) cell culture. This article reviews the application of both 2D and 3D microscopy approaches for monitoring and studying 3D cell cultures. We first summarize the most popular optical microscopy methods that have been used with 3D cell cultures. We then discuss the general advantages and disadvantages of various microscopy techniques for several broad categories of investigation involving 3D cell cultures. Finally, we provide perspectives on key areas of technical need in which there are clear opportunities for innovation. Our goal is to guide microscope engineers and biomedical end users toward optimal imaging methods for specific investigational scenarios and to identify use cases in which additional innovations in high-resolution imaging could be helpful.

Imaging cells and nanoparticles using modulated optically computed phase microscopy

Nanoparticles (NPs) have been successfully used as drug delivery systems. To develop and optimize NP-based drug delivery systems, it is essential to understand the dynamics of cell-NP interactions. Quantitative phase imaging techniques enable label-free imaging and have the potential to reveal how cells interact with NPs. To measure the subtle motion at cellular and subcellular scales, it requires a high phase sensitivity and a high spatial resolution. However, phase imaging techniques are limited by an intrinsic tradeoff between sensitivity and resolution. To overcome the tradeoff, we develop a technology termed as modulated optically computed phase microscopy (M-OCPM) based on low coherence interferometry and optical computation. The key innovation of M-OCPM is to utilize optical computation that performs Fourier transform of the interferometric spectra, imposes temporal modulation on the interference signal, and circumvents the sensitivity-resolution tradeoff. We evaluated the performance of M-OCPM using various samples, and demonstrated its label-free imaging capability, high sensitivity (nanometer scale displacement sensitivity) and high resolution (~ 250 nm). Particularly, we imaged NPs along with cultured cells and showed different signal characteristics for NPs that were adhered to the cells or in the cell culture medium. Our results clearly demonstrated the feasibility of M-OCPM in studying cell-NP interactions.

Deep Learning-Based Image Restoration and Super-Resolution for Fluorescence Microscopy: Overview and Resources

Fluorescence microscopy is a key method for the visualization of cellular, subcellular, and molecular live-cell dynamics, enabling access to novel insights into mechanisms of health and disease. However, effects like phototoxicity, the fugitive nature of signals, photo bleaching, and method-inherent noise can degrade the achievable signal-to-noise ratio and image resolution. In recent years, deep learning (DL) approaches have been increasingly applied to remove these degradations. In this review, we give a brief overview over existing classical and DL approaches for denoising, deconvolution, and computational super-resolution of fluorescence microscopy data. We summarize existing open-source databases within these fields as well as code repositories related to corresponding publications and further contribute an example project for DL-based image denoising, which provides a low barrier entry into DL coding and respective applications. In summary, we supply interested researchers with tools to apply or develop DL applications in live-cell imaging and foster research participation in this field.

Microscopy image reconstruction with physics-informed denoising diffusion probabilistic model

Light microscopy is a practical tool for advancing biomedical research and diagnostics, offering invaluable insights into the cellular and subcellular structures of living organisms. However, diffraction and optical imperfections actively hinder the attainment of high-quality images. In recent years, there has been a growing interest in applying deep learning techniques to overcome these challenges in light microscopy imaging. Nonetheless, the resulting reconstructions often suffer from undesirable artefacts and hallucinations. Here, we introduce a deep learning-based approach that incorporates the fundamental physics of light propagation in microscopy into the loss function. This model employs a conditioned diffusion model in a physics-informed architecture. To mitigate the issue of limited available data, we utilise synthetic datasets for training purposes. Our results demonstrate consistent enhancements in image quality and substantial reductions in artefacts when compared to state-of-the-art methods. The presented technique is intuitively accessible and allows obtaining higher quality microscopy images for biomedical studies.

High-resolution cell imaging using white light phase shifting interferometry and iterative phase deconvolution

An optimization algorithm is presented for the deconvolution of a complex field to improve the resolution and accuracy of quantitative phase imaging (QPI). A high-resolution phase map can be recovered by solving a constrained optimization problem of deconvolution using a complex gradient operator. The method is demonstrated on phase measurements of samples using a white light based phase shifting interferometry (WLPSI) method. The application of the algorithm on real and simulated objects shows a significant resolution and contrast improvement. Experiments performed on Escherichia coli bacterium have revealed its sub-cellular structures that were not visible in the raw WLPSI images obtained using a five phase shifting method. These features can give valuable insights into the structures and functioning of biological cells. The algorithm is simple in implementation and can be incorporated into other QPI modalities .

Volume holographic illuminator for Airy light-sheet microscopy

Airy light sheets combined with the deconvolution approach can provide multiple benefits, including large field of view (FOV), thin optical sectioning, and high axial resolution. The efficient design of an Airy light-sheet fluorescence microscope requires a compact illumination system. Here, we show that an Airy light sheet can be conveniently implemented in microscopy using a volume holographic grating (VHG). To verify the FOV and the axial resolution of the proposed VHG-based Airy light-sheet fluorescence microscope, ex-vivo fluorescently labeled Caenorhabditis elegans (C. elegans) embryos were imaged, and the Richardson-Lucy deconvolution method was used to improve the image contrast. Optimized parameters for deconvolution were compared with different methods. The experimental results show that the FOV and the axial resolution were 196 µm and 3 µm, respectively. The proposed method of using a compact VHG to replace the common spatial light modulator provides a direct solution to construct a compact light-sheet fluorescence microscope.

"MeiQuant": An Integrated Tool for Analyzing Meiotic Prophase I Spread Images

Immunocytochemical analysis of meiotic proteins on mouse chromosome spreads is one method of choice to study prophase I chromosome organization and homologous recombination. In recent decades, the development of microscopic approaches led to the production of a large number of images that monitor fluorescent proteins, defined as fluorescent objects, and a major challenge facing the community is the deep analysis of these fluorescent objects (measurement of object length, intensity, distance between objects, as well as foci identification, counting, and colocalization). We propose a set of tools designed from the macro language of the widely used image analysis software ImageJ (Schindelin et al., Nat Methods 9: 676-682, 2012), embedded in the "MeiQuant" macro, which are specifically designed for analyzing objects in the field of meiosis. Our aim is to propose a unified evolutive common tool for image analysis, with a specific focus on mouse prophase I meiotic events.

Technologies for depth scanning in miniature optical imaging systems [Invited]

Biomedical optical imaging has found numerous clinical and research applications. For achieving 3D imaging, depth scanning presents the most significant challenge, particularly in miniature imaging devices. This paper reviews the state-of-art technologies for depth scanning in miniature optical imaging systems, which include two general approaches: 1) physically shifting part of or the entire imaging device to allow imaging at different depths and 2) optically changing the focus of the imaging optics. We mainly focus on the second group of methods, introducing a wide variety of tunable microlenses, covering the underlying physics, actuation mechanisms, and imaging performance. Representative applications in clinical and neuroscience research are briefly presented. Major challenges and future perspectives of depth/focus scanning technologies for biomedical optical imaging are also discussed.

Deep self-learning enables fast, high-fidelity isotropic resolution restoration for volumetric fluorescence microscopy

One intrinsic yet critical issue that troubles the field of fluorescence microscopy ever since its introduction is the unmatched resolution in the lateral and axial directions (i.e., resolution anisotropy), which severely deteriorates the quality, reconstruction, and analysis of 3D volume images. By leveraging the natural anisotropy, we present a deep self-learning method termed Self-Net that significantly improves the resolution of axial images by using the lateral images from the same raw dataset as rational targets. By incorporating unsupervised learning for realistic anisotropic degradation and supervised learning for high-fidelity isotropic recovery, our method can effectively suppress the hallucination with substantially enhanced image quality compared to previously reported methods. In the experiments, we show that Self-Net can reconstruct high-fidelity isotropic 3D images from organelle to tissue levels via raw images from various microscopy platforms, e.g., wide-field, laser-scanning, or super-resolution microscopy. For the first time, Self-Net enables isotropic whole-brain imaging at a voxel resolution of 0.2 × 0.2 × 0.2 μm3, which addresses the last-mile problem of data quality in single-neuron morphology visualization and reconstruction with minimal effort and cost. Overall, Self-Net is a promising approach to overcoming the inherent resolution anisotropy for all classes of 3D fluorescence microscopy.

Volumetric light sheet imaging with adaptive optics correction

Light sheet microscopy has developed quickly over the past decades and become a popular method for imaging live model organisms and other thick biological tissues. For rapid volumetric imaging, an electrically tunable lens can be used to rapidly change the imaging plane in the sample. For larger fields of view and higher NA objectives, the electrically tunable lens introduces aberrations in the system, particularly away from the nominal focus and off-axis. Here, we describe a system that employs an electrically tunable lens and adaptive optics to image over a volume of 499 × 499 × 192 μm3 with close to diffraction-limited resolution. Compared to the system without adaptive optics, the performance shows an increase in signal to background ratio by a factor of 3.5. While the system currently requires 7s/volume, it should be straightforward to increase the imaging speed to under 1s per volume.

Uncovering the cytotoxic effects of air pollution with multi-modal imaging of in vitro respiratory models

Annually, an estimated seven million deaths are linked to exposure to airborne pollutants. Despite extensive epidemiological evidence supporting clear associations between poor air quality and a range of short- and long-term health effects, there are considerable gaps in our understanding of the specific mechanisms by which pollutant exposure induces adverse biological responses at the cellular and tissue levels. The development of more complex, predictive, in vitro respiratory models, including two- and three-dimensional cell cultures, spheroids, organoids and tissue cultures, along with more realistic aerosol exposure systems, offers new opportunities to investigate the cytotoxic effects of airborne particulates under controlled laboratory conditions. Parallel advances in high-resolution microscopy have resulted in a range of in vitro imaging tools capable of visualizing and analysing biological systems across unprecedented scales of length, time and complexity. This article considers state-of-the-art in vitro respiratory models and aerosol exposure systems and how they can be interrogated using high-resolution microscopy techniques to investigate cell-pollutant interactions, from the uptake and trafficking of particles to structural and functional modification of subcellular organelles and cells. These data can provide a mechanistic basis from which to advance our understanding of the health effects of airborne particulate pollution and develop improved mitigation measures.

Airy beam light sheet microscopy boosted by deep learning deconvolution

Common light sheet microscopy comes with a trade-off between light sheet width defining the optical sectioning and the usable field of view arising from the divergence of the illuminating Gaussian beam. To overcome this, low-diverging Airy beams have been introduced. Airy beams, however, exhibit side lobes degrading image contrast. Here, we constructed an Airy beam light sheet microscope, and developed a deep learning image deconvolution to remove the effects of the side lobes without knowledge of the point spread function. Using a generative adversarial network and high-quality training data, we significantly enhanced image contrast and improved the performance of a bicubic upscaling. We evaluated the performance with fluorescently labeled neurons in mouse brain tissue samples. We found that deep learning-based deconvolution was about 20-fold faster than the standard approach. The combination of Airy beam light sheet microscopy and deep learning deconvolution allows imaging large volumes rapidly and with high quality.

Regularization of dielectric tensor tomography

Dielectric tensor tomography reconstructs the three-dimensional dielectric tensors of microscopic objects and provides information about the crystalline structure orientations and principal refractive indices. Because dielectric tensor tomography is based on transmission measurement, it suffers from the missing cone problem, which causes poor axial resolution, underestimation of the refractive index, and halo artifacts. In this study, we study the application of total variation and positive semi-definiteness regularization to three-dimensional tensor distributions. In particular, we demonstrate the reduction of artifacts when applied to dielectric tensor tomography.

Practical considerations for quantitative light sheet fluorescence microscopy

Fluorescence microscopy has evolved from a purely observational tool to a platform for quantitative, hypothesis-driven research. As such, the demand for faster and less phototoxic imaging modalities has spurred a rapid growth in light sheet fluorescence microscopy (LSFM). By restricting the excitation to a thin plane, LSFM reduces the overall light dose to a specimen while simultaneously improving image contrast. However, the defining characteristics of light sheet microscopes subsequently warrant unique considerations in their use for quantitative experiments. In this Perspective, we outline many of the pitfalls in LSFM that can compromise analysis and confound interpretation. Moreover, we offer guidance in addressing these caveats when possible. In doing so, we hope to provide a useful resource for life scientists seeking to adopt LSFM to quantitatively address complex biological hypotheses.

Too bright for 2 dimensions: recent progress in advanced 3-dimensional microscopy of the kidney

The kidney is a structurally and functionally complex organ responsible for the control of water, ion, and other solute homeostasis. Moreover, the kidneys excrete metabolic waste products and produce hormones, such as renin and erythropoietin. The functional unit of the kidney is the nephron, which is composed by a serial arrangement of a filter unit called the renal corpuscle and several tubular segments that modulate the filtered fluid by reabsorption and secretion. Within each kidney, thousands of nephrons are closely intermingled and surrounded by an intricate network of blood vessels and various interstitial cell types, including fibroblasts and immune cells. This complex tissue architecture is essential for proper kidney function. In fact, kidney disease is often reflected or even caused by a derangement of the histologic structures. Frequently, kidney histology is studied using microscopic analysis of 2-dimensional tissue sections, which, however, misses important 3-dimensional spatial information. Reconstruction of serial sections tries to overcome this limitation, but is technically challenging, time-consuming, and often inherently linked to sectioning artifacts. In recent years, advances in tissue preparation (e.g., optical clearing) and new light- and electron-microscopic methods have provided novel avenues for 3-dimensional kidney imaging. Combined with novel machine-learning algorithms, these approaches offer unprecedented options for large-scale and automated analysis of kidney structure and function. This review provides a brief overview of these emerging imaging technologies and presents key examples of how these approaches are already used to study the normal and the diseased kidney.

Development of a 3D-immunofluorescence analysis for sensory nerve endings in human ligaments

BACKGROUND

The analysis of ligamentous mechanoreceptors is difficult due to a high amount of unclassifiable mechanoreceptors, which result from incomplete visualization through limited microscopic techniques.

NEW METHOD

The method was developed using dorsal intercarpal ligaments and dorsal regions of the scapholunate interosseous ligament from human cadaver wrists. Consecutive 70 µm thick cryosections were stained with immunofluorescence markers for protein S100B, neurotrophin receptor p75 (p75), protein gene product 9.5 (PGP 9.5) and 4',6-diamidino-2-phenylindole (DAPI). 3D images of sensory nerve endings were obtained using a confocal laser scanning microscope. Experimental point spread functions (PSF) were used to deconvolve images. Sensory nerve endings were localised in each section plane and classified according to Freeman and Wyke. Finally, confocal data was visualized as 3D-images.

RESULTS

The method produced excellent image quality, revealing detailed three-dimensional structures. The created 3D-model of sensory nerve endings could be analyzed in all three dimensions, augmenting visualization of the form and immunoreactive pattern of sensory nerve endings. Deconvolution with experimentally measured PSFs aided in enhancing image quality.

COMPARISON WITH EXISTING METHODS

Using a triple immunofluorescent staining method allows to visualize the structure of sensory nerve endings more precisely than techniques with serial analysis of different monostaining of neural markers. Imaging in three dimensions enhances morphologic details, which are limited in 2D-microscopy.

CONCLUSION

3D-triple immunofluorescence produces high quality visualization of mechanoreceptors, thereby improving their analysis. As an elaborate technique, it is ideal for defined research questions concerning the microstructure of sensory nerve endings.

Quality assessment in light microscopy for routine use through simple tools and robust metrics

Although there is a need to demonstrate reproducibility in light microscopy acquisitions, the lack of standardized guidelines monitoring microscope health status over time has so far impaired the widespread use of quality control (QC) measurements. As scientists from 10 imaging core facilities who encounter various types of projects, we provide affordable hardware and open source software tools, rigorous protocols, and define reference values to assess QC metrics for the most common fluorescence light microscopy modalities. Seven protocols specify metrics on the microscope resolution, field illumination flatness, chromatic aberrations, illumination power stability, stage drift, positioning repeatability, and spatial-temporal noise of camera sensors. We designed the MetroloJ_QC ImageJ/Fiji Java plugin to incorporate the metrics and automate analysis. Measurements allow us to propose an extensive characterization of the QC procedures that can be used by any seasoned microscope user, from research biologists with a specialized interest in fluorescence light microscopy through to core facility staff, to ensure reproducible and quantifiable microscopy results.

Experimentally unsupervised deconvolution for light-sheet microscopy with propagation-invariant beams

Deconvolution is a challenging inverse problem, particularly in techniques that employ complex engineered point-spread functions, such as microscopy with propagation-invariant beams. Here, we present a deep-learning method for deconvolution that, in lieu of end-to-end training with ground truths, is trained using known physics of the imaging system. Specifically, we train a generative adversarial network with images generated with the known point-spread function of the system, and combine this with unpaired experimental data that preserve perceptual content. Our method rapidly and robustly deconvolves and super-resolves microscopy images, demonstrating a two-fold improvement in image contrast to conventional deconvolution methods. In contrast to common end-to-end networks that often require 1000-10,000s paired images, our method is experimentally unsupervised and can be trained solely on a few hundred regions of interest. We demonstrate its performance on light-sheet microscopy with propagation-invariant Airy beams in oocytes, preimplantation embryos and excised brain tissue, as well as illustrate its utility for Bessel-beam LSM. This method aims to democratise learned methods for deconvolution, as it does not require data acquisition outwith the conventional imaging protocol.

Accurate unsupervised estimation of aberrations in digital holographic microscopy for improved quantitative reconstruction

In the context of digital in-line holographic microscopy, we describe an unsupervised methodology to estimate the aberrations of an optical microscopy system from a single hologram. The method is based on the Inverse Problems Approach reconstructions of holograms of spherical objects. The forward model is based on a Lorenz-Mie model distorted by optical aberrations described by Zernike polynomials. This methodology is thus able to characterize most varying aberrations in the field of view in order to take them into account to improve the reconstruction of any sample. We show that this approach increases the repeatability and quantitativity of the reconstructions in both simulations and experimental data. We use the Cramér-Rao lower bounds to study the accuracy of the reconstructions. Finally, we demonstrate the efficiency of this aberration calibration with image reconstructions using a phase retrieval algorithm as well as a regularized inverse problems algorithm.

Image Reconstruction in Light-Sheet Microscopy: Spatially Varying Deconvolution and Mixed Noise

We study the problem of deconvolution for light-sheet microscopy, where the data is corrupted by spatially varying blur and a combination of Poisson and Gaussian noise. The spatial variation of the point spread function of a light-sheet microscope is determined by the interaction between the excitation sheet and the detection objective PSF. We introduce a model of the image formation process that incorporates this interaction and we formulate a variational model that accounts for the combination of Poisson and Gaussian noise through a data fidelity term consisting of the infimal convolution of the single noise fidelities, first introduced in L. Calatroni et al. (SIAM J Imaging Sci 10(3):1196-1233, 2017). We establish convergence rates and a discrepancy principle for the infimal convolution fidelity and the inverse problem is solved by applying the primal-dual hybrid gradient (PDHG) algorithm in a novel way. Numerical experiments performed on simulated and real data show superior reconstruction results in comparison with other methods.

Novel in vitro Experimental Approaches to Study Myelination and Remyelination in the Central Nervous System

Myelin is the lipidic insulating structure enwrapping axons and allowing fast saltatory nerve conduction. In the central nervous system, myelin sheath is the result of the complex packaging of multilamellar extensions of oligodendrocyte (OL) membranes. Before reaching myelinating capabilities, OLs undergo a very precise program of differentiation and maturation that starts from OL precursor cells (OPCs). In the last 20 years, the biology of OPCs and their behavior under pathological conditions have been studied through several experimental models. When co-cultured with neurons, OPCs undergo terminal maturation and produce myelin tracts around axons, allowing to investigate myelination in response to exogenous stimuli in a very simple in vitro system. On the other hand, in vivo models more closely reproducing some of the features of human pathophysiology enabled to assess the consequences of demyelination and the molecular mechanisms of remyelination, and they are often used to validate the effect of pharmacological agents. However, they are very complex, and not suitable for large scale drug discovery screening. Recent advances in cell reprogramming, biophysics and bioengineering have allowed impressive improvements in the methodological approaches to study brain physiology and myelination. Rat and mouse OPCs can be replaced by human OPCs obtained by induced pluripotent stem cells (iPSCs) derived from healthy or diseased individuals, thus offering unprecedented possibilities for personalized disease modeling and treatment. OPCs and neural cells can be also artificially assembled, using 3D-printed culture chambers and biomaterial scaffolds, which allow modeling cell-to-cell interactions in a highly controlled manner. Interestingly, scaffold stiffness can be adopted to reproduce the mechanosensory properties assumed by tissues in physiological or pathological conditions. Moreover, the recent development of iPSC-derived 3D brain cultures, called organoids, has made it possible to study key aspects of embryonic brain development, such as neuronal differentiation, maturation and network formation in temporal dynamics that are inaccessible to traditional in vitro cultures. Despite the huge potential of organoids, their application to myelination studies is still in its infancy. In this review, we shall summarize the novel most relevant experimental approaches and their implications for the identification of remyelinating agents for human diseases such as multiple sclerosis.

DiLFM: an artifact-suppressed and noise-robust light-field microscopy through dictionary learning

Light field microscopy (LFM) has been widely used for recording 3D biological dynamics at camera frame rate. However, LFM suffers from artifact contaminations due to the illness of the reconstruction problem via naïve Richardson-Lucy (RL) deconvolution. Moreover, the performance of LFM significantly dropped in low-light conditions due to the absence of sample priors. In this paper, we thoroughly analyze different kinds of artifacts and present a new LFM technique termed dictionary LFM (DiLFM) that substantially suppresses various kinds of reconstruction artifacts and improves the noise robustness with an over-complete dictionary. We demonstrate artifact-suppressed reconstructions in scattering samples such as Drosophila embryos and brains. Furthermore, we show our DiLFM can achieve robust blood cell counting in noisy conditions by imaging blood cell dynamic at 100 Hz and unveil more neurons in whole-brain calcium recording of zebrafish with low illumination power in vivo.

Efficient computation of backprojection arrays for 3D light field deconvolution

Light field deconvolution allows three-dimensional investigations from a single snapshot recording of a plenoptic camera. It is based on a linear image formation model, and iterative volume reconstruction requires to define the backprojection of individual image pixels into object space. This is effectively a reversal of the point spread function (PSF), and backprojection arrays H' can be derived from the shift-variant PSFs H of the optical system, which is a very time consuming step for high resolution cameras. This paper illustrates the common structure of backprojection arrays and the significance of their efficient computation. A new algorithm is presented to determine H' from H, which is based on the distinct relation of the elements' positions within the two multi-dimensional arrays. It permits a pure array rearrangement, and while results are identical to those from published codes, computation times are drastically reduced. This is shown by benchmarking the new method using various sample PSF arrays against existing algorithms. The paper is complemented by practical hints for the experimental acquisition of light field PSFs in a photographic setup.

Regularization-parameter-free optimization approach for image deconvolution

Image deconvolution is often modeled as an optimization problem for a cost function involving two or more terms that represent the data fidelity and the image domain constraints (or penalties). While a number of choices for modeling the cost function and implementing the optimization algorithms exist, selection of the regularization parameter in the cost function usually involves empirical tuning, which is a tedious process. Any optimization framework provides a family of solutions, depending on the numerical value of the regularization parameter. The end-user has to perform the task of tuning the regularization parameter based on visual inspection of the recovered solutions and then use the suitable image for further applications. In this work, we present an image deconvolution framework using the methodology of mean gradient descent (MGD), which does not involve any regularization parameter. The aim of our approach is instead to arrive at a solution point where the different costs balance each other. This is achieved by progressing the solution in the direction that bisects the steepest descent directions corresponding to the two cost terms in each iteration. The methodology is illustrated with numerical simulations as well as with experimental image records from a bright-field microscope system and shows uniform deconvolution performance for data with different noise levels. MGD offers an efficient and user-friendly method that may be employed for a variety of image deconvolution tools. The MGD approach as discussed here may find applications in the context of more general optimization problems as well.

Snapshot three-dimensional absorption imaging of microscopic specimens

Snapshot projection optical tomography (SPOT) uses a micro-lens array (MLA) to simultaneously capture the projection images of a three-dimensional (3D) specimen corresponding to different viewing directions. Compared to other light-field imaging techniques using an MLA, SPOT is dual telecentric and can block high-angle stray rays without sacrificing the light collection efficiency. Using SPOT, we recently demonstrated snapshot 3D fluorescence imaging. Here we demonstrate snapshot 3D absorption imaging of microscopic specimens. For the illumination, we focus the incoherent light from a light-emitting diode onto a pinhole, which is placed at a conjugate plane to the sample plane. SPOT allows us to capture the ray bundles passing through the specimen along different directions. The images recorded by an array of lenslets can be related to the projections of 3D absorption coefficient along the viewing directions of lenslets. Using a tomographic reconstruction algorithm, we obtain the 3D map of absorption coefficient. We apply the developed system to different types of samples, which demonstrates the optical sectioning capability. The transverse and axial resolutions measured with gold nanoparticles are 1.3 μm and 2.3 μm, respectively.

Hyperspectral Three-Dimensional Fluorescence Imaging Using Snapshot Optical Tomography

Hyperspectral three-dimensional (3D) imaging can provide both 3D structural and functional information of a specimen. The imaging throughput is typically very low due to the requirement of scanning mechanisms for different depths and wavelengths. Here we demonstrate hyperspectral 3D imaging using Snapshot projection optical tomography (SPOT) and Fourier-transform spectroscopy (FTS). SPOT allows us to instantaneously acquire the projection images corresponding to different viewing angles, while FTS allows us to perform hyperspectral imaging at high spectral resolution. Using fluorescent beads and sunflower pollens, we demonstrate the imaging performance of the developed system.

A simple, inexpensive and multi-scale 3-D fluorescent test sample for optical sectioning microscopies

Fluorescence standards allow for quality control and for the comparison of data sets across instruments and laboratories in applications of quantitative fluorescence. For example, users of microscopy core facilities can expect a homogenous and time-invariant illumination and an uniform detection sensitivity, which are prerequisites for imaging analysis, tracking or fluorimetric pH or Ca²⁺ -concentration measurements. Similarly, confirming the three-dimensional (3-D) resolution of optical sectioning microscopes calls for a regular calibration with a standardized point source. The test samples required for such measurements are typically different ones, they are often expensive and they depend much on the very microscope technique used. Similarly, the ever-increasing choice among microscope techniques and geometries increases the demand for comparison across instruments. Here, we advocate and demonstrate the multiple uses of a surprisingly versatile and simple 3-D test sample that can complement existing and much more expensive calibration samples: commercial tissue paper labeled with a fluorescent highlighter pen. We provide relevant sample characteristics and show examples ranging from the sub-μm to cm scale, acquired on epifluorescence, confocal, image scanning, two-photon (2P) and light-sheet microscopes.

3D positioning of tagged DNA loci by widefield and super-resolution fluorescence imaging of fixed yeast nuclei

This protocol describes how to culture, image, and determine the nuclear position of a fluorescently tagged DNA locus in the 3D nucleoplasm of fixed Saccharomyces cerevisiae cells. Here, we propose a manual scoring method based on widefield images and an automated method based on 3D-SIM images. Yeast culture conditions have to be followed meticulously to get the best biological response in a given environment. For complete details on the use and execution of this protocol, please refer to Forey et al. (2020).

DVDeconv: An Open-Source MATLAB Toolbox for Depth-Variant Asymmetric Deconvolution of Fluorescence Micrographs

To investigate the cellular structure, biomedical researchers often obtain three-dimensional images by combining two-dimensional images taken along the z axis. However, these images are blurry in all directions due to diffraction limitations. This blur becomes more severe when focusing further inside the specimen as photons in deeper focus must traverse a longer distance within the specimen. This type of blur is called depth-variance. Moreover, due to lens imperfection, the blur has asymmetric shape. Most deconvolution solutions for removing blur assume depth-invariant or x-y symmetric blur, and presently, there is no open-source for depth-variant asymmetric deconvolution. In addition, existing datasets for deconvolution microscopy also assume invariant or x-y symmetric blur, which are insufficient to reflect actual imaging conditions. DVDeconv, that is a set of MATLAB functions with a user-friendly graphical interface, has been developed to address depth-variant asymmetric blur. DVDeconv includes dataset, depth-variant asymmetric point spread function generator, and deconvolution algorithms. Experimental results using DVDeconv reveal that depth-variant asymmetric deconvolution using DVDeconv removes blurs accurately. Furthermore, the dataset in DVDeconv constructed can be used to evaluate the performance of microscopy deconvolution to be developed in the future.

Three-dimensional volumetric deconvolution in coherent optics and holography

Methods of three-dimensional deconvolution (3DD) or volumetric deconvolution of optical complex-valued wavefronts diffracted by 3D samples with the 3D point spread function are presented. Particularly, the quantitative correctness of the recovered 3D sample distributions is addressed. Samples consisting of point-like objects can be retrieved from their 3D diffracted wavefronts with non-iterative (Wiener filter) 3DD. Continuous extended samples, including complex-valued (phase) samples, can be retrieved with iterative (Gold and Richardson-Lucy) 3DD algorithms. It is shown that quantitatively correct 3D sample distribution can be recovered only with iterative 3DD, and with the optimal protocols provided. It is demonstrated that 3DD can improve the lateral resolution to the resolution limit, and the axial resolution can be at least four times better than the resolution limit. The presented 3DD methods of complex-valued optical fields can be applied for 3D optical imaging and holography.

4D Widefield Fluorescence Imaging of Appressorium Morphogenesis by Magnaporthe oryzae

Fluorescence microscopy has become a widely used and indispensable tool for the M. oryzae research community, providing unique insight into appressorium formation and function. A common practice within the field is to acquire and present images of a number of different conidia, expressing a fluorescent fusion protein of interest, at various stages of infectious development, therein providing a representative "snapshot" of the population at a given point in time. Furthermore, these images typically show only a single focal plane through the specimen (2D) and therefore lack, often valuable, volumetric information. While this approach has its advantages, the continuous imaging of (multiple) single conidia in three dimensions (3D), and over time (4D), can provide additional insight into the spatial and temporal dynamics of fluorescent fusion proteins, and the subcellular structures and compartments they label, in living cells. Here we describe our typical workflow for the 4D live-cell imaging of appressorium morphogenesis in vitro using two-color widefield fluorescence microscopy and briefly outline some important considerations for strain construction, and downstream image processing and visualization.

Axially resolved volumetric two-photon microscopy with an extended field of view using depth localization under mirrored Airy beams

It is a great challenge in two-photon microscopy (2PM) to have a high volumetric imaging speed without sacrificing the spatial and temporal resolution in three dimensions (3D). The structure in 2PM images could be reconstructed with better spatial and temporal resolution by the proper choice of the data processing algorithm. Here, we propose a method to reconstruct 3D volume from 2D projections imaged by mirrored Airy beams. We verified that our approach can achieve high accuracy in 3D localization over a large axial range and is applicable to continuous and dense sample. The effective field of view after reconstruction is expanded. It is a promising technique for rapid volumetric 2PM with axial localization at high resolution.

Three-dimensional in situ morphometrics of Mycobacterium tuberculosis infection within lesions by optical mesoscopy and novel acid-fast staining

Tuberculosis (TB) preclinical testing relies on in vivo models including the mouse aerosol challenge model. The only method of determining colony morphometrics of TB infection in a tissue in situ is two-dimensional (2D) histopathology. 2D measurements consider heterogeneity within a single observable section but not above and below, which could contain critical information. Here we describe a novel approach, using optical clearing and a novel staining procedure with confocal microscopy and mesoscopy, for three-dimensional (3D) measurement of TB infection within lesions at sub-cellular resolution over a large field of view. We show TB morphometrics can be determined within lesion pathology, and differences in infection with different strains of Mycobacterium tuberculosis. Mesoscopy combined with the novel CUBIC Acid-Fast (CAF) staining procedure enables a quantitative approach to measure TB infection and allows 3D analysis of infection, providing a framework which could be used in the analysis of TB infection in situ.

3D projection electrophoresis for single-cell immunoblotting

Immunoassays and mass spectrometry are powerful single-cell protein analysis tools; however, interfacing and throughput bottlenecks remain. Here, we introduce three-dimensional single-cell immunoblots to detect both cytosolic and nuclear proteins. The 3D microfluidic device is a photoactive polyacrylamide gel with a microwell array-patterned face (xy) for cell isolation and lysis. Single-cell lysate in each microwell is "electrophoretically projected" into the 3rd dimension (z-axis), separated by size, and photo-captured in the gel for immunoprobing and confocal/light-sheet imaging. Design and analysis are informed by the physics of 3D diffusion. Electrophoresis throughput is > 2.5 cells/s (70× faster than published serial sampling), with 25 immunoblots/mm2 device area (>10× increase over previous immunoblots). The 3D microdevice design synchronizes analyses of hundreds of cells, compared to status quo serial analyses that impart hours-long delay between the first and last cells. Here, we introduce projection electrophoresis to augment the heavily genomic and transcriptomic single-cell atlases with protein-level profiling.

Rethinking resolution estimation in fluorescence microscopy: from theoretical resolution criteria to super-resolution microscopy

Resolution is undoubtedly the most important parameter in optical microscopy by providing an estimation on the maximum resolving power of a certain optical microscope. For centuries, the resolution of an optical microscope is generally considered to be limited only by the numerical aperture of the optical system and the wavelength of light. However, since the invention and popularity of various advanced fluorescence microscopy techniques, especially super-resolution fluorescence microscopy, many new methods have been proposed for estimating the resolution, leading to confusions for researchers who need to quantify the resolution of their fluorescence microscopes. In this paper, we firstly summarize the early concepts and criteria for predicting the resolution limit of an ideal optical system. Then, we discuss some important influence factors that deteriorate the resolution of a certain fluorescence microscope. Finally, we provide methods and examples on how to measure the resolution of a fluorescence microscope from captured fluorescence images. This paper aims to answer as best as possible the theoretical and practical issues regarding the resolution estimation in fluorescence microscopy.

BatchDeconvolution: a Fiji plugin for increasing deconvolution workflow

Deconvolution microscopy is a very useful, software-based technique allowing to deblur microscopy images and increase both lateral and axial resolutions. It can be used along with many of fluorescence microscopy imaging techniques. By increasing axial resolution, it also enables three-dimensional imaging using a basic wide-field fluorescence microscope. Unfortunately, commercially available deconvolution software is expensive, while freely available programs have limited capabilities of a batch file processing. In this work we present BatchDeconvolution, a Fiji plugin that bridges two programs that we used subsequently in an image deconvolution pipeline: PSF Generator and DeconvolutionLab2, both from Biomedical Imaging Group, EPFL. Our software provides a simple way to perform a batch processing of multiple microscopy files with minimal working time required from the user.

Pushing the boundaries of optoacoustic microscopy by total impulse response characterization

Optical microscopy improves in resolution and signal-to-noise ratio by correcting for the system’s point spread function; a measure of how a point source is resolved, typically determined by imaging nanospheres. Optical-resolution optoacoustic (photoacoustic) microscopy could be similarly corrected, especially to account for the spatially-dependent signal distortions induced by the acoustic detection and the time-resolved and bi-polar nature of optoacoustic signals. Correction algorithms must therefore include the spatial dependence of signals’ origins and profiles in time, i.e. the four-dimensional total impulse response (TIR). However, such corrections have been so far impeded by a lack of efficient TIR-characterization methods. We introduce high-quality TIR determination based on spatially-distributed optoacoustic point sources (SOAPs), produced by scanning an optical focus on an axially-translatable 250 nm gold layer. Using a spatially-dependent TIR-correction improves the signal-to-noise ratio by >10 dB and the axial resolution by ~30%. This accomplishment displays a new performance paradigm for optoacoustic microscopy.

Characterizing the total impulse response (TIR) of photoacoustic microscopes has been challenging due to difficulties distributing appropriate point sources. Here, the authors present a method for 3D generation of spatially-distributed optoacoustic point sources and show that subsequent TIR correction results in improved image quality.

Fundamentals of Microscopy

The light (or optical) microscope is the icon of science. The aphorism "seeing is believing" is often quoted in scientific papers involving microscopy. Unlike many scientific instruments, the light microscope will deliver an image however badly it is set up. Fluorescence microscopy is a widely used research tool across all disciplines of biological and biomedical science. Most universities and research institutions have microscopes, including confocal microscopes. This introductory paper in a series detailing advanced light microscopy techniques explains the foundations of both electron and light microscopy for biologists and life scientists working with the mouse. An explanation is given of how an image is formed. A description is given of how to set up a light microscope, whether it be a brightfield light microscope on the laboratory bench, a widefield fluorescence microscope, or a confocal microscope. These explanations are accompanied by operational protocols. A full explanation on how to set up and adjust a microscope according to the principles of Köhler illumination is given. The importance of Nyquist sampling is discussed. Guidelines are given on how to choose the best microscope to image the particular sample or slide preparation that you are working with. These are the basic principles of microscopy that a researcher must have an understanding of when operating core bioimaging facility instruments, in order to collect high-quality images. © 2020 The Authors. Basic Protocol 1: Setting up Köhler illumination for a brightfield microscope Basic Protocol 2: Aligning the fluorescence bulb and setting up Köhler illumination for a widefield fluorescence microscope Basic Protocol 3: Generic protocol for operating a confocal microscope.

Comparison of Multiscale Imaging Methods for Brain Research

A major challenge in neuroscience is how to study structural alterations in the brain. Even small changes in synaptic composition could have severe outcomes for body functions. Many neuropathological diseases are attributable to disorganization of particular synaptic proteins. Yet, to detect and comprehensively describe and evaluate such often rather subtle deviations from the normal physiological status in a detailed and quantitative manner is very challenging. Here, we have compared side-by-side several commercially available light microscopes for their suitability in visualizing synaptic components in larger parts of the brain at low resolution, at extended resolution as well as at super-resolution. Microscopic technologies included stereo, widefield, deconvolution, confocal, and super-resolution set-ups. We also analyzed the impact of adaptive optics, a motorized objective correction collar and CUDA graphics card technology on imaging quality and acquisition speed. Our observations evaluate a basic set of techniques, which allow for multi-color brain imaging from centimeter to nanometer scales. The comparative multi-modal strategy we established can be used as a guide for researchers to select the most appropriate light microscopy method in addressing specific questions in brain research, and we also give insights into recent developments such as optical aberration corrections.

Multi-target Detection with an Arbitrary Spacing Distribution

Motivated by the structure reconstruction problem in single-particle cryo-electron microscopy, we consider the multi-target detection model, where multiple copies of a target signal occur at unknown locations in a long measurement, further corrupted by additive Gaussian noise. At low noise levels, one can easily detect the signal occurrences and estimate the signal by averaging. However, in the presence of high noise, which is the focus of this paper, detection is impossible. Here, we propose two approaches-autocorrelation analysis and an approximate expectation maximization algorithm-to reconstruct the signal without the need to detect signal occurrences in the measurement. In particular, our methods apply to an arbitrary spacing distribution of signal occurrences. We demonstrate reconstructions with synthetic data and empirically show that the sample complexity of both methods scales as SNR^-3 in the low SNR regime.

Imaging Platelet Processes and Function-Current and Emerging Approaches for Imaging in vitro and in vivo

Platelets are small anucleate cells that are essential for many biological processes including hemostasis, thrombosis, inflammation, innate immunity, tumor metastasis, and wound healing. Platelets circulate in the blood and in order to perform all of their biological roles, platelets must be able to arrest their movement at an appropriate site and time. Our knowledge of how platelets achieve this has expanded as our ability to visualize and quantify discreet platelet events has improved. Platelets are exquisitely sensitive to changes in blood flow parameters and so the visualization of rapid intricate platelet processes under conditions found in flowing blood provides a substantial challenge to the platelet imaging field. The platelet's size (~2 μm), rapid activation (milliseconds), and unsuitability for genetic manipulation, means that appropriate imaging tools are limited. However, with the application of modern imaging systems to study platelet function, our understanding of molecular events mediating platelet adhesion from a single-cell perspective, to platelet recruitment and activation, leading to thrombus (clot) formation has expanded dramatically. This review will discuss current platelet imaging techniques in vitro and in vivo, describing how the advancements in imaging have helped answer/expand on platelet biology with a particular focus on hemostasis. We will focus on platelet aggregation and thrombus formation, and how platelet imaging has enhanced our understanding of key events, highlighting the knowledge gained through the application of imaging modalities to experimental models in vitro and in vivo. Furthermore, we will review the limitations of current imaging techniques, and questions in thrombosis research that remain to be addressed. Finally, we will speculate how the same imaging advancements might be applied to the imaging of other vascular cell biological functions and visualization of dynamic cell-cell interactions.

Fluorescence imaging reversion using spatially variant deconvolution

Fluorescence imaging opens new possibilities for intraoperative guidance and early cancer detection, in particular when using agents that target specific disease features. Nevertheless, photon scattering in tissue degrades image quality and leads to ambiguity in fluorescence image interpretation and challenges clinical translation. We introduce the concept of capturing the spatially-dependent impulse response of an image and investigate Spatially Adaptive Impulse Response Correction (SAIRC), a method that is proposed for improving the accuracy and sensitivity achieved. Unlike classical methods that presume a homogeneous spatial distribution of optical properties in tissue, SAIRC explicitly measures the optical heterogeneity in tissues. This information allows, for the first time, the application of spatially-dependent deconvolution to correct the fluorescence images captured in relation to their modification by photon scatter. Using experimental measurements from phantoms and animals, we investigate the improvement in resolution and quantification over non-corrected images. We discuss how the proposed method is essential for maximizing the performance of fluorescence molecular imaging in the clinic.

Numerical approach for reducing out-of-focus light in bright-field fluorescence microscopy and superresolution speckle microscopy

The standard two-dimensional (2D) image recorded in bright-field fluorescence microscopy is rigorously modeled by a convolution process involving a three-dimensional (3D) sample and a 3D point spread function. We show on synthetic and experimental data that deconvolving the 2D image using the appropriate 3D point spread function reduces the contribution of the out-of-focus fluorescence, resulting in a better image contrast and resolution. This approach is particularly interesting for superresolution speckle microscopy, in which the resolution gain stems directly from the efficiency of the deconvolution of each speckle image.

Live-Cell Fluorescence Microscopy to Investigate Subcellular Protein Localization and Cell Morphology Changes in Bacteria

Investigations of factors influencing cell division and cell shape in bacteria are commonly performed in conjunction with high-resolution fluorescence microscopy as observations made at a population level may not truly reflect what occurs at a single cell level. Live-cell timelapse microscopy allows investigators to monitor the changes in cell division or cell morphology which provide valuable insights regarding subcellular localization of proteins and timing of gene expression, as it happens, to potentially aid in answering important biological questions. Here, we describe our protocol to monitor phenotypic changes in Bacillus subtilis and Staphylococcus aureus using a high-resolution deconvolution microscope. The objective of this report is to provide a simple and clear protocol that can be adopted by other investigators interested in conducting fluorescence microscopy experiments to study different biological processes in bacteria as well as other organisms.

Fluorescence anisotropy imaging in drug discovery

Non-invasive measurement of drug-target engagement can provide critical insights in the molecular pharmacology of small molecule drugs. Fluorescence polarization/fluorescence anisotropy measurements are commonly employed in protein/cell screening assays. However, the expansion of such measurements to the in vivo setting has proven difficult until recently. With the advent of high-resolution fluorescence anisotropy microscopy it is now possible to perform kinetic measurements of intracellular drug distribution and target engagement in commonly used mouse models. In this review we discuss the background, current advances and future perspectives in intravital fluorescence anisotropy measurements to derive pharmacokinetic and pharmacodynamic measurements in single cells and whole organs.

Hyperspectral Imaging Bioinspired by Chromatic Blur Vision in Color Blind Animals

Hyperspectral imaging remote sensing is mutually restricted in terms of spatial and spectral resolutions, signal-to-noise ratio and exposure time. To deal with this trade-off properly, it is beneficial for imaging systems to have high light flux. In this paper, we put forward a novel hyperspectral imaging method with high light flux bioinspired by chromatic blur vision in color blind animals. We designed a camera lens with high degree of longitudinal chromatic aberration, a monochrome image sensor captured the chromatic blur images at different focal lengths. Finally, by using the known point spread functions of the chromatic blur imaging system, we process these chromatically blurred images by deconvolution based on singular value decomposition inverse filtering, and the spectral images of a target were restored. We constructed three different targets for validating image restoration based on a typical octopus eyeball imaging system. The results show that the proposed imaging method can effectively extract spectral images from the chromatically blurred images. This study can facilitate development of a novel bionic hyperspectral imaging, which may benefit from the high light flux of a large aperture and provide higher detection sensitivity.

Modeling Point Spread Function in Fluorescence Microscopy With a Sparse Gaussian Mixture: Tradeoff Between Accuracy and Efficiency

Deblurring is a fundamental inverse problem in bioimaging. It requires modeling the point spread function (PSF), which captures the optical distortions entailed by the image formation process. The PSF limits the spatial resolution attainable for a given microscope. However, recent applications require a higher resolution and have prompted the development of super-resolution techniques to achieve sub-pixel accuracy. This requirement restricts the class of suitable PSF models to analog ones. In addition, deblurring is computationally intensive, hence further requiring computationally efficient models. A custom candidate fitting both the requirements is the Gaussian model. However, this model cannot capture the rich tail structures found in both the theoretical and empirical PSFs. In this paper, we aim at improving the reconstruction accuracy beyond the Gaussian model, while preserving its computational efficiency. We introduce a new class of analog PSF models based on the Gaussian mixtures. The number of Gaussian kernels controls both the modeling accuracy and the computational efficiency of the model: the lower the number of kernels, the lower the accuracy and the higher the efficiency. To explore the accuracy-efficiency tradeoff, we propose a variational formulation of the PSF calibration problem, where a convex sparsity-inducing penalty on the number of Gaussian kernels allows trading accuracy for efficiency. We derive an efficient algorithm based on a fully split formulation of alternating split Bregman. We assess our framework on synthetic and real data, and demonstrate a better reconstruction accuracy in both geometry and photometry in point source localization-a fundamental inverse problem in fluorescence microscopy.

Superresolution method for a single wide-field image deconvolution by superposition of point sources

In this work, we present a new algorithm for wide-field fluorescent micrsocopy deconvolution from a single acquisition without a sparsity prior, which allows the retrieval of the target function with superresolution, with a simple approach that the measured data are fit by the convolution of a superposition of virtual point sources (SUPPOSe) of equal intensity with the point spread function. The cloud of virtual point sources approximates the actual distribution of sources that can be discrete or continuous. In this manner, only the positions of the sources need to be determined. An upper bound for the uncertainty in the position of the sources was derived, which provides a criteria to distinguish real facts from eventual artefacts and distortions. Two very different experimental situations were used for the test (an artificially synthesized image and fluorescent microscopy images), showing excellent reconstructions and agreement with the predicted uncertainties, achieving up to a fivefold improvement in the resolution for the microscope. The method also provides the optimum number of sources to be used for the fit. LAY DESCRIPTION: A new method is presented that allows the reconstruction of an image with superresolution from a single frame taken with a standard fluorescent microscope. An improvement in the resolution of a factor between 3 and 5 is achieved depending on the noise of the measurement and how precisely the instrument response function (point spread function) is measured. The complete mathematical description is presented showing how to estimate the quality of the reconstruction. The method is based in the approximation of the actual intensity distribution of the object being measured by a superposition of point sources of equal intensity. The problem is converted from determining the intensity of each point to determining the position of the virtual sources. The best fit is found using a genetic algorithm. To validate the method several results of different nature are presented including an artificially generated image, fluorescent beads and labelled mitochondria. The artificial image provides a prior knowledge of the actual system for comparison and validation. The beads were imaged with our highest numerical aperture objective to show method capabilities and also acquired with a low numerical aperture objective to compare the reconstructed image with that acquired with a high numerical aperture objective. This same strategy was followed with the biological sample to show the method working in real practical situations.