Deep learning-based autofocus method enhances image quality in light-sheet fluorescence microscopy

Active remote focus stabilization in oblique plane microscopy

Light-sheet fluorescence microscopy (LSFM) has demonstrated great potential in the life sciences owing to its efficient volumetric imaging capabilities. For long-term imaging, the light-sheet typically needs to be stabilized to the detection focal plane for the best imaging results. Current light-sheet stabilization methods rely on fluorescence emission from the sample, which may interrupt scientific imaging and add to sample photobleaching. Here, we show that for oblique plane microscopes (OPM), a subset of LSFM where a single primary objective is used for illumination and detection, light-sheet stabilization can be achieved without expending sample fluorescence. Our method achieves ∼21 nm axial precision and maintains the light-sheet well within the depth of focus of the detection system for hour-long acquisition runs in a lab environment that would otherwise detune the system. We demonstrate subcellular imaging of the actin skeleton in melanoma cancer cells with a stabilized OPM.

GJFocuser: a Gaussian difference and joint learning-based autofocus method for whole slide imaging

Whole slide imaging (WSI) provides tissue visualization at the cellular level, thereby enhancing the effectiveness of computer-aided diagnostic systems. High-precision autofocusing methods are essential for ensuring the quality of WSI. However, the accuracy of existing autofocusing techniques can be notably affected by variations in staining and sample heterogeneity, particularly without the addition of extra hardware. This study proposes a robust autofocusing method based on the difference between Gaussians (DoG) and joint learning. The DoG emphasizes image edge information that is closely related to focal distance, thereby mitigating the influence of staining variations. The joint learning framework constrains the network's sensitivity to defocus distance, effectively addressing the impact of the differences in sample morphology. We first conduct comparative experiments on public datasets against state-of-the-art methods, with results indicating that our approach achieves cutting-edge performance. Subsequently, we apply this method in a low-cost digital microscopy system, showcasing its effectiveness and versatility in practical scenarios.

Active Remote Focus Stabilization in Oblique Plane Microscopy

Light-sheet fluorescence microscopy (LSFM) has demonstrated great potential in the life sciences owing to its efficient volumetric imaging capabilities. For long term imaging, the light-sheet typically needs to be stabilized to the detection focal plane for the best imaging results. Current light-sheet stabilization methods rely on fluorescence emission from the sample, which may interrupt the scientific imaging and add to sample photobleaching. Here we show that for oblique plane microscopes (OPM), a subset of LSFM where a single primary objective is used for illumination and detection, light-sheet stabilization can be achieved without expending sample fluorescence. Our method achieves ~43nm axial precision and maintains the light-sheet well within the depth of focus of the detection system for hour-long acquisition runs in a lab environment that would otherwise detune the system. We demonstrate subcellular imaging of the actin skeleton in melanoma cancer cells with a stabilized OPM.

Posterior approach to correct for focal plane offsets in lattice light-sheet structured illumination microscopy

Significance

Lattice light-sheet structured illumination microscopy (latticeSIM) has proven highly effective in producing three-dimensional images with super resolution rapidly and with minimal photobleaching. However, due to the use of two separate objectives, sample-induced aberrations can result in an offset between the planes of excitation and detection, causing artifacts in the reconstructed images.

Aim

We introduce a posterior approach to detect and correct the axial offset between the excitation and detection focal planes in latticeSIM and provide a method to minimize artifacts in the reconstructed images.

Approach

We utilized the residual phase information within the overlap regions of the laterally shifted structured illumination microscopy information components in frequency space to retrieve the axial offset between the excitation and the detection focal planes in latticeSIM.

Results

We validated our technique through simulations and experiments, encompassing a range of samples from fluorescent beads to subcellular structures of adherent cells. We also show that using transfer functions with the same axial offset as the one present during data acquisition results in reconstructed images with minimal artifacts and salvages otherwise unusable data.

Conclusion

We envision that our method will be a valuable addition to restore image quality in latticeSIM datasets even for those acquired under non-ideal experimental conditions.

Intelligent Beam Optimization for Light-Sheet Fluorescence Microscopy through Deep Learning

Light-sheet fluorescence microscopy (LSFM) provides the benefit of optical sectioning coupled with rapid acquisition times, enabling high-resolution 3-dimensional imaging of large tissue-cleared samples. Inherent to LSFM, the quality of the imaging heavily relies on the characteristics of the illumination beam, which only illuminates a thin section of the sample. Therefore, substantial efforts are dedicated to identifying slender, nondiffracting beam profiles that yield uniform and high-contrast images. An ongoing debate concerns the identification of optimal illumination beams for different samples: Gaussian, Bessel, Airy patterns, and/or others. However, comparisons among different beam profiles are challenging as their optimization objectives are often different. Given that our large imaging datasets (approximately 0.5 TB of images per sample) are already analyzed using deep learning models, we envisioned a different approach to the problem by designing an illumination beam tailored to boost the performance of the deep learning model. We hypothesized that integrating the physical LSFM illumination model (after passing it through a variable phase mask) into the training of a cell detection network would achieve this goal. Here, we report that joint optimization continuously updates the phase mask and results in improved image quality for better cell detection. The efficacy of our method is demonstrated through both simulations and experiments that reveal substantial enhancements in imaging quality compared to the traditional Gaussian light sheet. We discuss how designing microscopy systems through a computational approach provides novel insights for advancing optical design that relies on deep learning models for the analysis of imaging datasets.

A posterior approach to correct for focal plane offsets in lattice light sheet structured illumination microscopy

Significance

Lattice light sheet structured illumination microscopy (latticeSIM) has proven highly effective in producing 3D images with super resolution rapidly and with minimal photobleaching. However, due to the use of two separate objectives, sample-induced aberrations can result in an offset between the planes of excitation and detection, causing artifacts in the reconstructed images.

Aim

We introduce a posterior approach to detect and correct for the axial offset between the excitation and detection focal planes in latticeSIM and provide a method to minimize artifacts in the reconstructed images.

Approach

We utilized the residual phase information within the overlap regions of the laterally shifted structured illumination microscopy (SIM) information components in frequency space to retrieve the axial offset between the excitation and the detection focal planes in latticeSIM.

Results

We validated our technique through simulations and experiments, encompassing a range of samples from fluorescent beads to subcellular structures of adherent cells. We also show utilizing transfer functions with the same axial offset as that which was present during the data acquisition results in reconstructed images with minimal artifacts and salvages otherwise unusable data.

Conclusion

We envision that our method will be a valuable addition to restore image quality in latticeSIM datasets even for those acquired under non-ideal experimental conditions.

DeepFocus: fast focus and astigmatism correction for electron microscopy

High-throughput 2D and 3D scanning electron microscopy, which relies on automation and dependable control algorithms, requires high image quality with minimal human intervention. Classical focus and astigmatism correction algorithms attempt to explicitly model image formation and subsequently aberration correction. Such models often require parameter adjustments by experts when deployed to new microscopes, challenging samples, or imaging conditions to prevent unstable convergence, making them hard to use in practice or unreliable. Here, we introduce DeepFocus, a purely data-driven method for aberration correction in scanning electron microscopy. DeepFocus works under very low signal-to-noise ratio conditions, reduces processing times by more than an order of magnitude compared to the state-of-the-art method, rapidly converges within a large aberration range, and is easily recalibrated to different microscopes or challenging samples.

Surgical procedure of intratympanic injection and inner ear pharmacokinetics simulation in domestic pigs

Introduction: One major obstacle in validating drugs for the treatment or prevention of hearing loss is the limited data available on the distribution and concentration of drugs in the human inner ear. Although small animal models offer some insights into inner ear pharmacokinetics, their smaller organ size and different barrier (round window membrane) permeabilities compared to humans can complicate study interpretation. Therefore, developing a reliable large animal model for inner ear drug delivery is crucial. The inner and middle ear anatomy of domestic pigs closely resembles that of humans, making them promising candidates for studying inner ear pharmacokinetics. However, unlike humans, the anatomical orientation and tortuosity of the porcine external ear canal frustrates local drug delivery to the inner ear. Methods: In this study, we developed a surgical technique to access the tympanic membrane of pigs. To assess hearing pre- and post-surgery, auditory brainstem responses to click and pure tones were measured. Additionally, we performed 3D segmentation of the porcine inner ear images and used this data to simulate the diffusion of dexamethasone within the inner ear through fluid simulation software (FluidSim). Results: We have successfully delivered dexamethasone and dexamethasone sodium phosphate to the porcine inner ear via the intratympanic injection. The recorded auditory brainstem measurements revealed no adverse effects on hearing thresholds attributable to the surgery. We have also simulated the diffusion rates for dexamethasone and dexamethasone sodium phosphate into the porcine inner ear and confirmed the accuracy of the simulations using in-vivo data. Discussion: We have developed and characterized a method for conducting pharmacokinetic studies of the inner ear using pigs. This animal model closely mirrors the size of the human cochlea and the thickness of its barriers. The diffusion time and drug concentrations we reported align closely with the limited data available from human studies. Therefore, we have demonstrated the potential of using pigs as a large animal model for studying inner ear pharmacokinetics.

Raman signal optimization based on residual network adaptive focusing

Due to its high sensitivity and specificity, Micro-Raman spectroscopy has emerged as a vital technique for molecular recognition and identification. As a weakly scattered signal, ensuring the accurate focus of the sample is essential for acquiring high quality Raman spectral signal and its analysis, especially in some complex microenvironments such as intracellular settings. Traditional autofocus methods are often time consuming or necessitate additional hardware, limiting real-time sample observation and device compatibility. Here, we propose an adaptive focusing method based on residual network to realize rapid and accurate focusing on Micro-Raman measurements. Using only a bright field image of the sample acquired on any image plane, we can predict the defocus distance with a residual network trained by Resnet50, in which the focus position is determined by combining the gradient and discrete cosine transform. Further, detailed regional division of the bright field map used for characterizing the height variation of actual sample surface is performed. As a result, a focus prediction map with 1μm accuracy is obtained from a bright field image in 120 ms. Based on this method, we successfully realize Raman signal optimization and the necessary correction of spectral information. This adaptive focusing method based on residual network is beneficial to further enhance the sensitivity and accuracy of Micro-Raman spectroscopy technology, which is of great significance in promoting the wide application of Raman spectroscopy.

High-precision microscopic autofocus with a single natural image

In industrial microscopic detection, learning-based autofocus methods have empowered operators to acquire high-quality images quickly. However, there are two parts of errors in Learning-based methods: the fitting error of the network model and the making error of the prior dataset, which limits the potential for further improvements in focusing accuracy. In this paper, a high-precision autofocus pipeline was introduced, which predicts the defocus distance from a single natural image. A new method for making datasets was proposed, which overcomes the limitations of the sharpness metric itself and improves the overall accuracy of the dataset. Furthermore, a lightweight regression network was built, namely Natural-image Defocus Prediction Model (NDPM), to improve the focusing accuracy. A realistic dataset of sufficient size was made to train all models. The experiment shows NDPM has better focusing performance compared with other models, with a mean focusing error of 0.422µm.

Enhancing Light-Sheet Fluorescence Microscopy Illumination Beams through Deep Design Optimization

Light sheet fluorescence microscopy (LSFM) provides the benefit of optical sectioning coupled with rapid acquisition times for imaging of tissue-cleared specimen. This allows for high-resolution 3D imaging of large tissue volumes. Inherently to LSFM, the quality of the imaging heavily relies on the characteristics of the illumination beam, with the notion that the illumination beam only illuminates a thin section that is being imaged. Therefore, substantial efforts are dedicated to identifying slender, non-diffracting beam profiles that can yield uniform and high-contrast images. An ongoing debate concerns the employment of the most optimal illumination beam; Gaussian, Bessel, Airy patterns and/or others. Comparisons among different beam profiles is challenging as their optimization objective is often different. Given that our large imaging datasets (~0.5TB images per sample) is already analyzed using deep learning models, we envisioned a different approach to this problem by hypothesizing that we can tailor the illumination beam to boost the deep learning models performance. We achieve this by integrating the physical LSFM illumination model after passing through a variable phase mask into the training of a cell detection network. Here we report that the joint optimization continuously updates the phase mask, improving the image quality for better cell detection. Our method's efficacy is demonstrated through both simulations and experiments, revealing substantial enhancements in imaging quality compared to traditional Gaussian light sheet. We offer valuable insights for designing microscopy systems through a computational approach that exhibits significant potential for advancing optics design that relies on deep learning models for analysis of imaging datasets.

Autofocus methods based on laser illumination

Autofocusing system plays an important role in microscopic measurement. However, natural-image-based autofocus methods encounter difficulties in improving focusing accuracy and robustness due to the diversity of detection objects. In this paper, a high-precision autofocus method with laser illumination was proposed, termed laser split-image autofocus (LSA), which actively endows the detection scene with image features. The common non-learning-based and learning-based methods for LSA were quantitatively analyzed and evaluated. Furthermore, a lightweight comparative framework model for LSA, termed split-image comparison model (SCM), was proposed to further improve the focusing accuracy and robustness, and a realistic split-image dataset of sufficient size was made to train all models. The experiment showed LSA has better focusing performance than natural-image-based method. In addition, SCM has a great improvement in accuracy and robustness compared with previous learning and non-learning methods, with a mean focusing error of 0.317µm in complex scenes. Therefore, SCM is more suitable for industrial measurement.

Focus prediction of medical microscopic images based on Lightweight Densely Connected with Squeeze-and-Excitation Network

Due to the demand for sample observation, optical microscopy has become an essential tool in the fields of biology and medicine. In addition, it is impossible to maintain the living sample in focus over long-time observation. Rapid focus prediction which involves moving a microscope stage along a vertical axis to find an optimal focus position, is a critical step for high-quality microscopic imaging of specimens. Current focus prediction algorithms, which are time-consuming, cannot support high frame rate imaging of dynamic living samples, and may introduce phototoxicity and photobleaching on the samples. In this paper, we propose Lightweight Densely Connected with Squeeze-and-Excitation Network (LDSE-NET). The results of the focusing algorithm are demonstrated on a public dataset and a self-built dataset. A complete evaluation system was constructed to compare and analyze the effectiveness of LDSE-NET, BotNet, and ResNet50 models in multi-region and multi-multiplier prediction. Experimental results show that LDSE-NET is reduced to 1E-05 of the root mean square error. The accuracy of the predicted focal length of the image is increased by 1 ~ 2 times. Training time is reduced by 33.3%. Moreover, the volume of the model only reaches the KB level, which has the characteristics of being lightweight.

Deep learning-based adaptive optics for light sheet fluorescence microscopy

Light sheet fluorescence microscopy (LSFM) is a high-speed imaging technique that is often used to image intact tissue-cleared specimens with cellular or subcellular resolution. Like other optical imaging systems, LSFM suffers from sample-induced optical aberrations that decrement imaging quality. Optical aberrations become more severe when imaging a few millimeters deep into tissue-cleared specimens, complicating subsequent analyses. Adaptive optics are commonly used to correct sample-induced aberrations using a deformable mirror. However, routinely used sensorless adaptive optics techniques are slow, as they require multiple images of the same region of interest to iteratively estimate the aberrations. In addition to the fading of fluorescent signal, this is a major limitation as thousands of images are required to image a single intact organ even without adaptive optics. Thus, a fast and accurate aberration estimation method is needed. Here, we used deep-learning techniques to estimate sample-induced aberrations from only two images of the same region of interest in cleared tissues. We show that the application of correction using a deformable mirror greatly improves image quality. We also introduce a sampling technique that requires a minimum number of images to train the network. Two conceptually different network architectures are compared; one that shares convolutional features and another that estimates each aberration independently. Overall, we have presented an efficient way to correct aberrations in LSFM and to improve image quality.

COMBINe enables automated detection and classification of neurons and astrocytes in tissue-cleared mouse brains

Tissue clearing renders entire organs transparent to accelerate whole-tissue imaging; for example, with light-sheet fluorescence microscopy. Yet, challenges remain in analyzing the large resulting 3D datasets that consist of terabytes of images and information on millions of labeled cells. Previous work has established pipelines for automated analysis of tissue-cleared mouse brains, but the focus there was on single-color channels and/or detection of nuclear localized signals in relatively low-resolution images. Here, we present an automated workflow (COMBINe, Cell detectiOn in Mouse BraIN) to map sparsely labeled neurons and astrocytes in genetically distinct mouse forebrains using mosaic analysis with double markers (MADM). COMBINe blends modules from multiple pipelines with RetinaNet at its core. We quantitatively analyzed the regional and subregional effects of MADM-based deletion of the epidermal growth factor receptor (EGFR) on neuronal and astrocyte populations in the mouse forebrain.

Tissue clearing and three-dimensional imaging of the whole cochlea and vestibular system from multiple large-animal models

The inner ear of humans and large animals is embedded in a thick and dense bone that makes dissection challenging. Here, we present a protocol that enables three-dimensional (3D) characterization of intact inner ears from large-animal models. We describe steps for decalcifying bone, using solvents to remove color and lipids, and imaging tissues in 3D using confocal and light sheet microscopy. We then detail a pipeline to count hair cells in antibody-stained and 3D imaged cochleae using open-source software. For complete details on the use and execution of this protocol, please refer to (Moatti et al., 2022).¹.

Bulk and mosaic deletions of Egfr reveal regionally defined gliogenesis in the developing mouse forebrain

The epidermal growth factor receptor (EGFR) plays a role in cell proliferation and differentiation during healthy development and tumor growth; however, its requirement for brain development remains unclear. Here we used a conditional mouse allele for Egfr to examine its contributions to perinatal forebrain development at the tissue level. Subtractive bulk ventral and dorsal forebrain deletions of Egfr uncovered significant and permanent decreases in oligodendrogenesis and myelination in the cortex and corpus callosum. Additionally, an increase in astrogenesis or reactive astrocytes in effected regions was evident in response to cortical scarring. Sparse deletion using mosaic analysis with double markers (MADM) surprisingly revealed a regional requirement for EGFR in rostrodorsal, but not ventrocaudal glial lineages including both astrocytes and oligodendrocytes. The EGFR-independent ventral glial progenitors may compensate for the missing EGFR-dependent dorsal glia in the bulk Egfr-deleted forebrain, potentially exposing a regenerative population of gliogenic progenitors in the mouse forebrain.

Armour G, Horman BM, Rock KD, Witchey SK, Greenbaum A, Patisaul HB. Maternal organophosphate flame retardant exposure alters the developing mesencephalic dopamine system in fetal rat

Organophosphate flame retardants (OPFRs) have become the predominant substitution for legacy brominated flame retardants but there is concern about their potential developmental neurotoxicity (DNT). OPFRs readily dissociate from the fireproofed substrate to the environment, and they (or their metabolites) have been detected in diverse matrices including air, water, soil, and biota, including human urine and breastmilk. Given this ubiquitous contamination, it becomes increasingly important to understand the potential effects of OPFRs on the developing nervous system. We have previously shown that maternal exposure to OPFRs results in neuroendocrine disruption, alterations to developmental metabolism of serotonin (5-HT) and axonal extension in male fetal rats, and potentiates adult anxiety-like behaviors. The development of the serotonin and dopamine systems occur in parallel and interact, therefore, we first sought to enhance our prior 5-HT work by first examining the ascending 5-HT system on embryonic day 14 using whole mount clearing of fetal heads and 3-dimensional (3D) brain imaging. We also investigated the effects of maternal OPFR exposure on the development of the mesocortical dopamine system in the same animals through 2-dimensional and 3D analysis following immunohistochemistry for tyrosine hydroxylase (TH). Maternal OPFR exposure induced morphological changes to the putative ventral tegmental area and substantia nigra in both sexes and reduced the overall volume of this structure in males, whereas 5-HT nuclei were unchanged. Additionally, dopaminergic axogenesis was disrupted in OPFR exposed animals, as the dorsoventral spread of ventral telencephalic TH afferents were greater at embryonic day 14, while sparing 5-HT fibers. These results indicate maternal exposure to OPFRs alters the development trajectory of the embryonic dopaminergic system and adds to growing evidence of OPFR DNT.

Structural and functional imaging of brains

Analyzing the complex structures and functions of brain is the key issue to understanding the physiological and pathological processes. Although neuronal morphology and local distribution of neurons/blood vessels in the brain have been known, the subcellular structures of cells remain challenging, especially in the live brain. In addition, the complicated brain functions involve numerous functional molecules, but the concentrations, distributions and interactions of these molecules in the brain are still poorly understood. In this review, frontier techniques available for multiscale structure imaging from organelles to the whole brain are first overviewed, including magnetic resonance imaging (MRI), computed tomography (CT), positron emission tomography (PET), serial-section electron microscopy (ssEM), light microscopy (LM) and synchrotron-based X-ray microscopy (XRM). Specially, XRM for three-dimensional (3D) imaging of large-scale brain tissue with high resolution and fast imaging speed is highlighted. Additionally, the development of elegant methods for acquisition of brain functions from electrical/chemical signals in the brain is outlined. In particular, the new electrophysiology technologies for neural recordings at the single-neuron level and in the brain are also summarized. We also focus on the construction of electrochemical probes based on dual-recognition strategy and surface/interface chemistry for determination of chemical species in the brain with high selectivity and long-term stability, as well as electrochemophysiological microarray for simultaneously recording of electrochemical and electrophysiological signals in the brain. Moreover, the recent development of brain MRI probes with high contrast-to-noise ratio (CNR) and sensitivity based on hyperpolarized techniques and multi-nuclear chemistry is introduced. Furthermore, multiple optical probes and instruments, especially the optophysiological Raman probes and fiber Raman photometry, for imaging and biosensing in live brain are emphasized. Finally, a brief perspective on existing challenges and further research development is provided.

Deep learning in single-molecule imaging and analysis: recent advances and prospects

Single-molecule microscopy is advantageous in characterizing heterogeneous dynamics at the molecular level. However, there are several challenges that currently hinder the wide application of single molecule imaging in bio-chemical studies, including how to perform single-molecule measurements efficiently with minimal run-to-run variations, how to analyze weak single-molecule signals efficiently and accurately without the influence of human bias, and how to extract complete information about dynamics of interest from single-molecule data. As a new class of computer algorithms that simulate the human brain to extract data features, deep learning networks excel in task parallelism and model generalization, and are well-suited for handling nonlinear functions and extracting weak features, which provide a promising approach for single-molecule experiment automation and data processing. In this perspective, we will highlight recent advances in the application of deep learning to single-molecule studies, discuss how deep learning has been used to address the challenges in the field as well as the pitfalls of existing applications, and outline the directions for future development.

Bionic vision autofocus method based on a liquid lens

Digital imaging systems (DISs) have been widely used in industrial process control, field monitoring, and other domains, and the autofocusing capability of DISs is a key factor affecting the imaging quality and intelligence of the system. In view of the deficiencies of focusing accuracy and speed in current imaging systems, this paper proposes a fast autofocus method of bionic vision on the basis of the liquid lens. First, the sharpness recognition network and sharpness comparison network are designed based on the consideration of a human visual focusing mechanism. Then a sharpness evaluation function combined with the distance-aware algorithm and an adaptive focusing search algorithm are proposed. These lead to the construction of our proposed autofocus method with the introduction of the memory mechanism. In order to verify the effectiveness of the proposed method, an experimental platform based on a liquid lens is built to test its performance. Experiment confirms that the proposed autofocus method has obvious advantages in robustness, accuracy, and speed compared with traditional methods.

Ontogeny of cellular organization and LGR5 expression in porcine cochlea revealed using tissue clearing and 3D imaging

Over 11% of the world's population experience hearing loss. Although there are promising studies to restore hearing in rodent models, the size, ontogeny, genetics, and frequency range of hearing of most rodents' cochlea do not match that of humans. The porcine cochlea can bridge this gap as it shares many anatomical, physiological, and genetic similarities with its human counterpart. Here, we provide a detailed methodology to process and image the porcine cochlea in 3D using tissue clearing and light-sheet microscopy. The resulting 3D images can be employed to compare cochleae across different ages and conditions, investigate the ontogeny of cochlear cytoarchitecture, and produce quantitative expression maps of LGR5, a marker of cochlear progenitors in mice. These data reveal that hair cell organization, inner ear morphology, cellular cartography in the organ of Corti, and spatiotemporal expression of LGR5 are dynamic over developmental stages in a pattern not previously documented.

Quantitative analysis of illumination and detection corrections in adaptive light sheet fluorescence microscopy

Light-sheet fluorescence microscopy (LSFM) is a high-speed, high-resolution and minimally phototoxic technique for 3D imaging of in vivo and in vitro specimens. LSFM exhibits optical sectioning and when combined with tissue clearing techniques, it facilitates imaging of centimeter scale specimens with micrometer resolution. Although LSFM is ubiquitous, it still faces two main challenges that effect image quality especially when imaging large volumes with high-resolution. First, the light-sheet illumination plane and detection lens focal plane need to be coplanar, however sample-induced aberrations can violate this requirement and degrade image quality. Second, introduction of sample-induced optical aberrations in the detection path. These challenges intensify when imaging whole organisms or structurally complex specimens like cochleae and bones that exhibit many transitions from soft to hard tissue or when imaging deep (> 2 mm). To resolve these challenges, various illumination and aberration correction methods have been developed, yet no adaptive correction in both the illumination and the detection path have been applied to improve LSFM imaging. Here, we bridge this gap, by implementing the two correction techniques on a custom built adaptive LSFM. The illumination beam angular properties are controlled by two galvanometer scanners, while a deformable mirror is positioned in the detection path to correct for aberrations. By imaging whole porcine cochlea, we compare and contrast these correction methods and their influence on the image quality. This knowledge will greatly contribute to the field of adaptive LSFM, and imaging of large volumes of tissue cleared specimens.

Microscopic Imaging Methods for Organ-on-a-Chip Platforms

Microscopic imaging is essential and the most popular method for in situ monitoring and evaluating the outcome of various organ-on-a-chip (OOC) platforms, including the number and morphology of mammalian cells, gene expression, protein secretions, etc. This review presents an overview of how various imaging methods can be used to image organ-on-a-chip platforms, including transillumination imaging (including brightfield, phase-contrast, and holographic optofluidic imaging), fluorescence imaging (including confocal fluorescence and light-sheet fluorescence imaging), and smartphone-based imaging (including microscope attachment-based, quantitative phase, and lens-free imaging). While various microscopic imaging methods have been demonstrated for conventional microfluidic devices, a relatively small number of microscopic imaging methods have been demonstrated for OOC platforms. Some methods have rarely been used to image OOCs. Specific requirements for imaging OOCs will be discussed in comparison to the conventional microfluidic devices and future directions will be introduced in this review.

Illumination angle correction during image acquisition in light-sheet fluorescence microscopy using deep learning

Light-sheet fluorescence microscopy (LSFM) is a high-speed imaging technique that provides optical sectioning with reduced photodamage. LSFM is routinely used in life sciences for live cell imaging and for capturing large volumes of cleared tissues. LSFM has a unique configuration, in which the illumination and detection paths are separated and perpendicular to each other. As such, the image quality, especially at high resolution, largely depends on the degree of overlap between the detection focal plane and the illuminating beam. However, spatial heterogeneity within the sample, curved specimen boundaries, and mismatch of refractive index between tissues and immersion media can refract the well-aligned illumination beam. This refraction can cause extensive blur and non-uniform image quality over the imaged field-of-view. To address these issues, we tested a deep learning-based approach to estimate the angular error of the illumination beam relative to the detection focal plane. The illumination beam was then corrected using a pair of galvo scanners, and the correction significantly improved the image quality across the entire field-of-view. The angular estimation was based on calculating the defocus level on a pixel level within the image using two defocused images. Overall, our study provides a framework that can correct the angle of the light-sheet and improve the overall image quality in high-resolution LSFM 3D image acquisition.

Erratum: Deep learning-based autofocus method enhances image quality in light-sheet fluorescence microscopy: publisher's note

[This corrects the article on p. 5214 in vol. 12, PMID: 34513252.].

Deep learning-based method for the continuous detection of heart rate in signals from a multi-fiber Bragg grating sensor compatible with magnetic resonance imaging

A method for the continuous detection of heart rate (HR) in signals acquired from patients using a sensor mat comprising a nine-element array of fiber Bragg gratings during routine magnetic resonance imaging (MRI) procedures is proposed. The method is based on a deep learning neural network model, which learned from signals acquired from 153 MRI patients. In addition, signals from 343 MRI patients were used for result verification. The proposed method provides automatic continuous extraction of HR with the root mean square error of 2.67 bpm, and the limits of agreement were -4.98-5.45 bpm relative to the reference HR.

Robust autofocusing for scanning electron microscopy based on a dual deep learning network

Scanning electron microscopy (SEM) is a high-resolution imaging technique with subnanometer spatial resolution that is widely used in materials science, basic science, and nanofabrication. However, conducting SEM is rather complex due to the nature of using an electron beam and the many parameters that must be adjusted to acquire high-quality images. Only trained operators can use SEM equipment properly, meaning that the use of SEM is restricted. To broaden the usability of SEM, we propose an autofocus method for a SEM system based on a dual deep learning network, which consists of an autofocusing-evaluation network (AENet) and an autofocusing-control network (ACNet). The AENet was designed to evaluate the quality of given images, with scores ranging from 0 to 9 regardless of the magnification. The ACNet can delicately control the focus of SEM online based on the AENet’s outputs for any lateral sample position and magnification. The results of these dual networks showed successful autofocus performance on three trained samples. Moreover, the robustness of the proposed method was demonstrated by autofocusing on unseen samples. We expect that our autofocusing system will not only contribute to expanding the versatility of SEM but will also be applicable to various microscopes.

Super-Resolution Microscopy: Shedding New Light on In Vivo Imaging

Over the past two decades, super-resolution microscopy (SRM), which offered a significant improvement in resolution over conventional light microscopy, has become a powerful tool to visualize biological activities in both fixed and living cells. However, completely understanding biological processes requires studying cells in a physiological context at high spatiotemporal resolution. Recently, SRM has showcased its ability to observe the detailed structures and dynamics in living species. Here we summarized recent technical advancements in SRM that have been successfully applied to in vivo imaging. Then, improvements in the labeling strategies are discussed together with the spectroscopic and chemical demands of the fluorophores. Finally, we broadly reviewed the current applications for super-resolution techniques in living species and highlighted some inherent challenges faced in this emerging field. We hope that this review could serve as an ideal reference for researchers as well as beginners in the relevant field of in vivo super resolution imaging.