Applications, promises, and pitfalls of deep learning for fluorescence image reconstruction

Distinct actin microfilament localization during early cell plate formation through deep learning-based image restoration

KEY MESSAGE

Using deep learning-based image restoration, we achieved high-resolution 4D imaging with minimal photodamage, revealing distinct localization and suggesting Lifeact-RFP-labeled actin microfilaments play a role in initiating cell plate formation. Phragmoplasts are plant-specific intracellular structures composed of microtubules, actin microfilaments (AFs), membranes, and associated proteins. Importantly, they are involved in the formation and the expansion of cell plates that partition daughter cells during cell division. While previous studies have revealed the important role of cytoskeletal dynamics in the proper functioning of the phragmoplast, the localization and the role of AFs in the initial phase of cell plate formation remain controversial. Here, we used deep learning-based image restoration to achieve high-resolution 4D imaging with minimal laser-induced damage, enabling us to investigate the dynamics of AFs during the initial phase of cell plate formation in transgenic tobacco BY-2 cells labeled with Lifeact-RFP or RFP-ABD2 (actin-binding domain 2). This computational approach overcame the limitation of conventional imaging, namely laser-induced photobleaching and phototoxicity. The restored images indicated that RFP-ABD2-labeled AFs were predominantly localized near the daughter nucleus, whereas Lifeact-RFP-labeled AFs were found not only near the daughter nucleus but also around the initial cell plate. These findings, validated by imaging with a long exposure time, highlight distinct localization patterns between the two AF probes and suggest that Lifeact-RFP-labeled AFs play a role in initiating cell plate formation.

Deep Equilibrium Unfolding Learning for Noise Estimation and Removal in Optical Molecular Imaging

In clinical optical molecular imaging, the need for real-time high frame rates and low excitation doses to ensure patient safety inherently increases susceptibility to detection noise. Faced with the challenge of image degradation caused by severe noise, image denoising is essential for mitigating the trade-off between acquisition cost and image quality. However, prevailing deep learning methods exhibit uncontrollable and suboptimal performance with limited interpretability, primarily due to neglecting underlying physical model and frequency information. In this work, we introduce an end-to-end model-driven Deep Equilibrium Unfolding Mamba (DEQ-UMamba) that integrates proximal gradient descent technique and learnt spatial-frequency characteristics to decouple complex noise structures into statistical distributions, enabling effective noise estimation and suppression in fluorescent images. Moreover, to address the computational limitations of unfolding networks, DEQ-UMamba trains an implicit mapping by directly differentiating the equilibrium point of the convergent solution, thereby ensuring stability and avoiding non-convergent behavior. With each network module aligned to a corresponding operation in the iterative optimization process, the proposed method achieves clear structural interpretability and strong performance. Comprehensive experiments conducted on both clinical and in vivo datasets demonstrate that DEQ-UMamba outperforms current state-of-the-art alternatives while utilizing fewer parameters, facilitating the advancement of cost-effective and high-quality clinical molecular imaging.

Data-driven signal-to-noise enhancement in scattering near-field infrared microscopy

This study introduces a machine-learning approach to enhance signal-to-noise ratios in scattering-type scanning near-field optical microscopy (s-SNOM). While s-SNOM offers a high spatial resolution, its effectiveness is often hindered by low signal levels, particularly in weakly absorbing samples. To address these challenges, we utilize a data-driven "patch-based" machine learning reconstruction method, incorporating modern generative adversarial neural networks (CycleGANs) for denoising s-SNOM images. This method allows for flexible reconstruction of images of arbitrary sizes, a critical capability given the variable nature of scanned sample areas in point-scanning probe-based microscopies. The CycleGAN model is trained on unpaired sets of images captured at both rapid and extended acquisition times, thereby modeling instrument noise while preserving essential topographical and molecular information. The results show significant improvements in image quality, as indicated by higher structural similarity index and peak signal-to-noise ratio values, comparable to those obtained from images captured with four times the integration time. This method not only enhances image quality but also has the potential to reduce the overall data acquisition time, making high-resolution s-SNOM imaging more feasible for a wide range of biological and materials science applications.

Deep-Optimal Leucorrhea Detection Through Fluorescent Benchmark Data Analysis

Vaginitis is a common condition in women that is described medically as irritation and/or inflammation of the vagina; it poses a significant health risk for women, necessitating precise diagnostic methods. Presently, conventional techniques for examining vaginal discharge involve the use of wet mounts and gram staining to identify vaginal diseases. In this research, we utilized fluorescent staining, which enables distinct visualization of cellular and pathogenic components, each exhibiting unique color characteristics when exposed to the same light source. We established a large, challenging multiple fluorescence leucorrhea dataset benchmark comprising 8 categories with a total of 343 K high-quality labels. We also presented a robust lightweight deep-learning network, LRNet. It includes a lightweight feature extraction network that employs Ghost modules, a feature pyramid network that incorporates deformable convolution in the neck, and a single detection head. The evaluation results indicate that this detection network surpasses conventional networks and can cut down the model parameters by up to 91.4% and floating-point operations (FLOPs) by 74%. The deep-optimal leucorrhea detection capability of LRNet significantly enhances its ability to detect various crucial indicators related to vaginal health.

Neural network-assisted localization of clustered point spread functions in single-molecule localization microscopy

Single-molecule localization microscopy (SMLM), which has revolutionized nanoscale imaging, faces challenges in densely labelled samples due to fluorophore clustering, leading to compromised localization accuracy. In this paper, we propose a novel convolutional neural network (CNN)-assisted approach to address the issue of locating the clustered fluorophores. Our CNN is trained on a diverse data set of simulated SMLM images where it learns to predict point spread function (PSF) locations by generating Gaussian blobs as output. Through rigorous evaluation, we demonstrate significant improvements in PSF localization accuracy, especially in densely labelled samples where traditional methods struggle. In addition, we employ blob detection as a post-processing technique to refine the predicted PSF locations and enhance localization precision. Our study underscores the efficacy of CNN in addressing clustering challenges in SMLM, thereby advancing spatial resolution and enabling deeper insights into complex biological structures.

In Vivo Neurodynamics Mapping via High-Speed Two-Photon Fluorescence Lifetime Volumetric Projection Microscopy

Monitoring the morphological and biochemical information of neurons and glial cells at high temporal resolution in three-dimensional (3D) volumes of in vivo is pivotal for understanding their structure and function, and quantifying the brain microenvironment. Conventional two-photon fluorescence lifetime volumetric imaging speed faces the acquisition speed challenges of slow serial focal tomographic scanning, complex post-processing procedures for lifetime images, and inherent trade-offs among contrast, signal-to-noise ratio, and speed. This study presents a two-photon fluorescence lifetime volumetric projection microscopy using an axially elongated Bessel focus and instant frequency-domain fluorescence lifetime technique, and integrating with a convolutional network to enhance the imaging speed for in vivo neurodynamics mapping. The proposed method is validated by monitoring intracellular Ca2+ concentration throughout whole volume, tracking microglia movement and microenvironmental changes following thermal injury in the zebrafish brain, analyzing structural and functional variations of gap junctions in astrocyte networks, and measuring the Ca2+ concentration in neurons in mouse brains. This innovative methodology enables quantitative in vivo visualization of neurodynamics and the cellular processes and interactions in the brain.

A neural network for long-term super-resolution imaging of live cells with reliable confidence quantification

Super-resolution (SR) neural networks transform low-resolution optical microscopy images into SR images. Application of single-image SR (SISR) methods to long-term imaging has not exploited the temporal dependencies between neighboring frames and has been subject to inference uncertainty that is difficult to quantify. Here, by building a large-scale fluorescence microscopy dataset and evaluating the propagation and alignment components of neural network models, we devise a deformable phase-space alignment (DPA) time-lapse image SR (TISR) neural network. DPA-TISR adaptively enhances the cross-frame alignment in the phase domain and outperforms existing state-of-the-art SISR and TISR models. We also develop Bayesian DPA-TISR and design an expected calibration error minimization framework that reliably infers inference confidence. We demonstrate multicolor live-cell SR imaging for more than 10,000 time points of various biological specimens with high fidelity, temporal consistency and accurate confidence quantification.

Aggregation-Induced Emission Luminogens Realizing High-Contrast Bioimaging

A revolutionary transformation in biomedical imaging is unfolding with the advent of aggregation-induced emission luminogens (AIEgens). These cutting-edge molecules not only overcome the limitations of traditional fluorescent probes but also improve the boundaries of high-contrast imaging. Unlike conventional fluorophores suffering from aggregation-caused quenching, AIEgens exhibit enhanced luminescence when aggregated, enabling superior imaging performance. This review delves into the molecular mechanisms of aggregation-induced emission (AIE), demonstrating how strategic molecular design unlocks exceptional luminescence and superior imaging contrast, which is crucial for distinguishing healthy and diseased tissues. This review also highlights key applications of AIEgens, such as time-resolved imaging, second near-infrared window (NIR-II), and the advancement of AIEgens in sensitivity to physical and biochemical cue-responsive imaging. The development of AIE technology promises to transform healthcare from early disease detection to targeted therapies, potentially reshaping personalized medicine. This paradigm shift in biophotonics offers efficient tools to decode the complexities of biological systems at the molecular level, bringing us closer to a future where the invisible becomes visible and the incurable becomes treatable.

Self-Driving Microscopes: AI Meets Super-Resolution Microscopy

The integration of Machine Learning (ML) with super-resolution microscopy represents a transformative advancement in biomedical research. Recent advances in ML, particularly deep learning (DL), have significantly enhanced image processing tasks, such as denoising and reconstruction. This review explores the growing potential of automation in super-resolution microscopy, focusing on how DL can enable autonomous imaging tasks. Overcoming the challenges of automation, particularly in adapting to dynamic biological processes and minimizing manual intervention, is crucial for the future of microscopy. Whilst still in its infancy, automation in super-resolution can revolutionize drug discovery and disease phenotyping leading to similar breakthroughs as have been recognized in this year's Nobel Prizes for Physics and Chemistry.

Unsupervised Learning-Assisted Acoustic-Driven Nano-Lens Holography for the Ultrasensitive and Amplification-Free Detection of Viable Bacteria

Bacterial infection is a crucial factor resulting in public health issues worldwide, often triggering epidemics and even fatalities. The accurate, rapid, and convenient detection of viable bacteria is an effective method for reducing infections and illness outbreaks. Here, an unsupervised learning-assisted and surface acoustic wave-interdigital transducer-driven nano-lens holography biosensing platform is developed for the ultrasensitive and amplification-free detection of viable bacteria. The monitoring device integrated with the nano-lens effect can achieve the holographic imaging of polystyrene microsphere probes in an ultra-wide field of view (∽28.28 mm2), with a sensitivity limit of as low as 99 nm. A lightweight unsupervised learning hologram processing algorithm considerably reduces training time and computing hardware requirements, without requiring datasets with manual labels. By combining phage-mediated viable bacterial DNA extraction and enhanced CRISPR-Cas12a systems, this strategy successfully achieves the ultrasensitive detection of viable Salmonella in various real samples, demonstrating enhanced accuracy validated with the qPCR benchmark method. This approach has a low cost (∽$500) and is rapid (∽1 h) and highly sensitive (∽38 CFU mL-1), allowing for the amplification-free detection of viable bacteria and emerging as a powerful tool for food safety inspection and clinical diagnosis.

Multi-Sensor Learning Enables Information Transfer Across Different Sensory Data and Augments Multi-Modality Imaging

Multi-modality imaging is widely used in clinical practice and biomedical research to gain a comprehensive understanding of an imaging subject. Currently, multi-modality imaging is accomplished by post hoc fusion of independently reconstructed images under the guidance of mutual information or spatially registered hardware, which limits the accuracy and utility of multi-modality imaging. Here, we investigate a data-driven multi-modality imaging (DMI) strategy for synergetic imaging of CT and MRI. We reveal two distinct types of features in multi-modality imaging, namely intra- and inter-modality features, and present a multi-sensor learning (MSL) framework to utilize the crossover inter-modality features for augmented multi-modality imaging. The MSL imaging approach breaks down the boundaries of traditional imaging modalities and allows for optimal hybridization of CT and MRI, which maximizes the use of sensory data. We showcase the effectiveness of our DMI strategy through synergetic CT-MRI brain imaging. The principle of DMI is quite general and holds enormous potential for various DMI applications across disciplines.

Unmixing Autoencoder for Image Reconstruction from Hyperspectral Data

Due to the complexity of samples and the limitations in spatial resolution, the spectra in hyperspectral imaging (HSI) are generally contributed to by multiple components, making univariate analysis ineffective. Although feature extraction methods have been applied, the chemical meaning of the compressed variables is difficult to interpret, limiting their further applications. An unmixing autoencoder (UAE) was developed in this work for the separation of the mixed spectra in HSI. The proposed model is composed of an encoder and a fully connected (FC) layer. The former is used to compress the input spectrum into several variables, and the latter is employed to reconstruct the spectrum. Combining reconstruction loss and sparse regularization, the weights and the spectral profiles of the components will be encoded in the compressed variables and the connection weights of FC, respectively. A simulated and three experimental HSI data sets were adopted to investigate the performance of the UAE model. The spectral components were successfully obtained, from which the handwriting under papers was revealed from the image of near-infrared (NIR) diffusive reflectance spectroscopy, and the images of lipids, proteins, and nucleic acids were reconstructed from the Raman and stimulated Raman scattering (SRS) images.

Unsupervised inter-domain transformation for virtually stained high-resolution mid-infrared photoacoustic microscopy using explainable deep learning

Mid-infrared photoacoustic microscopy can capture biochemical information without staining. However, the long mid-infrared optical wavelengths make the spatial resolution of photoacoustic microscopy significantly poorer than that of conventional confocal fluorescence microscopy. Here, we demonstrate an explainable deep learning-based unsupervised inter-domain transformation of low-resolution unlabeled mid-infrared photoacoustic microscopy images into confocal-like virtually fluorescence-stained high-resolution images. The explainable deep learning-based framework is proposed for this transformation, wherein an unsupervised generative adversarial network is primarily employed and then a saliency constraint is added for better explainability. We validate the performance of explainable deep learning-based mid-infrared photoacoustic microscopy by identifying cell nuclei and filamentous actins in cultured human cardiac fibroblasts and matching them with the corresponding CFM images. The XDL ensures similar saliency between the two domains, making the transformation process more stable and more reliable than existing networks. Our XDL-MIR-PAM enables label-free high-resolution duplexed cellular imaging, which can significantly benefit many research avenues in cell biology.

Berendzen K, Wanke F, Gabernet G, Harter K, Nahnsen S. Nf-Root: A Best-Practice Pipeline for Deep-Learning-Based Analysis of Apoplastic pH in Microscopy Images of Developmental Zones in Plant Root Tissue

Hormonal mechanisms associated with cell elongation play a vital role in the development and growth of plants. Here, we report Nextflow-root (nf-root), a novel best-practice pipeline for deep-learning-based analysis of fluorescence microscopy images of plant root tissue from A. thaliana. This bioinformatics pipeline performs automatic identification of developmental zones in root tissue images. This also includes apoplastic pH measurements, which is useful for modeling hormone signaling and cell physiological responses. We show that this nf-core standard-based pipeline successfully automates tissue zone segmentation and is both high-throughput and highly reproducible. In short, a deep-learning module deploys deterministically trained convolutional neural network models and augments the segmentation predictions with measures of prediction uncertainty and model interpretability, while aiming to facilitate result interpretation and verification by experienced plant biologists. We observed a high statistical similarity between the manually generated results and the output of the nf-root.

Neural space-time model for dynamic multi-shot imaging

Computational imaging reconstructions from multiple measurements that are captured sequentially often suffer from motion artifacts if the scene is dynamic. We propose a neural space-time model (NSTM) that jointly estimates the scene and its motion dynamics, without data priors or pre-training. Hence, we can both remove motion artifacts and resolve sample dynamics from the same set of raw measurements used for the conventional reconstruction. We demonstrate NSTM in three computational imaging systems: differential phase-contrast microscopy, three-dimensional structured illumination microscopy and rolling-shutter DiffuserCam. We show that NSTM can recover subcellular motion dynamics and thus reduce the misinterpretation of living systems caused by motion artifacts.

TAG-SPARK: Empowering High-Speed Volumetric Imaging With Deep Learning and Spatial Redundancy

Two-photon high-speed fluorescence calcium imaging stands as a mainstream technique in neuroscience for capturing neural activities with high spatiotemporal resolution. However, challenges arise from the inherent tradeoff between acquisition speed and image quality, grappling with a low signal-to-noise ratio (SNR) due to limited signal photon flux. Here, a contrast-enhanced video-rate volumetric system, integrating a tunable acoustic gradient (TAG) lens-based high-speed microscopy with a TAG-SPARK denoising algorithm is demonstrated. The former facilitates high-speed dense z-sampling at sub-micrometer-scale intervals, allowing the latter to exploit the spatial redundancy of z-slices for self-supervised model training. This spatial redundancy-based approach, tailored for 4D (xyzt) dataset, not only achieves >700% SNR enhancement but also retains fast-spiking functional profiles of neuronal activities. High-speed plus high-quality images are exemplified by in vivo Purkinje cells calcium observation, revealing intriguing dendritic-to-somatic signal convolution, i.e., similar dendritic signals lead to reverse somatic responses. This tailored technique allows for capturing neuronal activities with high SNR, thus advancing the fundamental comprehension of neuronal transduction pathways within 3D neuronal architecture.

Deep learning based local feature classification to automatically identify single molecule fluorescence events

Long-term single-molecule fluorescence measurements are widely used powerful tools to study the conformational dynamics of biomolecules in real time to further elucidate their conformational dynamics. Typically, thousands or even more single-molecule traces are analyzed to provide statistically meaningful information, which is labor-intensive and can introduce user bias. Recently, several deep-learning models have been developed to automatically classify single-molecule traces. In this study, we introduce DEBRIS (Deep lEarning Based fRagmentatIon approach for Single-molecule fluorescence event identification), a deep-learning model focusing on classifying local features and capable of automatically identifying steady fluorescence signals and dynamically emerging signals of different patterns. DEBRIS efficiently and accurately identifies both one-color and two-color single-molecule events, including their start and end points. By adjusting user-defined criteria, DEBRIS becomes the pioneer in using a deep learning model to accurately classify four different types of single-molecule fluorescence events using the same trained model, demonstrating its universality and ability to enrich the current toolbox.

Advances in Portable Optical Microscopy Using Cloud Technologies and Artificial Intelligence for Medical Applications

The need for faster and more accessible alternatives to laboratory microscopy is driving many innovations throughout the image and data acquisition chain in the biomedical field. Benchtop microscopes are bulky, lack communications capabilities, and require trained personnel for analysis. New technologies, such as compact 3D-printed devices integrated with the Internet of Things (IoT) for data sharing and cloud computing, as well as automated image processing using deep learning algorithms, can address these limitations and enhance the conventional imaging workflow. This review reports on recent advancements in microscope miniaturization, with a focus on emerging technologies such as photoacoustic microscopy and more established approaches like smartphone-based microscopy. The potential applications of IoT in microscopy are examined in detail. Furthermore, this review discusses the evolution of image processing in microscopy, transitioning from traditional to deep learning methods that facilitate image enhancement and data interpretation. Despite numerous advancements in the field, there is a noticeable lack of studies that holistically address the entire microscopy acquisition chain. This review aims to highlight the potential of IoT and artificial intelligence (AI) in combination with portable microscopy, emphasizing the importance of a comprehensive approach to the microscopy acquisition chain, from portability to image analysis.

Self-inspired learning for denoising live-cell super-resolution microscopy

Every collected photon is precious in live-cell super-resolution (SR) microscopy. Here, we describe a data-efficient, deep learning-based denoising solution to improve diverse SR imaging modalities. The method, SN2N, is a Self-inspired Noise2Noise module with self-supervised data generation and self-constrained learning process. SN2N is fully competitive with supervised learning methods and circumvents the need for large training set and clean ground truth, requiring only a single noisy frame for training. We show that SN2N improves photon efficiency by one-to-two orders of magnitude and is compatible with multiple imaging modalities for volumetric, multicolor, time-lapse SR microscopy. We further integrated SN2N into different SR reconstruction algorithms to effectively mitigate image artifacts. We anticipate SN2N will enable improved live-SR imaging and inspire further advances.

Fluorescent Reporters, Imaging, and Artificial Intelligence Toolkits to Monitor and Quantify Autophagy, Heterophagy, and Lysosomal Trafficking Fluxes

Lysosomal compartments control the clearance of cell-own material (autophagy) or of material that cells endocytose from the external environment (heterophagy) to warrant supply of nutrients, to eliminate macromolecules or parts of organelles present in excess, aged, or containing toxic material. Inherited or sporadic mutations in lysosomal proteins and enzymes may hamper their folding in the endoplasmic reticulum (ER) and their lysosomal transport via the Golgi compartment, resulting in lysosomal dysfunction and storage disorders. Defective cargo delivery to lysosomal compartments is harmful to cells and organs since it causes accumulation of toxic compounds and defective organellar homeostasis. Assessment of resident proteins and cargo fluxes to the lysosomal compartments is crucial for the mechanistic dissection of intracellular transport and catabolic events. It might be combined with high-throughput screenings to identify cellular, chemical, or pharmacological modulators of these events that may find therapeutic use for autophagy-related and lysosomal storage disorders. Here, discuss qualitative, quantitative and chronologic monitoring of autophagic, heterophagic and lysosomal protein trafficking in fixed and live cells, which relies on fluorescent single and tandem reporters used in combination with biochemical, flow cytometry, light and electron microscopy approaches implemented by artificial intelligence-based technology.

Frontiers in artificial intelligence-directed light-sheet microscopy for uncovering biological phenomena and multi-organ imaging

Light-sheet fluorescence microscopy (LSFM) introduces fast scanning of biological phenomena with deep photon penetration and minimal phototoxicity. This advancement represents a significant shift in 3-D imaging of large-scale biological tissues and 4-D (space + time) imaging of small live animals. The large data associated with LSFM requires efficient imaging acquisition and analysis with the use of artificial intelligence (AI)/machine learning (ML) algorithms. To this end, AI/ML-directed LSFM is an emerging area for multi-organ imaging and tumor diagnostics. This review will present the development of LSFM and highlight various LSFM configurations and designs for multi-scale imaging. Optical clearance techniques will be compared for effective reduction in light scattering and optimal deep-tissue imaging. This review will further depict a diverse range of research and translational applications, from small live organisms to multi-organ imaging to tumor diagnosis. In addition, this review will address AI/ML-directed imaging reconstruction, including the application of convolutional neural networks (CNNs) and generative adversarial networks (GANs). In summary, the advancements of LSFM have enabled effective and efficient post-imaging reconstruction and data analyses, underscoring LSFM's contribution to advancing fundamental and translational research.

Arkitekt: streaming analysis and real-time workflows for microscopy

Quantitative microscopy workflows have evolved dramatically over the past years, progressively becoming more complex with the emergence of deep learning. Long-standing challenges such as three-dimensional segmentation of complex microscopy data can finally be addressed, and new imaging modalities are breaking records in both resolution and acquisition speed, generating gigabytes if not terabytes of data per day. With this shift in bioimage workflows comes an increasing need for efficient orchestration and data management, necessitating multitool interoperability and the ability to span dedicated computing resources. However, existing solutions are still limited in their flexibility and scalability and are usually restricted to offline analysis. Here we introduce Arkitekt, an open-source middleman between users and bioimage apps that enables complex quantitative microscopy workflows in real time. It allows the orchestration of popular bioimage software locally or remotely in a reliable and efficient manner. It includes visualization and analysis modules, but also mechanisms to execute source code and pilot acquisition software, making 'smart microscopy' a reality.

Θ-Net: A Deep Neural Network Architecture for the Resolution Enhancement of Phase-Modulated Optical Micrographs In Silico

Optical microscopy is widely regarded to be an indispensable tool in healthcare and manufacturing quality control processes, although its inability to resolve structures separated by a lateral distance under ~200 nm has culminated in the emergence of a new field named fluorescence nanoscopy, while this too is prone to several caveats (namely phototoxicity, interference caused by exogenous probes and cost). In this regard, we present a triplet string of concatenated O-Net ('bead') architectures (termed 'Θ-Net' in the present study) as a cost-efficient and non-invasive approach to enhancing the resolution of non-fluorescent phase-modulated optical microscopical images in silico. The quality of the afore-mentioned enhanced resolution (ER) images was compared with that obtained via other popular frameworks (such as ANNA-PALM, BSRGAN and 3D RCAN), with the Θ-Net-generated ER images depicting an increased level of detail (unlike previous DNNs). In addition, the use of cross-domain (transfer) learning to enhance the capabilities of models trained on differential interference contrast (DIC) datasets [where phasic variations are not as prominently manifested as amplitude/intensity differences in the individual pixels unlike phase-contrast microscopy (PCM)] has resulted in the Θ-Net-generated images closely approximating that of the expected (ground truth) images for both the DIC and PCM datasets. This thus demonstrates the viability of our current Θ-Net architecture in attaining highly resolved images under poor signal-to-noise ratios while eliminating the need for a priori PSF and OTF information, thereby potentially impacting several engineering fronts (particularly biomedical imaging and sensing, precision engineering and optical metrology).

Image restoration in frequency space using complex-valued CNNs

Real-valued convolutional neural networks (RV-CNNs) in the spatial domain have outperformed classical approaches in many image restoration tasks such as image denoising and super-resolution. Fourier analysis of the results produced by these spatial domain models reveals the limitations of these models in properly processing the full frequency spectrum. This lack of complete spectral information can result in missing textural and structural elements. To address this limitation, we explore the potential of complex-valued convolutional neural networks (CV-CNNs) for image restoration tasks. CV-CNNs have shown remarkable performance in tasks such as image classification and segmentation. However, CV-CNNs for image restoration problems in the frequency domain have not been fully investigated to address the aforementioned issues. Here, we propose several novel CV-CNN-based models equipped with complex-valued attention gates for image denoising and super-resolution in the frequency domains. We also show that our CV-CNN-based models outperform their real-valued counterparts for denoising super-resolution structured illumination microscopy (SR-SIM) and conventional image datasets. Furthermore, the experimental results show that our proposed CV-CNN-based models preserve the frequency spectrum better than their real-valued counterparts in the denoising task. Based on these findings, we conclude that CV-CNN-based methods provide a plausible and beneficial deep learning approach for image restoration in the frequency domain.

Orientation selectivity mapping in the visual cortex

The orientation map is one of the most well-studied functional maps of the visual cortex. However, results from the literature are of different qualities. Clear boundaries among different orientation domains and blurred uncertain distinctions were shown in different studies. These unclear imaging results will lead to an inaccuracy in depicting cortical structures, and the lack of consideration in experimental design will also lead to biased depictions of the cortical features. How we accurately define orientation domains will impact the entire field of research. In this study, we test how spatial frequency (SF), stimulus size, location, chromatic, and data processing methods affect the orientation functional maps (including a large area of dorsal V4, and parts of dorsal V1) acquired by intrinsic signal optical imaging. Our results indicate that, for large imaging fields, large grating stimuli with mixed SF components should be considered to acquire the orientation map. A diffusion model image enhancement based on the difference map could further improve the map quality. In addition, the similar outcomes of achromatic and chromatic gratings indicate two alternative types of afferents from LGN, pooling in V1 to generate cue-invariant orientation selectivity.

Visual interpretability of image-based classification models by generative latent space disentanglement applied to in vitro fertilization

The success of deep learning in identifying complex patterns exceeding human intuition comes at the cost of interpretability. Non-linear entanglement of image features makes deep learning a "black box" lacking human meaningful explanations for the models' decision. We present DISCOVER, a generative model designed to discover the underlying visual properties driving image-based classification models. DISCOVER learns disentangled latent representations, where each latent feature encodes a unique classification-driving visual property. This design enables "human-in-the-loop" interpretation by generating disentangled exaggerated counterfactual explanations. We apply DISCOVER to interpret classification of in vitro fertilization embryo morphology quality. We quantitatively and systematically confirm the interpretation of known embryo properties, discover properties without previous explicit measurements, and quantitatively determine and empirically verify the classification decision of specific embryo instances. We show that DISCOVER provides human-interpretable understanding of "black box" classification models, proposes hypotheses to decipher underlying biomedical mechanisms, and provides transparency for the classification of individual predictions.

A deep learning method that identifies cellular heterogeneity using nanoscale nuclear features

Cellular phenotypic heterogeneity is an important hallmark of many biological processes and understanding its origins remains a substantial challenge. This heterogeneity often reflects variations in the chromatin structure, influenced by factors such as viral infections and cancer, which dramatically reshape the cellular landscape. To address the challenge of identifying distinct cell states, we developed artificial intelligence of the nucleus (AINU), a deep learning method that can identify specific nuclear signatures at the nanoscale resolution. AINU can distinguish different cell states based on the spatial arrangement of core histone H3, RNA polymerase II or DNA from super-resolution microscopy images. With only a small number of images as the training data, AINU correctly identifies human somatic cells, human-induced pluripotent stem cells, very early stage infected cells transduced with DNA herpes simplex virus type 1 and even cancer cells after appropriate retraining. Finally, using AI interpretability methods, we find that the RNA polymerase II localizations in the nucleoli aid in distinguishing human-induced pluripotent stem cells from their somatic cells. Overall, AINU coupled with super-resolution microscopy of nuclear structures provides a robust tool for the precise detection of cellular heterogeneity, with considerable potential for advancing diagnostics and therapies in regenerative medicine, virology and cancer biology.

Microvascular plasticity in mouse stroke model recovery: Anatomy statistics, dynamics measured by longitudinal in vivo two-photon angiography, network vectorization

This manuscript quantitatively investigates remodeling dynamics of the cortical microvascular network (thousands of connected capillaries) following photothrombotic ischemia (cubic millimeter volume, imaged weekly) using a novel in vivo two-photon angiography and high throughput vascular vectorization method. The results suggest distinct temporal patterns of cerebrovascular plasticity, with acute remodeling peaking at one week post-stroke. The network architecture then gradually stabilizes, returning to a new steady state after four weeks. These findings align with previous literature on neuronal plasticity, highlighting the correlation between neuronal and neurovascular remodeling. Quantitative analysis of neurovascular networks using length- and strand-based statistical measures reveals intricate changes in network anatomy and topology. The distance and strand-length statistics show significant alterations, with a peak of plasticity observed at one week post-stroke, followed by a gradual return to baseline. The orientation statistic plasticity peaks at two weeks, gradually approaching the (conserved across subjects) stroke signature. The underlying mechanism of the vascular response (angiogenesis vs. tissue deformation), however, is yet unexplored. Overall, the combination of chronic two-photon angiography, vascular vectorization, reconstruction/visualization, and statistical analysis enables both qualitative and quantitative assessments of neurovascular remodeling dynamics, demonstrating a method for investigating cortical microvascular network disorders and the therapeutic modes of action thereof.

Convolutional neural network transformer (CNNT) for fluorescence microscopy image denoising with improved generalization and fast adaptation

Deep neural networks can improve the quality of fluorescence microscopy images. Previous methods, based on Convolutional Neural Networks (CNNs), require time-consuming training of individual models for each experiment, impairing their applicability and generalization. In this study, we propose a novel imaging-transformer based model, Convolutional Neural Network Transformer (CNNT), that outperforms CNN based networks for image denoising. We train a general CNNT based backbone model from pairwise high-low Signal-to-Noise Ratio (SNR) image volumes, gathered from a single type of fluorescence microscope, an instant Structured Illumination Microscope. Fast adaptation to new microscopes is achieved by fine-tuning the backbone on only 5-10 image volume pairs per new experiment. Results show that the CNNT backbone and fine-tuning scheme significantly reduces training time and improves image quality, outperforming models trained using only CNNs such as 3D-RCAN and Noise2Fast. We show three examples of efficacy of this approach in wide-field, two-photon, and confocal fluorescence microscopy.

In silico labeling in cell biology: Potential and limitations

In silico labeling is the computational cross-modality image translation where the output modality is a subcellular marker that is not specifically encoded in the input image, for example, in silico localization of organelles from transmitted light images. In principle, in silico labeling has the potential to facilitate rapid live imaging of multiple organelles with reduced photobleaching and phototoxicity, a technology enabling a major leap toward understanding the cell as an integrated complex system. However, five years have passed since feasibility was attained, without any demonstration of using in silico labeling to uncover new biological insight. In here, we discuss the current state of in silico labeling, the limitations preventing it from becoming a practical tool, and how we can overcome these limitations to reach its full potential.

Pretraining a foundation model for generalizable fluorescence microscopy-based image restoration

Fluorescence microscopy-based image restoration has received widespread attention in the life sciences and has led to significant progress, benefiting from deep learning technology. However, most current task-specific methods have limited generalizability to different fluorescence microscopy-based image restoration problems. Here, we seek to improve generalizability and explore the potential of applying a pretrained foundation model to fluorescence microscopy-based image restoration. We provide a universal fluorescence microscopy-based image restoration (UniFMIR) model to address different restoration problems, and show that UniFMIR offers higher image restoration precision, better generalization and increased versatility. Demonstrations on five tasks and 14 datasets covering a wide range of microscopy imaging modalities and biological samples demonstrate that the pretrained UniFMIR can effectively transfer knowledge to a specific situation via fine-tuning, uncover clear nanoscale biomolecular structures and facilitate high-quality imaging. This work has the potential to inspire and trigger new research highlights for fluorescence microscopy-based image restoration.

Learning a stable approximation of an existing but unknown inverse mapping: application to the half-time circular Radon transform

Supervised deep learning-based methods have inspired a new wave of image reconstruction methods that implicitly learn effective regularization strategies from a set of training data. While they hold potential for improving image quality, they have also raised concerns regarding their robustness. Instabilities can manifest when learned methods are applied to find approximate solutions to ill-posed image reconstruction problems for which a unique and stable inverse mapping does not exist, which is a typical use case. In this study, we investigate the performance of supervised deep learning-based image reconstruction in an alternate use case in which a stable inverse mapping is known to exist but is not yet analytically available in closed form. For such problems, a deep learning-based method can learn a stable approximation of the unknown inverse mapping that generalizes well to data that differ significantly from the training set. The learned approximation of the inverse mapping eliminates the need to employ an implicit (optimization-based) reconstruction method and can potentially yield insights into the unknown analytic inverse formula. The specific problem addressed is image reconstruction from a particular case of radially truncated circular Radon transform (CRT) data, referred to as 'half-time' measurement data. For the half-time image reconstruction problem, we develop and investigate a learned filtered backprojection method that employs a convolutional neural network to approximate the unknown filtering operation. We demonstrate that this method behaves stably and readily generalizes to data that differ significantly from training data. The developed method may find application to wave-based imaging modalities that include photoacoustic computed tomography.

A Saturation Artifacts Inpainting Method Based on Two-Stage GAN for Fluorescence Microscope Images

Fluorescence microscopic images of cells contain a large number of morphological features that are used as an unbiased source of quantitative information about cell status, through which researchers can extract quantitative information about cells and study the biological phenomena of cells through statistical and analytical analysis. As an important research object of phenotypic analysis, images have a great influence on the research results. Saturation artifacts present in the image result in a loss of grayscale information that does not reveal the true value of fluorescence intensity. From the perspective of data post-processing, we propose a two-stage cell image recovery model based on a generative adversarial network to solve the problem of phenotypic feature loss caused by saturation artifacts. The model is capable of restoring large areas of missing phenotypic features. In the experiment, we adopt the strategy of progressive restoration to improve the robustness of the training effect and add the contextual attention structure to enhance the stability of the restoration effect. We hope to use deep learning methods to mitigate the effects of saturation artifacts to reveal how chemical, genetic, and environmental factors affect cell state, providing an effective tool for studying the field of biological variability and improving image quality in analysis.

Deep 3D histology powered by tissue clearing, omics and AI

To comprehensively understand tissue and organism physiology and pathophysiology, it is essential to create complete three-dimensional (3D) cellular maps. These maps require structural data, such as the 3D configuration and positioning of tissues and cells, and molecular data on the constitution of each cell, spanning from the DNA sequence to protein expression. While single-cell transcriptomics is illuminating the cellular and molecular diversity across species and tissues, the 3D spatial context of these molecular data is often overlooked. Here, I discuss emerging 3D tissue histology techniques that add the missing third spatial dimension to biomedical research. Through innovations in tissue-clearing chemistry, labeling and volumetric imaging that enhance 3D reconstructions and their synergy with molecular techniques, these technologies will provide detailed blueprints of entire organs or organisms at the cellular level. Machine learning, especially deep learning, will be essential for extracting meaningful insights from the vast data. Further development of integrated structural, molecular and computational methods will unlock the full potential of next-generation 3D histology.

Advanced Design of Soft Robots with Artificial Intelligence

A comprehensive review focused on the whole systems of the soft robotics with artificial intelligence, which can feel, think, react and interact with humans, is presented. The design strategies concerning about various aspects of the soft robotics, like component materials, device structures, prepared technologies, integrated method, and potential applications, are summarized. A broad outlook on the future considerations for the soft robots is proposed.

In recent years, breakthrough has been made in the field of artificial intelligence (AI), which has also revolutionized the industry of robotics. Soft robots featured with high-level safety, less weight, lower power consumption have always been one of the research hotspots. Recently, multifunctional sensors for perception of soft robotics have been rapidly developed, while more algorithms and models of machine learning with high accuracy have been optimized and proposed. Designs of soft robots with AI have also been advanced ranging from multimodal sensing, human–machine interaction to effective actuation in robotic systems. Nonetheless, comprehensive reviews concerning the new developments and strategies for the ingenious design of the soft robotic systems equipped with AI are rare. Here, the new development is systematically reviewed in the field of soft robots with AI. First, background and mechanisms of soft robotic systems are briefed, after which development focused on how to endow the soft robots with AI, including the aspects of feeling, thought and reaction, is illustrated. Next, applications of soft robots with AI are systematically summarized and discussed together with advanced strategies proposed for performance enhancement. Design thoughts for future intelligent soft robotics are pointed out. Finally, some perspectives are put forward.

A guide to artificial intelligence for cancer researchers

Artificial intelligence (AI) has been commoditized. It has evolved from a specialty resource to a readily accessible tool for cancer researchers. AI-based tools can boost research productivity in daily workflows, but can also extract hidden information from existing data, thereby enabling new scientific discoveries. Building a basic literacy in these tools is useful for every cancer researcher. Researchers with a traditional biological science focus can use AI-based tools through off-the-shelf software, whereas those who are more computationally inclined can develop their own AI-based software pipelines. In this article, we provide a practical guide for non-computational cancer researchers to understand how AI-based tools can benefit them. We convey general principles of AI for applications in image analysis, natural language processing and drug discovery. In addition, we give examples of how non-computational researchers can get started on the journey to productively use AI in their own work.

A comparative study of CARE 2D and N2V 2D for tissue-specific denoising in second harmonic generation imaging

This study explored the application of deep learning in second harmonic generation (SHG) microscopy, a rapidly growing area. This study focuses on the impact of glycerol concentration on image noise in SHG microscopy and compares two image restoration techniques: Noise-to-Void 2D (N2V 2D, no reference image restoration) and content-aware image restoration (CARE 2D, full reference image restoration). We demonstrated that N2V 2D effectively restored the images affected by high glycerol concentrations. To reduce sample exposure and damage, this study further addresses low-power SHG imaging by reducing the laser power by 70% using deep learning techniques. CARE 2D excels in preserving detailed structures, whereas N2V 2D maintains natural muscle structure. This study highlights the strengths and limitations of these models in specific SHG microscopy applications, offering valuable insights and potential advancements in the field .

Live-cell imaging powered by computation

The proliferation of microscopy methods for live-cell imaging offers many new possibilities for users but can also be challenging to navigate. The prevailing challenge in live-cell fluorescence microscopy is capturing intra-cellular dynamics while preserving cell viability. Computational methods can help to address this challenge and are now shifting the boundaries of what is possible to capture in living systems. In this Review, we discuss these computational methods focusing on artificial intelligence-based approaches that can be layered on top of commonly used existing microscopies as well as hybrid methods that integrate computation and microscope hardware. We specifically discuss how computational approaches can improve the signal-to-noise ratio, spatial resolution, temporal resolution and multi-colour capacity of live-cell imaging.

Zero-shot learning enables instant denoising and super-resolution in optical fluorescence microscopy

Computational super-resolution methods, including conventional analytical algorithms and deep learning models, have substantially improved optical microscopy. Among them, supervised deep neural networks have demonstrated outstanding performance, however, demanding abundant high-quality training data, which are laborious and even impractical to acquire due to the high dynamics of living cells. Here, we develop zero-shot deconvolution networks (ZS-DeconvNet) that instantly enhance the resolution of microscope images by more than 1.5-fold over the diffraction limit with 10-fold lower fluorescence than ordinary super-resolution imaging conditions, in an unsupervised manner without the need for either ground truths or additional data acquisition. We demonstrate the versatile applicability of ZS-DeconvNet on multiple imaging modalities, including total internal reflection fluorescence microscopy, three-dimensional wide-field microscopy, confocal microscopy, two-photon microscopy, lattice light-sheet microscopy, and multimodal structured illumination microscopy, which enables multi-color, long-term, super-resolution 2D/3D imaging of subcellular bioprocesses from mitotic single cells to multicellular embryos of mouse and C. elegans.

Implicit neural representations in light microscopy

Three-dimensional stacks acquired with confocal or two-photon microscopy are crucial for studying neuroanatomy. However, high-resolution image stacks acquired at multiple depths are time-consuming and susceptible to photobleaching. In vivo microscopy is further prone to motion artifacts. In this work, we suggest that deep neural networks with sine activation functions encoding implicit neural representations (SIRENs) are suitable for predicting intermediate planes and correcting motion artifacts, addressing the aforementioned shortcomings. We show that we can accurately estimate intermediate planes across multiple micrometers and fully automatically and unsupervised estimate a motion-corrected denoised picture. We show that noise statistics can be affected by SIRENs, however, rescued by a downstream denoising neural network, shown exemplarily with the recovery of dendritic spines. We believe that the application of these technologies will facilitate more efficient acquisition and superior post-processing in the future.

SpiDe-Sr: blind super-resolution network for precise cell segmentation and clustering in spatial proteomics imaging

Spatial proteomics elucidates cellular biochemical changes with unprecedented topological level. Imaging mass cytometry (IMC) is a high-dimensional single-cell resolution platform for targeted spatial proteomics. However, the precision of subsequent clinical analysis is constrained by imaging noise and resolution. Here, we propose SpiDe-Sr, a super-resolution network embedded with a denoising module for IMC spatial resolution enhancement. SpiDe-Sr effectively resists noise and improves resolution by 4 times. We demonstrate SpiDe-Sr respectively with cells, mouse and human tissues, resulting 18.95%/27.27%/21.16% increase in peak signal-to-noise ratio and 15.95%/31.63%/15.52% increase in cell extraction accuracy. We further apply SpiDe-Sr to study the tumor microenvironment of a 20-patient clinical breast cancer cohort with 269,556 single cells, and discover the invasion of Gram-negative bacteria is positively correlated with carcinogenesis markers and negatively correlated with immunological markers. Additionally, SpiDe-Sr is also compatible with fluorescence microscopy imaging, suggesting SpiDe-Sr an alternative tool for microscopy image super-resolution.

On-the-fly Raman microscopy guaranteeing the accuracy of discrimination

Accelerating the measurement for discrimination of samples, such as classification of cell phenotype, is crucial when faced with significant time and cost constraints. Spontaneous Raman microscopy offers label-free, rich chemical information but suffers from long acquisition time due to extremely small scattering cross-sections. One possible approach to accelerate the measurement is by measuring necessary parts with a suitable number of illumination points. However, how to design these points during measurement remains a challenge. To address this, we developed an imaging technique based on a reinforcement learning in machine learning (ML). This ML approach adaptively feeds back "optimal" illumination pattern during the measurement to detect the existence of specific characteristics of interest, allowing faster measurements while guaranteeing discrimination accuracy. Using a set of Raman images of human follicular thyroid and follicular thyroid carcinoma cells, we showed that our technique requires 3,333 to 31,683 times smaller number of illuminations for discriminating the phenotypes than raster scanning. To quantitatively evaluate the number of illuminations depending on the requisite discrimination accuracy, we prepared a set of polymer bead mixture samples to model anomalous and normal tissues. We then applied a home-built programmable-illumination microscope equipped with our algorithm, and confirmed that the system can discriminate the sample conditions with 104 to 4,350 times smaller number of illuminations compared to standard point illumination Raman microscopy. The proposed algorithm can be applied to other types of microscopy that can control measurement condition on the fly, offering an approach for the acceleration of accurate measurements in various applications including medical diagnosis.

Noninvasive Nonlinear Optical Computational Histology

Cancer remains a global health challenge, demanding early detection and accurate diagnosis for improved patient outcomes. An intelligent paradigm is introduced that elevates label-free nonlinear optical imaging with contrastive patch-wise learning, yielding stain-free nonlinear optical computational histology (NOCH). NOCH enables swift, precise diagnostic analysis of fresh tissues, reducing patient anxiety and healthcare costs. Nonlinear modalities are evaluated, including stimulated Raman scattering and multiphoton imaging, for their ability to enhance tumor microenvironment sensitivity, pathological analysis, and cancer examination. Quantitative analysis confirmed that NOCH images accurately reproduce nuclear morphometric features across different cancer stages. Key diagnostic features, such as nuclear morphology, size, and nuclear-cytoplasmic contrast, are well preserved. NOCH models also demonstrate promising generalization when applied to other pathological tissues. The study unites label-free nonlinear optical imaging with histopathology using contrastive learning to establish stain-free computational histology. NOCH provides a rapid, non-invasive, and precise approach to surgical pathology, holding immense potential for revolutionizing cancer diagnosis and surgical interventions.

Data-driven coordinated attention deep learning for high-fidelity brain imaging denoising and inpainting

Deep learning offers promise in enhancing low-quality images by addressing weak fluorescence signals, especially in deep in vivo mouse brain imaging. However, current methods struggle with photon scarcity and noise within in vivo deep mouse brains, and often neglecting tissue preservation. In this study, we propose an innovative in vivo cortical fluorescence image restoration approach, combining signal enhancement, denoising, and inpainting. We curated a deep brain cortical image dataset and developed a novel deep brain coordinate attention restoration network (DeepCAR), integrating coordinate attention with optimized residual networks. Our method swiftly and accurately restores deep cortex images exceeding 800 μm, preserving small-scale tissue structures. It boosts the peak signal-to-noise ratio (PSNR) by 6.94 dB for weak signals and 11.22 dB for large noisy images. Crucially, we validate the effectiveness on external datasets with diverse noise distributions, structural features compared to those in our training data, showcasing real-time high-performance image restoration capabilities.

Harnessing artificial intelligence to reduce phototoxicity in live imaging

Fluorescence microscopy is essential for studying living cells, tissues and organisms. However, the fluorescent light that switches on fluorescent molecules also harms the samples, jeopardizing the validity of results - particularly in techniques such as super-resolution microscopy, which demands extended illumination. Artificial intelligence (AI)-enabled software capable of denoising, image restoration, temporal interpolation or cross-modal style transfer has great potential to rescue live imaging data and limit photodamage. Yet we believe the focus should be on maintaining light-induced damage at levels that preserve natural cell behaviour. In this Opinion piece, we argue that a shift in role for AIs is needed - AI should be used to extract rich insights from gentle imaging rather than recover compromised data from harsh illumination. Although AI can enhance imaging, our ultimate goal should be to uncover biological truths, not just retrieve data. It is essential to prioritize minimizing photodamage over merely pushing technical limits. Our approach is aimed towards gentle acquisition and observation of undisturbed living systems, aligning with the essence of live-cell fluorescence microscopy.

Content-aware frame interpolation (CAFI): deep learning-based temporal super-resolution for fast bioimaging

The development of high-resolution microscopes has made it possible to investigate cellular processes in 3D and over time. However, observing fast cellular dynamics remains challenging because of photobleaching and phototoxicity. Here we report the implementation of two content-aware frame interpolation (CAFI) deep learning networks, Zooming SlowMo and Depth-Aware Video Frame Interpolation, that are highly suited for accurately predicting images in between image pairs, therefore improving the temporal resolution of image series post-acquisition. We show that CAFI is capable of understanding the motion context of biological structures and can perform better than standard interpolation methods. We benchmark CAFI's performance on 12 different datasets, obtained from four different microscopy modalities, and demonstrate its capabilities for single-particle tracking and nuclear segmentation. CAFI potentially allows for reduced light exposure and phototoxicity on the sample for improved long-term live-cell imaging. The models and the training and testing data are available via the ZeroCostDL4Mic platform.

Isotropic multi-scale neuronal reconstruction from high-ratio expansion microscopy with contrastive unsupervised deep generative models

BACKGROUND AND OBJECTIVE

Current methods for imaging reconstruction from high-ratio expansion microscopy (ExM) data are limited by anisotropic optical resolution and the requirement for extensive manual annotation, creating a significant bottleneck in the analysis of complex neuronal structures.

METHODS

We devised an innovative approach called the IsoGAN model, which utilizes a contrastive unsupervised generative adversarial network to sidestep these constraints. This model leverages multi-scale and isotropic neuron/protein/blood vessel morphology data to generate high-fidelity 3D representations of these structures, eliminating the need for rigorous manual annotation and supervision. The IsoGAN model introduces simplified structures with idealized morphologies as shape priors to ensure high consistency in the generated neuronal profiles across all points in space and scalability for arbitrarily large volumes.

RESULTS

The efficacy of the IsoGAN model in accurately reconstructing complex neuronal structures was quantitatively assessed by examining the consistency between the axial and lateral views and identifying a reduction in erroneous imaging artifacts. The IsoGAN model accurately reconstructed complex neuronal structures, as evidenced by the consistency between the axial and lateral views and a reduction in erroneous imaging artifacts, and can be further applied to various biological samples.

CONCLUSION

With its ability to generate detailed 3D neurons/proteins/blood vessel structures using significantly fewer axial view images, IsoGAN can streamline the process of imaging reconstruction while maintaining the necessary detail, offering a transformative solution to the existing limitations in high-throughput morphology analysis across different structures.

Image restoration of degraded time-lapse microscopy data mediated by near-infrared imaging

Time-lapse fluorescence microscopy is key to unraveling biological development and function; however, living systems, by their nature, permit only limited interrogation and contain untapped information that can only be captured by more invasive methods. Deep-tissue live imaging presents a particular challenge owing to the spectral range of live-cell imaging probes/fluorescent proteins, which offer only modest optical penetration into scattering tissues. Herein, we employ convolutional neural networks to augment live-imaging data with deep-tissue images taken on fixed samples. We demonstrate that convolutional neural networks may be used to restore deep-tissue contrast in GFP-based time-lapse imaging using paired final-state datasets acquired using near-infrared dyes, an approach termed InfraRed-mediated Image Restoration (IR2). Notably, the networks are remarkably robust over a wide range of developmental times. We employ IR2 to enhance the information content of green fluorescent protein time-lapse images of zebrafish and Drosophila embryo/larval development and demonstrate its quantitative potential in increasing the fidelity of cell tracking/lineaging in developing pescoids. Thus, IR2 is poised to extend live imaging to depths otherwise inaccessible.

A versatile Wavelet-Enhanced CNN-Transformer for improved fluorescence microscopy image restoration

Fluorescence microscopes are indispensable tools for the life science research community. Nevertheless, the presence of optical component limitations, coupled with the maximum photon budget that the specimen can tolerate, inevitably leads to a decline in imaging quality and a lack of useful signals. Therefore, image restoration becomes essential for ensuring high-quality and accurate analyses. This paper presents the Wavelet-Enhanced Convolutional-Transformer (WECT), a novel deep learning technique developed specifically for the purpose of reducing noise in microscopy images and attaining super-resolution. Unlike traditional approaches, WECT integrates wavelet transform and inverse-transform for multi-resolution image decomposition and reconstruction, resulting in an expanded receptive field for the network without compromising information integrity. Subsequently, multiple consecutive parallel CNN-Transformer modules are utilized to collaboratively model local and global dependencies, thus facilitating the extraction of more comprehensive and diversified deep features. In addition, the incorporation of generative adversarial networks (GANs) into WECT enhances its capacity to generate high perceptual quality microscopic images. Extensive experiments have demonstrated that the WECT framework outperforms current state-of-the-art restoration methods on real fluorescence microscopy data under various imaging modalities and conditions, in terms of quantitative and qualitative analysis.

Predicting Neuroblastoma Patient Risk Groups, Outcomes, and Treatment Response Using Machine Learning Methods: A Review

Neuroblastoma, a paediatric malignancy with high rates of cancer-related morbidity and mortality, is of significant interest to the field of paediatric cancers. High-risk NB tumours are usually metastatic and result in survival rates of less than 50%. Machine learning approaches have been applied to various neuroblastoma patient data to retrieve relevant clinical and biological information and develop predictive models. Given this background, this study will catalogue and summarise the literature that has used machine learning and statistical methods to analyse data such as multi-omics, histological sections, and medical images to make clinical predictions. Furthermore, the question will be turned on its head, and the use of machine learning to accurately stratify NB patients by risk groups and to predict outcomes, including survival and treatment response, will be summarised. Overall, this study aims to catalogue and summarise the important work conducted to date on the subject of expression-based predictor models and machine learning in neuroblastoma for risk stratification and patient outcomes including survival, and treatment response which may assist and direct future diagnostic and therapeutic efforts.