Democratising deep learning for microscopy with ZeroCostDL4Mic

Kinetic insights into agonist-dependent signalling bias at the pro-inflammatory G-protein coupled receptor GPR84

GPR84 is an orphan G-protein coupled receptor (GPCR) linked to inflammation. Strategies targeting GPR84 to prevent excessive inflammation in disease are hampered by a lack of understanding of its precise functional role. We have developed heterologous cell lines with low GPR84 expression levels that phenocopy the response of primary cells in a label-free cell electrical impedance (CEI) sensing system that measures cell morphology and adhesion. We then investigated the signalling profile and membrane localisation of GPR84 upon treatment with 6-OAU and DL-175, two agonists known to differentially influence immune cell function. When compared to 6-OAU, DL-175 was found to exhibit a delayed impedance response, a delayed and suppressed activation of Akt, which together correlated with an impaired ability to internalise GPR84 from the plasma membrane. The signalling differences were transient and occurred only at early time points in the low expressing cell lines, highlighting the importance of receptor number and kinetic readouts when evaluating signalling bias. Our findings open new ways to understand GPR84 signalling and evaluate the effect of newly developed agonists.

Harnessing computational spatial omics to explore the spatial biology intricacies

Spatially resolved transcriptomics (SRT) has unlocked new dimensions in our understanding of intricate tissue architectures. However, this rapidly expanding field produces a wealth of diverse and voluminous data, necessitating the evolution of sophisticated computational strategies to unravel inherent patterns. Two distinct methodologies, gene spatial pattern recognition (GSPR) and tissue spatial pattern recognition (TSPR), have emerged as vital tools in this process. GSPR methodologies are designed to identify and classify genes exhibiting noteworthy spatial patterns, while TSPR strategies aim to understand intercellular interactions and recognize tissue domains with molecular and spatial coherence. In this review, we provide a comprehensive exploration of SRT, highlighting crucial data modalities and resources that are instrumental for the development of methods and biological insights. We address the complexities and challenges posed by the use of heterogeneous data in developing GSPR and TSPR methodologies and propose an optimal workflow for both. We delve into the latest advancements in GSPR and TSPR, examining their interrelationships. Lastly, we peer into the future, envisaging the potential directions and perspectives in this dynamic field.

Addressing annotation and data scarcity when designing machine learning strategies for neurophotonics

Machine learning has revolutionized the way data are processed, allowing information to be extracted in a fraction of the time it would take an expert. In the field of neurophotonics, machine learning approaches are used to automatically detect and classify features of interest in complex images. One of the key challenges in applying machine learning methods to the field of neurophotonics is the scarcity of available data and the complexity associated with labeling them, which can limit the performance of data-driven algorithms. We present an overview of various strategies, such as weakly supervised learning, active learning, and domain adaptation that can be used to address the problem of labeled data scarcity in neurophotonics. We provide a comprehensive overview of the strengths and limitations of each approach and discuss their potential applications to bioimaging datasets. In addition, we highlight how different strategies can be combined to increase model performance on those datasets. The approaches we describe can help to improve the accessibility of machine learning-based analysis with limited number of annotated images for training and can enable researchers to extract more meaningful insights from small datasets.

Characterizing the blood-brain barrier and gut barrier with super-resolution imaging: opportunities and challenges

Brain and gut barriers have been receiving increasing attention in health and diseases including in psychiatry. Recent studies have highlighted changes in the blood-brain barrier and gut barrier structural properties, notably a loss of tight junctions, leading to hyperpermeability, passage of inflammatory mediators, stress vulnerability, and the development of depressive behaviors. To decipher the cellular processes actively contributing to brain and gut barrier function in health and disease, scientists can take advantage of neurophotonic tools and recent advances in super-resolution microscopy techniques to complement traditional imaging approaches like confocal and electron microscopy. Here, we summarize the challenges, pros, and cons of these innovative approaches, hoping that a growing number of scientists will integrate them in their study design exploring barrier-related properties and mechanisms.

Is AI essential? Examining the need for deep learning in image-activated sorting of Saccharomyces cerevisiae

Artificial intelligence (AI) has become a focal point across a multitude of societal sectors, with science not being an exception. Particularly in the life sciences, imaging flow cytometry has increasingly integrated AI for automated management and categorization of extensive cell image data. However, the necessity of AI over traditional classification methods when extending imaging flow cytometry to include cell sorting remains uncertain, primarily due to the time constraints between image acquisition and sorting actuation. AI-enabled image-activated cell sorting (IACS) methods remain substantially limited, even as recent advancements in IACS have found success while largely relying on traditional feature gating strategies. Here we assess the necessity of AI for image classification in IACS by contrasting the performance of feature gating, classical machine learning (ML), and deep learning (DL) with convolutional neural networks (CNNs) in the differentiation of Saccharomyces cerevisiae mutant images. We show that classical ML could only yield a 2.8-fold enhancement in target enrichment capability, albeit at the cost of a 13.7-fold increase in processing time. Conversely, a CNN could offer an 11.0-fold improvement in enrichment capability at an 11.5-fold increase in processing time. We further executed IACS on mixed mutant populations and quantified target strain enrichment via downstream DNA sequencing to substantiate the applicability of DL for the proposed study. Our findings validate the feasibility and value of employing DL in IACS for morphology-based genetic screening of S. cerevisiae, encouraging its incorporation in future advancements of similar technologies.

Machine learning for cross-scale microscopy of viruses

Despite advances in virological sciences and antiviral research, viruses continue to emerge, circulate, and threaten public health. We still lack a comprehensive understanding of how cells and individuals remain susceptible to infectious agents. This deficiency is in part due to the complexity of viruses, including the cell states controlling virus-host interactions. Microscopy samples distinct cellular infection stages in a multi-parametric, time-resolved manner at molecular resolution and is increasingly enhanced by machine learning and deep learning. Here we discuss how state-of-the-art artificial intelligence (AI) augments light and electron microscopy and advances virological research of cells. We describe current procedures for image denoising, object segmentation, tracking, classification, and super-resolution and showcase examples of how AI has improved the acquisition and analyses of microscopy data. The power of AI-enhanced microscopy will continue to help unravel virus infection mechanisms, develop antiviral agents, and improve viral vectors.

Neural network-assisted single-molecule localization microscopy with a weak-affinity protein tag

Single-molecule localization microscopy achieves nanometer spatial resolution by localizing single fluorophores separated in space and time. A major challenge of single-molecule localization microscopy is the long acquisition time, leading to low throughput, as well as to a poor temporal resolution that limits its use to visualize the dynamics of cellular structures in live cells. Another challenge is photobleaching, which reduces information density over time and limits throughput and the available observation time in live-cell applications. To address both challenges, we combine two concepts: first, we integrate the neural network DeepSTORM to predict super-resolution images from high-density imaging data, which increases acquisition speed. Second, we employ a direct protein label, HaloTag7, in combination with exchangeable ligands (xHTLs), for fluorescence labeling. This labeling method bypasses photobleaching by providing a constant signal over time and is compatible with live-cell imaging. The combination of both a neural network and a weak-affinity protein label reduced the acquisition time up to ∼25-fold. Furthermore, we demonstrate live-cell imaging with increased temporal resolution, and capture the dynamics of the endoplasmic reticulum over extended time without signal loss.

Near index matching enables solid diffractive optical element fabrication via additive manufacturing

Diffractive optical elements (DOEs) have a wide range of applications in optics and photonics, thanks to their capability to perform complex wavefront shaping in a compact form. However, widespread applicability of DOEs is still limited, because existing fabrication methods are cumbersome and expensive. Here, we present a simple and cost-effective fabrication approach for solid, high-performance DOEs. The method is based on conjugating two nearly refractive index-matched solidifiable transparent materials. The index matching allows for extreme scaling up of the elements in the axial dimension, which enables simple fabrication of a template using commercially available 3D printing at tens-of-micrometer resolution. We demonstrated the approach by fabricating and using DOEs serving as microlens arrays, vortex plates, including for highly sensitive applications such as vector beam generation and super-resolution microscopy using MINSTED, and phase-masks for three-dimensional single-molecule localization microscopy. Beyond the advantage of making DOEs widely accessible by drastically simplifying their production, the method also overcomes difficulties faced by existing methods in fabricating highly complex elements, such as high-order vortex plates, and spectrum-encoding phase masks for microscopy.

Detection of Frog Virus 3 by Integrating RPA-CRISPR/Cas12a-SPM with Deep Learning

A fast, easy-to-implement, highly sensitive, and point-of-care (POC) detection system for frog virus 3 (FV3) is proposed. Combining recombinase polymerase amplification (RPA) and CRISPR/Cas12a, a limit of detection (LoD) of 100 aM (60.2 copies/μL) is achieved by optimizing RPA primers and CRISPR RNAs (crRNAs). For POC detection, smartphone microscopy is implemented, and an LoD of 10 aM is achieved in 40 min. The proposed system detects four positive animal-derived samples with a quantitation cycle (Cq) value of quantitative PCR (qPCR) in the range of 13 to 32. In addition, deep learning models are deployed for binary classification (positive or negative samples) and multiclass classification (different concentrations of FV3 and negative samples), achieving 100 and 98.75% accuracy, respectively. Without temperature regulation and expensive equipment, the proposed RPA-CRISPR/Cas12a combined with smartphone readouts and artificial-intelligence-assisted classification showcases the great potential for FV3 detection, specifically POC detection of DNA virus.

The cellular response to extracellular vesicles is dependent on their cell source and dose

Extracellular vesicles (EVs) have been established to play important roles in cell-cell communication and shown promise as therapeutic agents. However, we still lack a basic understanding of how cells respond upon exposure to EVs from different cell sources at various doses. Thus, we treated fibroblasts with EVs from 12 different cell sources at doses between 20 and 200,000 per cell, analyzed their transcriptional effects, and functionally confirmed the findings in various cell types in vitro, and in vivo using single-cell RNA sequencing. Unbiased global analysis revealed EV dose to have a more significant effect than cell source, such that high doses down-regulated exocytosis and up-regulated lysosomal activity. However, EV cell source-specific responses were observed at low doses, and these reflected the activities of the EV's source cells. Last, we assessed EV-derived transcript abundance and found that immune cell-derived EVs were most associated with recipient cells. Together, this study provides important insights into the cellular response to EVs.

Optimal sparsity allows reliable system-aware restoration of fluorescence microscopy images

Fluorescence microscopy is one of the most indispensable and informative driving forces for biological research, but the extent of observable biological phenomena is essentially determined by the content and quality of the acquired images. To address the different noise sources that can degrade these images, we introduce an algorithm for multiscale image restoration through optimally sparse representation (MIRO). MIRO is a deterministic framework that models the acquisition process and uses pixelwise noise correction to improve image quality. Our study demonstrates that this approach yields a remarkable restoration of the fluorescence signal for a wide range of microscopy systems, regardless of the detector used (e.g., electron-multiplying charge-coupled device, scientific complementary metal-oxide semiconductor, or photomultiplier tube). MIRO improves current imaging capabilities, enabling fast, low-light optical microscopy, accurate image analysis, and robust machine intelligence when integrated with deep neural networks. This expands the range of biological knowledge that can be obtained from fluorescence microscopy.

Online citizen science with the Zooniverse for analysis of biological volumetric data

Public participation in research, also known as citizen science, is being increasingly adopted for the analysis of biological volumetric data. Researchers working in this domain are applying online citizen science as a scalable distributed data analysis approach, with recent research demonstrating that non-experts can productively contribute to tasks such as the segmentation of organelles in volume electron microscopy data. This, alongside the growing challenge to rapidly process the large amounts of biological volumetric data now routinely produced, means there is increasing interest within the research community to apply online citizen science for the analysis of data in this context. Here, we synthesise core methodological principles and practices for applying citizen science for analysis of biological volumetric data. We collate and share the knowledge and experience of multiple research teams who have applied online citizen science for the analysis of volumetric biological data using the Zooniverse platform ( www.zooniverse.org ). We hope this provides inspiration and practical guidance regarding how contributor effort via online citizen science may be usefully applied in this domain.

Big data in contemporary electron microscopy: challenges and opportunities in data transfer, compute and management

The second decade of the twenty-first century witnessed a new challenge in the handling of microscopy data. Big data, data deluge, large data, data compliance, data analytics, data integrity, data interoperability, data retention and data lifecycle are terms that have introduced themselves to the electron microscopy sciences. This is largely attributed to the booming development of new microscopy hardware tools. As a result, large digital image files with an average size of one terabyte within one single acquisition session is not uncommon nowadays, especially in the field of cryogenic electron microscopy. This brings along numerous challenges in data transfer, compute and management. In this review, we will discuss in detail the current state of international knowledge on big data in contemporary electron microscopy and how big data can be transferred, computed and managed efficiently and sustainably. Workflows, solutions, approaches and suggestions will be provided, with the example of the latest experiences in Australia. Finally, important principles such as data integrity, data lifetime and the FAIR and CARE principles will be considered.

Contributions of deep learning to automated numerical modelling of the interaction of electric fields and cartilage tissue based on 3D images

Electric fields find use in tissue engineering but also in sensor applications besides the broad classical application range. Accurate numerical models of electrical stimulation devices can pave the way for effective therapies in cartilage regeneration. To this end, the dielectric properties of the electrically stimulated tissue have to be known. However, knowledge of the dielectric properties is scarce. Electric field-based methods such as impedance spectroscopy enable determining the dielectric properties of tissue samples. To develop a detailed understanding of the interaction of the employed electric fields and the tissue, fine-grained numerical models based on tissue-specific 3D geometries are considered. A crucial ingredient in this approach is the automated generation of numerical models from biomedical images. In this work, we explore classical and artificial intelligence methods for volumetric image segmentation to generate model geometries. We find that deep learning, in particular the StarDist algorithm, permits fast and automatic model geometry and discretisation generation once a sufficient amount of training data is available. Our results suggest that already a small number of 3D images (23 images) is sufficient to achieve 80% accuracy on the test data. The proposed method enables the creation of high-quality meshes without the need for computer-aided design geometry post-processing. Particularly, the computational time for the geometrical model creation was reduced by half. Uncertainty quantification as well as a direct comparison between the deep learning and the classical approach reveal that the numerical results mainly depend on the cell volume. This result motivates further research into impedance sensors for tissue characterisation. The presented approach can significantly improve the accuracy and computational speed of image-based models of electrical stimulation for tissue engineering applications.

Sensing of Antibiotic-Bacteria Interactions

Sensing of antibiotic-bacteria interactions is an important area of research that has gained significant attention in recent years. Antibiotic resistance is a major public health concern, and it is essential to develop new strategies for detecting and monitoring bacterial responses to antibiotics in order to maintain effective antibiotic development and antibacterial treatment. This review summarizes recent advances in sensing strategies for antibiotic-bacteria interactions, which are divided into two main parts: studies on the mechanism of action for sensitive bacteria and interrogation of the defense mechanisms for resistant ones. In conclusion, this review provides an overview of the present research landscape concerning antibiotic-bacteria interactions, emphasizing the potential for method adaptation and the integration of machine learning techniques in data analysis, which could potentially lead to a transformative impact on mechanistic studies within the field.

Hapalosiphonacean cyanobacteria (Nostocales) thrived amid emerging embryophytes in an early Devonian (407-million-year-old) landscape

Cyanobacteria have a long evolutionary history, well documented in marine rocks. They are also abundant and diverse in terrestrial environments; however, although phylogenies suggest that the group colonized land early in its history, paleontological documentation of this remains limited. The Rhynie chert (407 Ma), our best preserved record of early terrestrial ecosystems, provides an opportunity to illuminate aspects of cyanobacterial diversity and ecology as plants began to radiate across the land surface. We used light microscopy and super-resolution confocal laser scanning microscopy to study a new population of Rhynie cyanobacteria; we also reinvestigated previously described specimens that resemble the new fossils. Our study demonstrates that all are part of a single fossil species belonging to the Hapalosiphonaceae (Nostocales). Along with other Rhynie microfossils, these remains show that the accommodation of morphologically complex cyanobacteria to terrestrial ecosystems transformed by embryophytes was well underway more than 400 million years ago.

Population-level antagonism between FGF and BMP signaling steers mesoderm differentiation in embryonic stem cells

The mesodermal precursor populations for different internal organ systems are specified during gastrulation by the combined activity of extracellular signaling systems such as BMP, Wnt, Nodal and FGF. The BMP, Wnt and Nodal signaling requirements for the differentiation of specific mesoderm subtypes in mammals have been mapped in detail, but how FGF shapes mesodermal cell type diversity is not precisely known. It is also not clear how FGF signaling integrates with the activity of other signaling systems involved in mesoderm differentiation. Here, we address these questions by analyzing the effects of targeted signaling manipulations in differentiating stem cell populations at single-cell resolution. We identify opposing functions of BMP and FGF, and map FGF-dependent and -independent mesodermal lineages. Stimulation with exogenous FGF boosts the expression of endogenous Fgf genes while repressing Bmp ligand genes. This positive autoregulation of FGF signaling, coupled with the repression of BMP signaling, may contribute to the specification of reproducible and coherent cohorts of cells with the same identity via a community effect, both in the embryo and in synthetic embryo-like systems.

Advancing Spectrally-Resolved Single Molecule Localization Microscopy with Deep Learning

Spectrally-resolved single molecule localization microscopy (srSMLM) is a recent technique enriching single molecule localization microscopy with the simultaneous recording of spectra of the single emitters. srSMLM resolution is limited by the number of photons collected per emitters. Sharing a photon budget to record the localization and the spectroscopic information results in a loss of spatial and spectral resolution-or forces the sacrifice of one at the expense of the other. Here, srUnet-a deep-learning Unet-based image processing routine trained to increase the spectral and spatial signals to compensate for the resolution loss inherent in additionally recording the spectral component is reported. Both localization and spectral precision are improved by srUnet-particularly for the low-emitting species. srUnet increases the fraction of localization whose signal can be both spatially and spectrally characterized. It preserves spectral shifts and the linearity of the dispersion of light. It strongly facilitates wavelength assignment in multicolor experiments. srUnet is a simple post-processing add-on boosting srSMLM performance close to conventional SMLM with the potential to turn srSMLM into the new standard for multicolor single molecule imaging.

Resolution enhancement with a task-assisted GAN to guide optical nanoscopy image analysis and acquisition

Super-resolution fluorescence microscopy methods enable the characterization of nanostructures in living and fixed biological tissues. However, they require the adjustment of multiple imaging parameters while attempting to satisfy conflicting objectives, such as maximizing spatial and temporal resolution while minimizing light exposure. To overcome the limitations imposed by these trade-offs, post-acquisition algorithmic approaches have been proposed for resolution enhancement and image-quality improvement. Here we introduce the task-assisted generative adversarial network (TA-GAN), which incorporates an auxiliary task (for example, segmentation, localization) closely related to the observed biological nanostructure characterization. We evaluate how the TA-GAN improves generative accuracy over unassisted methods, using images acquired with different modalities such as confocal, bright-field, stimulated emission depletion and structured illumination microscopy. The TA-GAN is incorporated directly into the acquisition pipeline of the microscope to predict the nanometric content of the field of view without requiring the acquisition of a super-resolved image. This information is used to automatically select the imaging modality and regions of interest, optimizing the acquisition sequence by reducing light exposure. Data-driven microscopy methods like the TA-GAN will enable the observation of dynamic molecular processes with spatial and temporal resolutions that surpass the limits currently imposed by the trade-offs constraining super-resolution microscopy.

Moving beyond the desktop: prospects for practical bioimage analysis via the web

As biological imaging continues to rapidly advance, it results in increasingly complex image data, necessitating a reevaluation of conventional bioimage analysis methods and their accessibility. This perspective underscores our belief that a transition from desktop-based tools to web-based bioimage analysis could unlock immense opportunities for improved accessibility, enhanced collaboration, and streamlined workflows. We outline the potential benefits, such as reduced local computational demands and solutions to common challenges, including software installation issues and limited reproducibility. Furthermore, we explore the present state of web-based tools, hurdles in implementation, and the significance of collective involvement from the scientific community in driving this transition. In acknowledging the potential roadblocks and complexity of data management, we suggest a combined approach of selective prototyping and large-scale workflow application for optimal usage. Embracing web-based bioimage analysis could pave the way for the life sciences community to accelerate biological research, offering a robust platform for a more collaborative, efficient, and democratized science.

MIA is an open-source standalone deep learning application for microscopic image analysis

In recent years, the amount of data generated by imaging techniques has grown rapidly, along with increasing computational power and the development of deep learning algorithms. To address the need for powerful automated image analysis tools for a broad range of applications in the biomedical sciences, the Microscopic Image Analyzer (MIA) was developed. MIA combines a graphical user interface that obviates the need for programming skills with state-of-the-art deep-learning algorithms for segmentation, object detection, and classification. It runs as a standalone, platform-independent application and uses open data formats, which are compatible with commonly used open-source software packages. The software provides a unified interface for easy image labeling, model training, and inference. Furthermore, the software was evaluated in a public competition and performed among the top three for all tested datasets.

Spatial topology of organelle is a new breast cancer cell classifier

Genomics and proteomics have been central to identify tumor cell populations, but more accurate approaches to classify cell subtypes are still lacking. We propose a new methodology to accurately classify cancer cells based on their organelle spatial topology. Herein, we developed an organelle topology-based cell classification pipeline (OTCCP), which integrates artificial intelligence (AI) and imaging quantification to analyze organelle spatial distribution and inter-organelle topology. OTCCP was used to classify a panel of human breast cancer cells, grown as 2D monolayer or 3D tumor spheroids using early endosomes, mitochondria, and their inter-organelle contacts. Organelle topology allows for a highly precise differentiation between cell lines of different subtypes and aggressiveness. These findings lay the groundwork for using organelle topological profiling as a fast and efficient method for phenotyping breast cancer function as well as a discovery tool to advance our understanding of cancer cell biology at the subcellular level.

Tissue optical clearing and 3D imaging of virus infections

Imaging pathogens within 3D environment of biological tissues provides spatial information about their localization and interactions with the host. Technological advances in fluorescence microscopy and 3D image analysis now permit visualization and quantification of pathogens directly in large tissue volumes and in great detail. In recent years large volume imaging became an important tool in virology research helping to understand the properties of viruses and the host response to infection. In this chapter we give a review of fluorescence microscopy modalities and tissue optical clearing methods used for large volume tissue imaging. A summary of recent applications for virus research is provided with particular emphasis on studies using light sheet fluorescence microscopy. We describe the challenges and approaches for volumetric image analysis. Practical examples of volumetric imaging implemented in virology laboratories and addressing specialized research questions, such as virus tropism and immune host response are described. We conclude with an overview of the emerging technologies and their potential for virus research.

Acquisition and annotation in high resolution in vivo digital biopsy by confocal microscopy for diagnosis in oral precancer and cancer

Introduction

Scanned fibre endomicroscopes are full point-scanning confocal microscopes with submicron lateral resolution with an optical slice thickness thin enough to isolate individual cell layers, allow active positioning of the optical slice in the z-axis and collection of megapixel images. Here we present descriptive findings and a brief atlas of an acquisition and annotation protocol high resolution in vivo capture of oral mucosal pathology including oral squamous cell carcinoma and dysplasia using a fluorescence scanned fibre endomicroscope with 3 topical fluorescent imaging agents: fluorescein, acriflavine and PARPi-FL.

Methods

Digital biopsy was successfully performed via an acquisition protocol in seventy-one patients presenting for investigation of oral mucosal abnormalities using a miniaturized, handheld scanned fibre endoscope. Multiple imaging agents were utilized and multiple time points sampled. Fifty-nine patients had a matched histopathology correlating in location with imaging. The images were annotated back to macrographic location using a purpose-built software, MouthMap™.

Results

Acquisition and annotation of cellular level resolved images was demonstrated with all 3 topical agents. Descriptive observations between clinically or histologically normal oral mucosa showed regular intranuclear distance, a regular nuclear profile and fluorescent homogeneity. This was dependent on the intraoral location and type of epithelium being observed. Key features of malignancy were a loss of intranuclear distance, disordered nuclear clustering and irregular nuclear fluorescence intensity and size. Perinuclear fluorescent granules were seen in the absence of irregular nuclear features in lichenoid inflammation.

Discussion

High resolution oral biopsy allows for painless and rapid capture of multiple mucosal sites, resulting in more data points to increase diagnostic precision. High resolution digital micrographs can be easily compared serially across multiple time points utilizing an annotation software. In the present study we have demonstrated realization of a high-resolution digital biopsy protocol of the oral mucosa for utility in the diagnosis of oral cancer and precancer..

Smart microscopes of the future

We dream of a future where light microscopes have new capabilities: language-guided image acquisition, automatic image analysis based on extensive prior training from biologist experts, and language-guided image analysis for custom analyses. Most capabilities have reached the proof-of-principle stage, but implementation would be accelerated by efforts to gather appropriate training sets and make user-friendly interfaces.

Motivation for using data-driven algorithms in research: A review of machine learning solutions for image analysis of micrographs in neuroscience

Machine learning is a powerful tool that is increasingly being used in many research areas, including neuroscience. The recent development of new algorithms and network architectures, especially in the field of deep learning, has made machine learning models more reliable and accurate and useful for the biomedical research sector. By minimizing the effort necessary to extract valuable features from datasets, they can be used to find trends in data automatically and make predictions about future data, thereby improving the reproducibility and efficiency of research. One application is the automatic evaluation of micrograph images, which is of great value in neuroscience research. While the development of novel models has enabled numerous new research applications, the barrier to use these new algorithms has also decreased by the integration of deep learning models into known applications such as microscopy image viewers. For researchers unfamiliar with machine learning algorithms, the steep learning curve can hinder the successful implementation of these methods into their workflows. This review explores the use of machine learning in neuroscience, including its potential applications and limitations, and provides some guidance on how to select a fitting framework to use in real-life research projects.

Label-free microscopy for virus infections

Microscopy has been essential to elucidate micro- and nano-scale processes in space and time and has provided insights into cell and organismic functions. It is widely employed in cell biology, microbiology, physiology, clinical sciences and virology. While label-dependent microscopy, such as fluorescence microscopy, provides molecular specificity, it has remained difficult to multiplex in live samples. In contrast, label-free microscopy reports on overall features of the specimen at minimal perturbation. Here, we discuss modalities of label-free imaging at the molecular, cellular and tissue levels, including transmitted light microscopy, quantitative phase imaging, cryogenic electron microscopy or tomography and atomic force microscopy. We highlight how label-free microscopy is used to probe the structural organization and mechanical properties of viruses, including virus particles and infected cells across a wide range of spatial scales. We discuss the working principles of imaging procedures and analyses and showcase how they open new avenues in virology. Finally, we discuss orthogonal approaches that enhance and complement label-free microscopy techniques.

Bridging Imaging Users to Imaging Analysis - A community survey

The "Bridging Imaging Users to Imaging Analysis" survey was conducted in 2022 by the Center for Open Bioimage Analysis (COBA), Bioimaging North America (BINA), and the Royal Microscopical Society Data Analysis in Imaging Section (RMS DAIM) to understand the needs of the imaging community. Through multi-choice and open-ended questions, the survey inquired about demographics, image analysis experiences, future needs, and suggestions on the role of tool developers and users. Participants of the survey were from diverse roles and domains of the life and physical sciences. To our knowledge, this is the first attempt to survey cross-community to bridge knowledge gaps between physical and life sciences imaging. Survey results indicate that respondents' overarching needs are documentation, detailed tutorials on the usage of image analysis tools, user-friendly intuitive software, and better solutions for segmentation, ideally in a format tailored to their specific use cases. The tool creators suggested the users familiarize themselves with the fundamentals of image analysis, provide constant feedback, and report the issues faced during image analysis while the users would like more documentation and an emphasis on tool friendliness. Regardless of the computational experience, there is a strong preference for 'written tutorials' to acquire knowledge on image analysis. We also observed that the interest in having 'office hours' to get an expert opinion on their image analysis methods has increased over the years. In addition, the community suggests the need for a common repository for the available image analysis tools and their applications. The opinions and suggestions of the community, released here in full, will help the image analysis tool creation and education communities to design and deliver the resources accordingly.

Cross-modality supervised image restoration enables nanoscale tracking of synaptic plasticity in living mice

Learning is thought to involve changes in glutamate receptors at synapses, submicron structures that mediate communication between neurons in the central nervous system. Due to their small size and high density, synapses are difficult to resolve in vivo, limiting our ability to directly relate receptor dynamics to animal behavior. Here we developed a combination of computational and biological methods to overcome these challenges. First, we trained a deep-learning image-restoration algorithm that combines the advantages of ex vivo super-resolution and in vivo imaging modalities to overcome limitations specific to each optical system. When applied to in vivo images from transgenic mice expressing fluorescently labeled glutamate receptors, this restoration algorithm super-resolved synapses, enabling the tracking of behavior-associated synaptic plasticity with high spatial resolution. This method demonstrates the capabilities of image enhancement to learn from ex vivo data and imaging techniques to improve in vivo imaging resolution.

Content aware image restoration improves spatiotemporal resolution in luminescence imaging

Luminescent reporters are due to their intrinsically high signal-to-noise ratio a powerful labelling tool for microscopy and macroscopic in vivo imaging in biomedical research. However, luminescence signal detection requires longer exposure times than fluorescence imaging and is consequently less suited for applications requiring high temporal resolution or throughput. Here we demonstrate that content aware image restoration can drastically reduce the exposure time requirements in luminescence imaging, thus overcoming one of the major limitations of the technique.

Quantitative Single-Molecule Localization Microscopy

Super-resolution fluorescence microscopy allows the investigation of cellular structures at nanoscale resolution using light. Current developments in super-resolution microscopy have focused on reliable quantification of the underlying biological data. In this review, we first describe the basic principles of super-resolution microscopy techniques such as stimulated emission depletion (STED) microscopy and single-molecule localization microscopy (SMLM), and then give a broad overview of methodological developments to quantify super-resolution data, particularly those geared toward SMLM data. We cover commonly used techniques such as spatial point pattern analysis, colocalization, and protein copy number quantification but also describe more advanced techniques such as structural modeling, single-particle tracking, and biosensing. Finally, we provide an outlook on exciting new research directions to which quantitative super-resolution microscopy might be applied.

EVL and MIM/MTSS1 regulate actin cytoskeletal remodeling to promote dendritic filopodia in neurons

Dendritic spines are the postsynaptic compartment of a neuronal synapse and are critical for synaptic connectivity and plasticity. A developmental precursor to dendritic spines, dendritic filopodia (DF), facilitate synapse formation by sampling the environment for suitable axon partners during neurodevelopment and learning. Despite the significance of the actin cytoskeleton in driving these dynamic protrusions, the actin elongation factors involved are not well characterized. We identified the Ena/VASP protein EVL as uniquely required for the morphogenesis and dynamics of DF. Using a combination of genetic and optogenetic manipulations, we demonstrated that EVL promotes protrusive motility through membrane-direct actin polymerization at DF tips. EVL forms a complex at nascent protrusions and DF tips with MIM/MTSS1, an I-BAR protein important for the initiation of DF. We proposed a model in which EVL cooperates with MIM to coalesce and elongate branched actin filaments, establishing the dynamic lamellipodia-like architecture of DF.

Deep learning techniques and mathematical modeling allow 3D analysis of mitotic spindle dynamics

Time-lapse microscopy movies have transformed the study of subcellular dynamics. However, manual analysis of movies can introduce bias and variability, obscuring important insights. While automation can overcome such limitations, spatial and temporal discontinuities in time-lapse movies render methods such as 3D object segmentation and tracking difficult. Here, we present SpinX, a framework for reconstructing gaps between successive image frames by combining deep learning and mathematical object modeling. By incorporating expert feedback through selective annotations, SpinX identifies subcellular structures, despite confounding neighbor-cell information, non-uniform illumination, and variable fluorophore marker intensities. The automation and continuity introduced here allows the precise 3D tracking and analysis of spindle movements with respect to the cell cortex for the first time. We demonstrate the utility of SpinX using distinct spindle markers, cell lines, microscopes, and drug treatments. In summary, SpinX provides an exciting opportunity to study spindle dynamics in a sophisticated way, creating a framework for step changes in studies using time-lapse microscopy.

The basolateral amygdala-anterior cingulate pathway contributes to depression-like behaviors and comorbidity with chronic pain behaviors in male mice

While depression and chronic pain are frequently comorbid, underlying neuronal circuits and their psychopathological relevance remain poorly defined. Here we show in mice that hyperactivity of the neuronal pathway linking the basolateral amygdala to the anterior cingulate cortex is essential for chronic pain-induced depression. Moreover, activation of this pathway in naive male mice, in the absence of on-going pain, is sufficient to trigger depressive-like behaviors, as well as transcriptomic alterations that recapitulate core molecular features of depression in the human brain. These alterations notably impact gene modules related to myelination and the oligodendrocyte lineage. Among these, we show that Sema4a, which was significantly upregulated in both male mice and humans in the context of altered mood, is necessary for the emergence of emotional dysfunction. Overall, these results place the amygdalo-cingulate pathway at the core of pain and depression comorbidity, and unravel the role of Sema4a and impaired myelination in mood control.

Neural networks determination of material elastic constants and structures in nematic complex fluids

Supervised machine learning and artificial neural network approaches can allow for the determination of selected material parameters or structures from a measurable signal without knowing the exact mathematical relationship between them. Here, we demonstrate that material nematic elastic constants and the initial structural material configuration can be found using sequential neural networks applied to the transmmited time-dependent light intensity through the nematic liquid crystal (NLC) sample under crossed polarizers. Specifically, we simulate multiple times the relaxation of the NLC from a random (qeunched) initial state to the equilibirum for random values of elastic constants and, simultaneously, the transmittance of the sample for monochromatic polarized light. The obtained time-dependent light transmittances and the corresponding elastic constants form a training data set on which the neural network is trained, which allows for the determination of the elastic constants, as well as the initial state of the director. Finally, we demonstrate that the neural network trained on numerically generated examples can also be used to determine elastic constants from experimentally measured data, finding good agreement between experiments and neural network predictions.

Blood quality evaluation via on-chip classification of cell morphology using a deep learning algorithm

The quality of red blood cells (RBCs) in stored blood has a direct impact on the recovery of patients treated by blood transfusion, which directly reflects the quality of blood. The traditional means for blood quality evaluation involve the use of reagents and multi-step and time-consuming operations. Here, a low-cost, multi-classification, label-free and high-precision method is developed, which combines microfluidic technology and a deep learning algorithm together to recognize and classify RBCs based on morphology. The microfluidic channel is designed to effectively and controllably solve the problem of cell overlap, which has a severe negative impact on the identification of cells. The object detection model in the deep learning algorithm is optimized and used to recognize multiple RBCs simultaneously in the whole field of view, so as to classify them into six morphological subcategories and count the numbers in each subgroup. The mean average precision of the developed object detection model reaches 89.24%. The blood quality can be evaluated by calculating the morphology index (MI) according to the numbers of cells in subgroups. The validation of the method is verified by evaluating three blood samples stored for 7 days, 21 days and 42 days, which have MIs of 84.53%, 73.33% and 24.34%, respectively, indicating good agreement with the actual blood quality. This method has the merits of cell identification in a wide channel, no need for single cell alignment as the image cytometry does and it is not only applicable to the quality evaluation of RBCs, but can also be used for general cell identifications with different morphologies.

Applications and Advances of Multicellular Tumor Spheroids: Challenges in Their Development and Analysis

Biomedical research requires both in vitro and in vivo studies in order to explore disease processes or drug interactions. Foundational investigations have been performed at the cellular level using two-dimensional cultures as the gold-standard method since the early 20th century. However, three-dimensional (3D) cultures have emerged as a new tool for tissue modeling over the last few years, bridging the gap between in vitro and animal model studies. Cancer has been a worldwide challenge for the biomedical community due to its high morbidity and mortality rates. Various methods have been developed to produce multicellular tumor spheroids (MCTSs), including scaffold-free and scaffold-based structures, which usually depend on the demands of the cells used and the related biological question. MCTSs are increasingly utilized in studies involving cancer cell metabolism and cell cycle defects. These studies produce massive amounts of data, which demand elaborate and complex tools for thorough analysis. In this review, we discuss the advantages and disadvantages of several up-to-date methods used to construct MCTSs. In addition, we also present advanced methods for analyzing MCTS features. As MCTSs more closely mimic the in vivo tumor environment, compared to 2D monolayers, they can evolve to be an appealing model for in vitro tumor biology studies.

An end-to-end pipeline based on open source deep learning tools for reliable analysis of complex 3D images of ovaries

Computational analysis of bio-images by deep learning (DL) algorithms has made exceptional progress in recent years and has become much more accessible to non-specialists with the development of ready-to-use tools. The study of oogenesis mechanisms and female reproductive success has also recently benefited from the development of efficient protocols for three-dimensional (3D) imaging of ovaries. Such datasets have a great potential for generating new quantitative data but are, however, complex to analyze due to the lack of efficient workflows for 3D image analysis. Here, we have integrated two existing open-source DL tools, Noise2Void and Cellpose, into an analysis pipeline dedicated to 3D follicular content analysis, which is available on Fiji. Our pipeline was developed on larvae and adult medaka ovaries but was also successfully applied to different types of ovaries (trout, zebrafish and mouse). Image enhancement, Cellpose segmentation and post-processing of labels enabled automatic and accurate quantification of these 3D images, which exhibited irregular fluorescent staining, low autofluorescence signal or heterogeneous follicles sizes. In the future, this pipeline will be useful for extensive cellular phenotyping in fish or mammals for developmental or toxicology studies.

Deep learning-enabled segmentation of ambiguous bioimages with deepflash2

Bioimages frequently exhibit low signal-to-noise ratios due to experimental conditions, specimen characteristics, and imaging trade-offs. Reliable segmentation of such ambiguous images is difficult and laborious. Here we introduce deepflash2, a deep learning-enabled segmentation tool for bioimage analysis. The tool addresses typical challenges that may arise during the training, evaluation, and application of deep learning models on ambiguous data. The tool's training and evaluation pipeline uses multiple expert annotations and deep model ensembles to achieve accurate results. The application pipeline supports various use-cases for expert annotations and includes a quality assurance mechanism in the form of uncertainty measures. Benchmarked against other tools, deepflash2 offers both high predictive accuracy and efficient computational resource usage. The tool is built upon established deep learning libraries and enables sharing of trained model ensembles with the research community. deepflash2 aims to simplify the integration of deep learning into bioimage analysis projects while improving accuracy and reliability.

Fluorescence based HTS-compatible ligand binding assays for dopamine D3 receptors in baculovirus preparations and live cells

Dopamine receptors are G-protein-coupled receptors that are connected to severe neurological disorders. The development of new ligands targeting these receptors enables gaining a deeper insight into the receptor functioning, including binding mechanisms, kinetics and oligomerization. Novel fluorescent probes allow the development of more efficient, cheaper, reliable and scalable high-throughput screening systems, which speeds up the drug development process. In this study, we used a novel Cy3B labelled commercially available fluorescent ligand CELT-419 for developing dopamine D3 receptor-ligand binding assays with fluorescence polarization and quantitative live cell epifluorescence microscopy. The fluorescence anisotropy assay using 384-well plates achieved Z’ value of 0.71, which is suitable for high-throughput screening of ligand binding. The assay can also be used to determine the kinetics of both the fluorescent ligand as well as some reference unlabeled ligands. Furthermore, CELT-419 was also used with live HEK293-D3R cells in epifluorescence microscopy imaging for deep-learning-based ligand binding quantification. This makes CELT-419 quite a universal fluorescence probe which has the potential to be also used in more advanced microscopy techniques resulting in more comparable studies.

Optofluidic imaging meets deep learning: from merging to emerging

Propelled by the striking advances in optical microscopy and deep learning (DL), the role of imaging in lab-on-a-chip has dramatically been transformed from a silo inspection tool to a quantitative "smart" engine. A suite of advanced optical microscopes now enables imaging over a range of spatial scales (from molecules to organisms) and temporal window (from microseconds to hours). On the other hand, the staggering diversity of DL algorithms has revolutionized image processing and analysis at the scale and complexity that were once inconceivable. Recognizing these exciting but overwhelming developments, we provide a timely review of their latest trends in the context of lab-on-a-chip imaging, or coined optofluidic imaging. More importantly, here we discuss the strengths and caveats of how to adopt, reinvent, and integrate these imaging techniques and DL algorithms in order to tailor different lab-on-a-chip applications. In particular, we highlight three areas where the latest advances in lab-on-a-chip imaging and DL can form unique synergisms: image formation, image analytics and intelligent image-guided autonomous lab-on-a-chip. Despite the on-going challenges, we anticipate that they will represent the next frontiers in lab-on-a-chip imaging that will spearhead new capabilities in advancing analytical chemistry research, accelerating biological discovery, and empowering new intelligent clinical applications.

Automated quantification and statistical assessment of proliferating cardiomyocyte rates in embryonic hearts

The use of digital image analysis and count regression models contributes to the reproducibility and rigor of histological studies in cardiovascular research. The use of formalized computer-based quantification strategies of histological images essentially removes potential researcher bias, allows for higher analysis throughput, and enables easy sharing of formalized quantification tools, contributing to research transparency, and data transferability. Moreover, the use of count regression models rather than ratios in statistical analysis of cell population data incorporates the extent of sampling into analysis and acknowledges the non-Gaussian nature of count distributions. Using quantification of proliferating cardiomyocytes in embryonic murine hearts as an example, we describe how these improvements can be implemented using open-source artificial intelligence-based image analysis tools and novel count regression models to efficiently analyze real-life data.

The anatomy of transcriptionally active chromatin loops in Drosophila primary spermatocytes using super-resolution microscopy

While the biochemistry of gene transcription has been well studied, our understanding of how this process is organised in 3D within the intact nucleus is less well understood. Here we investigate the structure of actively transcribed chromatin and the architecture of its interaction with active RNA polymerase. For this analysis, we have used super-resolution microscopy to image the Drosophila melanogaster Y loops which represent huge, several megabases long, single transcription units. The Y loops provide a particularly amenable model system for transcriptionally active chromatin. We find that, although these transcribed loops are decondensed they are not organised as extended 10nm fibres, but rather they largely consist of chains of nucleosome clusters. The average width of each cluster is around 50nm. We find that foci of active RNA polymerase are generally located off the main fibre axis on the periphery of the nucleosome clusters. Foci of RNA polymerase and nascent transcripts are distributed around the Y loops rather than being clustered in individual transcription factories. However, as the RNA polymerase foci are considerably less prevalent than the nucleosome clusters, the organisation of this active chromatin into chains of nucleosome clusters is unlikely to be determined by the activity of the polymerases transcribing the Y loops. These results provide a foundation for understanding the topological relationship between chromatin and the process of gene transcription.

Myosin-X recruits lamellipodin to filopodia tips

Myosin-X (MYO10), a molecular motor localizing to filopodia, is thought to transport various cargo to filopodia tips, modulating filopodia function. However, only a few MYO10 cargoes have been described. Here, using GFP-Trap and BioID approaches combined with mass spectrometry, we identified lamellipodin (RAPH1) as a novel MYO10 cargo. We report that the FERM domain of MYO10 is required for RAPH1 localization and accumulation at filopodia tips. Previous studies have mapped the RAPH1 interaction domain for adhesome components to its talin-binding and Ras-association domains. Surprisingly, we find that the RAPH1 MYO10-binding site is not within these domains. Instead, it comprises a conserved helix located just after the RAPH1 pleckstrin homology domain with previously unknown functions. Functionally, RAPH1 supports MYO10 filopodia formation and stability but is not required to activate integrins at filopodia tips. Taken together, our data indicate a feed-forward mechanism whereby MYO10 filopodia are positively regulated by MYO10-mediated transport of RAPH1 to the filopodium tip.

A mechanism of salt bridge-mediated resistance to FtsZ inhibitor PC190723 revealed by a cell-based screen

Bacterial cell division proteins, especially the tubulin homologue FtsZ, have emerged as strong targets for developing new antibiotics. Here, we have utilized the fission yeast heterologous expression system to develop a cell-based assay to screen for small molecules that directly and specifically target the bacterial cell division protein FtsZ. The strategy also allows for simultaneous assessment of the toxicity of the drugs to eukaryotic yeast cells. As a proof-of-concept of the utility of this assay, we demonstrate the effect of the inhibitors sanguinarine, berberine, and PC190723 on FtsZ. Though sanguinarine and berberine affect FtsZ polymerization, they exert a toxic effect on the cells. Further, using this assay system, we show that PC190723 affects Helicobacter pylori FtsZ function and gain new insights into the molecular determinants of resistance to PC190723. On the basis of sequence and structural analysis and site-specific mutations, we demonstrate that the presence of salt bridge interactions between the central H7 helix and β-strands S9 and S10 mediates resistance to PC190723 in FtsZ. The single-step in vivo cell-based assay using fission yeast enabled us to dissect the contribution of sequence-specific features of FtsZ and cell permeability effects associated with bacterial cell envelopes. Thus, our assay serves as a potent tool to rapidly identify novel compounds targeting polymeric bacterial cytoskeletal proteins like FtsZ to understand how they alter polymerization dynamics and address resistance determinants in targets.

Quanty-cFOS, a Novel ImageJ/Fiji Algorithm for Automated Counting of Immunoreactive Cells in Tissue Sections

Analysis of neural encoding and plasticity processes frequently relies on studying spatial patterns of activity-induced immediate early genes' expression, such as c-fos. Quantitatively analyzing the numbers of cells expressing the Fos protein or c-fos mRNA is a major challenge owing to large human bias, subjectivity and variability in baseline and activity-induced expression. Here, we describe a novel open-source ImageJ/Fiji tool, called 'Quanty-cFOS', with an easy-to-use, streamlined pipeline for the automated or semi-automated counting of cells positive for the Fos protein and/or c-fos mRNA on images derived from tissue sections. The algorithms compute the intensity cutoff for positive cells on a user-specified number of images and apply this on all the images to process. This allows for the overcoming of variations in the data and the deriving of cell counts registered to specific brain areas in a highly time-efficient and reliable manner. We validated the tool using data from brain sections in response to somatosensory stimuli in a user-interactive manner. Here, we demonstrate the application of the tool in a step-by-step manner, with video tutorials, making it easy for novice users to implement. Quanty-cFOS facilitates a rapid, accurate and unbiased spatial mapping of neural activity and can also be easily extended to count other types of labelled cells.

Tutorial: multiphoton microscopy to advance neuroscience research

Multiphoton microscopy (MPM) employs ultrafast infrared lasers for high-resolution deep three-dimensional imaging of live biological samples. The goal of this tutorial is to provide a practical guide to MPM imaging for novice microscopy developers and life-science users. Principles of MPM, microscope setup, and labeling strategies are discussed. Use of MPM to achieve unprecedented imaging depth of whole mounted explants and intravital imaging via implantable glass windows of the mammalian nervous system is demonstrated.

Fast4DReg - fast registration of 4D microscopy datasets

Unwanted sample drift is a common issue that plagues microscopy experiments, preventing accurate temporal visualization and quantification of biological processes. Although multiple methods and tools exist to correct images post acquisition, performing drift correction of three-dimensional (3D) videos using open-source solutions remains challenging and time consuming. Here, we present a new tool developed for ImageJ or Fiji called Fast4DReg that can quickly correct axial and lateral drift in 3D video-microscopy datasets. Fast4DReg works by creating intensity projections along multiple axes and estimating the drift between frames using two-dimensional cross-correlations. Using synthetic and acquired datasets, we demonstrate that Fast4DReg can perform better than other state-of-the-art open-source drift-correction tools and significantly outperforms them in speed. We also demonstrate that Fast4DReg can be used to register misaligned channels in 3D using either calibration slides or misaligned images directly. Altogether, Fast4DReg provides a quick and easy-to-use method to correct 3D imaging data before further visualization and analysis.