openFrame: A modular, sustainable, open microscopy platform with single-shot, dual-axis optical autofocus module providing high precision and long range of operation

'openFrame' is a modular, low-cost, open-hardware microscopy platform that can be configured or adapted to most light microscopy techniques and is easily upgradeable or expandable to multiple modalities. The ability to freely mix and interchange both open-source and proprietary hardware components or software enables low-cost, yet research-grade instruments to be assembled and maintained. It also enables rapid prototyping of advanced or novel microscope systems. For long-term time-lapse image data acquisition, slide-scanning or high content analysis, we have developed a novel optical autofocus incorporating orthogonal cylindrical optics to provide robust single-shot closed-loop focus lock, which we have demonstrated to accommodate defocus up to ±37 μm with <200 nm accuracy, and a two-step autofocus mode which we have shown can operate with defocus up to ±68 μm. We have used this to implement automated single molecule localisation microscopy (SMLM) in a relatively low-cost openFrame-based instrument using multimode diode lasers for excitation and cooled CMOS cameras.

Robust deep learning optical autofocus system applied to automated multiwell plate single molecule localization microscopy

We presenta robust, long-range optical autofocus system for microscopy utilizing machine learning. This can be useful for experiments with long image data acquisition times that may be impacted by defocusing resulting from drift of components, for example due to changes in temperature or mechanical drift. It is also useful for automated slide scanning or multiwell plate imaging where the sample(s) to be imaged may not be in the same horizontal plane throughout the image data acquisition. To address the impact of (thermal or mechanical) fluctuations over time in the optical autofocus system itself, we utilize a convolutional neural network (CNN) that is trained over multiple days to account for such fluctuations. To address the trade-off between axial precision and range of the autofocus, we implement orthogonal optical readouts with separate CNN training data, thereby achieving an accuracy well within the 600 nm depth of field of our 1.3 numerical aperture objective lens over a defocus range of up to approximately +/-100 μm. We characterize the performance of this autofocus system and demonstrate its application to automated multiwell plate single molecule localization microscopy.

Single-shot phase contrast microscopy using polarisation-resolved differential phase contrast

We present a robust, low-cost single-shot implementation of differential phase microscopy utilising a polarisation-sensitive camera to simultaneously acquire four images from which phase contrast images can be calculated. This polarisation-resolved differential phase contrast (pDPC) microscopy technique can be easily integrated with fluorescence microscopy.

Smad4 controls signaling robustness and morphogenesis by differentially contributing to the Nodal and BMP pathways

The transcriptional effector SMAD4 is a core component of the TGF-β family signaling pathways. However, its role in vertebrate embryo development remains unresolved. To address this, we deleted Smad4 in zebrafish and investigated the consequences of this on signaling by the TGF-β family morphogens, BMPs and Nodal. We demonstrate that in the absence of Smad4, dorsal/ventral embryo patterning is disrupted due to the loss of BMP signaling. However, unexpectedly, Nodal signaling is maintained, but lacks robustness. This Smad4-independent Nodal signaling is sufficient for mesoderm specification, but not for optimal endoderm specification. Furthermore, using Optical Projection Tomography in combination with 3D embryo morphometry, we have generated a BMP morphospace and demonstrate that Smad4 mutants are morphologically indistinguishable from embryos in which BMP signaling has been genetically/pharmacologically perturbed. Smad4 is thus differentially required for signaling by different TGF-β family ligands, which has implications for diseases where Smad4 is mutated or deleted.

Application of direct stochastic optical reconstruction microscopy (dSTORM) to the histological analysis of human glomerular disease

Electron microscopy (EM) following immunofluorescence (IF) imaging is a vital tool for the diagnosis of human glomerular diseases, but the implementation of EM is limited to specialised institutions and it is not available in many countries. Recent progress in fluorescence microscopy now enables conventional widefield fluorescence microscopes to be adapted at modest cost to provide resolution below 50 nm in biological specimens. We show that stochastically switched single-molecule localisation microscopy can be applied to clinical histological sections stained with standard IF techniques and that such super-resolved IF may provide an alternative means to resolve ultrastructure to aid the diagnosis of kidney disease where EM is not available. We have implemented the direct stochastic optical reconstruction microscopy technique with human kidney biopsy frozen sections stained with clinically approved immunofluorescent probes for the basal laminae and immunoglobulin G deposits. Using cases of membranous glomerulonephritis, thin basement membrane lesion, and lupus nephritis, we compare this approach to clinical EM images and demonstrate enhanced imaging compared to conventional IF microscopy. With minor modifications in established IF protocols of clinical frozen renal biopsies, we believe the cost-effective adaptation of conventional widefield microscopes can be widely implemented to provide super-resolved image information to aid diagnosis of human glomerular disease.

Convolutional neural networks for reconstruction of undersampled optical projection tomography data applied to in vivo imaging of zebrafish

Optical projection tomography (OPT) is a 3D mesoscopic imaging modality that can utilize absorption or fluorescence contrast. 3D images can be rapidly reconstructed from tomographic data sets sampled with sufficient numbers of projection angles using the Radon transform, as is typically implemented with optically cleared samples of the mm-to-cm scale. For in vivo imaging, considerations of phototoxicity and the need to maintain animals under anesthesia typically preclude the acquisition of OPT data at a sufficient number of angles to avoid artifacts in the reconstructed images. For sparse samples, this can be addressed with iterative algorithms to reconstruct 3D images from undersampled OPT data, but the data processing times present a significant challenge for studies imaging multiple animals. We show here that convolutional neural networks (CNN) can be used in place of iterative algorithms to remove artifacts-reducing processing time for an undersampled in vivo zebrafish dataset from 77 to 15 minutes. We also show that using CNN produces reconstructions of equivalent quality to compressed sensing with 40% fewer projections. We further show that diverse training data classes, for example, ex vivo mouse tissue data, can be used for CNN-based reconstructions of OPT data of other species including live zebrafish.

Automated Fluorescence Lifetime Imaging High-Content Analysis of Förster Resonance Energy Transfer between Endogenously Labeled Kinetochore Proteins in Live Budding Yeast Cells

We describe an open-source automated multiwell plate fluorescence lifetime imaging (FLIM) methodology to read out Förster resonance energy transfer (FRET) between fluorescent proteins (FPs) labeling endogenous kinetochore proteins (KPs) in live budding yeast cells. The low copy number of many KPs and their small spatial extent present significant challenges for the quantification of donor fluorescence lifetime in the presence of significant cellular autofluorescence and photobleaching. Automated FLIM data acquisition was controlled by µManager and incorporated wide-field time-gated imaging with optical sectioning to reduce background fluorescence. For data analysis, we used custom MATLAB-based software tools to perform kinetochore foci segmentation and local cellular background subtraction and fitted the fluorescence lifetime data using the open-source FLIMfit software. We validated the methodology using endogenous KPs labeled with mTurquoise2 FP and/or yellow FP and measured the donor fluorescence lifetimes for foci comprising 32 kinetochores with KP copy numbers as low as ~2 per kinetochore under an average labeling efficiency of 50%. We observed changes of median donor lifetime ≥250 ps for KPs known to form dimers. Thus, this FLIM high-content analysis platform enables the screening of relatively low-copy-number endogenous protein-protein interactions at spatially confined macromolecular complexes.

Quantitative time domain analysis of lifetime-based Förster resonant energy transfer measurements with fluorescent proteins: Static random isotropic fluorophore orientation distributions

Förster resonant energy transfer (FRET) measurements are widely used to obtain information about molecular interactions and conformations through the dependence of FRET efficiency on the proximity of donor and acceptor fluorophores. Fluorescence lifetime measurements can provide quantitative analysis of FRET efficiency and interacting population fraction. Many FRET experiments exploit the highly specific labelling of genetically expressed fluorescent proteins, applicable in live cells and organisms. Unfortunately, the typical assumption of fast randomization of fluorophore orientations in the analysis of fluorescence lifetime-based FRET readouts is not valid for fluorescent proteins due to their slow rotational mobility compared to their upper state lifetime. Here, previous analysis of effectively static isotropic distributions of fluorophore dipoles on FRET measurements is incorporated into new software for fitting donor emission decay profiles. Calculated FRET parameters, including molar population fractions, are compared for the analysis of simulated and experimental FRET data under the assumption of static and dynamic fluorophores and the intermediate regimes between fully dynamic and static fluorophores, and mixtures within FRET pairs, is explored. Finally, a method to correct the artefact resulting from fitting the emission from static FRET pairs with isotropic angular distributions to the (incorrect) typically assumed dynamic FRET decay model is presented.

In vivo multiphoton microscopy using a handheld scanner with lateral and axial motion compensation

This paper reports a handheld multiphoton fluorescence microscope designed for clinical imaging that incorporates axial motion compensation and lateral image stabilization. Spectral domain optical coherence tomography is employed to track the axial position of the skin surface, and lateral motion compensation is realised by imaging the speckle pattern arising from the optical coherence tomography beam illuminating the sample. Our system is able to correct lateral sample velocities of up to approximately 65 μm s^-1 . Combined with the use of negative curvature microstructured optical fibre to deliver tunable ultrafast radiation to the handheld multiphoton scanner without the need of a dispersion compensation unit, this instrument has potential for a range of clinical applications. The system is used to compensate for both lateral and axial motion of the sample when imaging human skin in vivo.

Open Source High Content Analysis Utilizing Automated Fluorescence Lifetime Imaging Microscopy

We present an open source high content analysis instrument utilizing automated fluorescence lifetime imaging (FLIM) for assaying protein interactions using Förster resonance energy transfer (FRET) based readouts of fixed or live cells in multiwell plates. This provides a means to screen for cell signaling processes read out using intramolecular FRET biosensors or intermolecular FRET of protein interactions such as oligomerization or heterodimerization, which can be used to identify binding partners. We describe here the functionality of this automated multiwell plate FLIM instrumentation and present exemplar data from our studies of HIV Gag protein oligomerization and a time course of a FRET biosensor in live cells. A detailed description of the practical implementation is then provided with reference to a list of hardware components and a description of the open source data acquisition software written in µManager. The application of FLIMfit, an open source MATLAB-based client for the OMERO platform, to analyze arrays of multiwell plate FLIM data is also presented. The protocols for imaging fixed and live cells are outlined and a demonstration of an automated multiwell plate FLIM experiment using cells expressing fluorescent protein-based FRET constructs is presented. This is complemented by a walk-through of the data analysis for this specific FLIM FRET data set.

Conformational transition of FGFR kinase activation revealed by site-specific unnatural amino acid reporter and single molecule FRET

Protein kinases share significant structural similarity; however, structural features alone are insufficient to explain their diverse functions. Thus, bridging the gap between static structure and function requires a more detailed understanding of their dynamic properties. For example, kinase activation may occur via a switch-like mechanism or by shifting a dynamic equilibrium between inactive and active states. Here, we utilize a combination of FRET and molecular dynamics (MD) simulations to probe the activation mechanism of the kinase domain of Fibroblast Growth Factor Receptor (FGFR). Using genetically-encoded, site-specific incorporation of unnatural amino acids in regions essential for activation, followed by specific labeling with fluorescent moieties, we generated a novel class of FRET-based reporter to monitor conformational differences corresponding to states sampled by non phosphorylated/inactive and phosphorylated/active forms of the kinase. Single molecule FRET analysis in vitro, combined with MD simulations, shows that for FGFR kinase, there are populations of inactive and active states separated by a high free energy barrier resulting in switch-like activation. Compared to recent studies, these findings support diversity in features of kinases that impact on their activation mechanisms. The properties of these FRET-based constructs will also allow further studies of kinase dynamics as well as applications in vivo.

Quantitative in vivo optical tomography of cancer progression & vasculature development in adult zebrafish

We describe a novel approach to study tumour progression and vasculature development in vivo via global 3-D fluorescence imaging of live non-pigmented adult zebrafish utilising angularly multiplexed optical projection tomography with compressive sensing (CS-OPT). This "mesoscopic" imaging method bridges a gap between established ~μm resolution 3-D fluorescence microscopy techniques and ~mm-resolved whole body planar imaging and diffuse tomography. Implementing angular multiplexing with CS-OPT, we demonstrate the in vivo global imaging of an inducible fluorescently labelled genetic model of liver cancer in adult non-pigmented zebrafish that also present fluorescently labelled vasculature. In this disease model, addition of a chemical inducer (doxycycline) drives expression of eGFP tagged oncogenic K-RASV12 in the liver of immune competent animals. We show that our novel in vivo global imaging methodology enables non-invasive quantitative imaging of the development of tumour and vasculature throughout the progression of the disease, which we have validated against established methods of pathology including immunohistochemistry. We have also demonstrated its potential for longitudinal imaging through a study of vascular development in the same zebrafish from early embryo to adulthood. We believe that this instrument, together with its associated analysis and data management tools, constitute a new platform for in vivo cancer studies and drug discovery in zebrafish disease models.

Visualising apoptosis in live zebrafish using fluorescence lifetime imaging with optical projection tomography to map FRET biosensor activity in space and time

Fluorescence lifetime imaging (FLIM) combined with optical projection tomography (OPT) has the potential to map Förster resonant energy transfer (FRET) readouts in space and time in intact transparent or near transparent live organisms such as zebrafish larvae, thereby providing a means to visualise cell signalling processes in their physiological context. Here the first application of FLIM OPT to read out biological function in live transgenic zebrafish larvae using a genetically expressed FRET biosensor is reported. Apoptosis, or programmed cell death, is mapped in 3-D by imaging the activity of a FRET biosensor that is cleaved by Caspase 3, which is a key effector of apoptosis. Although apoptosis is a naturally occurring process during development, it can also be triggered in a variety of ways, including through gamma irradiation. FLIM OPT is shown here to enable apoptosis to be monitored over time, in live zebrafish larvae via changes in Caspase 3 activation following gamma irradiation at 24 hours post fertilisation. Significant apoptosis was observed at 3.5 hours post irradiation, predominantly in the head region.

Mesoscopic in vivo 3-D tracking of sparse cell populations using angular multiplexed optical projection tomography

We describe an angular multiplexed imaging technique for 3-D in vivo cell tracking of sparse cell distributions and optical projection tomography (OPT) with superior time-lapse resolution and a significantly reduced light dose compared to volumetric time-lapse techniques. We demonstrate that using dual axis OPT, where two images are acquired simultaneously at different projection angles, can enable localization and tracking of features in 3-D with a time resolution equal to the camera frame rate. This is achieved with a 200x reduction in light dose compared to an equivalent volumetric time-lapse single camera OPT acquisition with 200 projection angles. We demonstrate the application of this technique to mapping the 3-D neutrophil migration pattern observed over ~25.5 minutes in a live 2 day post-fertilisation transgenic LysC:GFP zebrafish embryo following a tail wound.

Rapid global fitting of large fluorescence lifetime imaging microscopy datasets

Fluorescence lifetime imaging (FLIM) is widely applied to obtain quantitative information from fluorescence signals, particularly using Förster Resonant Energy Transfer (FRET) measurements to map, for example, protein-protein interactions. Extracting FRET efficiencies or population fractions typically entails fitting data to complex fluorescence decay models but such experiments are frequently photon constrained, particularly for live cell or in vivo imaging, and this leads to unacceptable errors when analysing data on a pixel-wise basis. Lifetimes and population fractions may, however, be more robustly extracted using global analysis to simultaneously fit the fluorescence decay data of all pixels in an image or dataset to a multi-exponential model under the assumption that the lifetime components are invariant across the image (dataset). This approach is often considered to be prohibitively slow and/or computationally expensive but we present here a computationally efficient global analysis algorithm for the analysis of time-correlated single photon counting (TCSPC) or time-gated FLIM data based on variable projection. It makes efficient use of both computer processor and memory resources, requiring less than a minute to analyse time series and multiwell plate datasets with hundreds of FLIM images on standard personal computers. This lifetime analysis takes account of repetitive excitation, including fluorescence photons excited by earlier pulses contributing to the fit, and is able to accommodate time-varying backgrounds and instrument response functions. We demonstrate that this global approach allows us to readily fit time-resolved fluorescence data to complex models including a four-exponential model of a FRET system, for which the FRET efficiencies of the two species of a bi-exponential donor are linked, and polarisation-resolved lifetime data, where a fluorescence intensity and bi-exponential anisotropy decay model is applied to the analysis of live cell homo-FRET data. A software package implementing this algorithm, FLIMfit, is available under an open source licence through the Open Microscopy Environment.

Automated fluorescence lifetime imaging plate reader and its application to Förster resonant energy transfer readout of Gag protein aggregation

Fluorescence lifetime measurements can provide quantitative readouts of local fluorophore environment and can be applied to biomolecular interactions via Förster resonant energy transfer (FRET). Fluorescence lifetime imaging (FLIM) can therefore provide a high content analysis (HCA) modality to map protein-protein interactions (PPIs) with applications in drug discovery, systems biology and basic research. We present here an automated multiwell plate reader able to perform rapid unsupervised optically sectioned FLIM of fixed and live biological samples and illustrate its potential to assay PPIs through application to Gag protein aggregation during the HIV life cycle. We demonstrate both hetero-FRET and homo-FRET readouts of protein aggregation and report the first quantitative evaluation of a FLIM HCA assay by generating dose response curves through addition of an inhibitor of Gag myristoylation. Z' factors exceeding 0.6 are realised for this FLIM FRET assay.

Automated fluorescence lifetime imaging plate reader and its application to Förster resonant energy transfer readout of Gag protein aggregation

Fluorescence lifetime measurements can provide quantitative readouts of local fluorophore environment and can be applied to biomolecular interactions via Förster resonant energy transfer (FRET). Fluorescence lifetime imaging (FLIM) can therefore provide a high content analysis (HCA) modality to map protein-protein interactions (PPIs) with applications in drug discovery, systems biology and basic research. We present here an automated multiwell plate reader able to perform rapid unsupervised optically sectioned FLIM of fixed and live biological samples and illustrate its potential to assay PPIs through application to Gag protein aggregation during the HIV life cycle. We demonstrate both hetero-FRET and homo-FRET readouts of protein aggregation and report the first quantitative evaluation of a FLIM HCA assay by generating dose response curves through addition of an inhibitor of Gag myristoylation. Z' factors exceeding 0.6 are realised for this FLIM FRET assay.

Multiphoton multispectral fluorescence lifetime tomography for the evaluation of basal cell carcinomas

We present the first detailed study using multispectral multiphoton fluorescence lifetime imaging to differentiate basal cell carcinoma cells (BCCs) from normal keratinocytes. Images were acquired from 19 freshly excised BCCs and 27 samples of normal skin (in & ex vivo). Features from fluorescence lifetime images were used to discriminate BCCs with a sensitivity/specificity of 79%/93% respectively. A mosaic of BCC fluorescence lifetime images covering >1 mm² is also presented, demonstrating the potential for tumour margin delineation.

Using 10,462 manually segmented cells from the image data, we quantify the cellular morphology and spectroscopic differences between BCCs and normal skin for the first time. Statistically significant increases were found in the fluorescence lifetimes of cells from BCCs in all spectral channels, ranging from 19.9% (425–515 nm spectral emission) to 39.8% (620–655 nm emission). A discriminant analysis based diagnostic algorithm allowed the fraction of cells classified as malignant to be calculated for each patient. This yielded a receiver operator characteristic area under the curve for the detection of BCC of 0.83.

We have used both morphological and spectroscopic parameters to discriminate BCC from normal skin, and provide a comprehensive base for how this technique could be used for BCC assessment in clinical practice.

Quantification of cellular autofluorescence of human skin using multiphoton tomography and fluorescence lifetime imaging in two spectral detection channels

We explore the diagnostic potential of imaging endogenous fluorophores using two photon microscopy and fluorescence lifetime imaging (FLIM) in human skin with two spectral detection channels. Freshly excised benign dysplastic nevi (DN) and malignant nodular Basal Cell Carcinomas (nBCCs) were excited at 760 nm. The resulting fluorescence signal was binned manually on a cell by cell basis. This improved the reliability of fitting using a double exponential decay model and allowed the fluorescence signatures from different cell populations within the tissue to be identified and studied. We also performed a direct comparison between different diagnostic groups. A statistically significant difference between the median mean fluorescence lifetime of 2.79 ns versus 2.52 ns (blue channel, 300-500 nm) and 2.08 ns versus 1.33 ns (green channel, 500-640 nm) was found between nBCCs and DN respectively, using the Mann-Whitney U test (p < 0.01). Further differences in the distribution of fluorescence lifetime parameters and inter-patient variability are also discussed.

FLIM FRET technology for drug discovery: automated multiwell-plate high-content analysis, multiplexed readouts and application in situ

A fluorescence lifetime imaging (FLIM) technology platform intended to read out changes in Förster resonance energy transfer (FRET) efficiency is presented for the study of protein interactions across the drug-discovery pipeline. FLIM provides a robust, inherently ratiometric imaging modality for drug discovery that could allow the same sensor constructs to be translated from automated cell-based assays through small transparent organisms such as zebrafish to mammals. To this end, an automated FLIM multiwell-plate reader is described for high content analysis of fixed and live cells, tomographic FLIM in zebrafish and FLIM FRET of live cells via confocal endomicroscopy. For cell-based assays, an exemplar application reading out protein aggregation using FLIM FRET is presented, and the potential for multiple simultaneous FLIM (FRET) readouts in microscopy is illustrated.

Microscopy-based HTS examines the mechanism of stress F-actin fiber disruption by cytochalasin D: orientation texture data collated with quantitative kinetic modeling

We report a drug dose-response, end-point study of intracellular filamentous actin (F-actin) by automated fluorescence microscopy, complemented with theoretical kinetic simulation of drug action. We highlight the use of an advanced orientation-sensitive image processing procedure (<cos(2)theta> transform), specially tailored for the detection of ordered filamentous "patches" in cell images. To examine the extent of stress F-actin disruption caused by the drug, we compare the measured response based on the above transformation with the theoretical data obtained from a quantitative model. We show that the assay data are consistent with the first-order mass action kinetics predicted by a basic reaction model. As a concluding remark, we briefly discuss advantages, perspectives, and challenges of conventional fluorescent microscopy within the context of the quantitative high-throughput screening paradigm.

Automated screening of neurite outgrowth

Outgrowth of neurites in culture is used for assessing neurotrophic activity. Neurite measurements have been performed very slowly using manual methods or more efficiently with interactive image analysis systems. In contrast, medium-throughput and noninteractive image analysis of neurite screens has not been well described. The authors report the performance of an automated image acquisition and analysis system (IN Cell Analyzer 1000) in the neurite assay. Neuro-2a (N2a) cells were plated in 96-well plates and were exposed to 6 conditions of retinoic acid. Immunofluorescence labeling of the cytoskeleton was used to detect neurites and cell bodies. Acquisition of the images was automatic. The image set was then analyzed by both manual tracing and automated algorithms. On 5 relevant parameters (number of neurites, neurite length, total cell area, number of cells, neurite length per cell), the authors did not observe a difference between the automated analysis and the manual analysis done by tracing. These data suggest that the automated system addresses the same biology as human scorers and with the same measurement precision for treatment effects. However, throughput of the automated system is orders of magnitude higher than with manual methods.

Differential effects of alcohol on the cortical processing of foreign and native language

The effect of alcohol (ethanol) on cortical processing of Finnish vs. English words in Finnish-speaking subjects was studied by recording auditory event-related potentials in 10 subjects who had started studying English at the age of 9-10 years. At the beginning of the block of 100 words, the subject heard an introductory sentence. Half of the words completed the sentence well and the other half did not. The subject pressed a reaction key immediately after hearing a proper word. After the control condition, the subject ingested alcohol (1 ml/kg). Alcohol attenuated the amplitude of N100 to both Finnish and English words, this attenuation being significantly stronger for English than for Finnish words. The early differential effect of alcohol suggests that language-specific information is extracted in the cortex already approximately 100 ms from the word onset. The results are in line with animal experiments demonstrating that alcohol selectively affects the activity of single units involved in newer forms of behavior.