BSSE: An open-source image processing tool for miniaturized microscopy

FPGA-Based In-Vivo Calcium Image Decoding for Closed-Loop Feedback Applications

Miniaturized calcium imaging is an emerging neural recording technique that has been widely used for monitoring neural activity on a large scale at a specific brain region of rats or mice. Most existing calcium-image analysis pipelines operate offline. This results in long processing latency, making it difficult to realize closed-loop feedback stimulation for brain research. In recent work, we have proposed an FPGA-based real-time calcium image processing pipeline for closed-loop feedback applications. It can perform real-time calcium image motion correction, enhancement, fast trace extraction, and real-time decoding from extracted traces. Here, we extend this work by proposing a variety of neural network based methods for real-time decoding and evaluate the tradeoff among these decoding methods and accelerator designs. We introduce the implementation of the neural network based decoders on the FPGA, and show their speedup against the implementation on the ARM processor. Our FPGA implementation enables the real-time calcium image decoding with sub-ms processing latency for closed-loop feedback applications.

Miniaturized modular-array fluorescence microscopy

Fluorescence live-cell imaging allows for continuous interrogation of cellular behaviors, and the recent development of portable live-cell imaging platforms has rapidly transformed conventional schemes with high adaptability, cost-effective functionalities and easy accessibility to cell-based assays. However, broader applications remain restrictive due to compatibility with conventional cell culture workflow and biochemical sensors, accessibility to up-right physiological imaging, or parallelization of data acquisition. Here, we introduce miniaturized modular-array fluorescence microscopy (MAM) for compact live-cell imaging in flexible formats. We advance the current miniscopy technology to devise an up-right modular architecture, each combining a gradient-index (GRIN) objective and individually-addressed illumination and acquisition components. Parallelization of an array of such modular devices allows for multi-site data acquisition in situ using conventional off-the-shelf cell chambers. Compared with existing methods, the device offers a high fluorescence sensitivity and efficiency, exquisite spatiotemporal resolution (∼3 µm and up to 60 Hz), a configuration compatible with conventional cell culture assays and physiological imaging, and an effective parallelization of data acquisition. The system has been demonstrated using various calibration and biological samples and experimental conditions, representing a promising solution to time-lapse in situ single-cell imaging and analysis.

Fast and accurate sCMOS noise correction for fluorescence microscopy

The rapid development of scientific CMOS (sCMOS) technology has greatly advanced optical microscopy for biomedical research with superior sensitivity, resolution, field-of-view, and frame rates. However, for sCMOS sensors, the parallel charge-voltage conversion and different responsivity at each pixel induces extra readout and pattern noise compared to charge-coupled devices (CCD) and electron-multiplying CCD (EM-CCD) sensors. This can produce artifacts, deteriorate imaging capability, and hinder quantification of fluorescent signals, thereby compromising strategies to reduce photo-damage to live samples. Here, we propose a content-adaptive algorithm for the automatic correction of sCMOS-related noise (ACsN) for fluorescence microscopy. ACsN combines camera physics and layered sparse filtering to significantly reduce the most relevant noise sources in a sCMOS sensor while preserving the fine details of the signal. The method improves the camera performance, enabling fast, low-light and quantitative optical microscopy with video-rate denoising for a broad range of imaging conditions and modalities.

Scientific complementary metal-oxide semiconductor (sCMOS) cameras have advanced the imaging field, but they often suffer from additional noise compared to CCD sensors. Here the authors present a content-adaptive algorithm for the automatic correction of sCMOS-related noise for fluorescence microscopy.