Dynamic range extension for photon counting arrays

LinoSPAD2: an FPGA-based, hardware-reconfigurable 512×1 single-photon camera system

We report on LinoSPAD2, a single-photon camera system, comprising a 512×1 single-photon avalanche diode (SPAD) front-end and one or two FPGA-based back-ends. Digital signals generated by the SPADs are processed by the FPGA in real time, whereas the FPGA offers full reconfigurability at a very high level of granularity both in time and space domains. The LinoSPAD2 camera system can process 512 SPADs simultaneously through 256 channels, duplicated on each FPGA-based back-end, with a bank of 64 time-to-digital converters (TDCs) operating at 133 MSa/s, whereas each TDC has a time resolution of 20 ps (LSB). To the best of our knowledge, LinoSPAD2 is the first fully reconfigurable SPAD camera system of large format. The SPAD front-end features a pitch of 26.2 μm, a native fill factor of 25.1%, and a microlens array achieving 2.3× concentration factor. At room temperature, the median dark count rate (DCR) is 80 cps at 7 V excess bias, the peak photon detection probability (PDP) is 53% at 520 nm wavelength, and the single-photon timing resolution (SPTR) is 50 ps FWHM. The instrument response function (IRF) is around 100 ps FWHM at system level. The LinoSPAD2 camera system is suitable for numerous applications, including LiDAR imaging, heralded spectroscopy, compressive Raman sensing, and other computational imaging techniques.

Single-photon microscopy to study biomolecular condensates

Biomolecular condensates serve as membrane-less compartments within cells, concentrating proteins and nucleic acids to facilitate precise spatial and temporal orchestration of various biological processes. The diversity of these processes and the substantial variability in condensate characteristics present a formidable challenge for quantifying their molecular dynamics, surpassing the capabilities of conventional microscopy. Here, we show that our single-photon microscope provides a comprehensive live-cell spectroscopy and imaging framework for investigating biomolecular condensation. Leveraging a single-photon detector array, single-photon microscopy enhances the potential of quantitative confocal microscopy by providing access to fluorescence signals at the single-photon level. Our platform incorporates photon spatiotemporal tagging, which allowed us to perform time-lapse super-resolved imaging for molecular sub-diffraction environment organization with simultaneous monitoring of molecular mobility, interactions, and nano-environment properties through fluorescence lifetime fluctuation spectroscopy. This integrated correlative study reveals the dynamics and interactions of RNA-binding proteins involved in forming stress granules, a specific type of biomolecular condensates, across a wide range of spatial and temporal scales. Our versatile framework opens up avenues for exploring a broad spectrum of biomolecular processes beyond the formation of membrane-less organelles.

Noise-resistant phase imaging with intensity correlation

Interferometric methods form the basis of highly sensitive measurement techniques from astronomy to bioimaging. Interferometry typically requires high stability between the measured and reference beams. The presence of rapid phase fluctuations washes out interference fringes, making phase profile recovery impossible. This challenge can be addressed by shortening the measurement time. However, such an approach reduces photon-counting rates, precluding applications in low-intensity imaging. We introduce a phase imaging technique which is immune to time-dependent phase fluctuations. Our technique, relying on intensity correlation instead of direct intensity measurements, allows one to obtain high interference visibility for arbitrarily long acquisition times. We prove the optimality of our method using the Cramér-Rao bound in the extreme case when no more than two photons are detected within the time window of phase stability. Our technique will broaden prospects in phase measurements, including emerging applications such as in infrared and x-ray imaging and quantum and matter-wave interferometry.

Correlated-photon imaging at 10 volumetric images per second

The correlation properties of light provide an outstanding tool to overcome the limitations of traditional imaging techniques. A relevant case is represented by correlation plenoptic imaging (CPI), a quantum-inspired volumetric imaging protocol employing spatio-temporally correlated photons from either entangled or chaotic sources to address the main limitations of conventional light-field imaging, namely, the poor spatial resolution and the reduced change of perspective for 3D imaging. However, the application potential of high-resolution imaging modalities relying on photon correlations is limited, in practice, by the need to collect a large number of frames. This creates a gap, unacceptable for many relevant tasks, between the time performance of correlated-light imaging and that of traditional imaging methods. In this article, we address this issue by exploiting the photon number correlations intrinsic in chaotic light, combined with a cutting-edge ultrafast sensor made of a large array of single-photon avalanche diodes (SPADs). This combination of source and sensor is embedded within a novel single-lens CPI scheme enabling to acquire 10 volumetric images per second. Our results place correlated-photon imaging at a competitive edge and prove its potential in practical applications.

Rapid single-photon color imaging of moving objects

This paper outlines an experimental demonstration of a Bayesian image reconstruction approach to achieve rapid single-photon color imaging of moving objects. The capacity to extract the color of objects is important in a variety of target identification and computer vision applications. Nonetheless, it remains challenging to achieve high-speed color imaging of moving objects in low-photon flux environments. The low-photon regime presents particular challenges for efficient spectral separation and identification, while unsupervised image reconstruction algorithms are often slow and computationally expensive. In this paper, we address both of these difficulties using a combination of hardware and computational solutions. We demonstrate color imaging using a Single-Photon Avalanche Diode (SPAD) detector array for rapid, low-light-level data acquisition, with an integrated color filter array (CFA) for efficient spectral unmixing. High-speed image reconstruction is achieved using a bespoke Bayesian algorithm to produce high-fidelity color videos. The analysis is conducted first on simulated data allowing different pixel formats and photon flux scenarios to be investigated. Experiments are then performed using a plasmonic metasurface-based CFA, integrated with a 64 × 64 pixel format SPAD array. Passive imaging is conducted using white-light illumination of multi-colored, moving targets. Intensity information is recorded in a series of 2D photon-counting SPAD frames, from which accurate color information is extracted using the fast Bayesian method introduced herein. The per-frame reconstruction rate proves to be hundreds of times faster than the previous computational method. Furthermore, this approach yields additional information in the form of uncertainty measures, which can be used to assist with imaging system optimization and decision-making in real-world applications. The techniques demonstrated point the way towards rapid video-rate single-photon color imaging. The developed Bayesian algorithm, along with more advanced SPAD technology and utilization of time-correlated single-photon counting (TCSPC) will permit live 3D, color videography in extremely low-photon flux environments.

Focus image scanning microscopy for sharp and gentle super-resolved microscopy

To date, the feasibility of super-resolution microscopy for imaging live and thick samples is still limited. Stimulated emission depletion (STED) microscopy requires high-intensity illumination to achieve sub-diffraction resolution, potentially introducing photodamage to live specimens. Moreover, the out-of-focus background may degrade the signal stemming from the focal plane. Here, we propose a new method to mitigate these limitations without drawbacks. First, we enhance a STED microscope with a detector array, enabling image scanning microscopy (ISM). Therefore, we implement STED-ISM, a method that exploits the working principle of ISM to reduce the depletion intensity and achieve a target resolution. Later, we develop Focus-ISM, a strategy to improve the optical sectioning and remove the background of any ISM-based imaging technique, with or without a STED beam. The proposed approach requires minimal architectural changes to a conventional microscope but provides substantial advantages for live and thick sample imaging.

The BrightEyes-TTM as an open-source time-tagging module for democratising single-photon microscopy

Fluorescence laser-scanning microscopy (LSM) is experiencing a revolution thanks to new single-photon (SP) array detectors, which give access to an entirely new set of single-photon information. Together with the blooming of new SP LSM techniques and the development of tailored SP array detectors, there is a growing need for (i) DAQ systems capable of handling the high-throughput and high-resolution photon information generated by these detectors, and (ii) incorporating these DAQ protocols in existing fluorescence LSMs. We developed an open-source, low-cost, multi-channel time-tagging module (TTM) based on a field-programmable gate array that can tag in parallel multiple single-photon events, with 30 ps precision, and multiple synchronisation events, with 4 ns precision. We use the TTM to demonstrate live-cell super-resolved fluorescence lifetime image scanning microscopy and fluorescence lifetime fluctuation spectroscopy. We expect that our BrightEyes-TTM will support the microscopy community in spreading SP-LSM in many life science laboratories.

Gate-Width Optimisation Based on Time-Gated Single Photon Avalanche Diode Receiver for Optical Wireless Communications

Using a single photon avalanche diode (SPAD) as a receiver in an optical wireless communications (OWC) system can effectively expand the transmission distance. However, the performance of the SPAD receiver is usually affected with the bit error rate (BER) lower bound determined by background light and the inter-symbol interference (ISI) distortion caused by dead time. In this paper, external time-gated technology is employed, and the SPAD is only activated within the set gate-ON time to alleviate the influence of background light and ISI distortion. The SPAD photon counting model and the communication BER model are established according to the external time-gated characteristics. Considering the functional relationship among the gate-ON time, signal light flux, background light flux and blocking time, we take the minimum BER as the optimization target, the optimal gate-ON time is derived. The numerical results show that whenever the signal light flux or the background light flux is higher, the BER performance of the time-gated mode is apparently better than the free-running mode. For example, when signal photons and background photons are 30 and 10 per symbol time, respectively, the BER of free-running SPAD converges to 0.1, while the BER of the time-gated scheme is about 10−6.

Resolving the Controversy in Biexciton Binding Energy of Cesium Lead Halide Perovskite Nanocrystals through Heralded Single-Particle Spectroscopy

Understanding exciton-exciton interaction in multiply excited nanocrystals is crucial to their utilization as functional materials. Yet, for lead halide perovskite nanocrystals, which are promising candidates for nanocrystal-based technologies, numerous contradicting values have been reported for the strength and sign of their exciton-exciton interaction. In this work, we unambiguously determine the biexciton binding energy in single cesium lead halide perovskite nanocrystals at room temperature. This is enabled by the recently introduced single-photon avalanche diode array spectrometer, capable of temporally isolating biexciton-exciton emission cascades while retaining spectral resolution. We demonstrate that CsPbBr₃ nanocrystals feature an attractive exciton-exciton interaction, with a mean biexciton binding energy of 10 meV. For CsPbI₃ nanocrystals, we observe a mean biexciton binding energy that is close to zero, and individual nanocrystals show either weakly attractive or weakly repulsive exciton-exciton interaction. We further show that, within ensembles of both materials, single-nanocrystal biexciton binding energies are correlated with the degree of charge-carrier confinement.

A Mechanically Flexible, Implantable Neural Interface for Computational Imaging and Optogenetic Stimulation Over 5.4×5.4mm2 FoV

Emerging optical functional imaging and optogenetics are among the most promising approaches in neuroscience to study neuronal circuits. Combining both methods into a single implantable device enables all-optical neural interrogation with immediate applications in freely-behaving animal studies. In this paper, we demonstrate such a device capable of optical neural recording and stimulation over large cortical areas. This implantable surface device exploits lens-less computational imaging and a novel packaging scheme to achieve an ultra-thin (250μm-thick), mechanically flexible form factor. The core of this device is a custom-designed CMOS integrated circuit containing a 160×160 array of time-gated single-photon avalanche photodiodes (SPAD) for low-light intensity imaging and an interspersed array of dual-color (blue and green) flip-chip bonded micro-LED (μLED) as light sources. We achieved 60μm lateral imaging resolution and 0.2mm³ volumetric precision for optogenetics over a 5.4×5.4mm² field of view (FoV). The device achieves a 125-fps frame-rate and consumes 40 mW of total power.

Heralded Spectroscopy Reveals Exciton-Exciton Correlations in Single Colloidal Quantum Dots

Multiply excited states in semiconductor quantum dots feature intriguing physics and play a crucial role in nanocrystal-based technologies. While photoluminescence provides a natural probe to investigate these states, room-temperature single-particle spectroscopy of their emission has proved elusive due to the temporal and spectral overlap with emission from the singly excited and charged states. Here, we introduce biexciton heralded spectroscopy enabled by a single-photon avalanche diode array based spectrometer. This allows us to directly observe biexciton-exciton emission cascades and measure the biexciton binding energy of single quantum dots at room temperature, even though it is well below the scale of thermal broadening and spectral diffusion. Furthermore, we uncover correlations hitherto masked in ensembles of the biexciton binding energy with both charge-carrier confinement and fluctuations of the local electrostatic potential. Heralded spectroscopy has the potential of greatly extending our understanding of charge-carrier dynamics in multielectron systems and of parallelization of quantum optics protocols.

Towards Quantum 3D Imaging Devices

We review the advancement of the research toward the design and implementation of quantum plenoptic cameras, radically novel 3D imaging devices that exploit both momentum–position entanglement and photon–number correlations to provide the typical refocusing and ultra-fast, scanning-free, 3D imaging capability of plenoptic devices, along with dramatically enhanced performances, unattainable in standard plenoptic cameras: diffraction-limited resolution, large depth of focus, and ultra-low noise. To further increase the volumetric resolution beyond the Rayleigh diffraction limit, and achieve the quantum limit, we are also developing dedicated protocols based on quantum Fisher information. However, for the quantum advantages of the proposed devices to be effective and appealing to end-users, two main challenges need to be tackled. First, due to the large number of frames required for correlation measurements to provide an acceptable signal-to-noise ratio, quantum plenoptic imaging (QPI) would require, if implemented with commercially available high-resolution cameras, acquisition times ranging from tens of seconds to a few minutes. Second, the elaboration of this large amount of data, in order to retrieve 3D images or refocusing 2D images, requires high-performance and time-consuming computation. To address these challenges, we are developing high-resolution single-photon avalanche photodiode (SPAD) arrays and high-performance low-level programming of ultra-fast electronics, combined with compressive sensing and quantum tomography algorithms, with the aim to reduce both the acquisition and the elaboration time by two orders of magnitude. Routes toward exploitation of the QPI devices will also be discussed.

Biochemical resolving power of fluorescence lifetime imaging: untangling the roles of the instrument response function and photon-statistics

A deeper understanding of spatial resolution has led to innovations in microscopy and the disruption of biomedical research, as with super-resolution microscopy. To foster similar advances in time-resolved and spectral imaging, we have previously introduced the concept of 'biochemical resolving power' in fluorescence microscopy. Here, we apply those concepts to investigate how the instrument response function (IRF), sampling conditions, and photon-statistics limit the biochemical resolution of fluorescence lifetime microscopy. Using Fisher information analysis and Monte Carlo simulations, we reveal the complex dependencies between photon-statistics and the IRF, permitting us to quantify resolution limits that have been poorly understood (e.g., the minimum resolvable decay time for a given width of the IRF and photon-statistics) or previously underappreciated (e.g., optimization of the IRF for biochemical detection). With this work, we unravel common misunderstandings on the role of the IRF and provide theoretical insights with significant practical implications on the design and use of time-resolved instrumentation.

Time-resolved spectral-domain optical coherence tomography with CMOS SPAD sensors

We present a first spectral-domain optical coherence tomography (SD-OCT) system deploying a complementary metal-oxide-semiconductor (CMOS) single-photon avalanche diode (SPAD) based, time-resolved line sensor. The sensor with 1024 pixels achieves a sensitivity of 87 dB at an A-scan rate of 1 kHz using a supercontinuum laser source with a repetition rate of 20 MHz, 38 nm bandwidth, and 2 mW power at 850 nm centre wavelength. In the time-resolved mode of the sensor, the system combines low-coherence interferometry (LCI) and massively parallel time-resolved single-photon counting to control the detection of interference spectra on the single-photon level based on the time-of-arrival of photons. For proof of concept demonstration of the combined detection scheme we show the acquisition of time-resolved interference spectra and the reconstruction of OCT images from selected time bins. Then, we exemplify the temporal discrimination feature with 50 ps time resolution and 249 ps timing uncertainty by removing unwanted reflections from along the optical path at a 30 mm distance from the sample. The current limitations of the proposed technique in terms of sensor parameters are analysed and potential improvements are identified for advanced photonic applications.

High fill-factor miniaturized SPAD arrays with a guard-ring-sharing technique

We present a novel guard-ring-sharing technique to push the limit of SPAD pixel miniaturization, and to demonstrate the operation of SPAD arrays with a 2.2 µm-pitch, the smallest ever reported. Device simulation and preliminary tests suggest that the optimized device design ensures the electrical isolation of SPADs with guard-ring sharing. 4×4 SPAD arrays with two parallel selective readout circuits are designed in 180 nm CMOS technology. SPAD characteristics for the pixel pitch of 2.2, 3, and 4 µm are systematically measured as a function of an active diameter, active-to-active distance, and excess bias. For a 4 µm-pitch, the fill factor is 42.4%, the maximum PDP 33.5%, the median DCR 2.5 cps, the timing jitter 88 ps, and the crosstalk probability is 3.57%, while the afterpulsing probability is 0.21%. Finally, we verified the feasibility of the proposed technique towards compact multi-megapixel 3D-stacked SPAD arrays.

How many photons are needed for FRET imaging?

Förster resonance energy transfer (FRET) imaging is an essential analytical method in biomedical research. The limited photon-budget experimentally available, however, imposes compromises between spatiotemporal and biochemical resolutions, photodamage and phototoxicity. The study of photon-statistics in biochemical imaging is thus important in guiding the efficient design of instrumentation and assays. Here, we show a comparative analysis of photon-statistics in FRET imaging demonstrating how the precision of FRET imaging varies vastly with imaging parameters. Therefore, we provide analytical and numerical tools for assay optimization. Fluorescence lifetime imaging microscopy (FLIM) is a very robust technique with excellent photon-efficiencies. However, we show that also intensity-based FRET imaging can reach high precision by utilizing information from both donor and acceptor fluorophores.

Quantum correlation measurement with single photon avalanche diode arrays

Temporal photon correlation measurement, instrumental to probing the quantum properties of light, typically requires multiple single photon detectors. Progress in single photon avalanche diode (SPAD) array technology highlights their potential as high-performance detector arrays for quantum imaging and photon number-resolving (PNR) experiments. Here, we demonstrate this potential by incorporating a novel on-chip SPAD array with 42% peak photon detection efficiency, low dark count rate and crosstalk probability of 0.14% per detection in a confocal microscope. This enables reliable measurements of second and third order photon correlations from a single quantum dot emitter. Our analysis overcomes the inter-detector optical crosstalk background even though it is over an order of magnitude larger than our faint signal. To showcase the vast application space of such an approach, we implement a recently introduced super-resolution imaging method, quantum image scanning microscopy (Q-ISM).

Single-photon avalanche diode imagers in biophotonics: review and outlook

Single-photon avalanche diode (SPAD) arrays are solid-state detectors that offer imaging capabilities at the level of individual photons, with unparalleled photon counting and time-resolved performance. This fascinating technology has progressed at a very fast pace in the past 15 years, since its inception in standard CMOS technology in 2003. A host of architectures have been investigated, ranging from simpler implementations, based solely on off-chip data processing, to progressively "smarter" sensors including on-chip, or even pixel level, time-stamping and processing capabilities. As the technology has matured, a range of biophotonics applications have been explored, including (endoscopic) FLIM, (multibeam multiphoton) FLIM-FRET, SPIM-FCS, super-resolution microscopy, time-resolved Raman spectroscopy, NIROT and PET. We will review some representative sensors and their corresponding applications, including the most relevant challenges faced by chip designers and end-users. Finally, we will provide an outlook on the future of this fascinating technology.

Fast single-cell biochemistry: theory, open source microscopy and applications

Fluorescence lifetime sensing enables researchers to probe the physicochemical environment of a fluorophore providing a window through which we can observe the complex molecular make-up of the cell. Fluorescence lifetime imaging microscopy (FLIM) quantifies and maps cell biochemistry, a complex ensemble of dynamic processes. Unfortunately, typical high-resolution FLIM systems exhibit rather limited acquisition speeds, often insufficient to capture the time evolution of biochemical processes in living cells. Here, we describe the theoretical background that justifies the developments of high-speed single photon counting systems. We show that systems with low dead-times not only result in faster acquisition throughputs but also improved dynamic range and spatial resolution. We also share the implementation of hardware and software as an open platform, show applications of fast FLIM biochemical imaging on living cells and discuss strategies to balance precision and accuracy in FLIM. The recent innovations and commercialisation of fast time-domain FLIM systems are likely to popularise FLIM within the biomedical community, to impact biomedical research positively and to foster the adoption of other FLIM techniques as well. While supporting and indeed pursuing these developments, with this work we also aim to warn the community about the possible shortcomings of fast single photon counting techniques and to highlight strategies to acquire data of high quality.

Phasor-based widefield FLIM using a gated 512×512 single-photon SPAD imager

Single-photon avalanche diode (SPAD) imagers can perform fast time-resolved imaging in a compact form factor, by exploiting the processing capability and speed of integrated CMOS electronics. Developments in SPAD imagers have recently made them compatible with widefield microscopy, thanks to array formats approaching one megapixel and sensitivity and noise levels approaching those of established technologies. In this paper, phasor-based FLIM is demonstrated with a gated binary 512×512 SPAD imager, which can operate with a gate length as short as 5.75 ns, a minimum gate step of 17.9 ps, and up to 98 kfps readout rate (1-bit frames). Lifetimes of ATTO 550 and Rhodamine 6G (R6G) solutions were measured across a 472×256 sub-array using phasor analysis, acquiring data by shifting a 13.1 ns gate window across the 50 ns laser period. The measurement accuracy obtained when employing such a scheme based on long, overlapping gates was validated by comparison with TCSPC measurements and fitting analysis results based on a standard Levenberg-Marquardt algorithm (>90% accuracy for the lifetime of R6G and ATTO 550). This demonstrates the ability of the proposed method to measure short lifetimes without minimum gate length requirements. The FLIM frame rate of the overall system can be increased up to a few fps for phasor-based widefield FLIM (moving closer to real-time operation) by FPGA-based parallel computation with continuous acquisition at the full speed of 98 kfps.

A 512×512 SPAD Image Sensor with Integrated Gating for Widefield FLIM

We report on SwissSPAD2, an image sensor with 512×512 photon-counting pixels, each comprising a single-photon avalanche diode (SPAD), a 1-bit memory, and a gating mechanism capable of turning the SPAD on and off, with a skew of 250ps and 344ps, respectively, for a minimum duration of 5.75ns. The sensor is designed to achieve a frame rate of up to 97,700 binary frames per second and sub-40ps gate shifts. By synchronizing it with a pulsed laser and using multiple successive overlapping gates, one can reconstruct a molecule's fluorescent response with picosecond temporal resolution. Thanks to the sensor's number of pixels (the largest to date) and the fully integrated gated operation, SwissSPAD2 enables widefield FLIM with an all-solid-state solution and at relatively high frame rates. This was demonstrated with preliminary results on organic dyes and semiconductor quantum dots using both decay fitting and phasor analysis. Furthermore, pixels with an exceptionally low dark count rate and high photon detection probability enable uniform and high quality imaging of biologically relevant fluorescent samples stained with multiple dyes. While future versions will feature the addition of microlenses and optimize firmware speed, our results open the way to low-cost alternatives to commercially available scientific time-resolved imagers.