Lensless high-resolution on-chip optofluidic microscopes for Caenorhabditis elegans and cell imaging

Raman spectroscopy as an alternative rapid microbial bioburden test method for continuous, automated detection of contamination in biopharmaceutical drug substance manufacturing

AIMS

To investigate an in-line Raman method capable of detecting accidental microbial contamination in pharmaceutical vessels, such as bioreactors producing monoclonal antibodies via cell culture.

METHODS AND RESULTS

The Raman method consists of a multivariate model built from Raman spectra collected in-line during reduced-scale bioreactor batches producing a monoclonal antibody, as well as a reduced-scale process with intentional spiking of representative compendial method microorganisms (n = 4). The orthogonal partial least squares regression discriminant analysis model (OPLS-DA) area under the curve (AUC), specificity and sensitivity were 0.96, 0.99, and 0.95, respectively. Furthermore, the model successfully detected contamination in an accidentally contaminated manufacturing-scale batch. In all cases, the time to detection (TTD) for Raman was superior compared to offline, traditional microbiological culturing.

CONCLUSIONS

The Raman OPLS-DA method met acceptance criteria for equivalent decision making to be considered a viable alternative to the compendial method for in-process bioburden testing. The in-line method is automated, non-destructive, and provides a continuous assessment of bioburden compared to an offline compendial method, which is manual, results in loss of product, and in practice is only collected once daily and requires 3-5 days for enumeration.

Lensless shadow microscopy-based shortcut analysis strategy for fast quantification of microplastic fibers released to water

Fast quantification is the primary challenge in monitoring microplastic fiber (MPF) pollution in water. The process of quantifying the number of MPFs in water typically involves filtration, imaging on a filter membrane, and manual counting. However, this routine workflow has limitations in terms of speed and accuracy. Here, we present an alternative analysis strategy based on our high-resolution lensless shadow microscope (LSM) for rapid imaging of MPFs on a chip and modified deep learning algorithms for automatic counting. Our LSM system was equipped with wide field-of-view submicron-pixel imaging sensors (>1 cm2; ∼500 nm/pixel) and could simultaneously capture the projection image of >3-μm microplastic spheres within 90 s. The algorithms enabled accurate classification and detection of the number and length of >10-μm linear and branched MPFs derived from melamine cleaning sponges in each image (∼0.4 gigapixels) within 60 s. Importantly, neither MPF morphology (dispersed or aggregated) nor environmental matrix had a notable impact on the automatic recognition of the MPFs by the algorithms. This new strategy had a detection limit of 10 particles/mL and significantly reduced the time of MPF imaging and counting from several hours with membrane-based methods to just a few minutes per sample. The strategy could be employed to monitor water pollution caused by microplastics if an efficient sample separation and a comprehensive sample image database were available.

Long-term imaging of three-dimensional hyphal development using the ePetri dish

Imaging three-dimensional microbial development and behavior over extended periods is crucial for advancing microbiological studies. Here, we introduce an upgraded ePetri dish system specifically designed for extended microbial culturing and 3D imaging, addressing the limitations of existing methods. Our approach includes a sealed growth chamber to enable long-term culturing, and a multi-step reconstruction algorithm that integrates 3D deconvolution, image filtering, ridge, and skeleton detection for detailed visualization of the hyphal network. The system effectively monitored the development of Aspergillus brasiliensis hyphae over a seven-day period, demonstrating the growth medium's stability within the chamber. The system's 3D imaging capability was validated in a volume of 5.5 mm × 4 mm × 0.5 mm, revealing a radial growth pattern of fungal hyphae. Additionally, we show that the system can identify potential filter failures that are undetectable with 2D imaging. With these capabilities, the upgraded ePetri dish represents a significant advancement in long-term 3D microbial imaging, promising new insights into microbial development and behavior across various microbiological research areas.

Lens-free auto-focusing imaging algorithm for the ultra-broadband light source

Auto-focusing is an essential task for lens-free holographic microscopy, which has developed many methods for high precision or fast refocusing. In this work, we derive the relationship among intensity derivation, the derivative of spectral distribution, as well as the distribution of the object, and propose a new auto-focusing criterion, the Robert critical function with axial difference (RCAD), to enhance the accuracy of distance estimation for lens-free imaging with the ultra-broadband light source. This method consists of three steps: image acquisition and preprocessing, axial-difference calculation, and distance estimation with sharpness analysis. The simulations and experiments demonstrate that the accuracy of this metric on auto-focusing with the ultra-broadband spectrum can effectively assist in determining the off-focus distance. The experiments are conducted in an ultra-broad-spectrum on-chip system, where the samples including the resolution target and the cross-section of the Tilia stem are employed to maximize the applicability of this method. We believe that the RCAD criterion is expected to be a useful auxiliary tool for lens-free on-chip microscopes with ultra-broadband spectrum illumination.

Microfluidics on lensless, semiconductor optical image sensors: challenges and opportunities for democratization of biosensing at the micro-and nano-scale

Optical image sensors are 2D arrays of pixels that integrate semiconductor photodiodes and field effect transistors for efficient photon conversion and processing of generated electrons. With technological advancements and subsequent democratization of these sensors, opportunities for integration with microfluidics devices are currently explored. 2D pixel arrays of such optical image sensors can reach dimensions larger than one centimeter with a sub-micrometer pixel size, for high spatial resolution lensless imaging with large field of view, a feat that cannot be achieved with lens-based optical microscopy. Moreover, with advancements in fabrication processes, the field of microfluidics has evolved to develop microfluidic devices with an overall size below one centimeter and individual components of sub-micrometer size, such that they can now be implemented onto optical image sensors. The convergence of these fields is discussed in this article, where we review fundamental principles, opportunities, challenges, and outlook for integration, with focus on contact-mode imaging configuration. Most recent developments and applications of microfluidic lensless contact-based imaging to the field of biosensors, in particular those related to the potential for point of need applications, are also discussed.

Improving flat fluorescence microscopy in scattering tissue through deep learning strategies

Intravital microscopy in small animals growingly contributes to the visualization of short- and long-term mammalian biological processes. Miniaturized fluorescence microscopy has revolutionized the observation of live animals' neural circuits. The technology's ability to further miniaturize to improve freely moving experimental settings is limited by its standard lens-based layout. Typical miniature microscope designs contain a stack of heavy and bulky optical components adjusted at relatively long distances. Computational lensless microscopy can overcome this limitation by replacing the lenses with a simple thin mask. Among other critical applications, Flat Fluorescence Microscope (FFM) holds promise to allow for real-time brain circuits imaging in freely moving animals, but recent research reports show that the quality needs to be improved, compared with imaging in clear tissue, for instance. Although promising results were reported with mask-based fluorescence microscopes in clear tissues, the impact of light scattering in biological tissue remains a major challenge. The outstanding performance of deep learning (DL) networks in computational flat cameras and imaging through scattering media studies motivates the development of deep learning models for FFMs. Our holistic ray-tracing and Monte Carlo FFM computational model assisted us in evaluating deep scattering medium imaging with DL techniques. We demonstrate that physics-based DL models combined with the classical reconstruction technique of the alternating direction method of multipliers (ADMM) perform a fast and robust image reconstruction, particularly in the scattering medium. The structural similarity indexes of the reconstructed images in scattering media recordings were increased by up to 20% compared with the prevalent iterative models. We also introduce and discuss the challenges of DL approaches for FFMs under physics-informed supervised and unsupervised learning.

A Multimodal Sensing CMOS Imager Based on Dual-Focus Imaging

Advanced machine intelligence is empowered not only by the ever-increasing computational capability for information processing but also by sensors for collecting multimodal information from complex environments. However, simply assembling different sensors can result in bulky systems and complex data processing. Herein, it is shown that a complementary metal-oxide-semiconductor (CMOS) imager can be transformed into a compact multimodal sensing platform through dual-focus imaging. By combining lens-based and lensless imaging, visual information, chemicals, temperature, and humidity can be detected with the same chip and output as a single image. As a proof of concept, the sensor is equipped on a micro-vehicle, and multimodal environmental sensing and mapping is demonstrated. A multimodal endoscope is also developed, and simultaneous imaging and chemical profiling along a porcine digestive tract is achieved. The multimodal CMOS imager is compact, versatile, and extensible and can be widely applied in microrobots, in vivo medical apparatuses, and other microdevices.

Diffraction-Net: a robust single-shot holography for multi-distance lensless imaging

Digital holography based on lensless imaging is a developing method adopted in microscopy and micro-scale measurement. To retrieve complex-amplitude on the sample surface, multiple images are required for common reconstruction methods. A promising single-shot approach points to deep learning, which has been used in lensless imaging but suffering from the unsatisfied generalization ability and stability. Here, we propose and construct a diffraction network (Diff-Net) to connect diffraction images at different distances, which breaks through the limitations of physical devices. The Diff-Net based single-shot holography is robust as there is no practical errors between the multiple images. An iterative complex-amplitude retrieval approach based on light transfer function through the Diff-Net generated multiple images is used for complex-amplitude recovery. This process indicates a hybrid-driven method including both physical model and deep learning, and the experimental results demonstrate that the Diff-Net possesses qualified generalization ability for samples with significantly different morphologies.

Living Sample Viability Measurement Methods from Traditional Assays to Nanomotion

Living sample viability measurement is an extremely common process in medical, pharmaceutical, and biological fields, especially drug pharmacology and toxicology detection. Nowadays, there are a number of chemical, optical, and mechanical methods that have been developed in response to the growing demand for simple, rapid, accurate, and reliable real-time living sample viability assessment. In parallel, the development trend of viability measurement methods (VMMs) has increasingly shifted from traditional assays towards the innovative atomic force microscope (AFM) oscillating sensor method (referred to as nanomotion), which takes advantage of the adhesion of living samples to an oscillating surface. Herein, we provide a comprehensive review of the common VMMs, laying emphasis on their benefits and drawbacks, as well as evaluating the potential utility of VMMs. In addition, we discuss the nanomotion technique, focusing on its applications, sample attachment protocols, and result display methods. Furthermore, the challenges and future perspectives on nanomotion are commented on, mainly emphasizing scientific restrictions and development orientations.

Development of a Photonic Switch via Electro-Capillarity-Induced Water Penetration Across a 10-nm Gap

With narrow and dense nanoarchitectures increasingly adopted to improve optical functionality, achieving the complete wetting of photonic devices is required when aiming at underwater molecule detection over the water-repellent optical materials. Despite continuous advances in photonic applications, real-time monitoring of nanoscale wetting transitions across nanostructures with 10-nm gaps, the distance at which photonic performance is maximized, remains a chronic hurdle when attempting to quantify the water influx and molecules therein. For this reason, the present study develops a photonic switch that transforms the wetting transition into perceivable color changes using a liquid-permeable Fabry-Perot resonator. Electro-capillary-induced Cassie-to-Wenzel transitions produce an optical memory effect in the photonic switch, as confirmed by surface-energy analysis, simulations, and an experimental demonstration. The results show that controlling the wetting behavior using the proposed photonic switch is a promising strategy for the integration of aqueous media with photonic hotspots in plasmonic nanostructures such as biochemical sensors.

Recent Advances in Lensless Imaging

Lensless imaging provides opportunities to design imaging systems free from the constraints imposed by traditional camera architectures. Thanks to advances in imaging hardware, fabrication techniques, and new algorithms, researchers have recently developed lensless imaging systems that are extremely compact, lightweight or able to image higher-dimensional quantities. Here we review these recent advances and describe the design principles and their effects that one should consider when developing and using lensless imaging systems.

Ptychographic sensor for large-scale lensless microbial monitoring with high spatiotemporal resolution

Traditional microbial detection methods often rely on the overall property of microbial cultures and cannot resolve individual growth event at high spatiotemporal resolution. As a result, they require bacteria to grow to confluence and then interpret the results. Here, we demonstrate the application of an integrated ptychographic sensor for lensless cytometric analysis of microbial cultures over a large scale and with high spatiotemporal resolution. The reported device can be placed within a regular incubator or used as a standalone incubating unit for long-term microbial monitoring. For longitudinal study where massive data are acquired at sequential time points, we report a new temporal-similarity constraint to increase the temporal resolution of ptychographic reconstruction by 7-fold. With this strategy, the reported device achieves a centimeter-scale field of view, a half-pitch spatial resolution of 488 nm, and a temporal resolution of 15-s intervals. For the first time, we report the direct observation of bacterial growth in a 15-s interval by tracking the phase wraps of the recovered images, with high phase sensitivity like that in interferometric measurements. We also characterize cell growth via longitudinal dry mass measurement and perform rapid bacterial detection at low concentrations. For drug-screening application, we demonstrate proof-of-concept antibiotic susceptibility testing and perform single-cell analysis of antibiotic-induced filamentation. The combination of high phase sensitivity, high spatiotemporal resolution, and large field of view is unique among existing microscopy techniques. As a quantitative and miniaturized platform, it can improve studies with microorganisms and other biospecimens at resource-limited settings.

Computational Optics for Point-of-Care Breast Cancer Profiling

With the global burden of cancer on the rise, it is critical to developing new modalities that could detect cancer and guide targeted treatments in fast and inexpensive ways. The need for such technologies is vital, especially in underserved regions where severe diagnostic bottlenecks exist. Recently, we developed a low-cost digital diagnostic system for breast cancer using fine-needle aspirates (FNAs). Named, AIDA (artificial intelligence diffraction analysis), the system combines lens-free digital diffraction imaging with deep-learning algorithms to achieve automated, rapid, and high-throughput cellular analyses for breast cancer diagnosis of FNA and subtype classification for better-guided treatments (Min et al. ACS Nano 12:9081-9090, 2018). Although primarily validated for breast cancer and lymphoma (Min et al. ACS Nano 12:9081-9090, 2018; Im et al. Nat Biomed Eng 2:666-674, 2018), the system could be easily adapted to diagnosing other prevalent cancers and thus find widespread use for global health.

Optofluidic ptychography on a chip

We report the implementation of a fully on-chip, lensless microscopy technique termed optofluidic ptychography. This imaging modality complements the miniaturization provided by microfluidics and allows the integration of ptychographic microscopy into various lab-on-a-chip devices. In our prototype, we place a microfluidic channel on the top surface of a coverslip and coat the bottom surface with a scattering layer. The channel and the coated coverslip substrate are then placed on top of an image sensor for diffraction data acquisition. Similar to the operation of a flow cytometer, the device utilizes microfluidic flow to deliver specimens across the channel. The diffracted light from the flowing objects is modulated by the scattering layer and recorded by the image sensor for ptychographic reconstruction, where high-resolution quantitative complex images are recovered from the diffraction measurements. By using an image sensor with a 1.85 μm pixel size, our device can resolve the 550 nm linewidth on the resolution target. We validate the device by imaging different types of biospecimens, including C. elegans, yeast cells, paramecium, and closterium sp. We also demonstrate a high-resolution ptychographic reconstruction at a video framerate of 30 frames per second. The reported technique can address a wide range of biomedical needs and engenders new ptychographic imaging innovations in a flow cytometer configuration.

Muscope: a miniature on-chip lensless microscope

We report the Muscope, a miniature lensless holographic microscope suitable for on-chip integration. The prototype of the Muscope measured approximately only 7 mm × 4 mm × 4 mm, and was capable of offering a sub-micron half-pitch resolution. We have used, for the first time, a microLED display as the light source in a microscope. The individual pixels of a microLED display chip are used as programmable, microscopic and intense LEDs which can be spatially moved in a two-dimensional plane with a 5 μm pitch. This unique feature set of the display was used to implement computational super-resolution and wide-field imaging without any extra hardware, unlike many other lensless microscopes. We also report a new method to evaluate the magnification in our setting. The Muscope surpasses the existing lensless microscopes in compactness, scalability for production, automated operation and system integration. It provides exciting opportunities for a new class of devices with in-built optical imaging and monitoring and/or sensing capabilities.

A Compact Raster Lensless Microscope Based on a Microdisplay

Lensless microscopy requires the simplest possible configuration, as it uses only a light source, the sample and an image sensor. The smallest practical microscope is demonstrated here. In contrast to standard lensless microscopy, the object is located near the lighting source. Raster optical microscopy is applied by using a single-pixel detector and a microdisplay. Maximum resolution relies on reduced LED size and the position of the sample respect the microdisplay. Contrarily to other sort of digital lensless holographic microscopes, light backpropagation is not required to reconstruct the images of the sample. In a mm-high microscope, resolutions down to 800 nm have been demonstrated even when measuring with detectors as large as 138 μm × 138 μm, with field of view given by the display size. Dedicated technology would shorten measuring time.

GEOMScope: Large Field-of-view 3D Lensless Microscopy with Low Computational Complexity

Imaging systems with miniaturized device footprint, real-time processing speed and high resolution three-dimensional (3D) visualization are critical to broad biomedical applications such as endoscopy. Most of existing imaging systems rely on bulky lenses and mechanically refocusing to perform 3D imaging. Here, we demonstrate GEOMScope, a lensless single-shot 3D microscope that forms image through a single layer of thin microlens array and reconstructs objects through an innovative algorithm combining geometrical-optics-based pixel back projection and background suppressions. We verify the effectiveness of GEOMScope on resolution target, fluorescent particles and volumetric objects. Comparing to other widefield lensless imaging devices, we significantly reduce the required computational resource and increase the reconstruction speed by orders of magnitude. This enables us to image and recover large volume 3D object in high resolution with near real-time processing speed. Such a low computational complexity is attributed to the joint design of imaging optics and reconstruction algorithms, and a joint application of geometrical optics and machine learning in the 3D reconstruction. More broadly, the excellent performance of GEOMScope in imaging resolution, volume, and reconstruction speed implicates that geometrical optics could greatly benefit and play an important role in computational imaging.

Multi-frequency impedance sensing for detection and sizing of DNA fragments

Electronic biosensors for DNA detection typically utilize immobilized oligonucleotide probes on a signal transducer, which outputs an electronic signal when target molecules bind to probes. However, limitation in probe selectivity and variable levels of non-target material in complex biological samples can lead to nonspecific binding and reduced sensitivity. Here we introduce the integration of 2.8 μm paramagnetic beads with DNA fragments. We apply a custom-made microfluidic chip to detect DNA molecules bound to beads by measuring Impedance Peak Response (IPR) at multiple frequencies. Technical and analytical performance was evaluated using beads containing purified Polymerase Chain Reaction (PCR) products of different lengths (157, 300, 613 bp) with DNA concentration ranging from 0.039 amol to 7.8 fmol. Multi-frequency IPR correlated positively with DNA amounts and was used to calculate a DNA quantification score. The minimum DNA amount of a 300 bp fragment coupled on beads that could be robustly detected was 0.0039 fmol (1.54 fg or 4750 copies/bead). Additionally, our approach allowed distinguishing beads with similar molar concentration DNA fragments of different lengths. Using this impedance sensor, purified PCR products could be analyzed within ten minutes to determine DNA fragment length and quantity based on comparison to a known DNA standard.

Automatic imaging of Drosophila embryos with light sheet fluorescence microscopy on chip

We present a microscope on chip for automated imaging of Drosophila embryos by light sheet fluorescence microscopy. This integrated device, constituted by both optical and microfluidic components, allows the automatic acquisition of a 3D stack of images for specimens diluted in a liquid suspension. The device has been fully optimized to address the challenges related to the specimens under investigation. Indeed, the thickness and the high ellipticity of Drosophila embryos can degrade the image quality. In this regard, optical and fluidic optimization has been carried out to implement dual-sided illumination and automatic sample orientation. In addition, we highlight the dual color investigation capabilities of this device, by processing two sample populations encoding different fluorescent proteins. This work was made possible by the versatility of the used fabrication technique, femtosecond laser micromachining, which allows straightforward fabrication of both optical and fluidic components in glass substrates.

Random access parallel microscopy

We introduce a random-access parallel (RAP) imaging modality that uses a novel design inspired by a Newtonian telescope to image multiple spatially separated samples without moving parts or robotics. This scheme enables near-simultaneous image capture of multiple petri dishes and random-access imaging with sub-millisecond switching times at the full resolution of the camera. This enables the RAP system to capture long-duration records from different samples in parallel, which is not possible using conventional automated microscopes. The system is demonstrated by continuously imaging multiple cardiac monolayer and Caenorhabditis elegans preparations.

Reducing time to pregnancy and facilitating the birth of healthy children through functional analysis of embryo physiology†

An ever-increasing number of couples rely on assisted reproductive technologies (ART) in order to conceive a child. Although advances in embryo culture have led to increases in the success rates of clinical ART, it often takes more than one treatment cycle to conceive a child. Ensuring patients conceive as soon as possible with a healthy embryo is a priority for reproductive medicine. Currently, selection of embryos for transfer relies predominantly on the morphological assessment of the preimplantation embryo; however, morphology is not an absolute link to embryo physiology, nor the health of the resulting child. Non-invasive quantitation of individual embryo physiology, a key regulator of both embryo viability and health, could provide valuable information to assist in the selection of the most viable embryo for transfer, hence reducing the time to pregnancy. Further, according to the Barker Hypothesis, the environment to which a fetus is exposed to during gestation affects subsequent offspring health. If the environment of the preimplantation period is capable of affecting metabolism, which in turn will affect gene expression through the metaboloepigenetic link, then assessment of embryo metabolism should represent an indirect measure of future offspring health. Previously, the term viable embryo has been used in association with the potential of an embryo to establish a pregnancy. Here, we propose the term healthy embryo to reflect the capacity of that embryo to lead to a healthy child and adult.

Lensless, reflection-based dark-field microscopy (RDFM) on a CMOS chip

We present for the first time a lens-free, oblique illumination imaging platform for on-sensor dark- field microscopy and shadow-based 3D object measurements. It consists of an LED point source that illuminates a 5-megapixel, 1.4 µm pixel size, back-illuminated CMOS sensor at angles between 0° and 90°. Analytes (polystyrene beads, microorganisms, and cells) were placed and imaged directly onto the sensor. The spatial resolution of this imaging system is limited by the pixel size (∼1.4 µm) over the whole area of the sensor (3.6×2.73 mm). We demonstrated two imaging modalities: (i) shadow imaging for estimation of 3D object dimensions (on polystyrene beads and microorganisms) when the illumination angle is between 0° and 85°, and (ii) dark-field imaging, at >85° illumination angles. In dark-field mode, a 3-4 times drop in background intensity and contrast reversal similar to traditional dark-field imaging was observed, due to larger reflection intensities at those angles. With this modality, we were able to detect and analyze morphological features of bacteria and single-celled algae clusters.

High-resolution and large field-of-view Fourier ptychographic microscopy and its applications in biomedicine

Fourier ptychographic microscopy (FPM) is a promising and fast-growing computational imaging technique with high resolution, wide field-of-view (FOV) and quantitative phase recovery, which effectively tackles the problems of phase loss, aberration-introduced artifacts, narrow depth-of-field and the trade-off between resolution and FOV in conventional microscopy simultaneously. In this review, we provide a comprehensive roadmap of microscopy, the fundamental principles, advantages, and drawbacks of existing imaging techniques, and the significant roles that FPM plays in the development of science. Since FPM is an optimization problem in nature, we discuss the framework and related work. We also reveal the connection of Euler's formula between FPM and structured illumination microscopy. We review recent advances in FPM, including the implementation of high-precision quantitative phase imaging, high-throughput imaging, high-speed imaging, three-dimensional imaging, mixed-state decoupling, and introduce the prosperous biomedical applications. We conclude by discussing the challenging problems and future applications. FPM can be extended to a kind of framework to tackle the phase loss and system limits in the imaging system. This insight can be used easily in speckle imaging, incoherent imaging for retina imaging, large-FOV fluorescence imaging, etc.

AI on a chip

Artificial intelligence (AI) has dramatically changed the landscape of science, industry, defence, and medicine in the last several years. Supported by considerably enhanced computational power and cloud storage, the field of AI has shifted from mostly theoretical studies in the discipline of computer science to diverse real-life applications such as drug design, material discovery, speech recognition, self-driving cars, advertising, finance, medical imaging, and astronomical observation, where AI-produced outcomes have been proven to be comparable or even superior to the performance of human experts. In these applications, what is essentially important for the development of AI is the data needed for machine learning. Despite its prominent importance, the very first process of the AI development, namely data collection and data preparation, is typically the most laborious task and is often a limiting factor of constructing functional AI algorithms. Lab-on-a-chip technology, in particular microfluidics, is a powerful platform for both the construction and implementation of AI in a large-scale, cost-effective, high-throughput, automated, and multiplexed manner, thereby overcoming the above bottleneck. On this platform, high-throughput imaging is a critical tool as it can generate high-content information (e.g., size, shape, structure, composition, interaction) of objects on a large scale. High-throughput imaging can also be paired with sorting and DNA/RNA sequencing to conduct a massive survey of phenotype-genotype relations whose data is too complex to analyze with traditional computational tools, but is analyzable with the power of AI. In addition to its function as a data provider, lab-on-a-chip technology can also be employed to implement the developed AI for accurate identification, characterization, classification, and prediction of objects in mixed, heterogeneous, or unknown samples. In this review article, motivated by the excellent synergy between AI and lab-on-a-chip technology, we outline fundamental elements, recent advances, future challenges, and emerging opportunities of AI with lab-on-a-chip technology or "AI on a chip" for short.

Microfluidic-based linear-optics label-free imager

Linear optics based nanoscopy previously reached resolution beyond the diffraction limit, illuminating samples in the visible light regime while allowing light to interact with freely moving metallic nanoparticles. However, the hydrodynamics governing the nanoparticle motion used to scan the sample is very complex and has low probability of achieving appropriate and fast mapping in practice. Hence, an implementation of the technique on real biological samples has not been demonstrated so far. Moreover, a suitable way to perform controlled nanoparticle scanning of biological samples is required. Here we show a solution where a microfluidic channel is used to flow and trap biological samples inside a water droplet along with suspended nanoparticles surrounded by silicone oil. The evanescent light scattered from the sample and is rescattered by the nanoparticles in the vicinity. This encodes the sub-wavelength features of the sample which can later on be decoded and reconstructed from measurements in the far field. The microfluidic system-controlled flow allows better nanoparticle scanning of the sample and maintains an isolated system for each sample in each droplet. A more localized scan at the droplet water/oil interface is also conducted using amphiphilic nanoparticles where their hydrophilic side is constrained to the droplet and their hydrophobic side is constrained to the oil. This allows higher probability of capturing evanescent fields closer to their origin, yielding better resolution and a higher signal to noise ratio. Using this system, we obtained images of an E. coli sample and demonstrated how the method yield fine resolution of the sample contours. To the best of our knowledge, this is the first time that a linear and label free optics imaging process was performed using a micro-fluidic device.

On-chip fluorescence microscopy with a random microlens diffuser

We present an on-chip, widefield fluorescence microscope, which consists of a diffuser placed a few millimeters away from a traditional image sensor. The diffuser replaces the optics of a microscope, resulting in a compact and easy-to-assemble system with a practical working distance of over 1.5 mm. Furthermore, the diffuser encodes volumetric information, enabling refocusability in post-processing and three-dimensional (3D) imaging of sparse samples from a single acquisition. Reconstruction of images from the raw data requires a precise model of the system, so we introduce a practical calibration scheme and a physics-based forward model to efficiently account for the spatially-varying point spread function (PSF). To improve performance in low-light, we propose a random microlens diffuser, which consists of many small lenslets randomly placed on the mask surface and yields PSFs that are robust to noise. We build an experimental prototype and demonstrate our system on both planar and 3D samples.

Harnessing the Power of Optical Microscopic and Macroscopic Imaging for Natural Products as Cancer Therapeutics

Natural products (NPs) are an important source for new drug discovery over the past decades, which have been demonstrated to be effectively used in cancer prevention, treatment, and adjuvant therapy. Many methods, such as the genomic and metabolomic approaches, immunochemistry, mass spectrometry, and chromatography, have been used to study the effects of NPs on cancer as well as themselves. Because of the advantages in specificity, sensitivity, high throughput, and cost-effectiveness, optical imaging (OI) approaches, including optical microscopic imaging and macroscopic imaging techniques have also been applied in the studies of NPs. Optical microscopic imaging can observe NPs as cancer therapeutics at the cellular level and analyze its cytotoxicity and mechanism of action. Optical macroscopic imaging observes the distribution, metabolic pathway, and target lesions of NPs in vivo, and evaluates NPs as cancer therapeutics at the whole-body level in small living animals. This review focuses on the recent advances in NPs as cancer therapeutics, with particular emphasis on the powerful use of optical microscopic and macroscopic imaging techniques, including the studies of observation of ingestion by cells, anticancer mechanism, and in vivo delivery. Finally, we prospect the wider application and future potential of OI approaches in NPs as cancer therapeutics.

Robust autofocusing method for multi-wavelength lensless imaging

Lensless imaging based on multi-wavelength phase retrieval becomes a promising technology widely used as it has simple acquisition, miniaturized size and low-cost setup. However, measuring the sample-to-sensor distance with high accuracy, which is the key for high-resolution reconstruction, is still a challenge. In this work, we propose a multi-wavelength criterion to realize autofocusing modulation, i.e., achieving much higher accuracy in determining the sample-to-sensor distance, compared to the conventional methods. Three beams in different spectrums are adopted to illuminate the sample, and the resulting holograms are recorded by a CCD camera. The patterns calculated by performing back propagation of the recorded holograms, with exhaustively searched sample-to-sensor distance value, are adopted to access the criterion. Image sharpness can be accessed and the optimal sample-to-sensor distance can be finely determined by targeting the valley of the curve given by the criterion. Through our novel multi-wavelength based autofocusing strategy and executing further phase retrieval process, high-resolution images can be finally retrieved. The applicability and robustness of our method is validated both in simulations and experiments. Our technique provides a useful tool for multi-wavelength lensless imaging under limited experimental conditions.

Optical Imaging Approaches to Monitor Static and Dynamic Cell-on-Chip Platforms: A Tutorial Review

Miniaturized laboratories on chip platforms play an important role in handling life sciences studies. The platforms may contain static or dynamic biological cells. Examples are a fixed medium of an organ-on-a-chip and individual cells moving in a microfluidic channel, respectively. Due to feasibility of control or investigation and ethical implications of live targets, both static and dynamic cell-on-chip platforms promise various applications in biology. To extract necessary information from the experiments, the demand for direct monitoring is rapidly increasing. Among different microscopy methods, optical imaging is a straightforward choice. Considering light interaction with biological agents, imaging signals may be generated as a result of scattering or emission effects from a sample. Thus, optical imaging techniques could be categorized into scattering-based and emission-based techniques. In this review, various optical imaging approaches used in monitoring static and dynamic platforms are introduced along with their optical systems, advantages, challenges, and applications. This review may help biologists to find a suitable imaging technique for different cell-on-chip studies and might also be useful for the people who are going to develop optical imaging systems in life sciences studies.

Micro-optical Components for Bioimaging on Tissues, Cells and Subcellular Structures

Bioimaging generally indicates imaging techniques that acquire biological information from living forms. Among different imaging techniques, optical microscopy plays a predominant role in observing tissues, cells and biomolecules. Along with the fast development of microtechnology, developing miniaturized and integrated optical imaging systems has become essential to provide new imaging solutions for point-of-care applications. In this review, we will introduce the basic micro-optical components and their fabrication technologies first, and further emphasize the development of integrated optical systems for in vitro and in vivo bioimaging, respectively. We will conclude by giving our perspectives on micro-optical components for bioimaging applications in the near future.

Mechanically scanned interference pattern structured illumination imaging

We present a fully lensless single pixel imaging technique using mechanically scanned interference patterns. The method uses only simple, flat optics; no lenses, curved mirrors, or acousto-optics are used in pattern formation or detection. The resolution is limited by the numerical aperture of the angular access to the object, with a fundamental limit of a quarter wavelength and no fundamental limit on working distance. While it is slower than some similar techniques, the lack of a lens objective and simplification of the required optics could make it more applicable in difficult wavelength regimes such as UV or X-ray.

High-throughput mechanotransduction in Drosophila embryos with mesofluidics

Developing embryos create complexity by expressing genes to coordinate movement which generates mechanical force. An emerging theory is that mechanical force can also serve as an input signal to regulate developmental gene expression. Experimental methods to apply mechanical stimulation to whole embryos have been limited, mainly to aspiration, indentation, or moving a coverslip; these approaches stimulate only a few embryos at a time and require manual alignment. A powerful approach for automation is microfluidic devices, which can precisely manipulate hundreds of samples. However, using microfluidics to apply mechanical stimulation has been limited to small cellular systems, with fewer applications for larger scale whole embryos. We developed a mesofluidic device that applies the precision and automation of microfluidics to the Drosophila embryo: high-throughput automatic alignment, immobilization, compression, real-time imaging, and recovery of hundreds of live embryos. We then use twist:eGFP embryos to show that the mechanical induction of twist depends on the dose and duration of compression. This device allows us to quantify responses to compression, map the distribution of ectopic twist, and measure embryo stiffness. For building mesofluidic devices, we describe modifications on ultra-thick photolithography, derive an analytical model that predicts the deflection of sidewalls, and discuss parametric calibration. This "mesomechanics" approach combines the high-throughput automation and precision of microfluidics with the biological relevance of live embryos to examine mechanotransduction. These analytical models facilitate the design of future devices to process multicellular organisms such as larvae, organoids, and mesoscale tissue samples.

Dual-phone illumination-imaging system for high resolution and large field of view multi-modal microscopy

In this paper we present for the first time a system comprised of two mobile phones, one for illumination and the other for microscopy, as a portable, user-friendly, and cost-effective microscopy platform for a wide range of applications. Versatile and adaptive illumination is made with a Retina display of an Apple mobile phone device. The phone screen is used to project various illumination patterns onto the specimen being imaged, each corresponding to a different illumination mode, such as bright-field, dark-field, point illumination, Rheinberg illumination, and fluorescence microscopy. The second phone (a Nokia phone) is modified to record microscopic images about the sample. This imaging platform provides a high spatial resolution of at least 2 μm, a large field-of-view of 3.6 × 2.7 mm, and a working distance of 0.6 mm. We demonstrate the performance of this platform for the visualization of microorganisms within microfluidic devices to gather qualitative and quantitative information regarding microorganism morphology, dimension, count, and velocity/trajectories in the x-y plane.

Open Source 3D-printed focussing mechanism for cellphone-based cellular microscopy

The need to improve access to microscopes in low-resource and educational settings coupled with the global proliferation of camera-enabled cellphones has recently led to an explosion in new developments in portable, low-cost microscopy. The availability of accurate ball lenses has resulted in many variants of van Leeuwenhoek-like microscopes. Combined with cellphones, they have the potential for use as portable microscopes in education and clinics. The need for reproducibility in such applications implies that control over focus is critical. Here, we describe a 3D-printed focussing mechanism based on a rack and pinion mechanism, coupled to a ball lens- based microscope. We quantify the time-stability of the focussing mechanism through an edge-based contrast measure used in autofocus cameras and apply it to 'thin smear' blood sample infected with Plasmodium as well as onion skin cells. We show that stability of the z-focus is in the micrometre range. This development could, we believe, serve to further enhance the utility of a low-cost and robust microscope and encourage further developments in field microscopes based on the Open Source principle. LAY DESCRIPTION: The wide spread of cellphones with cameras makes them an attractive platform for digital microscopy. Such microscopes could help improve microscope access in clinics and classrooms in the form of 'field microscopes', if they could be adapted for imaging cells. We integrate a 3D printed focussing mechanism made with recyclable plastic with ball-lens microscope of the Leeuwenhoek type. We demonstrate how the device can help stabilise to a focal plane for acquiring movies of a thin-smear of blood infected with Plasmodium and onion skin cells using a cellphone. The stability of focus is expectedly less precise as compared to research-grade microscopes, but is of the range of a few micrometers. We believe, the focussing device demonstrates it is possible to obtain reliable and reproducible images of typical samples used in clinics and classrooms. By making the design files of this device open-source we believe it could serve as a small step in improved, affordable and accurate 'field microscopes'.

Microfluidic Based Optical Microscopes on Chip

Last decade's advancements in optofluidics allowed obtaining an ever increasing integration of different functionalities in lab on chip devices to culture, analyze, and manipulate single cells and entire biological specimens. Despite the importance of optical imaging for biological sample monitoring in microfluidics, imaging is traditionally achieved by placing microfluidics channels in standard bench-top optical microscopes. Recently, the development of either integrated optical elements or lensless imaging methods allowed optical imaging techniques to be implemented in lab on chip systems, thus increasing their automation, compactness, and portability. In this review, we discuss known solutions to implement microscopes on chip that exploit different optical methods such as bright-field, phase contrast, holographic, and fluorescence microscopy.

Superresolution Line Scan Image Sensor for Multimodal Microscopy

A low-cost contact scanning microscope is presented which performs optical imaging of millimeter-scale samples with multiple sensory modalities at a spatial resolution better than the pixel size in both x and y dimensions. The 7.5 mm 3.2 mm 0.35 m CMOS image sensor is comprised of 214 scanning lines of 256 pixels, each line horizontally shifted by 300 nm with respect to the adjacent lines. When scanning in the y dimension, this results in a staircase-like staggered-pixels organization with an effective spatial resolution in the x dimension of less than the pixel size, with a theoretical limit of 300 nm, subject to the light diffraction limit and to photodiode size-dependent spatial aliasing. The height of the resulting pixel "staircases" is capped at 2.5 mm by wrapping the 215th row back to the first row, yielding an approximately 2 mm 2.5 mm instantaneous scanning window size. The spatial resolution in the y dimension is set by the sample scanning rate and the frame rate, subject to the same limitations. Integration of multiple scanning lines naturally lends itself to the inclusion of multiple sensory modalities, with five modalities included as an example: High-resolution (up to 300 nm), fluorescence-sensitive, and triple-orientation light polarization-sensitive pixels. The resulting modified scanning pattern is digitized by on-chip column-parallel 2nd order Delta-Sigma ADCs with ENOB of 9.1 and is reconstructed into a full-resolution image in software. Experimental measurements, where contact-scanning is emulated by the sample image moving on an LCD monitor and projected through a lens, support the validity of the presented concept.

Computational Optics Enables Breast Cancer Profiling in Point-of-Care Settings

The global burden of cancer, severe diagnostic bottlenecks in underserved regions, and underfunded health care systems are fueling the need for inexpensive, rapid, and treatment-informative diagnostics. On the basis of advances in computational optics and deep learning, we have developed a low-cost digital system, termed AIDA (artificial intelligence diffraction analysis), for breast cancer diagnosis of fine needle aspirates. Here, we show high accuracy (>90%) in (i) recognizing cells directly from diffraction patterns and (ii) classifying breast cancer types using deep-learning-based analysis of sample aspirates. The image algorithm is fast, enabling cellular analyses at high throughput (∼3 s per 1000 cells), and the unsupervised processing allows use by lower skill health care workers. AIDA can perform quantitative molecular profiling on individual cells, revealing intratumor molecular heterogeneity, and has the potential to improve cancer diagnosis and treatment. The system could be further developed for other cancers and thus find widespread use in global health.

Fast and robust Fourier domain-based classification for on-chip lens-free flow cytometry

The development of portable haematology analysers receives increased attention due to their deployability in resource-limited or emergency settings. Lens-free in-line holographic microscopy is one of the technologies that is being pushed forward in this regard as it eliminates complex and expensive optics, making miniaturisation and integration with microfluidics possible. On-chip flow cytometry enables high-speed capturing of individual cells in suspension, giving rise to high-throughput cell counting and classification. To perform a real-time analysis on this high-throughput content, we propose a fast and robust framework for the classification of leukocytes. The raw data consists of holographic acquisitions of leukocytes, captured with a high-speed camera as they are flowing through a microfluidic chip. Three different types of leukocytes are considered: granulocytes, monocytes and T-lymphocytes. The proposed method bypasses the reconstruction of the holographic data altogether by extracting Zernike moments directly from the frequency domain. By doing so, we introduce robustness to translations and rotations of cells, as well as to changes in distance of a cell with respect to the image sensor, achieving classification accuracies up to 96.8%. Furthermore, the reduced computational complexity of this approach, compared to traditional frameworks that involve the reconstruction of the holographic data, allows for very fast processing and classification, making it applicable in high-throughput flow cytometry setups.

Compact lensless subpixel resolution large field of view microscope

We report on a method to increase the spatial resolution in a compact lensless microscope. A compact side illumination is fabricated to illuminate the sample with a collimated beam by diffraction from a volume phase grating. The wavelength of a semi-conductor laser source (vertical-cavity surface-emitting laser) is tuned with the injection current to alter the illumination direction by wavelength selective diffraction from the volume phase grating. The angle tuning is such that several subpixel shifted digital inline holograms are obtained. The stack of holograms is then processed in a pixel super-resolution reconstruction algorithm. The amplitude of the sample is reconstructed with subpixel resolution over a large field of view (FOV). The technique is demonstrated on a 1951 USAF test target. A resolution of ∼2.76  μm, over a FOV of ∼28  mm², is demonstrated for a device of <2  cm height. The original pixel size was 5.2 μm demonstrating the subpixel resolution.

CMOS Enabled Microfluidic Systems for Healthcare Based Applications

With the increased global population, it is more important than ever to expand accessibility to affordable personalized healthcare. In this context, a seamless integration of microfluidic technology for bioanalysis and drug delivery and complementary metal oxide semiconductor (CMOS) technology enabled data-management circuitry is critical. Therefore, here, the fundamentals, integration aspects, and applications of CMOS-enabled microfluidic systems for affordable personalized healthcare systems are presented. Critical components, like sensors, actuators, and their fabrication and packaging, are discussed and reviewed in detail. With the emergence of the Internet-of-Things and the upcoming Internet-of-Everything for a people-process-data-device connected world, now is the time to take CMOS-enabled microfluidics technology to as many people as possible. There is enormous potential for microfluidic technologies in affordable healthcare for everyone, and CMOS technology will play a major role in making that happen.

Reflective lens-free imaging on high-density silicon microelectrode arrays for monitoring and evaluation of in vitro cardiac contractility

The high rate of drug attrition caused by cardiotoxicity is a major challenge for drug development. Here, we developed a reflective lens-free imaging (RLFI) approach to non-invasively record in vitro cell deformation in cardiac monolayers with high temporal (169 fps) and non-reconstructed spatial resolution (352 µm) over a field-of-view of maximally 57 mm². The method is compatible with opaque surfaces and silicon-based devices. Further, we demonstrated that the system can detect the impairment of both contractility and fast excitation waves in cardiac monolayers. Additionally, the RLFI device was implemented on a CMOS-based microelectrode array to retrieve multi-parametric information of cardiac cells, thereby offering more in-depth analysis of drug-induced (cardiomyopathic) effects for preclinical cardiotoxicity screening applications.

New live screening of plant-nematode interactions in the rhizosphere

Free living nematodes (FLN) are microscopic worms found in all soils. While many FLN species are beneficial to crops, some species cause significant damage by feeding on roots and vectoring viruses. With the planned legislative removal of traditionally used chemical treatments, identification of new ways to manage FLN populations has become a high priority. For this, more powerful screening systems are required to rapidly assess threats to crops and identify treatments efficiently. Here, we have developed new live assays for testing nematode responses to treatment by combining transparent soil microcosms, a new light sheet imaging technique termed Biospeckle Selective Plane Illumination Microscopy (BSPIM) for fast nematode detection, and Confocal Laser Scanning Microscopy for high resolution imaging. We show that BSPIM increased signal to noise ratios by up to 60 fold and allowed the automatic detection of FLN in transparent soil samples of 1.5 mL. Growing plant root systems were rapidly scanned for nematode abundance and activity, and FLN feeding behaviour and responses to chemical compounds observed in soil-like conditions. This approach could be used for direct monitoring of FLN activity either to develop new compounds that target economically damaging herbivorous nematodes or ensuring that beneficial species are not negatively impacted.

Micro-optics for microfluidic analytical applications

This critical review summarizes the developments in the integration of micro-optical elements with microfluidic platforms for facilitating detection and automation of bio-analytical applications.

Effect of counting chamber depth on the accuracy of lensless microscopy for the assessment of boar sperm motility

Sperm motility is one of the most significant parameters in the prediction of male fertility. Until now, both motility analysis using an optical microscope and computer-aided sperm analysis (CASA-Mot) entailed the use of counting chambers with a depth to 20 µm. Chamber depth significantly affects the intrinsic sperm movement, leading to an artificial motility pattern. For the first time, laser microscopy offers the possibility of avoiding this interference with sperm movement. The aims of the present study were to determine the different motility patterns observed in chambers with depths of 10, 20 and 100 µm using a new holographic approach and to compare the results obtained in the 20-µm chamber with those of the laser and optical CASA-Mot systems. The ISAS®3D-Track results showed that values for curvilinear velocity (VCL), straight line velocity, wobble and beat cross frequency were higher for the 100-µm chambers than for the 10- and 20-µm chambers. Only VCL showed a positive correlation between chambers. In addition, Bayesian analysis confirmed that the kinematic parameters observed with the 100-µm chamber were significantly different to those obtained using chambers with depths of 10 and 20 µm. When an optical analyser CASA-Mot system was used, all kinematic parameters, except VCL, were higher with ISAS®3D-Track, but were not relevant after Bayesian analysis. Finally, almost three different three-dimensional motility patterns were recognised. In conclusion, the use of the ISAS®3D-Track allows for the analysis of the natural three-dimensional pattern of sperm movement.

Optics-Integrated Microfluidic Platforms for Biomolecular Analyses

Compared with conventional optical methods, optics implemented on microfluidic chips provide small, and often much cheaper ways to interrogate biological systems from the level of single molecules up to small model organisms. The optical probing of single molecules has been used to investigate the mechanical properties of individual biological molecules; however, multiplexing of these measurements through microfluidics and nanofluidics confers many analytical advantages. Optics-integrated microfluidic systems can significantly simplify sample processing and allow a more user-friendly experience; alignments of on-chip optical components are predetermined during fabrication and many purely optical techniques are passively controlled. Furthermore, sample loss from complicated preparation and fluid transfer steps can be virtually eliminated, a particularly important attribute for biological molecules at very low concentrations. Excellent fluid handling and high surface area/volume ratios also contribute to faster detection times for low abundance molecules in small sample volumes. Although integration of optical systems with classical microfluidic analysis techniques has been limited, microfluidics offers a ready platform for interrogation of biophysical properties. By exploiting the ease with which fluids and particles can be precisely and dynamically controlled in microfluidic devices, optical sensors capable of unique imaging modes, single molecule manipulation, and detection of minute changes in concentration of an analyte are possible.

Compact, cost-effective and field-portable microscope prototype based on MISHELF microscopy

We report on a reduced cost, portable and compact prototype design of lensless holographic microscope with an illumination/detection scheme based on wavelength multiplexing, working with single hologram acquisition and using a fast convergence algorithm for image processing. All together, MISHELF (initials coming from Multi-Illumination Single-Holographic-Exposure Lensless Fresnel) microscopy allows the recording of three Fresnel domain diffraction patterns in a single camera snap-shot incoming from illuminating the sample with three coherent lights at once. Previous implementations have proposed an illumination/detection procedure based on a tuned (illumination wavelengths centered at the maximum sensitivity of the camera detection channels) configuration but here we report on a detuned (non-centered ones) scheme resulting in prototype miniaturization and cost reduction. Thus, MISHELF microscopy in combination with a novel and fast iterative algorithm allows high-resolution (μm range) phase-retrieved (twin image elimination) quantitative phase imaging of dynamic events (video rate recording speed). The performance of this microscope prototype is validated through experiments using both amplitude (USAF resolution test) and complex (live swine sperm cells and flowing microbeads) samples. The proposed method becomes in an alternative instrument improving some capabilities of existing lensless microscopes.

Meta-nanocavity model for dynamic super-resolution fluorescent imaging based on the plasmonic structure illumination microscopy method

Biological research requires dynamic and wide-field optical microscopy with resolution down to nanometer to study the biological process in a sub-cell or single molecular level. To address this issue, we propose a dynamic wide-field optical nanoimaging method based on a meta-nanocavity platform (MNCP) model which can be incorporated in micro/nano-fluidic systems so that the samples to be observed can be confined in a nano-scale space for the ease of imaging. It is found that this platform can support standing wave surface plasmons (SW-SPs) interference pattern with a period of 105 nm for a 532 nm incident wavelength. Furthermore, the potential application of the NCP for wide-field super-resolution imaging was discussed and the simulation results show that an imaging resolution of sub-80 nm can be achieved.