Computational cytometer based on magnetically modulated coherent imaging and deep learning

Roadmap on computational methods in optical imaging and holography [invited]

Computational methods have been established as cornerstones in optical imaging and holography in recent years. Every year, the dependence of optical imaging and holography on computational methods is increasing significantly to the extent that optical methods and components are being completely and efficiently replaced with computational methods at low cost. This roadmap reviews the current scenario in four major areas namely incoherent digital holography, quantitative phase imaging, imaging through scattering layers, and super-resolution imaging. In addition to registering the perspectives of the modern-day architects of the above research areas, the roadmap also reports some of the latest studies on the topic. Computational codes and pseudocodes are presented for computational methods in a plug-and-play fashion for readers to not only read and understand but also practice the latest algorithms with their data. We believe that this roadmap will be a valuable tool for analyzing the current trends in computational methods to predict and prepare the future of computational methods in optical imaging and holography.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00340-024-08280-3.

Fluorescence-Based Portable Assays for Detection of Biological and Chemical Analytes

Fluorescence-based detection techniques are part of an ever-expanding field and are widely used in biomedical and environmental research as a biosensing tool. These techniques have high sensitivity, selectivity, and a short response time, making them a valuable tool for developing bio-chemical assays. The endpoint of these assays is defined by changes in fluorescence signal, in terms of its intensity, lifetime, and/or shift in spectrum, which is monitored using readout devices such as microscopes, fluorometers, and cytometers. However, these devices are often bulky, expensive, and require supervision to operate, which makes them inaccessible in resource-limited settings. To address these issues, significant effort has been directed towards integrating fluorescence-based assays into miniature platforms based on papers, hydrogels, and microfluidic devices, and to couple these assays with portable readout devices like smartphones and wearable optical sensors, thereby enabling point-of-care detection of bio-chemical analytes. This review highlights some of the recently developed portable fluorescence-based assays by discussing the design of fluorescent sensor molecules, their sensing strategy, and the fabrication of point-of-care devices.

Microsystem Advances through Integration with Artificial Intelligence

Microfluidics is a rapidly growing discipline that involves studying and manipulating fluids at reduced length scale and volume, typically on the scale of micro- or nanoliters. Under the reduced length scale and larger surface-to-volume ratio, advantages of low reagent consumption, faster reaction kinetics, and more compact systems are evident in microfluidics. However, miniaturization of microfluidic chips and systems introduces challenges of stricter tolerances in designing and controlling them for interdisciplinary applications. Recent advances in artificial intelligence (AI) have brought innovation to microfluidics from design, simulation, automation, and optimization to bioanalysis and data analytics. In microfluidics, the Navier-Stokes equations, which are partial differential equations describing viscous fluid motion that in complete form are known to not have a general analytical solution, can be simplified and have fair performance through numerical approximation due to low inertia and laminar flow. Approximation using neural networks trained by rules of physical knowledge introduces a new possibility to predict the physicochemical nature. The combination of microfluidics and automation can produce large amounts of data, where features and patterns that are difficult to discern by a human can be extracted by machine learning. Therefore, integration with AI introduces the potential to revolutionize the microfluidic workflow by enabling the precision control and automation of data analysis. Deployment of smart microfluidics may be tremendously beneficial in various applications in the future, including high-throughput drug discovery, rapid point-of-care-testing (POCT), and personalized medicine. In this review, we summarize key microfluidic advances integrated with AI and discuss the outlook and possibilities of combining AI and microfluidics.

Deep learning assisted holography microscopy for in-flow enumeration of tumor cells in blood

Currently, detection of circulating tumor cells (CTCs) in cancer patient blood samples relies on immunostaining, which does not provide access to live CTCs, limiting the breadth of CTC-based applications. Here, we take the first steps to address this limitation, by demonstrating staining-free enumeration of tumor cells spiked into lysed blood samples using digital holographic microscopy (DHM), microfluidics and machine learning (ML). A 3D-printed module for laser assembly was developed to simplify the optical set up for holographic imaging of cells flowing through a sheath-based microfluidic device. Computational reconstruction of the holograms was performed to localize the cells in 3D and obtain the plane of best focus images to train deep learning models. We developed a custom-designed light-weight shallow Network dubbed s-Net and compared its performance against off-the-shelf CNN models including ResNet-50. The accuracy, sensitivity and specificity of the s-Net model was found to be higher than the off-the-shelf ML models. By applying an optimized decision threshold to mixed samples prepared in silico, the false positive rate was reduced from 1 × 10^-2 to 2.77 × 10^-4. Finally, the developed DHM-ML framework was successfully applied to enumerate spiked MCF-7 breast cancer cells and SkOV3 ovarian cancer cells from lysed blood samples containing white blood cells (WBCs) at concentrations typical of label-free enrichment techniques. We conclude by discussing the advances that need to be made to translate the DHM-ML approach to staining-free enumeration of actual CTCs in cancer patient blood samples.

Biosensors and machine learning for enhanced detection, stratification, and classification of cells: a review

Biological cells, by definition, are the basic units which contain the fundamental molecules of life of which all living things are composed. Understanding how they function and differentiating cells from one another, therefore, is of paramount importance for disease diagnostics as well as therapeutics. Sensors focusing on the detection and stratification of cells have gained popularity as technological advancements have allowed for the miniaturization of various components inching us closer to Point-of-Care (POC) solutions with each passing day. Furthermore, Machine Learning has allowed for enhancement in the analytical capabilities of these various biosensing modalities, especially the challenging task of classification of cells into various categories using a data-driven approach rather than physics-driven. In this review, we provide an account of how Machine Learning has been applied explicitly to sensors that detect and classify cells. We also provide a comparison of how different sensing modalities and algorithms affect the classifier accuracy and the dataset size required.

Detection of live breast cancer cells in bright-field microscopy images containing white blood cells by image analysis and deep learning

SIGNIFICANCE

Circulating tumor cells (CTCs) are important biomarkers for cancer management. Isolated CTCs from blood are stained to detect and enumerate CTCs. However, the staining process is laborious and moreover makes CTCs unsuitable for drug testing and molecular characterization.

AIM

The goal is to develop and test deep learning (DL) approaches to detect unstained breast cancer cells in bright-field microscopy images that contain white blood cells (WBCs).

APPROACH

We tested two convolutional neural network (CNN) approaches. The first approach allows investigation of the prominent features extracted by CNN to discriminate in vitro cancer cells from WBCs. The second approach is based on faster region-based convolutional neural network (Faster R-CNN).

RESULTS

Both approaches detected cancer cells with higher than 95% sensitivity and 99% specificity with the Faster R-CNN being more efficient and suitable for deployment presenting an improvement of 4% in sensitivity. The distinctive feature that CNN uses for discrimination is cell size, however, in the absence of size difference, the CNN was found to be capable of learning other features. The Faster R-CNN was found to be robust with respect to intensity and contrast image transformations.

CONCLUSIONS

CNN-based DL approaches could be potentially applied to detect patient-derived CTCs from images of blood samples.

MNP-Enhanced Microwave Medical Imaging by Means of Pseudo-Noise Sensing

Magnetic nanoparticles have been investigated for microwave imaging over the last decade. The use of functionalized magnetic nanoparticles, which are able to accumulate selectively within tumorous tissue, can increase the diagnostic reliability. This paper deals with the detecting and imaging of magnetic nanoparticles by means of ultra-wideband microwave sensing via pseudo-noise technology. The investigations were based on phantom measurements. In the first experiment, we analyzed the detectability of magnetic nanoparticles depending on the magnetic field intensity of the polarizing magnetic field, as well as the viscosity of the target and the surrounding medium in which the particles were embedded, respectively. The results show a nonlinear behavior of the magnetic nanoparticle response depending on the magnetic field intensity for magnetic nanoparticles diluted in distilled water and for magnetic nanoparticles embedded in a solid medium. Furthermore, the maximum amplitude of the magnetic nanoparticles responses varies for the different surrounding materials of the magnetic nanoparticles. In the second experiment, we investigated the influence of the target position on the three-dimensional imaging of the magnetic nanoparticles in a realistic measurement setup for breast cancer imaging. The results show that the magnetic nanoparticles can be detected successfully. However, the intensity of the particles in the image depends on its position due to the path-dependent attenuation, the inhomogeneous microwave illumination of the breast, and the inhomogeneity of the magnetic field. Regarding the last point, we present an approach to compensate for the inhomogeneity of the magnetic field by computing a position-dependent correction factor based on the measured magnetic field intensity and the magnetic susceptibility of the magnetic particles. Moreover, the results indicate an influence of the polarizing magnetic field on the measured ultra-wideband signals even without magnetic nanoparticles. Such a disturbing influence of the polarizing magnetic field on the measurements should be reduced for a robust magnetic nanoparticles detection. Therefore, we analyzed the two-state (ON/OFF) and the sinusoidal modulation of the external magnetic field concerning the detectability of the magnetic nanoparticles with respect to these spurious effects, as well as their practical application.

Diagnosis of Herpes Simplex Virus: Laboratory and Point-of-Care Techniques

Herpes is a widespread viral infection caused by the herpes simplex virus (HSV) that has no permanent cure to date. There are two subtypes, HSV-1 and HSV-2, that are known to cause a variety of symptoms, ranging from acute to chronic. HSV is highly contagious and can be transmitted via any type of physical contact. Additionally, viral shedding can also happen from asymptomatic infections. Thus, early and accurate detection of HSV is needed to prevent the transmission of this infection. Herpes can be diagnosed in two ways, by either detecting the presence of the virus in lesions or the antibodies in the blood. Different detection techniques are available based on both laboratory and point of care (POC) devices. Laboratory techniques include different biochemical assays, microscopy, and nucleic acid amplification. In contrast, POC techniques include microfluidics-based tests that enable on-spot testing. Here, we aim to review the different diagnostic techniques, both laboratory-based and POC, their limits of detection, sensitivity, and specificity, as well as their advantages and disadvantages.

Deep learning-enabled imaging flow cytometry for high-speed Cryptosporidium and Giardia detection

Imaging flow cytometry has become a popular technology for bioparticle image analysis because of its capability of capturing thousands of images per second. Nevertheless, the vast number of images generated by imaging flow cytometry imposes great challenges for data analysis especially when the species have similar morphologies. In this work, we report a deep learning-enabled high-throughput system for predicting Cryptosporidium and Giardia in drinking water. This system combines imaging flow cytometry and an efficient artificial neural network called MCellNet, which achieves a classification accuracy >99.6%. The system can detect Cryptosporidium and Giardia with a sensitivity of 97.37% and a specificity of 99.95%. The high-speed analysis reaches 346 frames per second, outperforming the state-of-the-art deep learning algorithm MobileNetV2 in speed (251 frames per second) with a comparable classification accuracy. The reported system empowers rapid, accurate, and high throughput bioparticle detection in clinical diagnostics, environmental monitoring and other potential biosensing applications.

Automatic detection and characterization of quantitative phase images of thalassemic red blood cells using a mask region-based convolutional neural network

SIGNIFICANCE

Label-free quantitative phase imaging is a promising technique for the automatic detection of abnormal red blood cells (RBCs) in real time. Although deep-learning techniques can accurately detect abnormal RBCs from quantitative phase images efficiently, their applications in diagnostic testing are limited by the lack of transparency. More interpretable results such as morphological and biochemical characteristics of individual RBCs are highly desirable.

AIM

An end-to-end deep-learning model was developed to efficiently discriminate thalassemic RBCs (tRBCs) from healthy RBCs (hRBCs) in quantitative phase images and segment RBCs for single-cell characterization.

APPROACH

Two-dimensional quantitative phase images of hRBCs and tRBCs were acquired using digital holographic microscopy. A mask region-based convolutional neural network (Mask R-CNN) model was trained to discriminate tRBCs and segment individual RBCs. Characterization of tRBCs was achieved utilizing SHapley Additive exPlanation analysis and canonical correlation analysis on automatically segmented RBC phase images.

RESULTS

The implemented model achieved 97.8% accuracy in detecting tRBCs. Phase-shift statistics showed the highest influence on the correct classification of tRBCs. Associations between the phase-shift features and three-dimensional morphological features were revealed.

CONCLUSIONS

The implemented Mask R-CNN model accurately identified tRBCs and segmented RBCs to provide single-RBC characterization, which has the potential to aid clinical decision-making.

Detection of Bacterial and Viral Pathogens Using Photonic Point-of-Care Devices

Infectious diseases caused by bacteria and viruses are highly contagious and can easily be transmitted via air, water, body fluids, etc. Throughout human civilization, there have been several pandemic outbreaks, such as the Plague, Spanish Flu, Swine-Flu, and, recently, COVID-19, amongst many others. Early diagnosis not only increases the chance of quick recovery but also helps prevent the spread of infections. Conventional diagnostic techniques can provide reliable results but have several drawbacks, including costly devices, lengthy wait time, and requirement of trained professionals to operate the devices, making them inaccessible in low-resource settings. Thus, a significant effort has been directed towards point-of-care (POC) devices that enable rapid diagnosis of bacterial and viral infections. A majority of the POC devices are based on plasmonics and/or microfluidics-based platforms integrated with mobile readers and imaging systems. These techniques have been shown to provide rapid, sensitive detection of pathogens. The advantages of POC devices include low-cost, rapid results, and portability, which enables on-site testing anywhere across the globe. Here we aim to review the recent advances in novel POC technologies in detecting bacteria and viruses that led to a breakthrough in the modern healthcare industry.

A neural network approach for real-time particle/cell characterization in microfluidic impedance cytometry

Microfluidic applications such as active particle sorting or selective enrichment require particle classification techniques that are capable of working in real time. In this paper, we explore the use of neural networks for fast label-free particle characterization during microfluidic impedance cytometry. A recurrent neural network is designed to process data from a novel impedance chip layout for enabling real-time multiparametric analysis of the measured impedance data streams. As demonstrated with both synthetic and experimental datasets, the trained network is able to characterize with good accuracy size, velocity, and cross-sectional position of beads, red blood cells, and yeasts, with a unitary prediction time of 0.4 ms. The proposed approach can be extended to other device designs and cell types for electrical parameter extraction. This combination of microfluidic impedance cytometry and machine learning can serve as a stepping stone to real-time single-cell analysis and sorting. Graphical Abstract.