High-Speed Chemical Imaging by Dense-Net Learning of Femtosecond Stimulated Raman Scattering

Emerging trends in SERS-based veterinary drug detection: multifunctional substrates and intelligent data approaches

Veterinary drug residues in poultry and livestock products present persistent challenges to food safety, necessitating precise and efficient detection methods. Surface-enhanced Raman scattering (SERS) has been identified as a powerful tool for veterinary drug residue analysis due to its high sensitivity and specificity. However, the development of reliable SERS substrates and the interpretation of complex spectral data remain significant obstacles. This review summarizes the development process of SERS substrates, categorizing them into metal-based, rigid, and flexible substrates, and highlighting the emerging trend of multifunctional substrates. The diverse application scenarios and detection requirements for these substrates are also discussed, with a focus on their use in veterinary drug detection. Furthermore, the integration of deep learning techniques into SERS-based detection is explored, including substrate structure design optimization, optical property prediction, spectral preprocessing, and both qualitative and quantitative spectral analyses. Finally, key limitations are briefly outlined, such as challenges in selecting reporter molecules, data imbalance, and computational demands. Future trends and directions for improving SERS-based veterinary drug detection are proposed.

Machine learning empowered coherent Raman imaging and analysis for biomedical applications

In situ and in vivo visualization and analysis of functional, endogenous biomolecules in living systems have generated a pivotal impact in our comprehension of biology and medicine. An increasingly adopted approach involves the utilization of molecular vibrational spectroscopy, which delivers notable advantages such as label-free imaging, high spectral density, high sensitivity, and molecule specificity. Nonetheless, analyzing and processing the intricate, multi-dimensional imaging data to extract interpretable and actionable information poses a fundamental obstacle. In contrast to conventional multivariate methods, machine learning has recently gained considerable attention for its capability of discerning essential features from massive datasets. Here, we present a comprehensive review of the latest advancements in the application of machine learning in the molecular spectroscopic imaging fields. We also discuss notable attributes of spectroscopic imaging modalities and explore their broader impact on other imaging techniques.

Unmixing Autoencoder for Image Reconstruction from Hyperspectral Data

Due to the complexity of samples and the limitations in spatial resolution, the spectra in hyperspectral imaging (HSI) are generally contributed to by multiple components, making univariate analysis ineffective. Although feature extraction methods have been applied, the chemical meaning of the compressed variables is difficult to interpret, limiting their further applications. An unmixing autoencoder (UAE) was developed in this work for the separation of the mixed spectra in HSI. The proposed model is composed of an encoder and a fully connected (FC) layer. The former is used to compress the input spectrum into several variables, and the latter is employed to reconstruct the spectrum. Combining reconstruction loss and sparse regularization, the weights and the spectral profiles of the components will be encoded in the compressed variables and the connection weights of FC, respectively. A simulated and three experimental HSI data sets were adopted to investigate the performance of the UAE model. The spectral components were successfully obtained, from which the handwriting under papers was revealed from the image of near-infrared (NIR) diffusive reflectance spectroscopy, and the images of lipids, proteins, and nucleic acids were reconstructed from the Raman and stimulated Raman scattering (SRS) images.

High-throughput single-cell sorting by stimulated Raman-activated cell ejection

Raman-activated cell sorting isolates single cells in a nondestructive and label-free manner, but its throughput is limited by small spontaneous Raman scattering cross section. Coherent Raman scattering integrated with microfluidics enables high-throughput cell analysis, but faces challenges with small cells (<3 μm) and tissue sections. Here, we report stimulated Raman-activated cell ejection (S-RACE) that enables high-throughput single-cell sorting by integrating stimulated Raman imaging, in situ image decomposition, and laser-induced cell ejection. S-RACE allows ejection of live bacteria or fungi guided by their Raman signatures. Furthermore, S-RACE successfully sorted lipid-rich Rhodotorula glutinis cells from a cell mixture with a throughput of ~13 cells per second, and the sorting results were confirmed by downstream quantitative polymerase chain reaction. Beyond single cells, S-RACE shows high compatibility with tissue sections. Incorporating a closed-loop feedback control circuit further enables real-time SRS imaging-identification-ejection. In summary, S-RACE opens exciting opportunities for diverse single-cell sorting applications.

Hyperspectral unmixing for Raman spectroscopy via physics-constrained autoencoders

Raman spectroscopy is widely used across scientific domains to characterize the chemical composition of samples in a nondestructive, label-free manner. Many applications entail the unmixing of signals from mixtures of molecular species to identify the individual components present and their proportions, yet conventional methods for chemometrics often struggle with complex mixture scenarios encountered in practice. Here, we develop hyperspectral unmixing algorithms based on autoencoder neural networks, and we systematically validate them using both synthetic and experimental benchmark datasets created in-house. Our results demonstrate that unmixing autoencoders provide improved accuracy, robustness, and efficiency compared to standard unmixing methods. We also showcase the applicability of autoencoders to complex biological settings by showing improved biochemical characterization of volumetric Raman imaging data from a monocytic cell.

Fast Real-Time Brain Tumor Detection Based on Stimulated Raman Histology and Self-Supervised Deep Learning Model

In intraoperative brain cancer procedures, real-time diagnosis is essential for ensuring safe and effective care. The prevailing workflow, which relies on histological staining with hematoxylin and eosin (H&E) for tissue processing, is resource-intensive, time-consuming, and requires considerable labor. Recently, an innovative approach combining stimulated Raman histology (SRH) and deep convolutional neural networks (CNN) has emerged, creating a new avenue for real-time cancer diagnosis during surgery. While this approach exhibits potential, there exists an opportunity for refinement in the domain of feature extraction. In this study, we employ coherent Raman scattering imaging method and a self-supervised deep learning model (VQVAE2) to enhance the speed of SRH image acquisition and feature representation, thereby enhancing the capability of automated real-time bedside diagnosis. Specifically, we propose the VQSRS network, which integrates vector quantization with a proxy task based on patch annotation for analysis of brain tumor subtypes. Training on images collected from the SRS microscopy system, our VQSRS demonstrates a significant speed enhancement over traditional techniques (e.g., 20-30 min). Comparative studies in dimensionality reduction clustering confirm the diagnostic capacity of VQSRS rivals that of CNN. By learning a hierarchical structure of recognizable histological features, VQSRS classifies major tissue pathological categories in brain tumors. Additionally, an external semantic segmentation method is applied for identifying tumor-infiltrated regions in SRH images. Collectively, these findings indicate that this automated real-time prediction technique holds the potential to streamline intraoperative cancer diagnosis, providing assistance to pathologists in simplifying the process.

Optical Photothermal Infrared - Fluorescence In Situ Hybridization (OPTIR-FISH)

Understanding the metabolic activities of individual cells within complex communities is critical for unraveling their role in human disease. Here, we present a comprehensive protocol for simultaneous cell identification and metabolic analysis with the OPTIR-FISH platform by combining rRNA-tagged FISH probes and isotope-labeled substrates. Fluorescence imaging provides cell identification by the specific binding of rRNA-tagged FISH probes, while OPTIR imaging provides metabolic activities within single cells by isotope-induced red shift on OPTIR spectra. Using bacteria cultured with 13C-glucose as a test bed, the protocol outlines microbial culture with isotopic labeling, fluorescence in situ hybridization (FISH), sample preparation, optimization of the OPTIR-FISH imaging setup, and data acquisition. We also demonstrate how to perform image analysis and interpret spectral data at the single-cell level with high throughput. This protocol's standardized and detailed nature will greatly facilitate its adoption by researchers from diverse backgrounds and disciplines within the broad single-cell metabolism research community.

Multi-molecular hyperspectral PRM-SRS microscopy

Lipids play crucial roles in many biological processes. Mapping spatial distributions and examining the metabolic dynamics of different lipid subtypes in cells and tissues are critical to better understanding their roles in aging and diseases. Commonly used imaging methods (such as mass spectrometry-based, fluorescence labeling, conventional optical imaging) can disrupt the native environment of cells/tissues, have limited spatial or spectral resolution, or cannot distinguish different lipid subtypes. Here we present a hyperspectral imaging platform that integrates a Penalized Reference Matching algorithm with Stimulated Raman Scattering (PRM-SRS) microscopy. Using this platform, we visualize and identify high density lipoprotein particles in human kidney, a high cholesterol to phosphatidylethanolamine ratio inside granule cells of mouse hippocampus, and subcellular distributions of sphingosine and cardiolipin in human brain. Our PRM-SRS displays unique advantages of enhanced chemical specificity, subcellular resolution, and fast data processing in distinguishing lipid subtypes in different organs and species.

Artificial Intelligence for Surface-Enhanced Raman Spectroscopy

Surface-enhanced Raman spectroscopy (SERS), well acknowledged as a fingerprinting and sensitive analytical technique, has exerted high applicational value in a broad range of fields including biomedicine, environmental protection, food safety among the others. In the endless pursuit of ever-sensitive, robust, and comprehensive sensing and imaging, advancements keep emerging in the whole pipeline of SERS, from the design of SERS substrates and reporter molecules, synthetic route planning, instrument refinement, to data preprocessing and analysis methods. Artificial intelligence (AI), which is created to imitate and eventually exceed human behaviors, has exhibited its power in learning high-level representations and recognizing complicated patterns with exceptional automaticity. Therefore, facing up with the intertwining influential factors and explosive data size, AI has been increasingly leveraged in all the above-mentioned aspects in SERS, presenting elite efficiency in accelerating systematic optimization and deepening understanding about the fundamental physics and spectral data, which far transcends human labors and conventional computations. In this review, the recent progresses in SERS are summarized through the integration of AI, and new insights of the challenges and perspectives are provided in aim to better gear SERS toward the fast track.

GAMMA challenge: Glaucoma grAding from Multi-Modality imAges

Glaucoma is a chronic neuro-degenerative condition that is one of the world's leading causes of irreversible but preventable blindness. The blindness is generally caused by the lack of timely detection and treatment. Early screening is thus essential for early treatment to preserve vision and maintain life quality. Colour fundus photography and Optical Coherence Tomography (OCT) are the two most cost-effective tools for glaucoma screening. Both imaging modalities have prominent biomarkers to indicate glaucoma suspects, such as the vertical cup-to-disc ratio (vCDR) on fundus images and retinal nerve fiber layer (RNFL) thickness on OCT volume. In clinical practice, it is often recommended to take both of the screenings for a more accurate and reliable diagnosis. However, although numerous algorithms are proposed based on fundus images or OCT volumes for the automated glaucoma detection, there are few methods that leverage both of the modalities to achieve the target. To fulfil the research gap, we set up the Glaucoma grAding from Multi-Modality imAges (GAMMA) Challenge to encourage the development of fundus & OCT-based glaucoma grading. The primary task of the challenge is to grade glaucoma from both the 2D fundus images and 3D OCT scanning volumes. As part of GAMMA, we have publicly released a glaucoma annotated dataset with both 2D fundus colour photography and 3D OCT volumes, which is the first multi-modality dataset for machine learning based glaucoma grading. In addition, an evaluation framework is also established to evaluate the performance of the submitted methods. During the challenge, 1272 results were submitted, and finally, ten best performing teams were selected for the final stage. We analyse their results and summarize their methods in the paper. Since all the teams submitted their source code in the challenge, we conducted a detailed ablation study to verify the effectiveness of the particular modules proposed. Finally, we identify the proposed techniques and strategies that could be of practical value for the clinical diagnosis of glaucoma. As the first in-depth study of fundus & OCT multi-modality glaucoma grading, we believe the GAMMA Challenge will serve as an essential guideline and benchmark for future research.

High-throughput single-cell sorting by stimulated Raman-activated cell ejection

Single-cell sorting is essential to explore cellular heterogeneity in biology and medicine. Recently developed Raman-activated cell sorting (RACS) circumvents the limitations of fluorescence-activated cell sorting, such as the cytotoxicity of labels. However, the sorting throughputs of all forms of RACS are limited by the intrinsically small cross-section of spontaneous Raman scattering. Here, we report a stimulated Raman-activated cell ejection (S-RACE) platform that enables high-throughput single-cell sorting based on high-resolution multi-channel stimulated Raman chemical imaging, in situ image decomposition, and laser-induced cell ejection. The performance of this platform was illustrated by sorting a mixture of 1 μm polymer beads, where 95% yield, 98% purity, and 14 events per second throughput were achieved. Notably, our platform allows live cell ejection, allowing for the growth of single colonies of bacteria and fungi after sorting. To further illustrate the chemical selectivity, lipid-rich Rhodotorula glutinis cells were successfully sorted from a mixture with Saccharomyces cerevisiae, confirmed by downstream quantitative PCR. Furthermore, by integrating a closed-loop feedback control circuit into the system, we realized real-time single-cell imaging and sorting, and applied this method to precisely eject regions of interest from a rat brain tissue section. The reported S-RACE platform opens exciting opportunities for a wide range of single-cell applications in biology and medicine.

Electronic Preresonance Stimulated Raman Scattering Spectromicroscopy Using Multiple-Plate Continuum

Stimulated Raman scattering (SRS) spectromicroscopy is a powerful technique that enables label-free detection of chemical bonds with high specificity. However, the low Raman cross section due to typical far-electronic resonance excitation seriously restricts the sensitivity and undermines its application to bio-imaging. To address this bottleneck, the electronic preresonance (EPR) SRS technique has been developed to enhance the Raman signals by shifting the excitation frequency toward the molecular absorption. A fundamental weakness of the previous demonstration is the lack of dual-wavelength tunability, making EPR-SRS only applicable to a limited number of species in the proof-of-concept experiment. Here, we demonstrate the EPR-SRS spectromicroscopy using a multiple-plate continuum (MPC) light source able to examine a single vibration mode with independently adjustable pump and Stokes wavelengths. In our experiments, the C═C vibration mode of Alexa 635 is interrogated by continuously scanning the pump-to-absorption frequency detuning throughout the entire EPR region enabled by MPC. The results exhibit 150-fold SRS signal enhancement and good agreement with the Albrecht A-term preresonance model. Signal enhancement is also observed in EPR-SRS images of the whole Drosophila brain stained with Alexa 635. With the improved sensitivity and potential to implement hyperspectral measurement, we envision that MPC-EPR-SRS spectromicroscopy can bring the Raman techniques closer to a routine in bio-imaging.

A weak-labelling and deep learning approach for in-focus object segmentation in 3D widefield microscopy

Three-dimensional information is crucial to our understanding of biological phenomena. The vast majority of biological microscopy specimens are inherently three-dimensional. However, conventional light microscopy is largely geared towards 2D images, while 3D microscopy and image reconstruction remain feasible only with specialised equipment and techniques. Inspired by the working principles of one such technique-confocal microscopy, we propose a novel approach to 3D widefield microscopy reconstruction through semantic segmentation of in-focus and out-of-focus pixels. For this, we explore a number of rule-based algorithms commonly used for software-based autofocusing and apply them to a dataset of widefield focal stacks. We propose a computation scheme allowing the calculation of lateral focus score maps of the slices of each stack using these algorithms. Furthermore, we identify algorithms preferable for obtaining such maps. Finally, to ensure the practicality of our approach, we propose a surrogate model based on a deep neural network, capable of segmenting in-focus pixels from the out-of-focus background in a fast and reliable fashion. The deep-neural-network-based approach allows a major speedup for data processing making it usable for online data processing.

Simultaneous Dual-Band Hyperspectral Stimulated Raman Scattering Microscopy with Femtosecond Optical Parametric Oscillators

Stimulated Raman scattering (SRS) microscopy is a label-free quantitative optical technique for imaging molecular distributions in cells and tissues by probing their intrinsic vibrational frequencies. Despite its usefulness, existing SRS imaging techniques have limited spectral coverage due to either a wavelength tuning constraint or narrow spectral bandwidth. High-wavenumber SRS imaging is commonly used to map lipid and protein distribution in biological cells and visualize cell morphology. However, to detect small molecules or Raman tags, imaging in the fingerprint region or "silent" region, respectively, is often required. For many applications, it is desirable to collect SRS images in two Raman spectral regions simultaneously for visualizing the distribution of specific molecules in cellular compartments or providing accurate ratiometric analysis. In this work, we present an SRS microscopy system using three beams generated by a femtosecond oscillator to acquire hyperspectral SRS image stacks in two arbitrary vibrational frequency bands, between 650-3280 cm-1, simultaneously. We demonstrate potential biomedical applications of the system in investigating fatty acid metabolism, cellular drug uptake and accumulation, and lipid unsaturation level in tissues. We also show that the dual-band hyperspectral SRS imaging system can be adapted for the broadband fingerprint region hyperspectral imaging (1100-1800 cm-1) by simply adding a modulator.

Visualization of a Limonene Synthesis Metabolon Inside Living Bacteria by Hyperspectral SRS Microscopy

Monitoring biosynthesis activity at single-cell level is key to metabolic engineering but is still difficult to achieve in a label-free manner. Using hyperspectral stimulated Raman scattering imaging in the 670-900 cm-1 region, localized limonene synthesis are visualized inside engineered Escherichia coli. The colocalization of limonene and GFP-fused limonene synthase is confirmed by co-registered stimulated Raman scattering and two-photon fluorescence images. The finding suggests a limonene synthesis metabolon with a polar distribution inside the cells. This finding expands the knowledge of de novo limonene biosynthesis in engineered bacteria and highlights the potential of SRS chemical imaging in metabolic engineering research.

Pixel-level multimodal fusion deep networks for predicting subcellular organelle localization from label-free live-cell imaging

Complex intracellular organizations are commonly represented by dividing the metabolic process of cells into different organelles. Therefore, identifying sub-cellular organelle architecture is significant for understanding intracellular structural properties, specific functions, and biological processes in cells. However, the discrimination of these structures in the natural organizational environment and their functional consequences are not clear. In this article, we propose a new pixel-level multimodal fusion (PLMF) deep network which can be used to predict the location of cellular organelle using label-free cell optical microscopy images followed by deep-learning-based automated image denoising. It provides valuable insights that can be of tremendous help in improving the specificity of label-free cell optical microscopy by using the Transformer-Unet network to predict the ground truth imaging which corresponds to different sub-cellular organelle architectures. The new prediction method proposed in this article combines the advantages of a transformer's global prediction and CNN's local detail analytic ability of background features for label-free cell optical microscopy images, so as to improve the prediction accuracy. Our experimental results showed that the PLMF network can achieve over 0.91 Pearson's correlation coefficient (PCC) correlation between estimated and true fractions on lung cancer cell-imaging datasets. In addition, we applied the PLMF network method on the cell images for label-free prediction of several different subcellular components simultaneously, rather than using several fluorescent labels. These results open up a new way for the time-resolved study of subcellular components in different cells, especially for cancer cells.

Multiple Parallel Fusion Network for Predicting Protein Subcellular Localization from Stimulated Raman Scattering (SRS) Microscopy Images in Living Cells

Stimulated Raman Scattering Microscopy (SRS) is a powerful tool for label-free detailed recognition and investigation of the cellular and subcellular structures of living cells. Determining subcellular protein localization from the cell level of SRS images is one of the basic goals of cell biology, which can not only provide useful clues for their functions and biological processes but also help to determine the priority and select the appropriate target for drug development. However, the bottleneck in predicting subcellular protein locations of SRS cell imaging lies in modeling complicated relationships concealed beneath the original cell imaging data owing to the spectral overlap information from different protein molecules. In this work, a multiple parallel fusion network, MPFnetwork, is proposed to study the subcellular locations from SRS images. This model used a multiple parallel fusion model to construct feature representations and combined multiple nonlinear decomposing algorithms as the automated subcellular detection method. Our experimental results showed that the MPFnetwork could achieve over 0.93 dice correlation between estimated and true fractions on SRS lung cancer cell datasets. In addition, we applied the MPFnetwork method to cell images for label-free prediction of several different subcellular components simultaneously, rather than using several fluorescent labels. These results open up a new method for the time-resolved study of subcellular components in different cells, especially cancer cells.

Instant diagnosis of gastroscopic biopsy via deep-learned single-shot femtosecond stimulated Raman histology

Gastroscopic biopsy provides the only effective method for gastric cancer diagnosis, but the gold standard histopathology is time-consuming and incompatible with gastroscopy. Conventional stimulated Raman scattering (SRS) microscopy has shown promise in label-free diagnosis on human tissues, yet it requires the tuning of picosecond lasers to achieve chemical specificity at the cost of time and complexity. Here, we demonstrate that single-shot femtosecond SRS (femto-SRS) reaches the maximum speed and sensitivity with preserved chemical resolution by integrating with U-Net. Fresh gastroscopic biopsy is imaged in <60 s, revealing essential histoarchitectural hallmarks perfectly agreed with standard histopathology. Moreover, a diagnostic neural network (CNN) is constructed based on images from 279 patients that predicts gastric cancer with accuracy >96%. We further demonstrate semantic segmentation of intratumor heterogeneity and evaluation of resection margins of endoscopic submucosal dissection (ESD) tissues to simulate rapid and automated intraoperative diagnosis. Our method holds potential for synchronizing gastroscopy and histopathological diagnosis.

Diagnosis of gastric cancer currently requires gastroscopic biopsy, which requires time and expertize to perform. Here, the authors demonstrate a femto-SRS imaging method which showed high accuracy in diagnosing gastric cancer without the need for pathologistbased diagnosis.

Quantitative Stimulated Raman Scattering Microscopy: Promises and Pitfalls

Since its first demonstration, stimulated Raman scattering (SRS) microscopy has become a powerful chemical imaging tool that shows promise in numerous biological and biomedical applications. The spectroscopic capability of SRS enables identification and tracking of specific molecules or classes of molecules, often without labeling. SRS microscopy also has the hallmark advantage of signal strength that is directly proportional to molecular concentration, allowing for in situ quantitative analysis of chemical composition of heterogeneous samples with submicron spatial resolution and subminute temporal resolution. However, it is important to recognize that quantification through SRS microscopy requires assumptions regarding both system and sample. Such assumptions are often taken axiomatically, which may lead to erroneous conclusions without proper validation. In this review, we focus on the tacitly accepted, yet complex, quantitative aspect of SRS microscopy. We discuss the various approaches to quantitative analysis, examples of such approaches, challenges in different systems, and potential solutions. Through our examination of published literature, we conclude that a scrupulous approach to experimental design can further expand the powerful and incisive quantitative capabilities of SRS microscopy.

Stimulated Raman scattering microscopy in chemistry and life science - Development, innovation, perspectives

In this review, we present a summary of the basics of the Stimulated Raman Scattering (SRS) phenomenon, methods of detecting the signal, and collection of the SRS images. We demonstrate the advantages of SRS imaging, and recent developments, but also the limitations, especially in image capture speeds and spatial resolution. We also compare the use of SRS microscopy in biological system studies with other techniques such as fluorescence microscopy, second-harmonic generation (SHG)-based microscopy, coherent anti-Stokes Raman scattering (CARS), and spontaneous Raman, and we show the compatibility of SRS-based systems with other discussed methods. The review is also focused on indicating innovations in SRS microscopy, on the background of which we present the layout and performance of our homemade setup built from commercially available elements enabling for imaging of the molecular structure of single cells over the spectral range of 800-3600 cm^-1. Methods of image analysis are discussed, including machine learning methods for obtaining images of the distribution of selected molecules and for the detection of pathological lesions in tissues or malignant cells in the context of clinical diagnosis of a wide range of diseases with the use of SRS microscopy. Finally, perspectives for the development of SRS microscopy are proposed.

High-Throughput Molecular Imaging via Deep-Learning-Enabled Raman Spectroscopy

Raman spectroscopy enables nondestructive, label-free imaging with unprecedented molecular contrast, but is limited by slow data acquisition, largely preventing high-throughput imaging applications. Here, we present a comprehensive framework for higher-throughput molecular imaging via deep-learning-enabled Raman spectroscopy, termed DeepeR, trained on a large data set of hyperspectral Raman images, with over 1.5 million spectra (400 h of acquisition) in total. We first perform denoising and reconstruction of low signal-to-noise ratio Raman molecular signatures via deep learning, with a 10× improvement in the mean-squared error over common Raman filtering methods. Next, we develop a neural network for robust 2-4× spatial super-resolution of hyperspectral Raman images that preserve molecular cellular information. Combining these approaches, we achieve Raman imaging speed-ups of up to 40-90×, enabling good-quality cellular imaging with a high-resolution, high signal-to-noise ratio in under 1 min. We further demonstrate Raman imaging speed-up of 160×, useful for lower resolution imaging applications such as the rapid screening of large areas or for spectral pathology. Finally, transfer learning is applied to extend DeepeR from cell to tissue-scale imaging. DeepeR provides a foundation that will enable a host of higher-throughput Raman spectroscopy and molecular imaging applications across biomedicine.

Unsupervised hyperspectral stimulated Raman microscopy image enhancement: denoising and segmentation via one-shot deep learning

Hyperspectral stimulated Raman scattering (SRS) microscopy is a label-free technique for biomedical and mineralogical imaging which can suffer from low signal-to-noise ratios. Here we demonstrate the use of an unsupervised deep learning neural network for rapid and automatic denoising of SRS images: UHRED (Unsupervised Hyperspectral Resolution Enhancement and Denoising). UHRED is capable of "one-shot" learning; only one hyperspectral image is needed, with no requirements for training on previously labelled datasets or images. Furthermore, by applying a k-means clustering algorithm to the processed data, we demonstrate automatic, unsupervised image segmentation, yielding, without prior knowledge of the sample, intuitive chemical species maps, as shown here for a lithium ore sample.

Molecular Dynamics and Machine Learning in Catalysts

Given the importance of catalysts in the chemical industry, they have been extensively investigated by experimental and numerical methods. With the development of computational algorithms and computer hardware, large-scale simulations have enabled influential studies with more atomic details reflecting microscopic mechanisms. This review provides a comprehensive summary of recent developments in molecular dynamics, including ab initio molecular dynamics and reaction force-field molecular dynamics. Recent research on both approaches to catalyst calculations is reviewed, including growth, dehydrogenation, hydrogenation, oxidation reactions, bias, and recombination of carbon materials that can guide catalyst calculations. Machine learning has attracted increasing interest in recent years, and its combination with the field of catalysts has inspired promising development approaches. Its applications in machine learning potential, catalyst design, performance prediction, structure optimization, and classification have been summarized in detail. This review hopes to shed light and perspective on ML approaches in catalysts.

Super-resolution label-free volumetric vibrational imaging

Innovations in high-resolution optical imaging have allowed visualization of nanoscale biological structures and connections. However, super-resolution fluorescence techniques, including both optics-oriented and sample-expansion based, are limited in quantification and throughput especially in tissues from photobleaching or quenching of the fluorophores, and low-efficiency or non-uniform delivery of the probes. Here, we report a general sample-expansion vibrational imaging strategy, termed VISTA, for scalable label-free high-resolution interrogations of protein-rich biological structures with resolution down to 78 nm. VISTA achieves decent three-dimensional image quality through optimal retention of endogenous proteins, isotropic sample expansion, and deprivation of scattering lipids. Free from probe-labeling associated issues, VISTA offers unbiased and high-throughput tissue investigations. With correlative VISTA and immunofluorescence, we further validated the imaging specificity of VISTA and trained an image-segmentation model for label-free multi-component and volumetric prediction of nucleus, blood vessels, neuronal cells and dendrites in complex mouse brain tissues. VISTA could hence open new avenues for versatile biomedical studies.

Dissecting lipid droplet biology with coherent Raman scattering microscopy

Lipid droplets (LDs) are lipid-rich organelles universally found in most cells. They serve as a key energy reservoir, actively participate in signal transduction and dynamically communicate with other organelles. LD dysfunction has been associated with a variety of diseases. The content level, composition and mobility of LDs are crucial for their physiological and pathological functions, and these different parameters of LDs are subject to regulation by genetic factors and environmental inputs. Coherent Raman scattering (CRS) microscopy utilizes optical nonlinear processes to probe the intrinsic chemical bond vibration, offering label-free, quantitative imaging of lipids in vivo with high chemical specificity and spatiotemporal resolution. In this Review, we provide an overview over the principle of CRS microscopy and its application in tracking different parameters of LDs in live cells and organisms. We also discuss the use of CRS microscopy in genetic screens to discover lipid regulatory mechanisms and in understanding disease-related lipid pathology.

A versatile deep learning architecture for classification and label-free prediction of hyperspectral images

Hyperspectral imaging is a technique that provides rich chemical or compositional information not regularly available to traditional imaging modalities such as intensity imaging or color imaging based on the reflection, transmission, or emission of light. Analysis of hyperspectral imaging often relies on machine learning methods to extract information. Here, we present a new flexible architecture, the U-within-U-Net, that can perform classification, segmentation, and prediction of orthogonal imaging modalities on a variety of hyperspectral imaging techniques. Specifically, we demonstrate feature segmentation and classification on the Indian Pines hyperspectral dataset and simultaneous location prediction of multiple drugs in mass spectrometry imaging of rat liver tissue. We further demonstrate label-free fluorescence image prediction from hyperspectral stimulated Raman scattering microscopy images. The applicability of the U-within-U-Net architecture on diverse datasets with widely varying input and output dimensions and data sources suggest that it has great potential in advancing the use of hyperspectral imaging across many different application areas ranging from remote sensing, to medical imaging, to microscopy.

Deep Learning for Biospectroscopy and Biospectral Imaging: State-of-the-Art and Perspectives

With the advances in instrumentation and sampling techniques, there is an explosive growth of data from molecular and cellular samples. The call to extract more information from the large data sets has greatly challenged the conventional chemometrics method. Deep learning, which utilizes very large data sets for finding hidden features therein and for making accurate predictions for a wide range of applications, has been applied in an unbelievable pace in biospectroscopy and biospectral imaging in the recent 3 years. In this Feature, we first introduce the background and basic knowledge of deep learning. We then focus on the emerging applications of deep learning in the data preprocessing, feature detection, and modeling of the biological samples for spectral analysis and spectroscopic imaging. Finally, we highlight the challenges and limitations in deep learning and the outlook for future directions.