Stain-free histopathology by programmable supercontinuum pulses

Toward cancer detection by label-free microscopic imaging in oncological surgery: Techniques, instrumentation and applications

This review examines the clinical application of label-free microscopy and spectroscopy, which are based on optical signals emitted by tissue components. Over the past three decades, a variety of techniques have been investigated with the aim of developing an in situ histopathology method that can rapidly and accurately identify tumor margins during surgical procedures. These techniques can be divided into two groups. One group encompasses techniques exploiting linear optical signals, and includes infrared and Raman microspectroscopy, and autofluorescence microscopy. The second group includes techniques based on nonlinear optical signals, including harmonic generation, coherent Raman scattering, and multiphoton autofluorescence microscopy. Some of these methods provide comparable information, while others are complementary. However, all of them have distinct advantages and disadvantages due to their inherent nature. The first part of the review provides an explanation of the underlying physics of the excitation mechanisms and a description of the instrumentation. It also covers endomicroscopy and data analysis, which are important for understanding the current limitations in implementing label-free techniques in clinical settings. The second part of the review describes the application of label-free microscopy imaging to improve oncological surgery with focus on brain tumors and selected gastrointestinal cancers, and provides a critical assessment of the current state of translation of these methods into clinical practice. Finally, the potential of confocal laser endomicroscopy for the acquisition of autofluorescence is discussed in the context of immediate clinical applications.

Accelerating biopharmaceutical cell line selection with label-free multimodal nonlinear optical microscopy and machine learning

The selection of high-performing cell lines is crucial for biopharmaceutical production but is often time-consuming and labor-intensive. We investigated label-free multimodal nonlinear optical microscopy for non-perturbative profiling of biopharmaceutical cell lines based on their intrinsic molecular contrast. Employing simultaneous label-free autofluorescence multiharmonic (SLAM) microscopy with fluorescence lifetime imaging microscopy (FLIM), we characterized Chinese hamster ovary (CHO) cell lines at early passages (0-2). A machine learning (ML)-assisted analysis pipeline leveraged high-dimensional information to classify single cells into their respective lines. Remarkably, the monoclonal cell line classifiers achieved balanced accuracies exceeding 96.8% as early as passage 2. Correlation features and FLIM modality played pivotal roles in early classification. This integrated optical bioimaging and machine learning approach presents a promising solution to expedite cell line selection process while ensuring identification of high-performing biopharmaceutical cell lines. The techniques have potential for broader single-cell characterization applications in stem cell research, immunology, cancer biology and beyond.

Impact of stain variation and color normalization for prognostic predictions in pathology

In recent years, deep neural networks (DNNs) have demonstrated remarkable performance in pathology applications, potentially even outperforming expert pathologists due to their ability to learn subtle features from large datasets. One complication in preparing digital pathology datasets for DNN tasks is the variation in tinctorial qualities. A common way to address this is to perform stain normalization on the images. In this study, we show that a well-trained DNN model trained on one batch of histological slides failed to generalize to another batch prepared at a different time from the same tissue blocks, even when stain normalization methods were applied. This study used sample data from a previously reported DNN that was able to identify patients with early-stage non-small cell lung cancer (NSCLC) whose tumors did and did not metastasize, with high accuracy, based on training and then testing of digital images from H&E stained primary tumor tissue sections processed at the same time. In this study, we obtained a new series of histologic slides from the adjacent recuts of the same tissue blocks processed in the same lab but at a different time. We found that the DNN trained on either batch of slides/images was unable to generalize and failed to predict progression in the other batch of slides/images (AUCcross-batch = 0.52 - 0.53 compared to AUCsame-batch = 0.74 - 0.81). The failure to generalize did not improve even when the tinctorial difference corrections were made through either traditional color-tuning or stain normalization with the help of a Cycle Generative Adversarial Network (CycleGAN) process. This highlights the need to develop an entirely new way to process and collect consistent microscopy images from histologic slides that can be used to both train and allow for the general application of predictive DNN algorithms.

INSPIRE: Single-beam probed complementary vibrational bioimaging

Molecular spectroscopy provides intrinsic contrast for in situ chemical imaging, linking the physiochemical properties of biomolecules to the functions of living systems. While stimulated Raman imaging has found successes in deciphering biological machinery, many vibrational modes are Raman inactive or weak, limiting the broader impact of the technique. It can potentially be mitigated by the spectral complementarity from infrared (IR) spectroscopy. However, the vastly different optical windows make it challenging to develop such a platform. Here, we introduce in situ pump-probe IR and Raman excitation (INSPIRE) microscopy, a nascent cross-modality spectroscopic imaging approach by encoding the ultrafast Raman and the IR photothermal relaxation into a single probe beam for simultaneous detection. INSPIRE inherits the merits of complementary modalities and demonstrates high-content molecular imaging of chemicals, cells, tissues, and organisms. Furthermore, INSPIRE applies to label-free and molecular tag imaging, offering possibilities for optical sensing and imaging in biomedicine and materials science.

Clinical-Grade Validation of an Autofluorescence Virtual Staining System With Human Experts and a Deep Learning System for Prostate Cancer

The tissue diagnosis of adenocarcinoma and intraductal carcinoma of the prostate includes Gleason grading of tumor morphology on the hematoxylin and eosin stain and immunohistochemistry markers on the prostatic intraepithelial neoplasia-4 stain (CK5/6, P63, and AMACR). In this work, we create an automated system for producing both virtual hematoxylin and eosin and prostatic intraepithelial neoplasia-4 immunohistochemistry stains from unstained prostate tissue using a high-throughput hyperspectral fluorescence microscope and artificial intelligence and machine learning. We demonstrate that the virtual stainer models can produce high-quality images suitable for diagnosis by genitourinary pathologists. Specifically, we validate our system through extensive human review and computational analysis, using a previously validated Gleason scoring model, and an expert panel, on a large data set of test slides. This study extends our previous work on virtual staining from autofluorescence, demonstrates the clinical utility of this technology for prostate cancer, and exemplifies a rigorous standard of qualitative and quantitative evaluation for digital pathology.

Fast Multiphoton Microscopic Imaging Joint Image Super-Resolution for Automated Gleason Grading of Prostate Cancers

Gleason grading system is dependable for quantifying prostate cancer. This paper introduces a fast multiphoton microscopic imaging method via deep learning for automatic Gleason grading. Due to the contradiction between multiphoton microscopy (MPM) imaging speed and quality, a deep learning architecture (SwinIR) is used for image super-resolution to address this issue. The quality of low-resolution image is improved, which increased the acquisition speed from 7.55 s per frame to 0.24 s per frame. A classification network (Swin Transformer) was introduced for automated Gleason grading. The classification accuracy and Macro-F1 achieved by training on high-resolution images are respectively 90.9% and 90.9%. For training on super-resolution images, the classification accuracy and Macro-F1 are respectively 89.9% and 89.9%. It shows that super-resolution image can provide a comparable performance to high-resolution image. Our results suggested that MPM joint image super-resolution and automatic classification methods hold the potential to be a real-time clinical diagnostic tool for prostate cancer diagnosis.

Pixelation with concentration-encoded effective photons for quantitative molecular optical sectioning microscopy

Irreproducibility in molecular optical sectioning microscopy has hindered the transformation of acquired digital images from qualitative descriptions to quantitative data. Although numerous tools, metrics, and phantoms have been developed, accurate quantitative comparisons of data from different microscopy systems with diverse acquisition conditions remains a challenge. Here, we develop a simple tool based on an absolute measurement of bulk fluorophore solutions with related Poisson photon statistics, to overcome this obstacle. Demonstrated in a prototypical multiphoton microscope, our tool unifies the unit of pixelated measurement to enable objective comparison of imaging performance across different modalities, microscopes, components/settings, and molecular targets. The application of this tool in live specimens identifies an attractive methodology for quantitative imaging, which rapidly acquires low signal-to-noise frames with either gentle illumination or low-concentration fluorescence labeling.

Towards next-generation diagnostic pathology: AI-empowered label-free multiphoton microscopy

Diagnostic pathology, historically dependent on visual scrutiny by experts, is essential for disease detection. Advances in digital pathology and developments in computer vision technology have led to the application of artificial intelligence (AI) in this field. Despite these advancements, the variability in pathologists' subjective interpretations of diagnostic criteria can lead to inconsistent outcomes. To meet the need for precision in cancer therapies, there is an increasing demand for accurate pathological diagnoses. Consequently, traditional diagnostic pathology is evolving towards "next-generation diagnostic pathology", prioritizing on the development of a multi-dimensional, intelligent diagnostic approach. Using nonlinear optical effects arising from the interaction of light with biological tissues, multiphoton microscopy (MPM) enables high-resolution label-free imaging of multiple intrinsic components across various human pathological tissues. AI-empowered MPM further improves the accuracy and efficiency of diagnosis, holding promise for providing auxiliary pathology diagnostic methods based on multiphoton diagnostic criteria. In this review, we systematically outline the applications of MPM in pathological diagnosis across various human diseases, and summarize common multiphoton diagnostic features. Moreover, we examine the significant role of AI in enhancing multiphoton pathological diagnosis, including aspects such as image preprocessing, refined differential diagnosis, and the prognostication of outcomes. We also discuss the challenges and perspectives faced by the integration of MPM and AI, encompassing equipment, datasets, analytical models, and integration into the existing clinical pathways. Finally, the review explores the synergy between AI and label-free MPM to forge novel diagnostic frameworks, aiming to accelerate the adoption and implementation of intelligent multiphoton pathology systems in clinical settings.

Impact of Stain Variation and Color Normalization for Prognostic Predictions in Pathology

In recent years, deep neural networks (DNNs) have demonstrated remarkable performance in pathology applications, potentially even outperforming expert pathologists due to their ability to learn subtle features from large datasets. One complication in preparing digital pathology datasets for DNN tasks is variation in tinctorial qualities. A common way to address this is to perform stain normalization on the images. In this study, we show that a well-trained DNN model trained on one batch of histological slides failed to generalize to another batch prepared at a different time from the same tissue blocks, even when stain normalization methods were applied. This study used sample data from a previously reported DNN that was able to identify patients with early stage non-small cell lung cancer (NSCLC) whose tumors did and did not metastasize, with high accuracy, based on training and then testing of digital images from H&E stained primary tumor tissue sections processed at the same time. In this study we obtained a new series of histologic slides from the adjacent recuts of same tissue blocks processed in the same lab but at a different time. We found that the DNN trained on the either batch of slides/images was unable to generalize and failed to predict progression in the other batch of slides/images (AUCcross-batch = 0.52 - 0.53 compared to AUCsame-batch = 0.74 - 0.81). The failure to generalize did not improve even when the tinctorial difference correction were made through either traditional color-tuning or stain normalization with the help of a Cycle Generative Adversarial Network (CycleGAN) process. This highlights the need to develop an entirely new way to process and collect consistent microscopy images from histologic slides that can be used to both train and allow for the general application of predictive DNN algorithms.

Predicting DNA damage response in non-small cell lung cancer organoids via simultaneous label-free autofluorescence multiharmonic microscopy

The DNA damage response (DDR) is a fundamental readout for evaluating efficacy of cancer therapeutics, many of which target DNA associated processes. Current techniques to evaluate DDR rely on immunostaining for phosphorylated histone H2AX (γH2AX), which is an indicator of DNA double-strand breaks. While γH2AX immunostaining can provide a snapshot of DDR in fixed cell and tissue samples, this method is technically cumbersome due to temporal monitoring of DDR requiring timepoint replicates, extensive assay development efforts for 3D cell culture samples such as organoids, and time-consuming protocols for γH2AX immunostaining and its evaluation. The goal of this current study is to reduce overall burden on assay duration and development in non-small cell lung cancer (NSCLC) organoids by leveraging label-free multiphoton imaging. In this study, simultaneous label-free autofluorescence multiharmonic (SLAM) microscopy was used to provide rich intracellular information based on endogenous contrasts. SLAM microscopy enables imaging of live samples eliminating the need to generate sacrificial sample replicates and has improved image acquisition in 3D space over conventional confocal microscopy. Predictive modeling between label-free SLAM microscopy and γH2AX immunostained images confirmed strong correlation between SLAM image features and γH2AX signal. Across multiple DNA targeting chemotherapeutics and multiple patient-derived NSCLC organoid lines, the optical redox ratio and third harmonic generation channels were used to robustly predict DDR. Imaging via SLAM microscopy can be used to more rapidly predict DDR in live 3D NSCLC organoids with minimal sample handling and without labeling.

Large field-of-view metabolic profiling of murine brain tissue following morphine incubation using label-free multiphoton microscopy

BACKGROUND

Although the effects on neural activation and glucose consumption caused by opiates such as morphine are known, the metabolic machinery underlying opioid use and misuse is not fully explored. Multiphoton microscopy (MPM) techniques have been developed for optical imaging at high spatial resolution. Despite the increased use of MPM for neural imaging, the use of intrinsic optical contrast has seen minimal use in neuroscience.

NEW METHOD

We present a label-free, multimodal microscopy technique for metabolic profiling of murine brain tissue following incubation with morphine sulfate (MSO4). We evaluate two- and three-photon excited autofluorescence, and second and third harmonic generation to determine meaningful intrinsic contrast mechanisms in brain tissue using simultaneous label-free, autofluorescence multi-harmonic (SLAM) microscopy.

RESULTS

Regional differences quantified in the cortex, caudate, and thalamus of the brain demonstrate region-specific changes to metabolic profiles measured from FAD intensity, along with brain-wide quantification. While the overall intensity of FAD signal significantly decreased after morphine incubation, this metabolic molecule accumulated near the nucleus accumbens.

COMPARISON WITH EXISTING METHODS

Histopathology requires tissue fixation and staining to determine cell type and morphology, lacking information about cellular metabolism. Tools such as fMRI or PET imaging have been widely used, but lack cellular resolution. SLAM microscopy obviates the need for tissue preparation, permitting immediate use and imaging of tissue with subcellular resolution in its native environment.

CONCLUSIONS

This study demonstrates the utility of SLAM microscopy for label-free investigations of neural metabolism, especially the intensity changes in FAD autofluorescence and structural morphology from third-harmonic generation.

Dispersion mismatch correction for evident chromatic anomaly in low coherence interferometry

The applications of ultrafast optics to biomedical microscopy have expanded rapidly in recent years, including interferometric techniques like optical coherence tomography and microscopy (OCT/OCM). The advances of ultra-high resolution OCT and the inclusion of OCT/OCM in multimodal systems combined with multiphoton microscopy have marked a transition from using pseudo-continuous broadband sources, such as superluminescent diodes, to ultrafast supercontinuum optical sources. We report anomalies in the dispersion profiles of low-coherence ultrafast pulses through long and non-identical arms of a Michelson interferometer that are well beyond group delay or third-order dispersions. This chromatic anomaly worsens the observed axial resolution and causes fringe artifacts in the reconstructed tomograms in OCT/OCM using traditional algorithms. We present DISpersion COmpensation Techniques for Evident Chromatic Anomalies (DISCOTECA) as a universal solution to address the problem of chromatic dispersion mismatch in interferometry, especially with ultrafast sources. First, we demonstrate the origin of these artifacts through the self-phase modulation of ultrafast pulses due to focusing elements in the beam path. Next, we present three solution paradigms for DISCOTECA: optical, optoelectronic, and computational, along with quantitative comparisons to traditional methods to highlight the improvements to the dynamic range and axial profile. We explain the piecewise reconstruction of the phase mismatch between the arms of the spectral-domain interferometer using a modified short-term Fourier transform algorithm inspired by spectroscopic OCT. Finally, we present a decision-making guide for evaluating the utility of DISCOTECA in interferometry and for the artifact-free reconstruction of OCT images using an ultrafast supercontinuum source for biomedical applications.

Inspiring a convergent engineering approach to measure and model the tissue microenvironment

Understanding the molecular and physical complexity of the tissue microenvironment (TiME) in the context of its spatiotemporal organization has remained an enduring challenge. Recent advances in engineering and data science are now promising the ability to study the structure, functions, and dynamics of the TiME in unprecedented detail; however, many advances still occur in silos that rarely integrate information to study the TiME in its full detail. This review provides an integrative overview of the engineering principles underlying chemical, optical, electrical, mechanical, and computational science to probe, sense, model, and fabricate the TiME. In individual sections, we first summarize the underlying principles, capabilities, and scope of emerging technologies, the breakthrough discoveries enabled by each technology and recent, promising innovations. We provide perspectives on the potential of these advances in answering critical questions about the TiME and its role in various disease and developmental processes. Finally, we present an integrative view that appreciates the major scientific and educational aspects in the study of the TiME.

Automated Whole Slide Imaging for Label-Free Histology Using Photon Absorption Remote Sensing Microscopy

OBJECTIVE

Pathologists rely on histochemical stains to impart contrast in thin translucent tissue samples, revealing tissue features necessary for identifying pathological conditions. However, the chemical labeling process is destructive and often irreversible or challenging to undo, imposing practical limits on the number of stains that can be applied to the same tissue section. Here we present an automated label-free whole slide scanner using a PARS microscope designed for imaging thin, transmissible samples.

METHODS

Peak SNR and in-focus acquisitions are achieved across entire tissue sections using the scattering signal from the PARS detection beam to measure the optimal focal plane. Whole slide images (WSI) are seamlessly stitched together using a custom contrast leveling algorithm. Identical tissue sections are subsequently H&E stained and brightfield imaged. The one-to-one WSIs from both modalities are visually and quantitatively compared.

RESULTS

PARS WSIs are presented at standard 40x magnification in malignant human breast and skin samples. We show correspondence of subcellular diagnostic details in both PARS and H&E WSIs and demonstrate virtual H&E staining of an entire PARS WSI. The one-to-one WSI from both modalities show quantitative similarity in nuclear features and structural information.

CONCLUSION

PARS WSIs are compatible with existing digital pathology tools, and samples remain suitable for histochemical, immunohistochemical, and other staining techniques.

SIGNIFICANCE

This work is a critical advance for integrating label-free optical methods into standard histopathology workflows.

Analysis of 3D pathology samples using weakly supervised AI

Human tissue, which is inherently three-dimensional (3D), is traditionally examined through standard-of-care histopathology as limited two-dimensional (2D) cross-sections that can insufficiently represent the tissue due to sampling bias. To holistically characterize histomorphology, 3D imaging modalities have been developed, but clinical translation is hampered by complex manual evaluation and lack of computational platforms to distill clinical insights from large, high-resolution datasets. We present TriPath, a deep-learning platform for processing tissue volumes and efficiently predicting clinical outcomes based on 3D morphological features. Recurrence risk-stratification models were trained on prostate cancer specimens imaged with open-top light-sheet microscopy or microcomputed tomography. By comprehensively capturing 3D morphologies, 3D volume-based prognostication achieves superior performance to traditional 2D slice-based approaches, including clinical/histopathological baselines from six certified genitourinary pathologists. Incorporating greater tissue volume improves prognostic performance and mitigates risk prediction variability from sampling bias, further emphasizing the value of capturing larger extents of heterogeneous morphology.

Dispersive wave generation in single higher-order modes of a large-core silica step-index fiber with pulse energies up to 12 nJ

The generation of light in a laser system is constrained by the gain medium, limiting the available wavelengths. We demonstrate in-fiber generation of ultrafast pulses between ∼550 and 800 nm via dispersive wave generation (DWG), in higher-order modes (HOMs). Using higher-order modes enables power scaling, due to their large effective area compared to the fundamental modes of single-mode fibers and dispersion engineering, even in simple step-index fibers. The process occurs in a single higher-order mode, which we excite using passive glass components (an axicon and two telescopes). The output pulses have energies up to 12 nJ at the biologically relevant wavelength of 705 nm.

Computer-Aided Multiphoton Microscopy Diagnosis of 5 Different Primary Architecture Subtypes of Meningiomas

Meningiomas rank among the most common intracranial tumors, and surgery stands as the primary treatment modality for meningiomas. The precise subtyping and diagnosis of meningiomas, both before and during surgery, play a pivotal role in enabling neurosurgeons choose the optimal surgical program. In this study, we utilized multiphoton microscopy (MPM) based on 2-photon excited fluorescence and second-harmonic generation to identify 5 common meningioma subtypes. The morphological features of these subtypes were depicted using the MPM multichannel mode. Additionally, we developed 2 distinct programs to quantify collagen content and blood vessel density. Furthermore, the lambda mode of the MPM characterized architectural and spectral features, from which 3 quantitative indicators were extracted. Moreover, we employed machine learning to differentiate meningioma subtypes automatically, achieving high classification accuracy. These findings demonstrate the potential of MPM as a noninvasive diagnostic tool for meningioma subtyping and diagnosis, offering improved accuracy and resolution compared with traditional methods.

An end-to-end workflow for nondestructive 3D pathology

Recent advances in 3D pathology offer the ability to image orders of magnitude more tissue than conventional pathology methods while also providing a volumetric context that is not achievable with 2D tissue sections, and all without requiring destructive tissue sectioning. Generating high-quality 3D pathology datasets on a consistent basis, however, is not trivial and requires careful attention to a series of details during tissue preparation, imaging and initial data processing, as well as iterative optimization of the entire process. Here, we provide an end-to-end procedure covering all aspects of a 3D pathology workflow (using light-sheet microscopy as an illustrative imaging platform) with sufficient detail to perform well-controlled preclinical and clinical studies. Although 3D pathology is compatible with diverse staining protocols and computationally generated color palettes for visual analysis, this protocol focuses on the use of a fluorescent analog of hematoxylin and eosin, which remains the most common stain used for gold-standard pathological reports. We present our guidelines for a broad range of end users (e.g., biologists, clinical researchers and engineers) in a simple format. The end-to-end workflow requires 3-6 d to complete, bearing in mind that data analysis may take longer.

Temporally optimized and spectrally shaped hyperspectral coherent anti-Stokes Raman scattering microscopy

Coherent anti-Stokes Raman scattering (CARS) microscopy offers label-free chemical contrasts based on molecular vibrations. Hyperspectral CARS (HS-CARS) microscopy enables comprehensive microscale chemical characterization of biological samples. Various HS-CARS methods have been developed with individual advantages and disadvantages. We present what we believe to be a new temporally optimized and spectrally shaped (TOSS) HS-CARS method to overcome the limitations of existing techniques by providing precise control of the spatial and temporal profiles of the excitation beams for efficient and accurate measurements. This method uniquely uses Fourier transform pulse shaping based on a two-dimensional spatial light modulator to control the phase and amplitude of the excitation beams. TOSS-HS-CARS achieves fast, stable, and flexible acquisition, minimizes photodamage, and is highly adaptable to a multimodal multiphoton imaging system.

Compact simultaneous label-free autofluorescence multi-harmonic microscopy for user-friendly photodamage-monitored imaging

Significance

Label-free nonlinear optical microscopy has become a powerful tool for biomedical research. However, the possible photodamage risk hinders further clinical applications.

Aim

To reduce these adverse effects, we constructed a new platform of simultaneous label-free autofluorescence multi-harmonic (SLAM) microscopy, featuring four-channel multimodal imaging, inline photodamage monitoring, and pulse repetition-rate tuning.

Approach

Using a large-core birefringent photonic crystal fiber for spectral broadening and a prism compressor for pulse pre-chirping, this system allows users to independently adjust pulse width, repetition rate, and energy, which is useful for optimizing imaging conditions towards no/minimal photodamage.

Results

It demonstrates label-free multichannel imaging at one excitation pulse per image pixel and thus paves the way for improving the imaging speed by a faster optical scanner with a low risk of nonlinear photodamage. Moreover, the system grants users the flexibility to autonomously fine-tune repetition rate, pulse width, and average power, free from interference, ensuring the discovery of optimal imaging conditions with high SNR and minimal phototoxicity across various applications.

Conclusions

The combination of a stable laser source, independently tunable ultrashort pulse, photodamage monitoring features, and a compact design makes this new system a robust, powerful, and user-friendly imaging platform.

Noninvasive Nonlinear Optical Computational Histology

Cancer remains a global health challenge, demanding early detection and accurate diagnosis for improved patient outcomes. An intelligent paradigm is introduced that elevates label-free nonlinear optical imaging with contrastive patch-wise learning, yielding stain-free nonlinear optical computational histology (NOCH). NOCH enables swift, precise diagnostic analysis of fresh tissues, reducing patient anxiety and healthcare costs. Nonlinear modalities are evaluated, including stimulated Raman scattering and multiphoton imaging, for their ability to enhance tumor microenvironment sensitivity, pathological analysis, and cancer examination. Quantitative analysis confirmed that NOCH images accurately reproduce nuclear morphometric features across different cancer stages. Key diagnostic features, such as nuclear morphology, size, and nuclear-cytoplasmic contrast, are well preserved. NOCH models also demonstrate promising generalization when applied to other pathological tissues. The study unites label-free nonlinear optical imaging with histopathology using contrastive learning to establish stain-free computational histology. NOCH provides a rapid, non-invasive, and precise approach to surgical pathology, holding immense potential for revolutionizing cancer diagnosis and surgical interventions.

Virtual histological staining of unlabeled autopsy tissue

Traditional histochemical staining of post-mortem samples often confronts inferior staining quality due to autolysis caused by delayed fixation of cadaver tissue, and such chemical staining procedures covering large tissue areas demand substantial labor, cost and time. Here, we demonstrate virtual staining of autopsy tissue using a trained neural network to rapidly transform autofluorescence images of label-free autopsy tissue sections into brightfield equivalent images, matching hematoxylin and eosin (H&E) stained versions of the same samples. The trained model can effectively accentuate nuclear, cytoplasmic and extracellular features in new autopsy tissue samples that experienced severe autolysis, such as COVID-19 samples never seen before, where the traditional histochemical staining fails to provide consistent staining quality. This virtual autopsy staining technique provides a rapid and resource-efficient solution to generate artifact-free H&E stains despite severe autolysis and cell death, also reducing labor, cost and infrastructure requirements associated with the standard histochemical staining.

Supercontinuum intrinsic fluorescence imaging heralds free view of living systems

Optimal imaging strategies remain underdeveloped to maximize information for fluorescence microscopy while minimizing the harm to fragile living systems. Taking hint from the supercontinuum generation in ultrafast laser physics, we generated supercontinuum fluorescence from untreated unlabeled live samples before nonlinear photodamage onset. Our imaging achieved high-content cell phenotyping and tissue histology, identified bovine embryo polarization, quantified aging-related stress across cell types and species, demystified embryogenesis before and after implantation, sensed drug cytotoxicity in real-time, scanned brain area for targeted patching, optimized machine learning to track small moving organisms, induced two-photon phototropism of leaf chloroplasts under two-photon photosynthesis, unraveled microscopic origin of autumn colors, and interrogated intestinal microbiome. The results enable a facility-type microscope to freely explore vital molecular biology across life sciences.

Simultaneous label-free autofluorescence multi-harmonic microscopy driven by the supercontinuum generated from a bulk nonlinear crystal

Nonlinear microscopy encompasses several imaging techniques that leverage laser technology to probe intrinsic molecules of biological specimens. These native molecules produce optical fingerprints that allow nonlinear microscopes to reveal the chemical composition and structure of cells and tissues in a label-free and non-destructive fashion, information that enables a plethora of applications, e.g., real-time digital histopathology or image-guided surgery. Because state-of-the-art lasers exhibit either a limited bandwidth or reduced wavelength tunability, nonlinear microscopes lack the spectral support to probe different biomolecules simultaneously, thus losing analytical potential. Therefore, a conventional nonlinear microscope requires multiple or tunable lasers to individually excite endogenous molecules, increasing both the cost and complexity of the system. A solution to this problem is supercontinuum generation, a nonlinear optical phenomenon that supplies broadband femtosecond radiation, granting a wide spectrum for concurrent molecular excitation. This study introduces a source for nonlinear multiphoton microscopy based on the supercontinuum generation from a yttrium aluminum garnet (YAG) crystal, an approach that allows simultaneous label-free autofluorescence multi-harmonic imaging of biological samples and offers a practical and compact alternative for the clinical translation of nonlinear microscopy. While this supercontinuum covered the visible spectrum (550-900 nm) and the near-infrared region (950-1200 nm), the pulses within 1030-1150 nm produced label-free volumetric chemical images of ex vivo chinchilla kidney, thus validating the supercontinuum from bulk crystals as a powerful source for multimodal nonlinear microscopy.

An End-to-End Platform for Digital Pathology Using Hyperspectral Autofluorescence Microscopy and Deep Learning-Based Virtual Histology

Conventional histopathology involves expensive and labor-intensive processes that often consume tissue samples, rendering them unavailable for other analyses. We present a novel end-to-end workflow for pathology powered by hyperspectral microscopy and deep learning. First, we developed a custom hyperspectral microscope to nondestructively image the autofluorescence of unstained tissue sections. We then trained a deep learning model to use autofluorescence to generate virtual histologic stains, which avoids the cost and variability of chemical staining procedures and conserves tissue samples. We showed that the virtual images reproduce the histologic features present in the real-stained images using a randomized nonalcoholic steatohepatitis (NASH) scoring comparison study, where both real and virtual stains are scored by pathologists (D.T., A.D.B., R.K.P.). The test showed moderate-to-good concordance between pathologists' scoring on corresponding real and virtual stains. Finally, we developed deep learning-based models for automated NASH Clinical Research Network score prediction. We showed that the end-to-end automated pathology platform is comparable with an independent panel of pathologists for NASH Clinical Research Network scoring when evaluated against the expert pathologist consensus scores. This study provides proof of concept for this virtual staining strategy, which could improve cost, efficiency, and reliability in pathology and enable novel approaches to spatial biology research.

Label- and slide-free tissue histology using 3D epi-mode quantitative phase imaging and virtual hematoxylin and eosin staining

Histological staining of tissue biopsies, especially hematoxylin and eosin (H&E) staining, serves as the benchmark for disease diagnosis and comprehensive clinical assessment of tissue. However, the typical formalin-fixation, paraffin-embedding (FFPE) process is laborious and time consuming, often limiting its usage in time-sensitive applications such as surgical margin assessment. To address these challenges, we combine an emerging 3D quantitative phase imaging technology, termed quantitative oblique back illumination microscopy (qOBM), with an unsupervised generative adversarial network pipeline to map qOBM phase images of unaltered thick tissues (i.e., label- and slide-free) to virtually stained H&E-like (vH&E) images. We demonstrate that the approach achieves high-fidelity conversions to H&E with subcellular detail using fresh tissue specimens from mouse liver, rat gliosarcoma, and human gliomas. We also show that the framework directly enables additional capabilities such as H&E-like contrast for volumetric imaging. The quality and fidelity of the vH&E images are validated using both a neural network classifier trained on real H&E images and tested on virtual H&E images, and a user study with neuropathologists. Given its simple and low-cost embodiment and ability to provide real-time feedback in vivo, this deep-learning-enabled qOBM approach could enable new workflows for histopathology with the potential to significantly save time, labor, and costs in cancer screening, detection, treatment guidance, and more.

Visualizing vasculature and its response to therapy in the tumor microenvironment

Tumor vasculature plays a critical role in the progression and metastasis of tumors, antitumor immunity, drug delivery, and resistance to therapies. The morphological and functional changes of tumor vasculature in response to therapy take place in a spatiotemporal-dependent manner, which can be predictive of treatment outcomes. Dynamic monitoring of intratumor vasculature contributes to an improved understanding of the mechanisms of action of specific therapies or reasons for treatment failure, leading to therapy optimization. There is a rich history of methods used to image the vasculature. This review describes recent advances in imaging technologies to visualize the tumor vasculature, with a focus on enhanced intravital imaging techniques and tumor window models. We summarize new insights on spatial-temporal vascular responses to various therapies, including changes in vascular perfusion and permeability and immune-vascular crosstalk, obtained from intravital imaging. Finally, we briefly discuss the clinical applications of intravital imaging techniques.

An end-to-end workflow for non-destructive 3D pathology

Recent advances in 3D pathology offer the ability to image orders-of-magnitude more tissue than conventional pathology while providing a volumetric context that is lacking with 2D tissue sections, all without requiring destructive tissue sectioning. Generating high-quality 3D pathology datasets on a consistent basis is non-trivial, requiring careful attention to many details regarding tissue preparation, imaging, and data/image processing in an iterative process. Here we provide an end-to-end protocol covering all aspects of a 3D pathology workflow (using light-sheet microscopy as an illustrative imaging platform) with sufficient detail to perform well-controlled preclinical and clinical studies. While 3D pathology is compatible with diverse staining protocols and computationally generated color palettes for visual analysis, this protocol will focus on a fluorescent analog of hematoxylin and eosin (H&E), which remains the most common stain for gold-standard diagnostic determinations. We present our guidelines for a broad range of end-users (e.g., biologists, clinical researchers, and engineers) in a simple tutorial format.

Weakly Supervised AI for Efficient Analysis of 3D Pathology Samples

Human tissue consists of complex structures that display a diversity of morphologies, forming a tissue microenvironment that is, by nature, three-dimensional (3D). However, the current standard-of-care involves slicing 3D tissue specimens into two-dimensional (2D) sections and selecting a few for microscopic evaluation1,2, with concomitant risks of sampling bias and misdiagnosis3-6. To this end, there have been intense efforts to capture 3D tissue morphology and transition to 3D pathology, with the development of multiple high-resolution 3D imaging modalities7-18. However, these tools have had little translation to clinical practice as manual evaluation of such large data by pathologists is impractical and there is a lack of computational platforms that can efficiently process the 3D images and provide patient-level clinical insights. Here we present Modality-Agnostic Multiple instance learning for volumetric Block Analysis (MAMBA), a deep-learning-based platform for processing 3D tissue images from diverse imaging modalities and predicting patient outcomes. Archived prostate cancer specimens were imaged with open-top light-sheet microscopy12-14 or microcomputed tomography15,16 and the resulting 3D datasets were used to train risk-stratification networks based on 5-year biochemical recurrence outcomes via MAMBA. With the 3D block-based approach, MAMBA achieves an area under the receiver operating characteristic curve (AUC) of 0.86 and 0.74, superior to 2D traditional single-slice-based prognostication (AUC of 0.79 and 0.57), suggesting superior prognostication with 3D morphological features. Further analyses reveal that the incorporation of greater tissue volume improves prognostic performance and mitigates risk prediction variability from sampling bias, suggesting that there is value in capturing larger extents of spatially heterogeneous 3D morphology. With the rapid growth and adoption of 3D spatial biology and pathology techniques by researchers and clinicians, MAMBA provides a general and efficient framework for 3D weakly supervised learning for clinical decision support and can help to reveal novel 3D morphological biomarkers for prognosis and therapeutic response.

Nonlinear Imaging Histopathology: A Pipeline to Correlate Gold-Standard Hematoxylin and Eosin Staining With Modern Nonlinear Microscopy

Hematoxylin and eosin (H&E) staining, the century-old technique, has been the gold standard tool for pathologists to detect anomalies in tissues and diseases such as cancer. H&E staining is a cumbersome, time-consuming process that delays and wastes precious minutes during an intraoperative diagnosis. However, even in the modern era, real-time label-free imaging techniques such as simultaneous label-free autofluorescence multiharmonic (SLAM) microscopy have delivered several more layers of information to characterize a tissue with high precision. Still, they have yet to translate to the clinic. The slow translation rate can be attributed to the lack of direct comparisons between the old and new techniques. Our approach to solving this problem is to: 1) reduce dimensions by pre-sectioning the tissue in 500 μm slices, and 2) produce fiducial laser markings which appear in both SLAM and histological imaging. High peak-power femtosecond laser pulses enable ablation in a controlled and contained manner. We perform laser marking on a grid of points encompassing the SLAM region of interest. We optimize laser power, numerical aperture, and timing to produce axially extended marking, hence multilayered fiducial markers, with minimal damage to the surrounding tissues. We performed this co-registration over an area of 3 × 3 mm2 of freshly excised mouse kidney and intestine, followed by standard H&E staining. Reduced dimensionality and the use of laser markings provided a comparison of the old and new techniques, giving a wealth of correlative information and elevating the potential of translating nonlinear microscopy to the clinic for rapid pathological assessment.

Nondestructive 3D Pathology with Light-Sheet Fluorescence Microscopy for Translational Research and Clinical Assays

In recent years, there has been a revived appreciation for the importance of spatial context and morphological phenotypes for both understanding disease progression and guiding treatment decisions. Compared with conventional 2D histopathology, which is the current gold standard of medical diagnostics, nondestructive 3D pathology offers researchers and clinicians the ability to visualize orders of magnitude more tissue within their natural volumetric context. This has been enabled by rapid advances in tissue-preparation methods, high-throughput 3D microscopy instrumentation, and computational tools for processing these massive feature-rich data sets. Here, we provide a brief overview of many of these technical advances along with remaining challenges to be overcome. We also speculate on the future of 3D pathology as applied in translational investigations, preclinical drug development, and clinical decision-support assays.

Label-Free Optical Metabolic Imaging in Cells and Tissues

Over the last half century, the autofluorescence of the metabolic cofactors NADH (reduced nicotinamide adenine dinucleotide) and FAD (flavin adenine dinucleotide) has been quantified in a variety of cell types and disease states. With the spread of nonlinear optical microscopy techniques in biomedical research, NADH and FAD imaging has offered an attractive solution to noninvasively monitor cell and tissue status and elucidate dynamic changes in cell or tissue metabolism. Various tools and methods to measure the temporal, spectral, and spatial properties of NADH and FAD autofluorescence have been developed. Specifically, an optical redox ratio of cofactor fluorescence intensities and NADH fluorescence lifetime parameters have been used in numerous applications, but significant work remains to mature this technology for understanding dynamic changes in metabolism. This article describes the current understanding of our optical sensitivity to different metabolic pathways and highlights current challenges in the field. Recent progress in addressing these challenges and acquiring more quantitative information in faster and more metabolically relevant formats is also discussed.

CoADS: Cross attention based dual-space graph network for survival prediction of lung cancer using whole slide images

BACKGROUND AND OBJECTIVE

Accurate overall survival (OS) prediction for lung cancer patients is of great significance, which can help classify patients into different risk groups to benefit from personalized treatment. Histopathology slides are considered the gold standard for cancer diagnosis and prognosis, and many algorithms have been proposed to predict the OS risk. Most methods rely on selecting key patches or morphological phenotypes from whole slide images (WSIs). However, OS prediction using the existing methods exhibits limited accuracy and remains challenging.

METHODS

In this paper, we propose a novel cross-attention-based dual-space graph convolutional neural network model (CoADS). To facilitate the improvement of survival prediction, we fully take into account the heterogeneity of tumor sections from different perspectives. CoADS utilizes the information from both physical and latent spaces. With the guidance of cross-attention, both the spatial proximity in physical space and the feature similarity in latent space between different patches from WSIs are integrated effectively.

RESULTS

We evaluated our approach on two large lung cancer datasets of 1044 patients. The extensive experimental results demonstrated that the proposed model outperforms state-of-the-art methods with the highest concordance index.

CONCLUSIONS

The qualitative and quantitative results show that the proposed method is more powerful for identifying the pathology features associated with prognosis. Furthermore, the proposed framework can be extended to other pathological images for predicting OS or other prognosis indicators, and thus delivering individualized treatment.

Weakly supervised identification of microscopic human breast cancer-related optical signatures from normal-appearing breast tissue

With the latest advancements in optical bioimaging, rich structural and functional information has been generated from biological samples, which calls for capable computational tools to identify patterns and uncover relationships between optical characteristics and various biomedical conditions. Constrained by the existing knowledge of the novel signals obtained by those bioimaging techniques, precise and accurate ground truth annotations can be difficult to obtain. Here we present a weakly supervised deep learning framework for optical signature discovery based on inexact and incomplete supervision. The framework consists of a multiple instance learning-based classifier for the identification of regions of interest in coarsely labeled images and model interpretation techniques for optical signature discovery. We applied this framework to investigate human breast cancer-related optical signatures based on virtual histopathology enabled by simultaneous label-free autofluorescence multiharmonic microscopy (SLAM), with the goal of exploring unconventional cancer-related optical signatures from normal-appearing breast tissues. The framework has achieved an average area under the curve (AUC) of 0.975 on the cancer diagnosis task. In addition to well-known cancer biomarkers, non-obvious cancer-related patterns were revealed by the framework, including NAD(P)H-rich extracellular vesicles observed in normal-appearing breast cancer tissue, which facilitate new insights into the tumor microenvironment and field cancerization. This framework can be further extended to diverse imaging modalities and optical signature discovery tasks.

Benefits of cascaded nonlinear dynamics in hybrid fibers for low-noise supercontinuum generation

The recent development of fiber supercontinuum (SC) sources with ultra-low noise levels has been instrumental in advancing the state-of-the-art in a wide range of research topics. However, simultaneously satisfying the application demands of maximizing spectral bandwidth and minimizing noise is a major challenge that so far has been addressed with compromise, found by fine-tuning the characteristics of a single nonlinear fiber transforming the injected laser pulses into a broadband SC. In this work, we investigate a hybrid approach that splits the nonlinear dynamics into two discrete fibers optimized for nonlinear temporal compression and spectral broadening, respectively. This introduces new design degrees of freedom, making it possible to select the best fiber for each stage of the SC generation process. With experiments and simulations we study the benefits of this hybrid approach for three common and commercially available highly nonlinear fiber (HNLF) designs, focusing on flatness, bandwidth and relative intensity noise of the generated SC. In our results, hybrid all-normal dispersion (ANDi) HNLF stand out as they combine the broad spectral bandwidths associated with soliton dynamics with extremely low noise and smooth spectra known from normal dispersion nonlinearities. Hybrid ANDi HNLF are a simple and low-cost route for implementing ultra-low noise SC sources and scaling their repetition rate for various applications such as biophotonic imaging, coherent optical communications, or ultrafast photonics.

Variability of fluorescence intensity distribution measured by flow cytometry is influenced by cell size and cell cycle progression

The distribution of fluorescence signals measured with flow cytometry can be influenced by several factors, including qualitative and quantitative properties of the used fluorochromes, optical properties of the detection system, as well as the variability within the analyzed cell population itself. Most of the single cell samples prepared from in vitrocultures or clinical specimens contain a variable cell cycle component. Cell cycle, together with changes in the cell size, are two of the factors that alter the functional properties of analyzed cells and thus affect the interpretation of obtained results. Here, we describe the association between cell cycle status and cell size, and the variability in the distribution of fluorescence intensity as determined with flow cytometry, at population scale. We show that variability in the distribution of background and specific fluorescence signals is related to the cell cycle state of the selected population, with the 10% low fluorescence signal fraction enriched mainly in cells in their G0/G1 cell cycle phase, and the 10% high fraction containing cells mostly in the G2/M phase. Therefore we advise using caution and additional experimental validation when comparing populations defined by fractions at both ends of fluorescence signal distribution to avoid biases caused by the effect of cell cycle and cell size.

Nonlinear optical imaging by detection with optical parametric amplification (invited paper)

Nonlinear optical imaging is a versatile tool that has been proven to be exceptionally useful in various research fields. However, due to the use of photomultiplier tubes (PMTs), the wide application of nonlinear optical imaging is limited by the incapability of imaging under ambient light. In this paper, we propose and demonstrate a new optical imaging detection method based on optical parametric amplification (OPA). As a nonlinear optical process, OPA intrinsically rejects ambient light photons by coherence gating. Periodical poled lithium niobate (PPLN) crystals are used in this study as the media for OPA. Compared to bulk nonlinear optical crystals, PPLN crystals support the generation of OPA signal with lower pump power. Therefore, this characteristic of PPLN crystals is particularly beneficial when using high-repetition-rate lasers, which facilitate high-speed optical signal detection, such as in spectroscopy and imaging. A PPLN-based OPA system was built to amplify the emitted imaging signal from second harmonic generation (SHG) and coherent anti-Stokes Raman scattering (CARS) microscopy imaging, and the amplified optical signal was strong enough to be detected by a biased photodiode under ordinary room light conditions. With OPA detection, ambient-light-on SHG and CARS imaging becomes possible, and achieves a similar result as PMT detection under strictly dark environments. These results demonstrate that OPA can be used as a substitute for PMTs in nonlinear optical imaging to adapt it to various applications with complex lighting conditions.

Consecutive 1015-1105-nm wavelength tunable "figure-of-9" mode-locked Yb:fiber oscillator

A widely wavelength tunable mode-locked Yb-doped fiber oscillator based on nonlinear amplifier loop mirror (NALM) is reported, in which only a piece of short (∼0.5 m) single-mode polarization-maintaining (PM) Yb-doped fiber is employed, instead of the frequently used long (a few meters) double cladding (DC) fiber in previous papers. Experimentally, the center wavelength can be consecutively tuned from 1015 to 1105 nm by tilting the silver mirror, corresponding to a tuning range of 90 nm. To the best of our knowledge, this is the broadest consecutive tuning range in Yb:fiber mode-locked fiber oscillator. In addition, the mechanism of wavelength tuning is tentatively analyzed and attributed to the combined action of the spatial dispersion induced by a tilting silver mirror and the limited aperture in the system. Specific to the wavelength of 1045 nm, the output pulses with 13-nm spectral bandwidth can be compressed to 154 fs.

Multidimensional quantitative characterization of the tumor microenvironment by multicontrast nonlinear microscopy

Characterization of the microenvironment features of tumors, such as its microstructures, biomolecular metabolism, and functional dynamics, may provide essential pathologic information about the tumor, tumor margin, and adjacent normal tissue for early and intraoperative diagnosis. However, it can be particularly challenging to obtain faithful and comprehensive pathological information simultaneously from unperturbed tissues due to the complexity of the microenvironment in organisms. Super-multiplex nonlinear optical imaging system emerged and matured as an attractive tool for acquisition and elucidation of the nonlinear properties correlated with tumor microenvironment. Here, we introduced a nonlinear effects-based multidimensional optical imaging platform and methodology to simultaneously and efficiently capture contrasting and complementary nonlinear optical signatures of freshly excised human skin tissues. The qualitative and quantitative analysis of autofluorescence (FAD), collagen fiber, and intracellular components (lipids and proteins) illustrated the differences about morphological changes and biomolecular metabolic processes of the epidermis and dermis in different skin carcinogenic types. Interpretation of multi-parameter stain-free histological findings complements conventional H&E-stained slides for investigating basal cell carcinoma and pigmented nevus, validates the platform's versatility and efficiency for classifying subtypes of skin carcinoma, and provides the potential to translate endogenous molecule into biomarker for assisting in rapid cancer screening and diagnosis.

Label-free analysis of inflammatory tissue remodeling in murine lung tissue based on multiphoton microscopy, Raman spectroscopy and machine learning

Inflammatory fibrotic tissue remodeling can lead to severe morbidity. Histopathology grading requires extraction of biopsies and elaborate tissue processing. Label-free optical technologies can provide diagnostic readout without preparation and artificial stainings and show potential for in vivo applications. Here, we present an integration of Raman spectroscopy (RS) and multiphoton microscopy for joint investigation of the bio-chemical composition and morphological features related to cellular components and connective tissue. Both modalities show that collagen signatures were significantly increased in a murine fibrosis model. Furthermore, autofluorescence signatures assigned to immune cells show high correlation with disease severity. RS indicates increased levels of elastin and lipids. Further, we investigated the effect of joint data sets on prediction performance in machine learning models. Although binary classification did not benefit from adding more features, multi-class classification was improved by integrated data sets.

Tracking the binding of multi-functional fluorescent tags for Alzheimer's disease using quantitative multiphoton microscopy

A recent theranostic approach to address Alzheimer's disease (AD) utilizes multifunctional targets that both tag and negate the toxicity of AD biomarkers. These compounds, which emit fluorescence with both an activation and a spectral shift in the presence of Aβ, were previously characterized with traditional fluorescence imaging for binary characterization. However, these multifunctional compounds have broad and dynamic emission spectra that are dependent on factors such as the local environment, presence of Aβ deposits, etc. Since quantitative multiphoton microscopy is sensitive to the binding dynamics of molecules, we characterized the performance of two such compounds, LS-4 and ZY-12-OMe, using Simultaneous Label-free Autofluorescence Multi-harmonic (SLAM) microscopy and Fast Optical Coherence, Autofluorescence Lifetime imaging and Second harmonic generation (FOCALS) microscopy. This study shows that the combination of quantitative multiphoton imaging with multifunctional tags for AD offers new insights into the interaction of these tags with AD biomarkers and the theranostic mechanisms.

Optical coherence tomography holds promise to transform the diagnostic anatomic pathology gross evaluation process

SIGNIFICANCE

Real-time histology can close a variety of gaps in tissue diagnostics. Currently, gross pathology analysis of excised tissue is dependent upon visual inspection and palpation to identify regions of interest for histopathological processing. Such analysis is limited by the variable correlation between macroscopic and microscopic findings. The current standard of care is costly, burdensome, and inefficient.

AIM

We are the first to address this gap by introducing optical coherence tomography (OCT) to be integrated in real-time during the pathology grossing process.

APPROACH

This is achieved by our high-resolution, ultrahigh-speed, large field-of-view OCT device designed for this clinical application.

RESULTS

We demonstrate the feasibility of imaging tissue sections from multiple human organs (breast, prostate, lung, and pancreas) in a clinical gross pathology setting without interrupting standard workflows.

CONCLUSIONS

OCT-based real-time histology evaluation holds promise for addressing a gap that has been present for >100 years.

Label-free multimodal nonlinear optical imaging of needle biopsy cores for intraoperative cancer diagnosis

SIGNIFICANCE

Needle biopsy (NB) procedures are important for the initial diagnosis of many types of cancer. However, the possibility of NB specimens being unable to provide diagnostic information, (i.e., non-diagnostic sampling) and the time-consuming histological evaluation process can cause delays in diagnoses that affect patient care.

AIM

We aim to demonstrate the advantages of this label-free multimodal nonlinear optical imaging (NLOI) technique as a non-destructive point-of-procedure evaluation method for NB tissue cores, for the visualization and characterization of the tissue microenvironment.

APPROACH

A portable, label-free, multimodal NLOI system combined second-harmonic generation (SHG) and third-harmonic generation and two- and three-photon autofluorescence (2PF, 3PF) microscopy. It was used for intraoperative imaging of fresh NB tissue cores acquired during canine cancer surgeries, which involved liver, lung, and mammary tumors as well as soft-tissue sarcoma; in total, eight canine patients were recruited. An added tissue culture chamber enabled the use of this NLOI system for longitudinal imaging of fresh NB tissue cores taken from an induced rat mammary tumor and healthy mouse livers.

RESULTS

The intraoperative NLOI system was used to assess fresh canine NB specimens during veterinary cancer surgeries. Histology-like morphological features were visualized by the combination of four NLOI modalities at the point-of-procedure. The NLOI results provided quantitative information on the tissue microenvironment such as the collagen fiber orientation using Fourier-domain SHG analysis and metabolic profiling by optical redox ratio (ORR) defined by 2PF/(2PF + 3PF). The analyses showed that the canine mammary tumor had more randomly oriented collagen fibers compared to the tumor margin, and hepatocarcinoma had a wider distribution of ORR with a lower mean value compared to the liver fibrosis and the normal-appearing liver. Moreover, the loss of metabolic information during tissue degradation of fresh murine NB specimens was shown by overall intensity decreases in all channels and an increase of mean ORR from 0.94 (standard deviation 0.099) to 0.97 (standard deviation 0.077) during 1-h longitudinal imaging of a rat mammary tumor NB specimen. The tissue response to staurosporine (STS), an apoptotic inducer, from fresh murine liver NB specimens was also observed. The mean ORR decreased from 0.86 to 0.74 in the first 40 min and then increased to 0.8 during the rest of the hour of imaging, compared to the imaging results without the addition of STS, which showed a continuous increase of ORR from 0.72 to 0.75.

CONCLUSIONS

A label-free, multimodal NLOI platform reveals microstructural and metabolic information of the fresh NB cores during intraoperative cancer imaging. This system has been demonstrated on animal models to show its potential to provide a more comprehensive histological assessment and a better understanding of the unperturbed tumor microenvironment. Considering tissue degradation, or loss of viability upon fixation, this intraoperative NLOI system has the advantage of immediate assessment of freshly excised tissue specimens at the point of procedure.

Rapid slide-free and non-destructive histological imaging using wide-field optical-sectioning microscopy

Histopathology based on formalin-fixed and paraffin-embedded tissues has long been the gold standard for surgical margin assessment (SMA). However, routine pathological practice is lengthy and laborious, failing to guide surgeons intraoperatively. In this report, we propose a practical and low-cost histological imaging method with wide-field optical-sectioning microscopy (i.e., High-and-Low-frequency (HiLo) microscopy). HiLo can achieve rapid and non-destructive imaging of freshly-excised tissues at an extremely high acquisition speed of 5 cm²/min with a spatial resolution of 1.3 µm (lateral) and 5.8 µm (axial), showing great potential as an SMA tool that can provide immediate feedback to surgeons and pathologists for intraoperative decision-making. We demonstrate that HiLo enables rapid extraction of diagnostic features for different subtypes of human lung adenocarcinoma and hepatocellular carcinoma, producing surface images of rough specimens with large field-of-views and cellular features that are comparable to the clinical standard. Our results show promising clinical translations of HiLo microscopy to improve the current standard of care.

Deep learning autofluorescence-harmonic microscopy

Laser scanning microscopy has inherent tradeoffs between imaging speed, field of view (FOV), and spatial resolution due to the limitations of sophisticated mechanical and optical setups, and deep learning networks have emerged to overcome these limitations without changing the system. Here, we demonstrate deep learning autofluorescence-harmonic microscopy (DLAM) based on self-alignment attention-guided residual-in-residual dense generative adversarial networks to close the gap between speed, FOV, and quality. Using the framework, we demonstrate label-free large-field multimodal imaging of clinicopathological tissues with enhanced spatial resolution and running time advantages. Statistical quality assessments show that the attention-guided residual dense connections minimize the persistent noise, distortions, and scanning fringes that degrade the autofluorescence-harmonic images and avoid reconstruction artifacts in the output images. With the advantages of high contrast, high fidelity, and high speed in image reconstruction, DLAM can act as a powerful tool for the noninvasive evaluation of diseases, neural activity, and embryogenesis.

Super-Multiplex Nonlinear Optical Imaging Unscrambles the Statistical Complexity of Cancer Subtypes and Tumor Microenvironment

Label-free nonlinear optical imaging (NLOI) has made tremendous inroads toward unscrambling the microcosmic complexity of cancers. However, harmonic and Raman microscopy offers throughput without redox information to reveal metabolic differentiation, and fluorescence lifetime microscopy lacks the vibrational response of molecules to visualize specific molecular constituents such as lipid. Here, a flexible, robust simultaneous multi-nonlinear imaging and cross-modality system that combines complementary imaging contrast mechanisms is demonstrated. This system, utilizing multiplexed ultrashort pulses, ingeniously integrates typical nonlinear processes, and high-dimension lifetime extension in a single setup to enhance the imaging dimensions and quality. Using this system, the authors perform label-free comprehensive evaluation of clinicopathological tissues of ovarian carcinoma due to its statistical complexity. The results show that the technology provides statistically rich, insightful information with high accuracy, sensitivity, and specificity, in contrast to standard histopathology, and can potentially be a powerful tool for fundamental cancer research and clinical applications.

High-Throughput, Label-Free and Slide-Free Histological Imaging by Computational Microscopy and Unsupervised Learning

Rapid and high-resolution histological imaging with minimal tissue preparation has long been a challenging and yet captivating medical pursuit. Here, the authors propose a promising and transformative histological imaging method, termed computational high-throughput autofluorescence microscopy by pattern illumination (CHAMP). With the assistance of computational microscopy, CHAMP enables high-throughput and label-free imaging of thick and unprocessed tissues with large surface irregularity at an acquisition speed of 10 mm2 /10 s with 1.1-µm lateral resolution. Moreover, the CHAMP image can be transformed into a virtually stained histological image (Deep-CHAMP) through unsupervised learning within 15 s, where significant cellular features are quantitatively extracted with high accuracy. The versatility of CHAMP is experimentally demonstrated using mouse brain/kidney and human lung tissues prepared with various clinical protocols, which enables a rapid and accurate intraoperative/postoperative pathological examination without tissue processing or staining, demonstrating its great potential as an assistive imaging platform for surgeons and pathologists to provide optimal adjuvant treatment.