High-speed light-sheet microscopy for the in-situ acquisition of volumetric histological images of living tissue

Multi-Instance Multi-Task Learning for Joint Clinical Outcome and Genomic Profile Predictions From the Histopathological Images

With the remarkable success of digital histopathology and the deep learning technology, many whole-slide pathological images (WSIs) based deep learning models are designed to help pathologists diagnose human cancers. Recently, rather than predicting categorical variables as in cancer diagnosis, several deep learning studies are also proposed to estimate the continuous variables such as the patients' survival or their transcriptional profile. However, most of the existing studies focus on conducting these predicting tasks separately, which overlooks the useful intrinsic correlation among them that can boost the prediction performance of each individual task. In addition, it is sill challenge to design the WSI-based deep learning models, since a WSI is with huge size but annotated with coarse label. In this study, we propose a general multi-instance multi-task learning framework (HistMIMT) for multi-purpose prediction from WSIs. Specifically, we firstly propose a novel multi-instance learning module (TMICS) considering both common and specific task information across different tasks to generate bag representation for each individual task. Then, a soft-mask based fusion module with channel attention (SFCA) is developed to leverage useful information from the related tasks to help improve the prediction performance on target task. We evaluate our method on three cancer cohorts derived from the Cancer Genome Atlas (TCGA). For each cohort, our multi-purpose prediction tasks range from cancer diagnosis, survival prediction and estimating the transcriptional profile of gene TP53. The experimental results demonstrated that HistMIMT can yield better outcome on all clinical prediction tasks than its competitors.

Analysis of 3D pathology samples using weakly supervised AI

Human tissue, which is inherently three-dimensional (3D), is traditionally examined through standard-of-care histopathology as limited two-dimensional (2D) cross-sections that can insufficiently represent the tissue due to sampling bias. To holistically characterize histomorphology, 3D imaging modalities have been developed, but clinical translation is hampered by complex manual evaluation and lack of computational platforms to distill clinical insights from large, high-resolution datasets. We present TriPath, a deep-learning platform for processing tissue volumes and efficiently predicting clinical outcomes based on 3D morphological features. Recurrence risk-stratification models were trained on prostate cancer specimens imaged with open-top light-sheet microscopy or microcomputed tomography. By comprehensively capturing 3D morphologies, 3D volume-based prognostication achieves superior performance to traditional 2D slice-based approaches, including clinical/histopathological baselines from six certified genitourinary pathologists. Incorporating greater tissue volume improves prognostic performance and mitigates risk prediction variability from sampling bias, further emphasizing the value of capturing larger extents of heterogeneous morphology.

Open-top Bessel beam two-photon light sheet microscopy for three-dimensional pathology

Nondestructive pathology based on three-dimensional (3D) optical microscopy holds promise as a complement to traditional destructive hematoxylin and eosin (H&E) stained slide-based pathology by providing cellular information in high throughput manner. However, conventional techniques provided superficial information only due to shallow imaging depths. Herein, we developed open-top two-photon light sheet microscopy (OT-TP-LSM) for intraoperative 3D pathology. An extended depth of field two-photon excitation light sheet was generated by scanning a nondiffractive Bessel beam, and selective planar imaging was conducted with cameras at 400 frames/s max during the lateral translation of tissue specimens. Intrinsic second harmonic generation was collected for additional extracellular matrix (ECM) visualization. OT-TP-LSM was tested in various human cancer specimens including skin, pancreas, and prostate. High imaging depths were achieved owing to long excitation wavelengths and long wavelength fluorophores. 3D visualization of both cells and ECM enhanced the ability of cancer detection. Furthermore, an unsupervised deep learning network was employed for the style transfer of OT-TP-LSM images to virtual H&E images. The virtual H&E images exhibited comparable histological characteristics to real ones. OT-TP-LSM may have the potential for histopathological examination in surgical and biopsy applications by rapidly providing 3D information.

The LMIT: Light-mediated minimally-invasive theranostics in oncology

Minimally-invasive diagnosis and therapy have gradually become the trend and research hotspot of current medical applications. The integration of intraoperative diagnosis and treatment is a development important direction for real-time detection, minimally-invasive diagnosis and therapy to reduce mortality and improve the quality of life of patients, so called minimally-invasive theranostics (MIT). Light is an important theranostic tool for the treatment of cancerous tissues. Light-mediated minimally-invasive theranostics (LMIT) is a novel evolutionary technology that integrates diagnosis and therapeutics for the less invasive treatment of diseased tissues. Intelligent theranostics would promote precision surgery based on the optical characterization of cancerous tissues. Furthermore, MIT also requires the assistance of smart medical devices or robots. And, optical multimodality lay a solid foundation for intelligent MIT. In this review, we summarize the important state-of-the-arts of optical MIT or LMIT in oncology. Multimodal optical image-guided intelligent treatment is another focus. Intraoperative imaging and real-time analysis-guided optical treatment are also systemically discussed. Finally, the potential challenges and future perspectives of intelligent optical MIT are discussed.

Fluid preservation in brain banking: a review

Fluid preservation is nearly universally used in brain banking to store fixed tissue specimens for future research applications. However, the effects of long-term immersion on neural circuitry and biomolecules are not well characterized. As a result, there is a need to synthesize studies investigating fluid preservation of brain tissue. We searched PubMed and other databases to identify studies measuring the effects of fluid preservation in nervous system tissue. We categorized studies based on the fluid preservative used: formaldehyde solutions, buffer solutions, alcohol solutions, storage after tissue clearing, and cryoprotectant solutions. We identified 91 studies containing 197 independent observations of the effects of long-term storage on cellular morphology. Most studies did not report any significant alterations due to long-term storage. When present, the most frequent alteration was decreased antigenicity, commonly attributed to progressive crosslinking by aldehydes that renders biomolecules increasingly inaccessible over time. To build a mechanistic understanding, we discuss biochemical aspects of long-term fluid preservation. A subset of lipids appears to be chemical altered or extracted over time due to incomplete retention in the crosslinked gel. Alternative storage fluids mitigate the problem of antigen masking but have not been extensively characterized and may have other downsides. We also compare fluid preservation to cryopreservation, paraffin embedding, and resin embedding. Overall, existing evidence suggests that fluid preservation provides maintenance of neural architecture for decades, including precise structural details. However, to avoid the well-established problem of overfixation caused by storage in high concentration formaldehyde solutions, fluid preservation procedures can use an initial fixation step followed by an alternative long-term storage fluid. Further research is warranted on optimizing protocols and characterizing the generalizability of the storage artifacts that have been identified.

Fluorescence lifetime of injected indocyanine green as a universal marker of solid tumours in patients

The surgical resection of solid tumours can be enhanced by fluorescence-guided imaging. However, variable tumour uptake and incomplete clearance of fluorescent dyes reduces the accuracy of distinguishing tumour from normal tissue via conventional fluorescence intensity-based imaging. Here we show that, after systemic injection of the near-infrared dye indocyanine green in patients with various types of solid tumour, the fluorescence lifetime (FLT) of tumour tissue is longer than the FLT of non-cancerous tissue. This tumour-specific shift in FLT can be used to distinguish tumours from normal tissue with an accuracy of over 97% across tumour types, and can be visualized at the cellular level using microscopy and in larger specimens through wide-field imaging. Unlike fluorescence intensity, which depends on imaging-system parameters, tissue depth and the amount of dye taken up by tumours, FLT is a photophysical property that is largely independent of these factors. FLT imaging with indocyanine green may improve the accuracy of cancer surgeries.

Optical imaging technologies for in vivo cancer detection in low-resource settings

Cancer continues to affect underserved populations disproportionately. Novel optical imaging technologies, which can provide rapid, non-invasive, and accurate cancer detection at the point of care, have great potential to improve global cancer care. This article reviews the recent technical innovations and clinical translation of low-cost optical imaging technologies, highlighting the advances in both hardware and software, especially the integration of artificial intelligence, to improve in vivo cancer detection in low-resource settings. Additionally, this article provides an overview of existing challenges and future perspectives of adapting optical imaging technologies into clinical practice, which can potentially contribute to novel insights and programs that effectively improve cancer detection in low-resource settings.

High-resolution open-top axially swept light sheet microscopy

BACKGROUND

Open-top light-sheet microscopy (OT-LSM) is a specialized microscopic technique for the high-throughput cellular imaging of optically cleared, large-sized specimens, such as the brain. Despite the development of various OT-LSM techniques, achieving submicron resolution in all dimensions remains.

RESULTS

We developed a high-resolution open-top axially swept LSM (HR-OTAS-LSM) for high-throughput and high-resolution imaging in all dimensions. High axial and lateral resolutions were achieved by using an aberration-corrected axially swept excitation light sheet in the illumination arm and a high numerical aperture (NA) immersion objective lens in the imaging arm, respectively. The high-resolution, high-throughput visualization of neuronal networks in mouse brain and retina specimens validated the performance of HR-OTAS-LSM.

CONCLUSIONS

The proposed HR-OTAS-LSM method represents a significant advancement in the high-resolution mapping of cellular networks in biological systems such as the brain and retina.

Bridging live-cell imaging and next-generation cancer treatment

By providing spatial, molecular and morphological data over time, live-cell imaging can provide a deeper understanding of the cellular and signalling events that determine cancer response to treatment. Understanding this dynamic response has the potential to enhance clinical outcome by identifying biomarkers or actionable targets to improve therapeutic efficacy. Here, we review recent applications of live-cell imaging for uncovering both tumour heterogeneity in treatment response and the mode of action of cancer-targeting drugs. Given the increasing uses of T cell therapies, we discuss the unique opportunity of time-lapse imaging for capturing the interactivity and motility of immunotherapies. Although traditionally limited in the number of molecular features captured, novel developments in multidimensional imaging and multi-omics data integration offer strategies to connect single-cell dynamics to molecular phenotypes. We review the effect of these recent technological advances on our understanding of the cellular dynamics of tumour targeting and discuss their implication for next-generation precision medicine.

Deep learning-enabled realistic virtual histology with ultraviolet photoacoustic remote sensing microscopy

The goal of oncologic surgeries is complete tumor resection, yet positive margins are frequently found postoperatively using gold standard H&E-stained histology methods. Frozen section analysis is sometimes performed for rapid intraoperative margin evaluation, albeit with known inaccuracies. Here, we introduce a label-free histological imaging method based on an ultraviolet photoacoustic remote sensing and scattering microscope, combined with unsupervised deep learning using a cycle-consistent generative adversarial network for realistic virtual staining. Unstained tissues are scanned at rates of up to 7 mins/cm2, at resolution equivalent to 400x digital histopathology. Quantitative validation suggests strong concordance with conventional histology in benign and malignant prostate and breast tissues. In diagnostic utility studies we demonstrate a mean sensitivity and specificity of 0.96 and 0.91 in breast specimens, and respectively 0.87 and 0.94 in prostate specimens. We also find virtual stain quality is preferred (P = 0.03) compared to frozen section analysis in a blinded survey of pathologists.

Weakly Supervised AI for Efficient Analysis of 3D Pathology Samples

Human tissue consists of complex structures that display a diversity of morphologies, forming a tissue microenvironment that is, by nature, three-dimensional (3D). However, the current standard-of-care involves slicing 3D tissue specimens into two-dimensional (2D) sections and selecting a few for microscopic evaluation1,2, with concomitant risks of sampling bias and misdiagnosis3-6. To this end, there have been intense efforts to capture 3D tissue morphology and transition to 3D pathology, with the development of multiple high-resolution 3D imaging modalities7-18. However, these tools have had little translation to clinical practice as manual evaluation of such large data by pathologists is impractical and there is a lack of computational platforms that can efficiently process the 3D images and provide patient-level clinical insights. Here we present Modality-Agnostic Multiple instance learning for volumetric Block Analysis (MAMBA), a deep-learning-based platform for processing 3D tissue images from diverse imaging modalities and predicting patient outcomes. Archived prostate cancer specimens were imaged with open-top light-sheet microscopy12-14 or microcomputed tomography15,16 and the resulting 3D datasets were used to train risk-stratification networks based on 5-year biochemical recurrence outcomes via MAMBA. With the 3D block-based approach, MAMBA achieves an area under the receiver operating characteristic curve (AUC) of 0.86 and 0.74, superior to 2D traditional single-slice-based prognostication (AUC of 0.79 and 0.57), suggesting superior prognostication with 3D morphological features. Further analyses reveal that the incorporation of greater tissue volume improves prognostic performance and mitigates risk prediction variability from sampling bias, suggesting that there is value in capturing larger extents of spatially heterogeneous 3D morphology. With the rapid growth and adoption of 3D spatial biology and pathology techniques by researchers and clinicians, MAMBA provides a general and efficient framework for 3D weakly supervised learning for clinical decision support and can help to reveal novel 3D morphological biomarkers for prognosis and therapeutic response.

Motionless volumetric structured light sheet microscopy

To meet the increasing need for low-cost, compact imaging technology with cellular resolution, we have developed a microLED-based structured light sheet microscope for three-dimensional ex vivo and in vivo imaging of biological tissue in multiple modalities. All the illumination structure is generated directly at the microLED panel-which serves as the source-so light sheet scanning and modulation is completely digital, yielding a system that is simpler and less prone to error than previously reported methods. Volumetric images with optical sectioning are thus achieved in an inexpensive, compact form factor without any moving parts. We demonstrate the unique properties and general applicability of our technique by ex vivo imaging of porcine and murine tissue from the gastrointestinal tract, kidney, and brain.

Parallelized computational 3D video microscopy of freely moving organisms at multiple gigapixels per second

Wide field of view microscopy that can resolve 3D information at high speed and spatial resolution is highly desirable for studying the behaviour of freely moving model organisms. However, it is challenging to design an optical instrument that optimises all these properties simultaneously. Existing techniques typically require the acquisition of sequential image snapshots to observe large areas or measure 3D information, thus compromising on speed and throughput. Here, we present 3D-RAPID, a computational microscope based on a synchronized array of 54 cameras that can capture high-speed 3D topographic videos over an area of 135 cm2, achieving up to 230 frames per second at spatiotemporal throughputs exceeding 5 gigapixels per second. 3D-RAPID employs a 3D reconstruction algorithm that, for each synchronized snapshot, fuses all 54 images into a composite that includes a co-registered 3D height map. The self-supervised 3D reconstruction algorithm trains a neural network to map raw photometric images to 3D topography using stereo overlap redundancy and ray-propagation physics as the only supervision mechanism. The resulting reconstruction process is thus robust to generalization errors and scales to arbitrarily long videos from arbitrarily sized camera arrays. We demonstrate the broad applicability of 3D-RAPID with collections of several freely behaving organisms, including ants, fruit flies, and zebrafish larvae.

Label-free cleared tissue microscopy and machine learning for 3D histopathology of biomaterial implants

Tissue clearing of whole intact organs has enhanced imaging by enabling the exploration of tissue structure at a subcellular level in three-dimensional space. Although clearing and imaging of the whole organ have been used to study tissue biology, the microenvironment in which cells evolve to adapt to biomaterial implants or allografts in the body is poorly understood. Obtaining high-resolution information from complex cell-biomaterial interactions with volumetric landscapes represents a key challenge in the fields of biomaterials and regenerative medicine. To provide a new approach to examine how tissue responds to biomaterial implants, we apply cleared tissue light-sheet microscopy and three-dimensional reconstruction to utilize the wealth of autofluorescence information for visualizing and contrasting anatomical structures. This study demonstrates the adaptability of the clearing and imaging technique to provide sub-cellular resolution (0.6 μm isotropic) 3D maps of various tissue types, using samples from fully intact peritoneal organs to volumetric muscle loss injury specimens. Specifically, in the volumetric muscle loss injury model, we provide 3D visualization of the implanted extracellular matrix biomaterial in the wound bed of the quadricep muscle groups and further apply computational-driven image classification to analyze the autofluorescence spectrum at multiple emission wavelengths to categorize tissue types at the injured site interacting with the biomaterial scaffolds.

Recent progress of second near-infrared (NIR-II) fluorescence microscopy in bioimaging

Visualizing biological tissues in vivo at a cellular or subcellular resolution to explore molecular signaling and cell behaviors is a crucial direction for research into biological processes. In vivo imaging can provide quantitative and dynamic visualization/mapping in biology and immunology. New microscopy techniques combined with near-infrared region fluorophores provide additional avenues for further progress in vivo bioimaging. Based on the development of chemical materials and physical optoelectronics, new NIR-II microscopy techniques are emerging, such as confocal and multiphoton microscopy, light-sheet fluorescence microscopy (LSFM), and wide-field microscopy. In this review, we introduce the characteristics of in vivo imaging using NIR-II fluorescence microscopy. We also cover the recent advances in NIR-II fluorescence microscopy techniques in bioimaging and the potential for overcoming current challenges.

Optical-resolution photoacoustic microscopy with a needle-shaped beam

Optical-resolution photoacoustic microscopy (OR-PAM) can visualize wavelength-dependent optical absorption at the cellular level. However, OR-PAM suffers from a limited depth of field (DOF) due to the tight focus of the optical excitation beam, making it challenging to acquire high-resolution images of samples with uneven surfaces or high-quality volumetric images without z-scanning. To overcome this limitation, we propose needle-shaped beam photoacoustic microscopy (NB-PAM), which can extend the DOF to up to ~28-fold Rayleigh lengths via customized diffractive optical elements (DOEs). The DOE generate a needle beam with a well-maintained beam diameter, a uniform axial intensity distribution, and negligible sidelobes. The advantage of using NB-PAM is demonstrated by both histology-like imaging of fresh slide-free organs using a 266 nm laser and in vivo mouse brain vasculature imaging using a 532 nm laser. The approach provides new perspectives for slide-free intraoperative pathological imaging and in-vivo organ-level imaging.

Structural and functional imaging of brains

Analyzing the complex structures and functions of brain is the key issue to understanding the physiological and pathological processes. Although neuronal morphology and local distribution of neurons/blood vessels in the brain have been known, the subcellular structures of cells remain challenging, especially in the live brain. In addition, the complicated brain functions involve numerous functional molecules, but the concentrations, distributions and interactions of these molecules in the brain are still poorly understood. In this review, frontier techniques available for multiscale structure imaging from organelles to the whole brain are first overviewed, including magnetic resonance imaging (MRI), computed tomography (CT), positron emission tomography (PET), serial-section electron microscopy (ssEM), light microscopy (LM) and synchrotron-based X-ray microscopy (XRM). Specially, XRM for three-dimensional (3D) imaging of large-scale brain tissue with high resolution and fast imaging speed is highlighted. Additionally, the development of elegant methods for acquisition of brain functions from electrical/chemical signals in the brain is outlined. In particular, the new electrophysiology technologies for neural recordings at the single-neuron level and in the brain are also summarized. We also focus on the construction of electrochemical probes based on dual-recognition strategy and surface/interface chemistry for determination of chemical species in the brain with high selectivity and long-term stability, as well as electrochemophysiological microarray for simultaneously recording of electrochemical and electrophysiological signals in the brain. Moreover, the recent development of brain MRI probes with high contrast-to-noise ratio (CNR) and sensitivity based on hyperpolarized techniques and multi-nuclear chemistry is introduced. Furthermore, multiple optical probes and instruments, especially the optophysiological Raman probes and fiber Raman photometry, for imaging and biosensing in live brain are emphasized. Finally, a brief perspective on existing challenges and further research development is provided.

Discrimination of normal and cancerous human skin tissues based on laser-induced spectral shift fluorescence microscopy

A homemade spectral shift fluorescence microscope (SSFM) is coupled with a spectrometer to record the spectral images of specimens based on the emission wavelength. Here a reliable diagnosis of neoplasia is achieved according to the spectral fluorescence properties of ex-vivo skin tissues after rhodamine6G (Rd6G) staining. It is shown that certain spectral shifts occur for nonmelanoma/melanoma lesions against normal/benign nevus, leading to spectral micrographs. In fact, there is a strong correlation between the emission wavelength and the sort of skin lesions, mainly due to the Rd6G interaction with the mitochondria of cancerous cells. The normal tissues generally enjoy a significant red shift regarding the laser line (37 nm). Conversely, plenty of fluorophores are conjugated to unhealthy cells giving rise to a relative blue shift i.e., typically SCC (6 nm), BCC (14 nm), and melanoma (19 nm) against healthy tissues. In other words, the redshift takes place with respect to the excitation wavelength i.e., melanoma (18 nm), BCC (23 nm), and SCC (31 nm) with respect to the laser line. Consequently, three data sets are available in the form of micrographs, addressing pixel-by-pixel signal intensity, emission wavelength, and fluorophore concentration of specimens for prompt diagnosis.

Ex situ perfusion fixation for brain banking: a technical report

Perfusion fixation is a well-established technique in animal research to improve preservation quality in the study of many tissues, including the brain. There is a growing interest in using perfusion to fix postmortem human brain tissue to achieve the highest fidelity preservation for downstream high-resolution morphomolecular brain mapping studies. Numerous practical barriers arise when applying perfusion fixation in brain banking settings, including the large mass of the organ, degradation of vascular integrity and patency prior to the start of the procedure, and differing investigator goals sometimes necessitating part of the brain to be frozen. As a result, there is a critical need to establish a perfusion fixation procedure in brain banking that is flexible and scalable. This technical report describes our approach to developing an ex situ perfusion fixation protocol. We discuss the challenges encountered and lessons learned while implementing this procedure. Routine morphological staining and RNA in situ hybridization data show that the perfused brains have well-preserved tissue cytoarchitecture and intact biomolecular signal. However, it remains uncertain whether this procedure leads to improved histology quality compared to immersion fixation. Additionally, ex vivo magnetic resonance imaging (MRI) data suggest that the perfusion fixation protocol may introduce imaging artifacts in the form of air bubbles in the vasculature. We conclude with further research directions to investigate the use of perfusion fixation as a rigorous and reproducible alternative to immersion fixation for the preparation of postmortem human brains.