Label-free detection of invasive micropapillary carcinoma of the breast using multiphoton microscopy

Invasive micropapillary carcinoma of the breast (IMPC) is a rare form of breast cancer with unique histological features, and is associated with high axillary lymph node metastasis and poor clinical prognosis. Thus, IMPC should be diagnosed in time to improve the treatment and management of patients. In this study, multiphoton microscopy (MPM) is used to label-free visualize the morphological features of IMPC. Our results demonstrate that MPM images are well in agreement with hematoxylin and eosin staining and epithelial membrane antigen staining, indicating MPM is comparable to traditional histological analysis in identifying the tissue structure and cell morphology. Statistical analysis shows significant differences in the circumference and area of the glandular lumen and cancer nest between the different IMPC cell clusters with complete glandular lumen morphology, and also shows difference in collagen length, width, and orientation, indicating the invasive ability of different morphologies of IMPC may be different.

Label-free imaging of human brain tissue at subcellular resolution for potential rapid intra-operative assessment of glioma surgery

Background: Frozen section and smear preparation are the current standard for intraoperative histopathology during cancer surgery. However, these methods are time-consuming and subject to limited sampling. Multiphoton microscopy (MPM) is a high-resolution non-destructive imaging technique capable of optical sectioning in real time with subcellular resolution. In this report, we systematically investigated the feasibility and translation potential of MPM for rapid histopathological assessment of label- and processing-free surgical specimens. Methods: We employed a customized MPM platform to capture architectural and cytological features of biological tissues based on two-photon excited NADH and FAD autofluorescence and second harmonic generation from collagen. Infiltrating glioma, an aggressive disease that requires subcellular resolution for definitive characterization during surgery, was chosen as an example for this validation study. MPM images were collected from resected brain specimens of 19 patients and correlated with histopathology. Deep learning was introduced to assist with image feature recognition. Results: MPM robustly captures diagnostic features of glioma including increased cellularity, cellular and nuclear pleomorphism, microvascular proliferation, necrosis, and collagen deposition. Preliminary application of deep learning to MPM images achieves high accuracy in distinguishing gray from white matter and cancer from non-cancer. We also demonstrate the ability to obtain such images from intact brain tissue with a multiphoton endomicroscope for intraoperative application. Conclusion: Multiphoton imaging correlates well with histopathology and is a promising tool for characterization of cancer and delineation of infiltration within seconds during brain surgery.

Dense-UNet: a novel multiphoton in vivo cellular image segmentation model based on a convolutional neural network

Background

Multiphoton microscopy (MPM) offers a feasible approach for the biopsy in clinical medicine, but it has not been used in clinical applications due to the lack of efficient image processing methods, especially the automatic segmentation technology. Segmentation technology is still one of the most challenging assignments of the MPM imaging technique.

Methods

The MPM imaging segmentation model based on deep learning is one of the most effective methods to address this problem. In this paper, the practicability of using a convolutional neural network (CNN) model to segment the MPM image of skin cells in vivo was explored. A set of MPM in vivo skin cells images with a resolution of 128×128 was successfully segmented under the Python environment with TensorFlow. A novel deep-learning segmentation model named Dense-UNet was proposed. The Dense-UNet, which is based on U-net structure, employed the dense concatenation to deepen the depth of the network architecture and achieve feature reuse. This model included four expansion modules (each module consisted of four down-sampling layers) to extract features.

Results

Sixty training images were taken from the dorsal forearm using a femtosecond Ti:Sa laser running at 735 nm. The resolution of the images is 128×128 pixels. Experimental results confirmed that the accuracy of Dense-UNet (92.54%) was higher than that of U-Net (88.59%), with a significantly lower loss value of 0.1681. The 90.60% Dice coefficient value of Dense-UNet outperformed U-Net by 11.07%. The F1-Score of Dense-UNet, U-Net, and Seg-Net was 93.35%, 90.02%, and 85.04%, respectively.

Conclusions

The deepened down-sampling path improved the ability of the model to capture cellular fined-detailed boundary features, while the symmetrical up-sampling path provided a more accurate location based on the test result. These results were the first time that the segmentation of MPM in vivo images had been adopted by introducing a deep CNN to bridge this gap in Dense-UNet technology. Dense-UNet has reached ultramodern performance for MPM images, especially for in vivo images with low resolution. This implementation supplies an automatic segmentation model based on deep learning for high-precision segmentation of MPM images in vivo.

Automatic and label-free identification of blood vessels in gliomas using the combination of multiphoton microscopy and image analysis

Currently, the targeted treatment of tumor based on the tumor microenvironment is newly developed. Blood vessels are the key parts in the tumor microenvironment, which is taken as a new visible target for tumor therapy. Multiphoton microscopy (MPM), based on the second harmonic generation and two-photon excited fluorescence, is available to make the label-free analysis on the blood vessels in human gliomas. MPM can reveal the vascular morphological characteristics in gliomas, including vascular malformation, intense vascular proliferation, perivascular collagen deposition, perivascular lymphocytes aggregation and microvascular proliferation. In addition, the image analysis algorithms were developed to automatically calculate the perivascular collagen content, vascular cavity area, lumen area, wall area and vessel number. Thus, the vascular morphology, the perivascular collagen deposition and intense vascular proliferation degree can be further quantitatively characterized. Compared with the pathological analysis, the combination of MPM and image analysis has potential advantages in making a quantitative and qualitative analyzing on vascular morphology in glioma microenvironment. As micro-endoscope and two-photon fiberscope are technologically improved, this combined method will be a useful imaging way to make the real-time research on the targeting tumor microenvironment in gliomas.

Automated classification of hepatocellular carcinoma differentiation using multiphoton microscopy and deep learning

In the case of hepatocellular carcinoma (HCC) samples, classification of differentiation is crucial for determining prognosis and treatment strategy decisions. However, a label-free and automated classification system for HCC grading has not been yet developed. Hence, in this study, we demonstrate the fusion of multiphoton microscopy and a deep-learning algorithm for classifying HCC differentiation to produce an innovative computer-aided diagnostic method. Convolutional neural networks based on the VGG-16 framework were trained using 217 combined two-photon excitation fluorescence and second-harmonic generation images; the resulting classification accuracy of the HCC differentiation grade was over 90%. Our results suggest that a combination of multiphoton microscopy and deep learning can realize label-free, automated methods for various tissues, diseases and other related classification problems.

Rapid, label-free identification of cerebellar structures using multiphoton microscopy

The cerebellum is the prominent laminar structure of the mammalian brain that has been implicated in various psychiatric and neurological diseases. Although clinical brain imaging techniques have provided precise anatomic images of cerebellar structures, a definitive diagnosis still requires adequate resolution to identify individual layers in cerebellar cortex, the extent of tumor, even requires the histological tissue examination during surgical procedures. In this study, multiphoton microscopy (MPM), based on second harmonic generation (SHG) and two-photon excited fluorescence (TPEF), was perform on the rat cerebellar structures and pathology with the combination of image analysis methods. Results show that MPM can reveal the cerebellar vermis, hemispheres, medulla, and ventricle, as well as axon bundles, Purkinje cells, capillaries, and the pia mater of the cerebellum. Together with custom-developed image processing algorithms, MPM could further differentiate between the gray and white matter, as well as evaluate the Purkinje cell layer, identify the cerebellar tumor boundary, and distinguish between the tumor core and peritumor regions. Our results establish a direct visualization and rapid assessment approach for the cerebellar structures, as well as suggest the feasibility of in vivo multiphoton microendoscopes and fiberscopes as clinical tools for neuropathological diagnoses.

Rapid, label-free identification of cerebellar structures using multiphoton microscopy

The cerebellum is the prominent laminar structure of the mammalian brain that has been implicated in various psychiatric and neurological diseases. Although clinical brain imaging techniques have provided precise anatomic images of cerebellar structures, a definitive diagnosis still requires adequate resolution to identify individual layers in cerebellar cortex, the extent of tumor, even requires the histological tissue examination during surgical procedures. In this study, multiphoton microscopy (MPM), based on second harmonic generation (SHG) and two-photon excited fluorescence (TPEF), was perform on the rat cerebellar structures and pathology with the combination of image analysis methods. Results show that MPM can reveal the cerebellar vermis, hemispheres, medulla, and ventricle, as well as axon bundles, Purkinje cells, capillaries, and the pia mater of the cerebellum. Together with custom-developed image processing algorithms, MPM could further differentiate between the gray and white matter, as well as evaluate the Purkinje cell layer, identify the cerebellar tumor boundary, and distinguish between the tumor core and peritumor regions. Our results establish a direct visualization and rapid assessment approach for the cerebellar structures, as well as suggest the feasibility of in vivo multiphoton microendoscopes and fiberscopes as clinical tools for neuropathological diagnoses.

Study on the diagnosis of rabbit VX2 esophageal cancer and stent-therapy efficacy based on multiphoton microscopy

Histopathology is the gold standard for diagnosis of esophageal carcinoma. Based on two photon excited fluorescence (TPEF) and second harmonic generation (SHG), multiphoton microscopy (MPM) has become a novel optical tool adjunct to current histopathological techniques without any exogenous contrast agents. We thus investigated the potential of using TPEF and SHG techniques for differentiating cancer tissues from cancer tissues after paclitaxel-eluting stent implantation and normal esophageal tissues which are fresh and unstained without dying from the VX2 esophageal carcinoma rabbit models. Comparisons were made between MPM imaging and gold standard sections for each specimen stained with hematoxylin-eosin (H&E). Our results indicated that the MPM imaging technique could identify and distinguish among normal esophageal tissues, cancer tissues, as well as cancer tissues after stent implantation. Therefore, MPM potentially offers a powerful tool to not only diagnose esophageal cancer but also monitor stent-therapy efficacy.

Image segmentation for integrated multiphoton microscopy and reflectance confocal microscopy imaging of human skin in vivo

BACKGROUND

Non-invasive cellular imaging of the skin in vivo can be achieved in reflectance confocal microscopy (RCM) and multiphoton microscopy (MPM) modalities to yield complementary images of the skin based on different optical properties. One of the challenges of in vivo microscopy is the delineation (i.e., segmentation) of cellular and subcellular architectural features.

METHODS

In this work we present a method for combining watershed and level-set models for segmentation of multimodality images obtained by an integrated MPM and RCM imaging system from human skin in vivo.

RESULTS

Firstly, a segmentation model based on watershed is introduced for obtaining the accurate structure of cell borders from the RCM image. Secondly,, a global region based energy level-set model is constructed for extracting the nucleus of each cell from the MPM image. Thirdly, a local region-based Lagrange Continuous level-set approach is used for segmenting cytoplasm from the MPM image.

CONCLUSIONS

Experimental results demonstrated that cell borders from RCM image and boundaries of cytoplasm and nucleus from MPM image can be obtained by our segmentation method with better accuracy and effectiveness. We are planning to use this method to perform quantitative analysis of MPM and RCM images of in vivo human skin to study the variations of cellular parameters such as cell size, nucleus size and other mophormetric features with skin pathologies.

High Resolution Phase-Sensitive Magnetomotive Optical Coherence Microscopy for Tracking Magnetic Microbeads and Cellular Mechanics

We present a real-time multimodal near-infrared imaging technology that tracks externally induced axial motion of magnetic microbeads in single cells in culture. The integrated multimodal imaging technique consists of phase-sensitive magnetomotive optical coherence microscopy (MM-OCM) and multiphoton microscopy (MPM).MPMis utilized for the visualization of multifunctional fluorescent and magnetic microbeads, while MM-OCM detects, with nanometer-scale sensitivity, periodic displacements of the microbeads induced by the modulation of an external magnetic field. Magnetomotive signals are measured from mouse macrophages, human breast primary ductal carcinoma cells, and human breast epithelial cells in culture, and validated with full-field phase-sensitive microscopy. This methodology demonstrates the capability for imaging controlled cell dynamics and has the potential for measuring cell biomechanical properties, which are important in assessing the health and pathological state of cells.

Multiphoton microscopic imaging of esophagus during the early phase of tumor progression

Esophageal cancer is one of the most common cancer and leading cause of cancer death worldwide. Multiphoton microscopy (MPM) has become a novel optical tool of choice for imaging tissue architecture and cellular morphology based on two-photon excited fluorescence and second harmonic generation. In this study, we used MPM to image microstructure of human normal esophagus, carcinoma in situ, and early invasive carcinoma in order to investigate the morphological change of tissue structure during the early phase of tumor progression. The diagnostic features such as the appearance of cancerous cells, the absence of the basement membrane were extracted to distinguish between normal and cancerous esophagus tissue. The infiltration depth during tumor progression was determined by the appearance of cancerous cells. The significant change of layer structure between cancerous tissue and normal esophagus was described. We also quantitatively described the differences of morphology between normal and cancerous cells. These results correlated well with the corresponding histological findings. With the advancement of clinically miniaturized MPM and the multi-photon probe, combining MPM with standard endoscopy will therefore allow us to make a real-time in vivo diagnosis of early esophageal cancer at the cellular level.

Label-free imaging characteristics of colonic mucinous adenocarcinoma using multiphoton microscopy

Colorectal carcinoma (CRC) has high mortality and increased incidence rates. An early detection of CRC is very important. Multiphoton microscopy (MPM) with high resolution and high sensitivity is used to effectively distinguish the microstructure changes of normal and mucinous adenocarcinoma slices of ex vivo human colonic tissues. In mucinous adenocarcinoma mucosa, the glands are distorted and elongated, the gland cavity is indistinct, and the mesh collagen fibers are diminished. In the submucosa, the collagens are seriously disordered, elongated, pushed aside, and sparsely visible, the content of elastic fibers is also broken and almost disappearing. Many cancer cells, some in cavity-like shape full of mucus surrounded by some collagen fibers, occupied the submucosa, which are comparable to hematoxylin-eosin (HE) stained images. Second harmonic generation and two-photon excitation fluorescence (SHG/TPEF) intensity ratio can be used further to quantitatively evaluate normality and abnormality. The fast Fourier transform (FFT) images show that the normal collagen fibrils are dense and in random order, and the cancerous collagen is certainly organized. The exploratory results show that it has potential for the development of multiphoton mini-endoscopy in real-time early diagnosis of CRC.

Multimodal In Vivo Skin Imaging with Integrated Optical Coherence and Multiphoton Microscopy

In this paper, we demonstrate high-resolution, multimodal in vivo imaging of human skin using optical coherence (OCM) and multiphoton microscopy (MPM). These two modalities are integrated into a single instrument to enable simultaneous acquisition and coregistration. The system design and the OCM image processing architecture enable sufficient performance of both modalities for in vivo imaging of human skin. Examples of multimodal in vivo imaging are presented as well as time lapse imaging of blood flow in single capillary loops. By making use of multiple intrinsic contrast mechanisms this integrated technique improves the ability to noninvasively visualize living tissue. Integrated OCM and MPM has potential applications for in vivo diagnosis of various pathological skin conditions, such as skin cancer, as well as potential pharmaceutical and cosmetic research applications.