Automated classification of multiphoton microscopy images of ovarian tissue using deep learning

Medical image identification methods: A review

The identification of medical images is an essential task in computer-aided diagnosis, medical image retrieval and mining. Medical image data mainly include electronic health record data and gene information data, etc. Although intelligent imaging provided a good scheme for medical image analysis over traditional methods that rely on the handcrafted features, it remains challenging due to the diversity of imaging modalities and clinical pathologies. Many medical image identification methods provide a good scheme for medical image analysis. The concepts pertinent of methods, such as the machine learning, deep learning, convolutional neural networks, transfer learning, and other image processing technologies for medical image are analyzed and summarized in this paper. We reviewed these recent studies to provide a comprehensive overview of applying these methods in various medical image analysis tasks, such as object detection, image classification, image registration, segmentation, and other tasks. Especially, we emphasized the latest progress and contributions of different methods in medical image analysis, which are summarized base on different application scenarios, including classification, segmentation, detection, and image registration. In addition, the applications of different methods are summarized in different application area, such as pulmonary, brain, digital pathology, brain, skin, lung, renal, breast, neuromyelitis, vertebrae, and musculoskeletal, etc. Critical discussion of open challenges and directions for future research are finally summarized. Especially, excellent algorithms in computer vision, natural language processing, and unmanned driving will be applied to medical image recognition in the future.

Toward next-generation endoscopes integrating biomimetic video systems, nonlinear optical microscopy, and deep learning

According to the World Health Organization, the proportion of the world's population over 60 years will approximately double by 2050. This progressive increase in the elderly population will lead to a dramatic growth of age-related diseases, resulting in tremendous pressure on the sustainability of healthcare systems globally. In this context, finding more efficient ways to address cancers, a set of diseases whose incidence is correlated with age, is of utmost importance. Prevention of cancers to decrease morbidity relies on the identification of precursor lesions before the onset of the disease, or at least diagnosis at an early stage. In this article, after briefly discussing some of the most prominent endoscopic approaches for gastric cancer diagnostics, we review relevant progress in three emerging technologies that have significant potential to play pivotal roles in next-generation endoscopy systems: biomimetic vision (with special focus on compound eye cameras), non-linear optical microscopies, and Deep Learning. Such systems are urgently needed to enhance the three major steps required for the successful diagnostics of gastrointestinal cancers: detection, characterization, and confirmation of suspicious lesions. In the final part, we discuss challenges that lie en route to translating these technologies to next-generation endoscopes that could enhance gastrointestinal imaging, and depict a possible configuration of a system capable of (i) biomimetic endoscopic vision enabling easier detection of lesions, (ii) label-free in vivo tissue characterization, and (iii) intelligently automated gastrointestinal cancer diagnostic.

A Deep Learning Framework for the Prediction and Diagnosis of Ovarian Cancer in Pre- and Post-Menopausal Women

Ovarian cancer ranks as the fifth leading cause of cancer-related mortality in women. Late-stage diagnosis (stages III and IV) is a major challenge due to the often vague and inconsistent initial symptoms. Current diagnostic methods, such as biomarkers, biopsy, and imaging tests, face limitations, including subjectivity, inter-observer variability, and extended testing times. This study proposes a novel convolutional neural network (CNN) algorithm for predicting and diagnosing ovarian cancer, addressing these limitations. In this paper, CNN was trained on a histopathological image dataset, divided into training and validation subsets and augmented before training. The model achieved a remarkable accuracy of 94%, with 95.12% of cancerous cases correctly identified and 93.02% of healthy cells accurately classified. The significance of this study lies in overcoming the challenges associated with the human expert examination, such as higher misclassification rates, inter-observer variability, and extended analysis times. This study presents a more accurate, efficient, and reliable approach to predicting and diagnosing ovarian cancer. Future research should explore recent advances in this field to enhance the effectiveness of the proposed method further.

Nonlinear microscopy and deep learning classification for mammary gland microenvironment studies

Tumors, their microenvironment, and the mechanisms by which collagen morphology changes throughout cancer progression have recently been a topic of interest. Second harmonic generation (SHG) and polarization second harmonic (P-SHG) microscopy are label-free, hallmark methods that can highlight this alteration in the extracellular matrix (ECM). This article uses automated sample scanning SHG and P-SHG microscopy to investigate ECM deposition associated with tumors residing in the mammary gland. We show two different analysis approaches using the acquired images to distinguish collagen fibrillar orientation changes in the ECM. Lastly, we apply a supervised deep-learning model to classify naïve and tumor-bearing mammary gland SHG images. We benchmark the trained model using transfer learning with the well-known MobileNetV2 architecture. By fine-tuning the different parameters of these models, we show a trained deep-learning model that suits such a small dataset with 73% accuracy.

High-precision automatic identification method for dicentric chromosome images using two-stage convolutional neural network

Dicentric chromosome analysis is the gold standard for biological dose assessment. To enhance the efficiency of biological dose assessment in large-scale radiation catastrophes, automatic identification of dicentric chromosome images is a promising and objective method. In this paper, an automatic identification method for dicentric chromosome images using two-stage convolutional neural network is proposed based on Giemsa-stained automatic microscopic imaging. To automatically segment the adhesive chromosome masses, a k-means based adaptive image segmentation and watershed segmentation algorithm is applied. The first-stage CNN is used to identify the dicentric chromosome images from all the images and the second-stage CNN works to specifically identify the dicentric chromosome images. This two-stage CNN identification method can effectively detects chromosome images with concealed centromeres, poorly expanded and long-armed entangled chromosomes, and tricentric chromosomes. The novel two-stage CNN method has a chromosome identification accuracy of 99.4%, a sensitivity of 85.8% sensitivity, and a specificity of 99.6%, effectively reducing the false positive rate of dicentric chromosome. The analysis speed of this automatic identification method can be 20 times quicker than manual detection, providing a valuable reference for other image identification situations with small target rates.

Second harmonic generation microscopy: a powerful tool for bio-imaging

Second harmonic generation (SHG) microscopy is an important optical imaging technique in a variety of applications. This article describes the history and physical principles of SHG microscopy and its more advanced variants, as well as their strengths and weaknesses in biomedical applications. It also provides an overview of SHG and advanced SHG imaging in neuroscience and microtubule imaging and how these methods can aid in understanding microtubule formation, structuration, and involvement in neuronal function. Finally, we offer a perspective on the future of these methods and how technological advancements can help make SHG microscopy a more widely adopted imaging technique.

Deep learning-based high-speed, large-field, and high-resolution multiphoton imaging

Multiphoton microscopy is a formidable tool for the pathological analysis of tumors. The physical limitations of imaging systems and the low efficiencies inherent in nonlinear processes have prevented the simultaneous achievement of high imaging speed and high resolution. We demonstrate a self-alignment dual-attention-guided residual-in-residual generative adversarial network trained with various multiphoton images. The network enhances image contrast and spatial resolution, suppresses noise, and scanning fringe artifacts, and eliminates the mutual exclusion between field of view, image quality, and imaging speed. The network may be integrated into commercial microscopes for large-scale, high-resolution, and low photobleaching studies of tumor environments.

Automatic Detection and Segmentation of Ovarian Cancer Using a Multitask Model in Pelvic CT Images

Ovarian cancer is one of the most common malignant tumours of female reproductive organs in the world. The pelvic CT scan is a common examination method used for the screening of ovarian cancer, which shows the advantages in safety, efficiency, and providing high-resolution images. Recently, deep learning applications in medical imaging attract more and more attention in the research field of tumour diagnostics. However, due to the limited number of relevant datasets and reliable deep learning models, it remains a challenging problem to detect ovarian tumours on CT images. In this work, we first collected CT images of 223 ovarian cancer patients in the Affiliated Hospital of Qingdao University. A new end-to-end network based on YOLOv5 is proposed, namely, YOLO-OCv2 (ovarian cancer). We improved the previous work YOLO-OC firstly, including balanced mosaic data enhancement and decoupled detection head. Then, based on the detection model, a multitask model is proposed, which can simultaneously complete the detection and segmentation tasks.

Artificial Intelligence-Powered Prediction of ALK Gene Rearrangement in Patients With Non-Small-Cell Lung Cancer

PURPOSE

Several studies reported the possibility of predicting genetic abnormalities in non-small-cell lung cancer by deep learning (DL). However, there are no data of predicting ALK gene rearrangement (ALKr) using DL. We evaluated the ALKr predictability using the DL platform.

MATERIALS AND METHODS

We selected 66 ALKr-positive cases and 142 ALKr-negative cases, which were diagnosed by ALKr immunohistochemical staining in our institution from January 2009 to March 2019. We generated virtual slide of 300 slides (150 ALKr-positive slides and 150 ALKr-negative slides) using NanoZoomer. HALO-AI was used to analyze the whole-slide imaging data, and the DenseNet network was used to build the learning model. Of the 300 slides, we randomly assigned 172 slides to the training cohort and 128 slides to the test cohort to ensure no duplication of cases. In four resolutions (16.0/4.0/1.0/0.25 μm/pix), ALKr prediction models were built in the training cohort and ALKr prediction performance was evaluated in the test cohort. We evaluated the diagnostic probability of ALKr by receiver operating characteristic analysis in each ALKr probability threshold (50%, 60%, 70%, 80%, 90%, and 95%). We expected the area under the curve to be 0.64-0.85 in the model of a previous study. Furthermore, in the test cohort data, an expert pathologist also evaluated the presence of ALKr by hematoxylin and eosin staining on whole-slide imaging.

RESULTS

The maximum area under the curve was 0.73 (50% threshold: 95% CI, 0.65 to 0.82) in the resolution of 1.0 μm/pix. In this resolution, with an ALKr probability of 50% threshold, the sensitivity and specificity were 73% and 73%, respectively. The expert pathologist's sensitivity and specificity in the same test cohort were 13% and 94%.

CONCLUSION

The ALKr prediction by DL was feasible. Further study should be addressed to improve accuracy of ALKr prediction.

Artificial intelligence and machine learning in precision and genomic medicine

The advancement of precision medicine in medical care has led behind the conventional symptom-driven treatment process by allowing early risk prediction of disease through improved diagnostics and customization of more effective treatments. It is necessary to scrutinize overall patient data alongside broad factors to observe and differentiate between ill and relatively healthy people to take the most appropriate path toward precision medicine, resulting in an improved vision of biological indicators that can signal health changes. Precision and genomic medicine combined with artificial intelligence have the potential to improve patient healthcare. Patients with less common therapeutic responses or unique healthcare demands are using genomic medicine technologies. AI provides insights through advanced computation and inference, enabling the system to reason and learn while enhancing physician decision making. Many cell characteristics, including gene up-regulation, proteins binding to nucleic acids, and splicing, can be measured at high throughput and used as training objectives for predictive models. Researchers can create a new era of effective genomic medicine with the improved availability of a broad range of datasets and modern computer techniques such as machine learning. This review article has elucidated the contributions of ML algorithms in precision and genome medicine.

TAIM: Tool for Analyzing Root Images to Calculate the Infection Rate of Arbuscular Mycorrhizal Fungi

Arbuscular mycorrhizal fungi (AMF) infect plant roots and are hypothesized to improve plant growth. Recently, AMF is now available for axenic culture. Therefore, AMF is expected to be used as a microbial fertilizer. To evaluate the usefulness of AMF as a microbial fertilizer, we need to investigate the relationship between the degree of root colonization of AMF and plant growth. The method popularly used for calculation of the degree of root colonization, termed the magnified intersections method, is performed manually and is too labor-intensive to enable an extensive survey to be undertaken. Therefore, we automated the magnified intersections method by developing an application named "Tool for Analyzing root images to calculate the Infection rate of arbuscular Mycorrhizal fungi: TAIM." TAIM is a web-based application that calculates the degree of AMF colonization from images using automated computer vision and pattern recognition techniques. Experimental results showed that TAIM correctly detected sampling areas for calculation of the degree of infection and classified the sampling areas with 87.4% accuracy. TAIM is publicly accessible at http://taim.imlab.jp/.

Transfer learning for medical image classification: a literature review

BACKGROUND

Transfer learning (TL) with convolutional neural networks aims to improve performances on a new task by leveraging the knowledge of similar tasks learned in advance. It has made a major contribution to medical image analysis as it overcomes the data scarcity problem as well as it saves time and hardware resources. However, transfer learning has been arbitrarily configured in the majority of studies. This review paper attempts to provide guidance for selecting a model and TL approaches for the medical image classification task.

METHODS

425 peer-reviewed articles were retrieved from two databases, PubMed and Web of Science, published in English, up until December 31, 2020. Articles were assessed by two independent reviewers, with the aid of a third reviewer in the case of discrepancies. We followed the PRISMA guidelines for the paper selection and 121 studies were regarded as eligible for the scope of this review. We investigated articles focused on selecting backbone models and TL approaches including feature extractor, feature extractor hybrid, fine-tuning and fine-tuning from scratch.

RESULTS

The majority of studies (n = 57) empirically evaluated multiple models followed by deep models (n = 33) and shallow (n = 24) models. Inception, one of the deep models, was the most employed in literature (n = 26). With respect to the TL, the majority of studies (n = 46) empirically benchmarked multiple approaches to identify the optimal configuration. The rest of the studies applied only a single approach for which feature extractor (n = 38) and fine-tuning from scratch (n = 27) were the two most favored approaches. Only a few studies applied feature extractor hybrid (n = 7) and fine-tuning (n = 3) with pretrained models.

CONCLUSION

The investigated studies demonstrated the efficacy of transfer learning despite the data scarcity. We encourage data scientists and practitioners to use deep models (e.g. ResNet or Inception) as feature extractors, which can save computational costs and time without degrading the predictive power.

Deep learning-based image processing in optical microscopy

Optical microscopy has emerged as a key driver of fundamental research since it provides the ability to probe into imperceptible structures in the biomedical world. For the detailed investigation of samples, a high-resolution image with enhanced contrast and minimal damage is preferred. To achieve this, an automated image analysis method is preferable over manual analysis in terms of both speed of acquisition and reduced error accumulation. In this regard, deep learning (DL)-based image processing can be highly beneficial. The review summarises and critiques the use of DL in image processing for the data collected using various optical microscopic techniques. In tandem with optical microscopy, DL has already found applications in various problems related to image classification and segmentation. It has also performed well in enhancing image resolution in smartphone-based microscopy, which in turn enablse crucial medical assistance in remote places.

Graphical abstract

A deep hybrid learning pipeline for accurate diagnosis of ovarian cancer based on nuclear morphology

Nuclear morphological features are potent determining factors for clinical diagnostic approaches adopted by pathologists to analyze the malignant potential of cancer cells. Considering the structural alteration of the nucleus in cancer cells, various groups have developed machine learning techniques based on variation in nuclear morphometric information like nuclear shape, size, nucleus-cytoplasm ratio and various non-parametric methods like deep learning have also been tested for analyzing immunohistochemistry images of tissue samples for diagnosing various cancers. We aim to correlate the morphometric features of the nucleus along with the distribution of nuclear lamin proteins with classical machine learning to differentiate between normal and ovarian cancer tissues. It has already been elucidated that in ovarian cancer, the extent of alteration in nuclear shape and morphology can modulate genetic changes and thus can be utilized to predict the outcome of low to a high form of serous carcinoma. In this work, we have performed exhaustive imaging of ovarian cancer versus normal tissue and developed a dual pipeline architecture that combines the matrices of morphometric parameters with deep learning techniques of auto feature extraction from pre-processed images. This novel Deep Hybrid Learning model, though derived from classical machine learning algorithms and standard CNN, showed a training and validation AUC score of 0.99 whereas the test AUC score turned out to be 1.00. The improved feature engineering enabled us to differentiate between cancerous and non-cancerous samples successfully from this pilot study.

Analysis on the Characterization of Multiphoton Microscopy Images for Malignant Neoplastic Colon Lesion Detection under Deep Learning Methods

Background:

Colorectal cancer has a high incidence rate worldwide, with over 1.8 million new cases and 880,792 deaths in 2018. Fortunately, its early detection significantly increases the survival rate, reaching a cure rate of 90% when diagnosed at a localized stage. Colonoscopy is the gold standard technique for detection and removal of colorectal lesions with potential to evolve into cancer. When polyps are found in a patient, the current procedure is their complete removal. However, in this process, gastroenterologists cannot assure complete resection and clean margins which are given by the histopathology analysis of the removed tissue, which is performed at laboratory.

Aims:

In this paper, we demonstrate the capabilities of multiphoton microscopy (MPM) technology to provide imaging biomarkers that can be extracted by deep learning techniques to identify malignant neoplastic colon lesions and distinguish them from healthy, hyperplastic, or benign neoplastic tissue, without the need for histopathological staining.

Materials and Methods:

To this end, we present a novel MPM public dataset containing 14,712 images obtained from 42 patients and grouped into 2 classes. A convolutional neural network is trained on this dataset and a spatially coherent predictions scheme is applied for performance improvement.

Results:

We obtained a sensitivity of 0.8228 ± 0.1575 and a specificity of 0.9114 ± 0.0814 on detecting malignant neoplastic lesions. We also validated this approach to estimate the self-confidence of the network on its own predictions, obtaining a mean sensitivity of 0.8697 and a mean specificity of 0.9524 with the 18.67% of the images classified as uncertain.

Conclusions:

This work lays the foundations for performing in vivo optical colon biopsies by combining this novel imaging technology together with deep learning algorithms, hence avoiding unnecessary polyp resection and allowing in situ diagnosis assessment.

Navigating the Collagen Jungle: The Biomedical Potential of Fiber Organization in Cancer

Recent research has highlighted the importance of key tumor microenvironment features, notably the collagen-rich extracellular matrix (ECM) in characterizing tumor invasion and progression. This led to great interest from both basic researchers and clinicians, including pathologists, to include collagen fiber evaluation as part of the investigation of cancer development and progression. Fibrillar collagen is the most abundant in the normal extracellular matrix, and was revealed to be upregulated in many cancers. Recent studies suggested an emerging theme across multiple cancer types in which specific collagen fiber organization patterns differ between benign and malignant tissue and also appear to be associated with disease stage, prognosis, treatment response, and other clinical features. There is great potential for developing image-based collagen fiber biomarkers for clinical applications, but its adoption in standard clinical practice is dependent on further translational and clinical evaluations. Here, we offer a comprehensive review of the current literature of fibrillar collagen structure and organization as a candidate cancer biomarker, and new perspectives on the challenges and next steps for researchers and clinicians seeking to exploit this information in biomedical research and clinical workflows.

Single image super-resolution for whole slide image using convolutional neural networks and self-supervised color normalization

High-quality whole slide scanners used for animal and human pathology scanning are expensive and can produce massive datasets, which limits the access to and adoption of this technique. As a potential solution to these challenges, we present a deep learning-based approach making use of single image super-resolution (SISR) to reconstruct high-resolution histology images from low-resolution inputs. Such low-resolution images can easily be shared, require less storage, and can be acquired quickly using widely available low-cost slide scanners. The network consists of multi-scale fully convolutional networks capable of capturing hierarchical features. Conditional generative adversarial loss is incorporated to penalize blurriness in the output images. The network is trained using a progressive strategy where the scaling factor is sampled from a normal distribution with an increasing mean. The results are evaluated with quantitative metrics and are used in a clinical histopathology diagnosis procedure which shows that the SISR framework can be used to reconstruct high-resolution images with clinical level quality. We further propose a self-supervised color normalization method that can remove staining variation artifacts. Quantitative evaluations show that the SISR framework can generalize well on unseen data collected from other patient tissue cohorts by incorporating the color normalization method.

Artificial Intelligence to Decode Cancer Mechanism: Beyond Patient Stratification for Precision Oncology

The multitude of multi-omics data generated cost-effectively using advanced high-throughput technologies has imposed challenging domain for research in Artificial Intelligence (AI). Data curation poses a significant challenge as different parameters, instruments, and sample preparations approaches are employed for generating these big data sets. AI could reduce the fuzziness and randomness in data handling and build a platform for the data ecosystem, and thus serve as the primary choice for data mining and big data analysis to make informed decisions. However, AI implication remains intricate for researchers/clinicians lacking specific training in computational tools and informatics. Cancer is a major cause of death worldwide, accounting for an estimated 9.6 million deaths in 2018. Certain cancers, such as pancreatic and gastric cancers, are detected only after they have reached their advanced stages with frequent relapses. Cancer is one of the most complex diseases affecting a range of organs with diverse disease progression mechanisms and the effectors ranging from gene-epigenetics to a wide array of metabolites. Hence a comprehensive study, including genomics, epi-genomics, transcriptomics, proteomics, and metabolomics, along with the medical/mass-spectrometry imaging, patient clinical history, treatments provided, genetics, and disease endemicity, is essential. Cancer Moonshot℠ Research Initiatives by NIH National Cancer Institute aims to collect as much information as possible from different regions of the world and make a cancer data repository. AI could play an immense role in (a) analysis of complex and heterogeneous data sets (multi-omics and/or inter-omics), (b) data integration to provide a holistic disease molecular mechanism, (c) identification of diagnostic and prognostic markers, and (d) monitor patient's response to drugs/treatments and recovery. AI enables precision disease management well beyond the prevalent disease stratification patterns, such as differential expression and supervised classification. This review highlights critical advances and challenges in omics data analysis, dealing with data variability from lab-to-lab, and data integration. We also describe methods used in data mining and AI methods to obtain robust results for precision medicine from "big" data. In the future, AI could be expanded to achieve ground-breaking progress in disease management.

Non-disruptive collagen characterization in clinical histopathology using cross-modality image synthesis

The importance of fibrillar collagen topology and organization in disease progression and prognostication in different types of cancer has been characterized extensively in many research studies. These explorations have either used specialized imaging approaches, such as specific stains (e.g., picrosirius red), or advanced and costly imaging modalities (e.g., second harmonic generation imaging (SHG)) that are not currently in the clinical workflow. To facilitate the analysis of stromal biomarkers in clinical workflows, it would be ideal to have technical approaches that can characterize fibrillar collagen on standard H&E stained slides produced during routine diagnostic work. Here, we present a machine learning-based stromal collagen image synthesis algorithm that can be incorporated into existing H&E-based histopathology workflow. Specifically, this solution applies a convolutional neural network (CNN) directly onto clinically standard H&E bright field images to extract information about collagen fiber arrangement and alignment, without requiring additional specialized imaging stains, systems or equipment.

Epithelium segmentation and automated Gleason grading of prostate cancer via deep learning in label-free multiphoton microscopic images

In the current clinical care practice, Gleason grading system is one of the most powerful prognostic predictors for prostate cancer (PCa). The grading system is based on the architectural pattern of cancerous epithelium in histological images. However, the standard procedure of histological examination often involves complicated tissue fixation and staining, which are time-consuming and may delay the diagnosis and surgery. In this study, label-free multiphoton microscopy (MPM) was used to acquire subcellular-resolution images of unstained prostate tissues. Then, a deep learning architecture (U-net) was introduced for epithelium segmentation of prostate tissues in MPM images. The obtained segmentation results were then merged with the original MPM images to train a classification network (AlexNet) for automated Gleason grading. The developed method achieved an overall pixel accuracy of 92.3% with a mean F1 score of 0.839 for epithelium segmentation. By merging the segmentation results with the MPM images, the accuracy of Gleason grading was improved from 72.42% to 81.13% in hold-out test set. Our results suggest that MPM in combination with deep learning holds the potential to be used as a fast and powerful clinical tool for PCa diagnosis.

Multiphoton microscopy of the dermoepidermal junction and automated identification of dysplastic tissues with deep learning

Histopathological image analysis performed by a trained expert is currently regarded as the gold-standard for the diagnostics of many pathologies, including cancers. However, such approaches are laborious, time consuming and contain a risk for bias or human error. There is thus a clear need for faster, less intrusive and more accurate diagnostic solutions, requiring also minimal human intervention. Multiphoton microscopy (MPM) can alleviate some of the drawbacks specific to traditional histopathology by exploiting various endogenous optical signals to provide virtual biopsies that reflect the architecture and composition of tissues, both in-vivo or ex-vivo. Here we show that MPM imaging of the dermoepidermal junction (DEJ) in unstained fixed tissues provides useful cues for a histopathologist to identify the onset of non-melanoma skin cancers. Furthermore, we show that MPM images collected on the DEJ, besides being easy to interpret by a trained specialist, can be automatically classified into healthy and dysplastic classes with high precision using a Deep Learning method and existing pre-trained convolutional neural networks. Our results suggest that deep learning enhanced MPM for in-vivo skin cancer screening could facilitate timely diagnosis and intervention, enabling thus more optimal therapeutic approaches.

Automated label-free detection of injured neuron with deep learning by two-photon microscopy

Stroke is a significant cause of morbidity and long-term disability globally. Detection of injured neuron is a prerequisite for defining the degree of focal ischemic brain injury, which can be used to guide further therapy. Here, we demonstrate the capability of two-photon microscopy (TPM) to label-freely identify injured neurons on unstained thin section and fresh tissue of rat cerebral ischemia-reperfusion model, revealing definite diagnostic features compared with conventional staining images. Moreover, a deep learning model based on convolutional neural network is developed to automatically detect the location of injured neurons on TPM images. We then apply deep learning-assisted TPM to evaluate the ischemic regions based on tissue edema, two-photon excited fluorescence signal intensity, as well as neuronal injury, presenting a novel manner for identifying the infarct core, peri-infarct area, and remote area. These results propose an automated and label-free method that could provide supplementary information to augment the diagnostic accuracy, as well as hold the potential to be used as an intravital diagnostic tool for evaluating the effectiveness of drug interventions and predicting potential therapeutics.

Label-Free Identification of Early Stages of Breast Ductal Carcinoma via Multiphoton Microscopy

Breast cancer can be cured by early diagnosis. Appropriate and effective clinical treatment benefits from accurate pathological diagnosis. However, due to the lack of effective screening and diagnostic imaging methods, early stages of breast cancer often progress to malignant breast cancer. In this study, multiphoton microscopy (MPM) via two-photon excited fluorescence combined with second-harmonic generation was used for identifying the early stages of breast ductal carcinoma. The results showed differences in both cytological features and collagen distribution among normal breast tissue, atypical ductal hyperplasia, low-grade ductal carcinoma in situ, and high-grade ductal carcinoma in situ with microinvasion. Furthermore, three features extracted from the MPM images were used to describe differences in cytological features, collagen density, and basement membrane circumference in the early stages of breast ductal carcinoma. They revealed that MPM has the ability to identify early stages of breast ductal carcinoma label-free, which would contribute to the early diagnosis and treatment of breast cancer. This study may provide the groundwork for the further application of MPM in the clinic.

Multiphoton microscopy in surgical oncology- a systematic review and guide for clinical translatability

BACKGROUND

Multiphoton microscopy (MPM) facilitates three-dimensional, high-resolution functional imaging of unlabeled tissues in vivo and ex vivo. This systematic review discusses the diagnostic value, advantages and challenges in the practical use of MPM in surgical oncology.

METHOD AND FINDINGS

A Medline search was conducted in April 2019. Fifty-three original research papers investigating MPM compared to standard histology in human patients with solid tumors were identified. A qualitative synopsis and meta-analysis of 14 blinded studies was performed. Risk of bias and applicability were evaluated. MPM can image fresh, frozen or fixed tissues up to a depth 1000 μm in the z-plane. Best results including functional imaging and virtual histochemistry are obtained by in vivo imaging or scanning fresh tissue immediately after excision. Two-photon excited fluorescence by natural fluorophores of the cytoplasm and second harmonic generation signals by fluorophores of the extracellular matrix can be scanned simultaneously, providing high resolution optical histochemistry comparable to standard histology. Functional parameters like fluorescence lifetime imaging or optical redox ratio provide additional objective information. A major concern is inability to visualize the nucleus. However, in a subpopulation analysis of 440 specimens, MPM yielded a sensitivity of 94%, specificity of 96% and accuracy of 95% for the detection of malignant tissue.

CONCLUSION

MPM is a promising emerging technique in surgical oncology. Ex vivo imaging has high sensitivity, specificity and accuracy for the detection of tumor cells. For broad clinical application in vivo, technical challenges need to be resolved.

Metformin Abrogates Age-Associated Ovarian Fibrosis

PURPOSE

The ovarian cancer risk factors of age and ovulation are curious because ovarian cancer incidence increases in postmenopausal women, long after ovulations have ceased. To determine how age and ovulation underlie ovarian cancer risk, we assessed the effects of these risk factors on the ovarian microenvironment.

EXPERIMENTAL DESIGN

Aged C57/lcrfa mice (0-33 months old) were generated to assess the aged ovarian microenvironment. To expand our findings into human aging, we assembled a cohort of normal human ovaries (n = 18, 21-71 years old). To validate our findings, an independent cohort of normal human ovaries was assembled (n = 9, 41-82 years old).

RESULTS

We first validated the presence of age-associated murine ovarian fibrosis. Using interdisciplinary methodologies, we provide novel evidence that ovarian fibrosis also develops in human postmenopausal ovaries across two independent cohorts (n = 27). Fibrotic ovaries have an increased CD206⁺:CD68⁺ cell ratio, CD8⁺ T-cell infiltration, and profibrotic DPP4⁺αSMA⁺ fibroblasts. Metformin use was associated with attenuated CD8⁺ T-cell infiltration and reduced CD206⁺:CD68⁺ cell ratio.

CONCLUSIONS

These data support a novel hypothesis that unifies the primary nonhereditary ovarian cancer risk factors through the development of ovarian fibrosis and the formation of a premetastatic niche, and suggests a potential use for metformin in ovarian cancer prophylaxis.See related commentary by Madariaga et al., p. 523.

Automated classification of hepatocellular carcinoma differentiation using multiphoton microscopy and deep learning

In the case of hepatocellular carcinoma (HCC) samples, classification of differentiation is crucial for determining prognosis and treatment strategy decisions. However, a label-free and automated classification system for HCC grading has not been yet developed. Hence, in this study, we demonstrate the fusion of multiphoton microscopy and a deep-learning algorithm for classifying HCC differentiation to produce an innovative computer-aided diagnostic method. Convolutional neural networks based on the VGG-16 framework were trained using 217 combined two-photon excitation fluorescence and second-harmonic generation images; the resulting classification accuracy of the HCC differentiation grade was over 90%. Our results suggest that a combination of multiphoton microscopy and deep learning can realize label-free, automated methods for various tissues, diseases and other related classification problems.

Label-free optical hemogram of granulocytes enhanced by artificial neural networks

An outstanding challenge for immunology is the classification of immune cells in a label-free fashion with high speed. For this purpose, optical techniques such as Raman spectroscopy or digital holographic microscopy have been used successfully to identify immune cell subsets. To achieve high accuracy, these techniques require a post-processing step using linear methods of multivariate processing, such as principal component analysis. Here we demonstrate for the first time a comparison between artificial neural networks and principal component analysis (PCA) to classify the key granulocyte cell lineages of neutrophils and eosinophils using both digital holographic microscopy and Raman spectroscopy. Artificial neural networks can offer advantages in terms of classification accuracy and speed over a PCA approach. We conclude that digital holographic microscopy with convolutional neural networks based analysis provides a route to a robust, stand-alone and high-throughput hemogram with a classification accuracy of 91.3 % at a throughput rate of greater than 100 cells per second.

Multiphoton microscopy in the diagnostic assessment of pediatric solid tissue in comparison to conventional histopathology: results of the first international online interobserver trial

Purpose: Clear resection margins are paramount for good outcome in children undergoing solid tumor resections. Multiphoton microscopy (MPM) can provide high-resolution, real-time, intraoperative microscopic images of tumor tissue. Objective: This prospective international multicenter study evaluates the diagnostic accuracy, feasibility, and interobserver congruence of MPM in diagnosing solid pediatric tissue and tumors for the first time. Material and methods: Representative fresh sections from six different neonatal solid tissues (liver, lung, kidney, adrenal gland, heart muscle, testicle) and two types of typical pediatric solid tumors (neuroblastoma, rhabdomyosarcoma) with adjacent nonneoplastic tissue were imaged with MPM and then presented online with corresponding H&E stained slides of the exact same tissue region. Both image sets of each tissue type were interpreted by 38 randomly selected international attending pediatric pathologists via an online evaluation software. Results: The quality of MPM was sufficient to make the diagnosis of all normal tissue types except cardiac muscle in >94% of assessors with high interobserver congruence and 95% sensitivity. Heart muscle was interpreted as skeletal muscle in 55% of cases. Based on MPM imaging, participating pathologists diagnosed the presented pediatric neoplasms with 100% specificity, although the sensitivity reached only about 50%. Conclusion: Even without prior training, pathologists are able to diagnose normal pediatric tissues with valuable accuracy using MPM. While current MPM imaging protocols are not yet sensitive enough to reliably rule out neuroblastoma or rhabdomyosarcoma, they seem to be specific and therefore useful to confirm a diagnosis intraoperatively. We are confident that improved algorithms, specific training, and more experience with the method will make MPM a valuable future alternative to frozen section analysis. Registration: The trial was registered at www.researchregistry.com, registration number 2967.