Deep learning automated pathology in ex vivo microscopy

Breast tissue imaging atlas using ultra-fast confocal microscopy to identify cancer lesions

New generation ultra-fast fluorescence confocal microscopy (UFCM) allows to image histological architecture of fresh breast tissue and may be used for ex vivo intraoperative analysis for margin status. The criteria to identify breast tumoral and non-tumoral tissues in UFCM images are still objects of investigation. The objective of the study was to create an atlas of ex vivo UFCM images of breast tissues and breast carcinomas based on the first extensive collection of large field-of-view UFCM breast images. One hundred sixty patients who underwent conserving surgery for breast cancer were included. Their fresh surgical specimens were sliced, stained with acridine orange, and imaged at high resolution with large-field-of-view UFCM. The resulting images were digitally false colored to resemble frozen sections. Each UFCM image was correlated with the corresponding definitive histology. Representative images of normal tissue, inflammation, benign lesions, invasive carcinoma (IC), and ductal carcinoma in situ (DCIS) were collected. A total of 320 large-field images were recorded from 58 IC of no special type, 44 invasive lobular carcinomas, 1 invasive mucinous carcinoma, 47 DCIS, 2 lobular carcinomas in situ, and 8 specimens without cancer. Representative images of the main components of the normal breast and the main types of ICs and DCIS were annotated to establish an UFCM atlas. UFCM enables the imaging of the fresh breast tissue sections. Main morphological criteria defined in traditional histopathology such as tissue architecture and cell features can be applied to describe UFCM images content. The generated atlas of the main normal or tumoral tissue features will support the adoption of this optical technology for the intraoperative examination of breast specimens in clinical practice as it can be used to train physicians on UFCM images and develop artificial intelligence algorithms. Further studies are needed to document rare breast lesions.

High-resolution full-field optical coherence tomography microscope for the evaluation of freshly excised skin specimens during Mohs surgery: A feasibility study

Histopathology for tumor margin assessment is time-consuming and expensive. High-resolution full-field optical coherence tomography (FF-OCT) images fresh tissues rapidly at cellular resolution and potentially facilitates evaluation. Here, we define FF-OCT features of normal and neoplastic skin lesions in fresh ex vivo tissues and assess its diagnostic accuracy for malignancies. For this, normal and neoplastic tissues were obtained from Mohs surgery, imaged using FF-OCT, and their features were described. Two expert OCT readers conducted a blinded analysis to evaluate their diagnostic accuracies, using histopathology as the ground truth. A convolutional neural network was built to distinguish and outline normal structures and tumors. Of the 113 tissues imaged, 95 (84%) had a tumor (75 basal cell carcinomas [BCCs] and 17 squamous cell carcinomas [SCCs]). The average reader diagnostic accuracy was 88.1%, with a sensitivity of 93.7%, and a specificity of 58.3%. The artificial intelligence (AI) model achieved a diagnostic accuracy of 87.6 ± 5.9%, sensitivity of 93.2 ± 2.1%, and specificity of 81.2 ± 9.2%. A mean intersection-over-union of 60.3 ± 10.1% was achieved when delineating the nodular BCC from normal structures. Limitation of the study was the small sample size for all tumors, especially SCCs. However, based on our preliminary results, we envision FF-OCT to rapidly image fresh tissues, facilitating surgical margin assessment. AI algorithms can aid in automated tumor detection, enabling widespread adoption of this technique.

Harnessing the power of optical microscopy for visualization and analysis of histopathological images

Histopathology is the foundation and gold standard for identifying diseases, and precise quantification of histopathological images can provide the pathologist with objective clues to make a more convincing diagnosis. Optical microscopy (OM), an important branch of optical imaging technology that provides high-resolution images of tissue cytology and structural morphology, has been used in the diagnosis of histopathology and evolved into a new disciplinary direction of optical microscopic histopathology (OMH). There are a number of ex-vivo studies providing applicability of different OMH approaches, and a transfer of these techniques toward in vivo diagnosis is currently in progress. Furthermore, combined with advanced artificial intelligence algorithms, OMH allows for improved diagnostic reliability and convenience due to the complementarity of retrieval information. In this review, we cover recent advances in OMH, including the exploration of new techniques in OMH as well as their applications, and look ahead to new challenges in OMH. These typical application examples well demonstrate the application potential and clinical value of OMH techniques in histopathological diagnosis.

Novel method for predicting nonvisible symptoms using machine learning in cancer palliative care

End-of-life patients with cancer may find expressing their symptoms difficult if they can no longer communicate verbally because of deteriorating health. In this study, we assessed these symptoms using machine learning, which has excellent predictive capabilities and has recently been applied in healthcare. We performed a retrospective clinical survey involving 213 patients with cancer from August 2015 to August 2016. We divided the reported symptoms into two groups-visible and nonvisible symptoms. We used decision tree analysis, an analytical machine learning method that organizes and analyzes information in the form of a tree diagram to visually represent the information structure. Our machine learning model used patient background data and visible symptoms to predict nonvisible symptoms: pain, dyspnea, fatigue, drowsiness, anxiety, delirium, inadequate informed consent, and spiritual issues. The highest and/or lowest values for prediction accuracy, sensitivity, and specificity were 88.0%/55.5%, 84.9%/3.3%, and 96.7%/24.1%, respectively. This work will facilitate better assessment and management of symptoms in patients with cancer. This study was the first to predict nonvisible symptoms using decision tree analyses for patients with cancer receiving palliative care. Notably, applications based on our results may assess symptoms to the same extent as healthcare professionals.

Artificial Intelligence-Aided Optical Imaging for Cancer Theranostics

The use of artificial intelligence (AI) to assist biomedical imaging have demonstrated its high accuracy and high efficiency in medical decision-making for individualized cancer medicine. In particular, optical imaging methods are able to visualize both the structural and functional information of tumors tissues with high contrast, low cost, and noninvasive property. However, no systematic work has been performed to inspect the recent advances on AI-aided optical imaging for cancer theranostics. In this review, we demonstrated how AI can guide optical imaging methods to improve the accuracy on tumor detection, automated analysis and prediction of its histopathological section, its monitoring during treatment, and its prognosis by using computer vision, deep learning and natural language processing. By contrast, the optical imaging techniques involved mainly consisted of various tomography and microscopy imaging methods such as optical endoscopy imaging, optical coherence tomography, photoacoustic imaging, diffuse optical tomography, optical microscopy imaging, Raman imaging, and fluorescent imaging. Meanwhile, existing problems, possible challenges and future prospects for AI-aided optical imaging protocol for cancer theranostics were also discussed. It is expected that the present work can open a new avenue for precision oncology by using AI and optical imaging tools.

Confocal Microscopy for Diagnosis and Management of Cutaneous Malignancies: Clinical Impacts and Innovation

Cutaneous malignancies are common malignancies worldwide, with rising incidence. Most skin cancers, including melanoma, can be cured if diagnosed correctly at an early stage. Thus, millions of biopsies are performed annually, posing a major economic burden. Non-invasive skin imaging techniques can aid in early diagnosis and save unnecessary benign biopsies. In this review article, we will discuss in vivo and ex vivo confocal microscopy (CM) techniques that are currently being utilized in dermatology clinics for skin cancer diagnosis. We will discuss their current applications and clinical impact. Additionally, we will provide a comprehensive review of the advances in the field of CM, including multi-modal approaches, the integration of fluorescent targeted dyes, and the role of artificial intelligence for improved diagnosis and management.

Deep learning on reflectance confocal microscopy improves Raman spectral diagnosis of basal cell carcinoma

SIGNIFICANCE

Raman spectroscopy (RS) provides an automated approach for assisting Mohs micrographic surgery for skin cancer diagnosis; however, the specificity of RS is limited by the high spectral similarity between tumors and normal tissues structures. Reflectance confocal microscopy (RCM) provides morphological and cytological details by which many features of epidermis and hair follicles can be readily identified. Combining RS with deep-learning-aided RCM has the potential to improve the diagnostic accuracy of RS in an automated fashion, without requiring additional input from the clinician.

AIM

The aim of this study is to improve the specificity of RS for detecting basal cell carcinoma (BCC) using an artificial neural network trained on RCM images to identify false positive normal skin structures (hair follicles and epidermis).

APPROACH

Our approach was to build a two-step classification model. In the first step, a Raman biophysical model that was used in prior work classified BCC tumors from normal tissue structures with high sensitivity. In the second step, 191 RCM images were collected from the same site as the Raman data and served as inputs for two ResNet50 networks. The networks selected the hair structure and epidermis images, respectively, within all images corresponding to the positive predictions of the Raman biophysical model with high specificity. The specificity of the BCC biophysical model was improved by moving the Raman spectra corresponding to these selected images from false positive to true negative.

RESULTS

Deep-learning trained on RCM images removed 52% of false positive predictions from the Raman biophysical model result while maintaining a sensitivity of 100%. The specificity was improved from 84.2% using Raman spectra alone to 92.4% by integrating Raman spectra with RCM images.

CONCLUSIONS

Combining RS with deep-learning-aided RCM imaging is a promising tool for guiding tumor resection surgery.

Machine Learning Based Prediction of Squamous Cell Carcinoma in Ex Vivo Confocal Laser Scanning Microscopy

Image classification with convolutional neural networks (CNN) offers an unprecedented opportunity to medical imaging. Regulatory agencies in the USA and Europe have already cleared numerous deep learning/machine learning based medical devices and algorithms. While the field of radiology is on the forefront of artificial intelligence (AI) revolution, conventional pathology, which commonly relies on examination of tissue samples on a glass slide, is falling behind in leveraging this technology. On the other hand, ex vivo confocal laser scanning microscopy (ex vivo CLSM), owing to its digital workflow features, has a high potential to benefit from integrating AI tools into the assessment and decision-making process. Aim of this work was to explore a preliminary application of CNN in digitally stained ex vivo CLSM images of cutaneous squamous cell carcinoma (cSCC) for automated detection of tumor tissue. Thirty-four freshly excised tissue samples were prospectively collected and examined immediately after resection. After the histologically confirmed ex vivo CLSM diagnosis, the tumor tissue was annotated for segmentation by experts, in order to train the MobileNet CNN. The model was then trained and evaluated using cross validation. The overall sensitivity and specificity of the deep neural network for detecting cSCC and tumor free areas on ex vivo CLSM slides compared to expert evaluation were 0.76 and 0.91, respectively. The area under the ROC curve was equal to 0.90 and the area under the precision-recall curve was 0.85. The results demonstrate a high potential of deep learning models to detect cSCC regions on digitally stained ex vivo CLSM slides and to distinguish them from tumor-free skin.

[Artificial intelligence in ex vivo confocal laser scanning microscopy]

BACKGROUND

Visual data, such as clinical photographs or pictures from imaging examination methods, such as ex vivo confocal laser scanning microscopy (CLSM), are particularly suitable for machine learning techniques.

OBJECTIVES

The aim was to find out whether data have already been published on this innovative application in ex vivo CLSM and what potential challenges and limitations could arise.

MATERIAL AND METHODS

Review of the literature and summary of current knowledge and personal experience on the use of artificial intelligence (AI) in ex vivo CLSM.

RESULTS

Successful integration of digital hematoxylin-eosin-like staining has made ex vivo CLSM significantly more accessible for digital assessments. Several machine learning techniques have been developed to date in such a way that they have been able to identify malignant skin lesions on clinical photographs and pathological microscopic images with similar accuracy compared to experts, or even find visual patterns that have been overlooked by experts and that correlate with certain dermatological diseases. One study on the use of AI in ex vivo CLSM for automated tumor detection has been published to date. Several challenges and limitations can arise when using AI in ex vivo CLSM.

CONCLUSIONS

The already digitized ex vivo CLSM, which was established for rapid section examination purposes, is a predestined method for the development and use of new applications with machine learning in the healthcare sector. The results of further studies on this topic are anticipated with great hope.