A pulse coupled neural network segmentation algorithm for reflectance confocal images of epithelial tissue

Age-dependent changes in epidermal architecture explored using an automated image analysis algorithm on in vivo reflectance confocal microscopy images

BACKGROUND

Reflectance confocal microscopy (RCM) allows for real-time in vivo visualization of the epidermis at the cellular level noninvasively. Parameters relating to tissue architecture can be extracted from RCM images, however, analysis of such images requires manual identification of cells to derive these parameters, which can be time-consuming and subject to human error, highlighting the need for an automated cell identification method.

METHODS

First, the region-of-interest (ROI) containing cells needs to be identified, followed by the identification of individual cells within the ROI. To perform this task, we use successive applications of Sato and Gabor filters. The final step is post-processing improvement of cell detection and removal of size outliers. The proposed algorithm is evaluated on manually annotated real data. It is then applied to 5345 images to study the evolution of epidermal architecture in children and adults. The images were acquired on the volar forearm of healthy children (3 months to 10 years) and women (25-80 years), and on the volar forearm and cheek of women (40-80 years). Following the identification of cell locations, parameters such as cell area, cell perimeter, and cell density are calculated, as well as the probability distribution of the number of nearest neighbors per cell. The thicknesses of the Stratum Corneum and supra-papillary epidermis are also calculated using a hybrid deep-learning method.

RESULTS

Epidermal keratinocytes are significantly larger (area and perimeter) in the granular layer than in the spinous layer and they get progressively larger with a child's age. Skin continues to mature dynamically during adulthood, as keratinocyte size continues to increase with age on both the cheeks and volar forearm, but the topology and cell aspect ratio remain unchanged across different epidermal layers, body sites, and age. Stratum Corneum and supra-papillary epidermis thicknesses increase with age, at a faster rate in children than in adults.

CONCLUSIONS

The proposed methodology can be applied to large datasets to automate image analysis and the calculation of parameters relevant to skin physiology. These data validate the dynamic nature of skin maturation during childhood and skin aging in adulthood.

Automatic granular and spinous epidermal cell identification and analysis on in vivo reflectance confocal microscopy images using cell morphological features

Significance

Reflectance confocal microscopy (RCM) allows for real-time in vivo visualization of the skin at the cellular level. The study of RCM images provides information on the structural properties of the epidermis. These may change in each layer of the epidermis, depending on the subject's age and the presence of certain dermatological conditions. Studying RCM images requires manual identification of cells to derive these properties, which is time consuming and subject to human error, highlighting the need for an automated cell identification method.

Aim

We aim to design an automated pipeline for the analysis of the structure of the epidermis from RCM images of the Stratum granulosum and Stratum spinosum.

Approach

We identified the region of interest containing the epidermal cells and the individual cells in the segmented tissue area using tubeness filters to highlight membranes. We used prior biological knowledge on cell size to process the resulting detected cells, removing cells that were too small and reapplying the used filters locally on detected regions that were too big to be considered a single cell. The proposed full image analysis pipeline (FIAP) was compared with machine learning-based approaches (cell cutter, different U-Net configurations, and loss functions).

Results

All methods were evaluated both on simulated data (four images) and on manually annotated RCM data (seven images). Accuracy was measured using recall and precision metrics. Both accuracy metrics were higher in the proposed FIAP for both real ( precision = 0.720 ± 0.068 , recall = 0.850 ± 0.11 ) and synthetic images ( precision = 0.835 ± 0.067 , recall = 0.925 ± 0.012 ). The tested machine learning methods failed to identify and segment keratinocytes on RCM images with a satisfactory accuracy.

Conclusions

We showed that automatic cell segmentation can be achieved using a pipeline based on membrane detection, with an accuracy that matches expert manual cell identification. To our knowledge, this is the first method based on membrane detection to study healthy skin using RCM images evaluated against manually identified cell positions.

Automating reflectance confocal microscopy image analysis for dermatological research: a review

SIGNIFICANCE

Reflectance confocal microscopy (RCM) is a noninvasive, in vivo technology that offers near histopathological resolution at the cellular level. It is useful in the study of phenomena for which obtaining a biopsy is impractical or would cause unnecessary tissue damage and trauma to the patient.

AIM

This review covers the use of RCM in the study of skin and the use of machine learning to automate information extraction. It has two goals: (1) an overview of information provided by RCM on skin structure and how it changes over time in response to stimuli and in disease and (2) an overview of machine learning approaches developed to automate the extraction of key morphological features from RCM images.

APPROACH

A PubMed search was conducted with additional literature obtained from references lists.

RESULTS

The application of RCM as an in vivo tool in dermatological research and the biologically relevant information derived from it are presented. Algorithms for image classification to epidermal layers, delineation of the dermal-epidermal junction, classification of skin lesions, and demarcation of individual cells within an image, all important factors in the makeup of the skin barrier, were reviewed. Application of image analysis methods in RCM is hindered by low image quality due to noise and/or poor contrast. Use of supervised machine learning is limited by time-consuming manual labeling of RCM images.

CONCLUSIONS

RCM has great potential in the study of skin structures. The use of artificial intelligence could enable an easier, more reproducible, precise, and rigorous study of RCM images for the understanding of skin structures, skin barrier, and skin inflammation and lesions. Although several attempts have been made, further work is still needed to provide a definite gold standard and overcome issues related to image quality, limited labeled datasets, and lack of phenotype variability in available databases.

How to Analyze the Neurodynamic Characteristics of Pulse-Coupled Neural Networks? A Theoretical Analysis and Case Study of Intersecting Cortical Model

The intersecting cortical model (ICM), initially designed for image processing, is a special case of the biologically inspired pulse-coupled neural-network (PCNN) models. Although the ICM has been widely used, few studies concern the internal activities and firing conditions of the neuron, which may lead to an invalid model in the application. Furthermore, the lack of theoretical analysis has led to inappropriate parameter settings and consequent limitations on ICM applications. To address this deficiency, we first study the continuous firing condition of ICM neurons to determine the restrictions that exist between network parameters and the input signal. Second, we investigate the neuron pulse period to understand the neural firing mechanism. Third, we derive the relationship between the continuous firing condition and the neural pulse period, and the relationship can prove the validity of the continuous firing condition and the neural pulse period as well. A solid understanding of the neural firing mechanism is helpful in setting appropriate parameters and in providing a theoretical basis for widespread applications to use the ICM model effectively. Extensive experiments of numerical tests with a common image reveal the rationality of our theoretical results.

Automated mesenchymal stem cell segmentation and machine learning-based phenotype classification using morphometric and textural analysis

Purpose: Mesenchymal stem cells (MSCs) have demonstrated clinically relevant therapeutic effects for treatment of trauma and chronic diseases. The proliferative potential, immunomodulatory characteristics, and multipotentiality of MSCs in monolayer culture is reflected by their morphological phenotype. Standard techniques to evaluate culture viability are subjective, destructive, or time-consuming. We present an image analysis approach to objectively determine morphological phenotype of MSCs for prediction of culture efficacy. Approach: The algorithm was trained using phase-contrast micrographs acquired during the early and mid-logarithmic stages of MSC expansion. Cell regions are localized using edge detection, thresholding, and morphological operations, followed by cell marker identification using H-minima transform within each region to differentiate individual cells from cell clusters. Clusters are segmented using marker-controlled watershed to obtain single cells. Morphometric and textural features are extracted to classify cells based on phenotype using machine learning. Results: Algorithm performance was validated using an independent test dataset of 186 MSCs in 36 culture images. Results show 88% sensitivity and 86% precision for overall cell detection and a mean Sorensen-Dice coefficient of 0.849 ± 0.106 for segmentation per image. The algorithm exhibited an area under the curve of 0.816 (CI 95 = 0.769 to 0.886) and 0.787 (CI 95 = 0.716 to 0.851) for classifying MSCs according to their phenotype at early and mid-logarithmic expansion, respectively. Conclusions: The proposed method shows potential to segment and classify low and moderately dense MSCs based on phenotype with high accuracy and robustness. It enables quantifiable and consistent morphology-based quality assessment for various culture protocols to facilitate cytotherapy development.

Algorithm to quantify nuclear features and confidence intervals for classification of oral neoplasia from high-resolution optical images

Purpose: In vivo optical imaging technologies like high-resolution microendoscopy (HRME) can image nuclei of the oral epithelium. In principle, automated algorithms can then calculate nuclear features to distinguish neoplastic from benign tissue. However, images frequently contain regions without visible nuclei, due to biological and technical factors, decreasing the data available to and accuracy of image analysis algorithms. Approach: We developed the nuclear density-confidence interval (ND-CI) algorithm to determine if an HRME image contains sufficient nuclei for classification, or if a better image is required. The algorithm uses a convolutional neural network to exclude image regions without visible nuclei. Then the remaining regions are used to estimate a confidence interval (CI) for the number of abnormal nuclei per   mm 2 , a feature used by a previously developed algorithm (called the ND algorithm), to classify images as benign or neoplastic. The range of the CI determines whether the ND-CI algorithm can classify an image with confidence, and if so, the predicted category. The ND and ND-CI algorithm were compared by calculating their positive predictive value (PPV) and negative predictive value (NPV) on 82 oral biopsies with histopathologically confirmed diagnoses. Results: After excluding the images that could not be classified with confidence, the ND-CI algorithm had higher PPV (65% versus 59%) and NPV (78% versus 75%) than the ND algorithm. Conclusions: The ND-CI algorithm could improve the real-time classification of HRME images of the oral epithelium by informing the user if an improved image is required for diagnosis.

A Kinect-Based Segmentation of Touching-Pigs for Real-Time Monitoring

Segmenting touching-pigs in real-time is an important issue for surveillance cameras intended for the 24-h tracking of individual pigs. However, methods to do so have not yet been reported. We particularly focus on the segmentation of touching-pigs in a crowded pig room with low-contrast images obtained using a Kinect depth sensor. We reduce the execution time by combining object detection techniques based on a convolutional neural network (CNN) with image processing techniques instead of applying time-consuming operations, such as optimization-based segmentation. We first apply the fastest CNN-based object detection technique (i.e., You Only Look Once, YOLO) to solve the separation problem for touching-pigs. If the quality of the YOLO output is not satisfied, then we try to find the possible boundary line between the touching-pigs by analyzing the shape. Our experimental results show that this method is effective to separate touching-pigs in terms of both accuracy (i.e., 91.96%) and execution time (i.e., real-time execution), even with low-contrast images obtained using a Kinect depth sensor.

Noninvasive diagnostic adjuncts for the evaluation of potentially premalignant oral epithelial lesions: current limitations and future directions

Potentially premalignant oral epithelial lesions (PPOELs) are a group of clinically suspicious conditions, of which a small percentage will undergo malignant transformation. PPOELs are suboptimally diagnosed and managed under the current standard of care. Dysplasia is the most well-established marker to distinguish high-risk PPOELs from low-risk PPOELs, and performing a biopsy to establish dysplasia is the diagnostic gold standard. However, a biopsy is limited by morbidity, resource requirements, and the potential for underdiagnosis. Diagnostic adjuncts may help clinicians better evaluate PPOELs before definitive biopsy, but existing adjuncts, such as toluidine blue, acetowhitening, and autofluorescence imaging, have poor accuracy and are not generally recommended. Recently, in vivo microscopy technologies, such as high-resolution microendoscopy, optical coherence tomography, reflectance confocal microscopy, and multiphoton imaging, have shown promise for improving PPOEL patient care. These technologies allow clinicians to visualize many of the same microscopic features used for histopathologic assessment at the point of care.

Wavelet-based statistical classification of skin images acquired with reflectance confocal microscopy

Detecting skin lentigo in reflectance confocal microscopy images is an important and challenging problem. This imaging modality has not yet been widely investigated for this problem and there are a few automatic processing techniques. They are mostly based on machine learning approaches and rely on numerous classical image features that lead to high computational costs given the very large resolution of these images. This paper presents a detection method with very low computational complexity that is able to identify the skin depth at which the lentigo can be detected. The proposed method performs multiresolution decomposition of the image obtained at each skin depth. The distribution of image pixels at a given depth can be approximated accurately by a generalized Gaussian distribution whose parameters depend on the decomposition scale, resulting in a very-low-dimension parameter space. SVM classifiers are then investigated to classify the scale parameter of this distribution allowing real-time detection of lentigo. The method is applied to 45 healthy and lentigo patients from a clinical study, where sensitivity of 81.4% and specificity of 83.3% are achieved. Our results show that lentigo is identifiable at depths between 50μm and 60μm, corresponding to the average location of the the dermoepidermal junction. This result is in agreement with the clinical practices that characterize the lentigo by assessing the disorganization of the dermoepidermal junction.

Handheld tunable focus confocal microscope utilizing a double-clad fiber coupler for in vivo imaging of oral epithelium

A reflectance confocal endomicroscope with double-clad fiber coupler and electrically tunable focus lens is applied to imaging of the oral mucosa. The instrument is designed to be lightweight and robust for clinical use. The tunable lens allows axial scanning through > 250 ?? ? m in the epithelium when the probe tip is placed in contact with tissue. Images are acquired at 6.6 frames per second with a field of view diameter up to 850 ?? ? m . In vivo imaging of a wide range of normal sites in the oral cavity demonstrates the accessibility of the handheld probe. In vivo imaging of clinical lesions diagnosed as inflammation and dysplasia illustrates the ability of reflectance confocal endomicroscopy to image cellular changes associated with pathology.

A novel multimodal optical imaging system for early detection of oral cancer

OBJECTIVES

Several imaging techniques have been advocated as clinical adjuncts to improve identification of suspicious oral lesions. However, these have not yet shown superior sensitivity or specificity over conventional oral examination techniques. We developed a multimodal, multi-scale optical imaging system that combines macroscopic biochemical imaging of fluorescence lifetime imaging with subcellular morphologic imaging of reflectance confocal microscopy for early detection of oral cancer. We tested our system on excised human oral tissues.

STUDY DESIGN

In total, 4 tissue specimens were imaged. These specimens were diagnosed as either clinically normal, oral lichen planus, gingival hyperplasia, or superficially invasive squamous cell carcinoma. The optical and fluorescence lifetime properties of each specimen were recorded.

RESULTS

Both quantitative and qualitative differences among normal, benign, and squamous cell carcinoma lesions can be resolved with fluorescence lifetime imaging reflectance confocal microscopy. The results demonstrate that an integrated approach based on these two methods can potentially enable rapid screening and evaluation of large areas of oral epithelial tissue.

CONCLUSIONS

Early results from ongoing studies of imaging human oral cavity illustrate the synergistic combination of the 2 modalities. An adjunct device based on such optical characterization of oral mucosa can potentially be used to detect oral carcinogenesis in early stages.