ComPRePS: An Automated Cloud-based Image Analysis tool to democratize AI in Digital Pathology

Artificial intelligence (AI) has extensive applications in a wide range of disciplines including healthcare and clinical practice. Advances in high-resolution whole-slide brightfield microscopy allow for the digitization of histologically stained tissue sections, producing gigapixel-scale whole-slide images (WSI). The significant improvement in computing and revolution of deep neural network (DNN)-based AI technologies over the last decade allow us to integrate massively parallelized computational power, cutting-edge AI algorithms, and big data storage, management, and processing. Applied to WSIs, AI has created opportunities for improved disease diagnostics and prognostics with the ultimate goal of enhancing precision medicine and resulting patient care. The National Institutes of Health (NIH) has recognized the importance of developing standardized principles for data management and discovery for the advancement of science and proposed the Findable, Accessible, Interoperable, Reusable, (FAIR) Data Principles1 with the goal of building a modernized biomedical data resource ecosystem to establish collaborative research communities. In line with this mission and to democratize AI-based image analysis in digital pathology, we propose ComPRePS: an end-to-end automated Computational Renal Pathology Suite which combines massive scalability, on-demand cloud computing, and an easy-to-use web-based user interface for data upload, storage, management, slide-level visualization, and domain expert interaction. Moreover, our platform is equipped with both in-house and collaborator developed sophisticated AI algorithms in the back-end server for image analysis to identify clinically relevant micro-anatomic functional tissue units (FTU) and to extract image features.

Correlating Deep Learning-Based Automated Reference Kidney Histomorphometry with Patient Demographics and Creatinine

Key Points

The authors leverage the unique benefits of panoptic segmentation to perform the largest ever quantitation of reference kidney morphometry. Kidney features vary with age and sex; and glomeruli size may intricately link to creatinine, defying prior notions.

Background

Reference histomorphometric data of healthy human kidneys are largely lacking because of laborious quantitation requirements. Correlating histomorphometric features with clinical parameters through machine learning approaches can provide valuable information about natural population variance. To this end, we leveraged deep learning (DL), computational image analysis, and feature analysis to associate the relationship of histomorphometry with patient age, sex, serum creatinine (SCr), and eGFR in a multinational set of reference kidney tissue sections.

Methods

A panoptic segmentation neural network was developed and used to segment viable and sclerotic glomeruli, cortical and medullary interstitia, tubules, and arteries/arterioles in the digitized images of 79 periodic acid–Schiff-stained human nephrectomy sections showing minimal pathologic changes. Simple morphometrics (e.g. , area, radius, density) were quantified from the segmented classes. Regression analysis aided in determining the association of histomorphometric parameters with age, sex, SCr, and eGFR.

Results

Our DL model achieved high segmentation performance for all test compartments. The size and density of glomeruli, tubules, and arteries/arterioles varied significantly among healthy humans, with potentially large differences between geographically diverse patients. Glomerular size was significantly correlated with SCr and eGFR. Slight, albeit significant, differences in renal vasculature were observed between sexes. Glomerulosclerosis percentage increased, and cortical density of arteries/arterioles decreased, as a function of increasing age.

Conclusions

Using DL, we automated precise measurements of kidney histomorphometric features. In the reference kidney tissue, several histomorphometric features demonstrated significant correlation to patient demographics, SCr, and eGFR. DL tools can increase the efficiency and rigor of histomorphometric analysis.

Correlating Deep Learning-Based Automated Reference Kidney Histomorphometry with Patient Demographics and Creatinine

Background

Reference histomorphometric data of healthy human kidneys are largely lacking due to laborious quantitation requirements. Correlating histomorphometric features with clinical parameters through machine learning approaches can provide valuable information about natural population variance. To this end, we leveraged deep learning, computational image analysis, and feature analysis to investigate the relationship of histomorphometry with patient age, sex, and serum creatinine (SCr) in a multinational set of reference kidney tissue sections.

Methods

A panoptic segmentation neural network was developed and used to segment viable and sclerotic glomeruli, cortical and medullary interstitia, tubules, and arteries/arterioles in the digitized images of 79 periodic acid-Schiff-stained human nephrectomy sections showing minimal pathologic changes. Simple morphometrics (e.g., area, radius, density) were quantified from the segmented classes. Regression analysis aided in determining the relationship of histomorphometric parameters with age, sex, and SCr.

Results

Our deep-learning model achieved high segmentation performance for all test compartments. The size and density of nephrons and arteries/arterioles varied significantly among healthy humans, with potentially large differences between geographically diverse patients. Nephron size was significantly dependent on SCr. Slight, albeit significant, differences in renal vasculature were observed between sexes. Glomerulosclerosis percentage increased, and cortical density of arteries/arterioles decreased, as a function of age.

Conclusions

Using deep learning, we automated precise measurements of kidney histomorphometric features. In the reference kidney tissue, several histomorphometric features demonstrated significant correlation to patient demographics and SCr. Deep learning tools can increase the efficiency and rigor of histomorphometric analysis.

Discovery of Novel Digital Biomarkers for Type 2 Diabetic Nephropathy Classification via Integration of Urinary Proteomics and Pathology

Background

The heterogeneous phenotype of diabetic nephropathy (DN) from type 2 diabetes complicates appropriate treatment approaches and outcome prediction. Kidney histology helps diagnose DN and predict its outcomes, and an artificial intelligence (AI)-based approach will maximize clinical utility of histopathological evaluation. Herein, we addressed whether AI-based integration of urine proteomics and image features improves DN classification and its outcome prediction, altogether augmenting and advancing pathology practice.

Methods

We studied whole slide images (WSIs) of periodic acid-Schiff-stained kidney biopsies from 56 DN patients with associated urinary proteomics data. We identified urinary proteins differentially expressed in patients who developed end-stage kidney disease (ESKD) within two years of biopsy. Extending our previously published human-AI-loop pipeline, six renal sub-compartments were computationally segmented from each WSI. Hand-engineered image features for glomeruli and tubules, and urinary protein measurements, were used as inputs to deep-learning frameworks to predict ESKD outcome. Differential expression was correlated with digital image features using the Spearman rank sum coefficient.

Results

A total of 45 urinary proteins were differentially detected in progressors, which was most predictive of ESKD (AUC=0.95), while tubular and glomerular features were less predictive (AUC=0.71 and AUC=0.63, respectively). Accordingly, a correlation map between canonical cell-type proteins, such as epidermal growth factor and secreted phosphoprotein 1, and AI-based image features was obtained, which supports previous pathobiological results.

Conclusions

Computational method-based integration of urinary and image biomarkers may improve the pathophysiological understanding of DN progression as well as carry clinical implications in histopathological evaluation.

Spatially Aware Transformer Networks for Contextual Prediction of Diabetic Nephropathy Progression from Whole Slide Images

Diabetic nephropathy (DN) in the context of type 2 diabetes is the leading cause of end-stage renal disease (ESRD) in the United States. DN is graded based on glomerular morphology and has a spatially heterogeneous presentation in kidney biopsies that complicates pathologists' predictions of disease progression. Artificial intelligence and deep learning methods for pathology have shown promise for quantitative pathological evaluation and clinical trajectory estimation; but, they often fail to capture large-scale spatial anatomy and relationships found in whole slide images (WSIs). In this study, we present a transformer-based, multi-stage ESRD prediction framework built upon nonlinear dimensionality reduction, relative Euclidean pixel distance embeddings between every pair of observable glomeruli, and a corresponding spatial self-attention mechanism for a robust contextual representation. We developed a deep transformer network for encoding WSI and predicting future ESRD using a dataset of 56 kidney biopsy WSIs from DN patients at Seoul National University Hospital. Using a leave-one-out cross-validation scheme, our modified transformer framework outperformed RNNs, XGBoost, and logistic regression baseline models, and resulted in an area under the receiver operating characteristic curve (AUC) of 0.97 (95% CI: 0.90-1.00) for predicting two-year ESRD, compared with an AUC of 0.86 (95% CI: 0.66-0.99) without our relative distance embedding, and an AUC of 0.76 (95% CI: 0.59-0.92) without a denoising autoencoder module. While the variability and generalizability induced by smaller sample sizes are challenging, our distance-based embedding approach and overfitting mitigation techniques yielded results that sugest opportunities for future spatially aware WSI research using limited pathology datasets.

Automated Reference Kidney Histomorphometry using a Panoptic Segmentation Neural Network Correlates to Patient Demographics and Creatinine

Reference histomorphometric data of healthy human kidneys are lacking due to laborious quantitation requirements. We leveraged deep learning to investigate the relationship of histomorphometry with patient age, sex, and serum creatinine in a multinational set of reference kidney tissue sections. A panoptic segmentation neural network was developed and used to segment viable and sclerotic glomeruli, cortical and medullary interstitia, tubules, and arteries/arterioles in digitized images of 79 periodic acid-Schiff (PAS)-stained human nephrectomy sections showing minimal pathologic changes. Simple morphometrics (e.g., area, radius, density) were measured from the segmented classes. Regression analysis was used to determine the relationship of histomorphometric parameters with age, sex, and serum creatinine. The model achieved high segmentation performance for all test compartments. We found that the size and density of nephrons, arteries/arterioles, and the baseline level of interstitium vary significantly among healthy humans, with potentially large differences between subjects from different geographic locations. Nephron size in any region of the kidney was significantly dependent on patient creatinine. Slight differences in renal vasculature and interstitium were observed between sexes. Finally, glomerulosclerosis percentage increased and cortical density of arteries/arterioles decreased as a function of age. We show that precise measurements of kidney histomorphometric parameters can be automated. Even in reference kidney tissue sections with minimal pathologic changes, several histomorphometric parameters demonstrated significant correlation to patient demographics and serum creatinine. These robust tools support the feasibility of deep learning to increase efficiency and rigor in histomorphometric analysis and pave the way for future large-scale studies.

Spatially Aware Transformer Networks for Contextual Prediction of Diabetic Nephropathy Progression from Whole Slide Images

Diabetic nephropathy (DN) in the context of type 2 diabetes is the leading cause of end-stage renal disease (ESRD) in the United States. DN is graded based on glomerular morphology and has a spatially heterogeneous presentation in kidney biopsies that complicates pathologists' predictions of disease progression. Artificial intelligence and deep learning methods for pathology have shown promise for quantitative pathological evaluation and clinical trajectory estimation; but, they often fail to capture large-scale spatial anatomy and relationships found in whole slide images (WSIs). In this study, we present a transformer-based, multi-stage ESRD prediction framework built upon nonlinear dimensionality reduction, relative Euclidean pixel distance embeddings between every pair of observable glomeruli, and a corresponding spatial self-attention mechanism for a robust contextual representation. We developed a deep transformer network for encoding WSI and predicting future ESRD using a dataset of 56 kidney biopsy WSIs from DN patients at Seoul National University Hospital. Using a leave-one-out cross-validation scheme, our modified transformer framework outperformed RNNs, XGBoost, and logistic regression baseline models, and resulted in an area under the receiver operating characteristic curve (AUC) of 0.97 (95% CI: 0.90-1.00) for predicting two-year ESRD, compared with an AUC of 0.86 (95% CI: 0.66-0.99) without our relative distance embedding, and an AUC of 0.76 (95% CI: 0.59-0.92) without a denoising autoencoder module. While the variability and generalizability induced by smaller sample sizes are challenging, our distance-based embedding approach and overfitting mitigation techniques yielded results that suggest opportunities for future spatially aware WSI research using limited pathology datasets.

Generative Modeling of Histology Tissue Reduces Human Annotation Effort for Segmentation Model Development

Segmentation of histology tissue whole side images is an important step for tissue analysis. Given enough annotated training data, modern neural networks are capable of accurate reproducible segmentation; however, the annotation of training datasets is time consuming. Techniques such as human-in-the-loop annotation attempt to reduce this annotation burden, but still require vast initial annotation. Semi-supervised learning-a technique which leverages both labeled and unlabeled data to learn features-has shown promise for easing the burden of annotation. Towards this goal, we employ a recently published semi-supervised method, datasetGAN, for the segmentation of glomeruli from renal biopsy images. We compare the performance of models trained using datasetGAN and traditional annotation and show that datasetGAN significantly reduces the amount of annotation required to develop a highly performing segmentation model. We also explore the usefulness of datasetGAN for transfer learning and find that this method greatly enhances the performance when a limited number of whole slide images are used for training.

Automated Tubular Morphometric Visualization for Whole Kidney Biopsy

One of the strongest prognostic predictors of chronic kidney disease is interstitial fibrosis and tubular atrophy (IFTA). The ultimate goal of IFTA calculation is an estimation of the functional nephritic area. However, the clinical gold standard of estimation by pathologist is imprecise, primarily due to the overwhelming number of tubules sampled in a standard kidney biopsy. Artificial intelligence algorithms could provide significant benefit in this aspect as their high-throughput could identify and quantitatively measure thousands of tubules in mere minutes. Towards this goal, we use a custom panoptic convolutional network similar to Panoptic-DeepLab to detect tubules from 87 WSIs of biopsies from native diabetic kidneys and transplant kidneys. We measure 206 features on each tubule, including commonly understood features like tubular basement membrane thickness and tubular diameter. Finally, we have developed a tool which allows a user to select a range of tubule morphometric features to be highlighted in corresponding WSIs. The tool can also highlight tubules in WSI leveraging multiple morphometric features through selection of regions-of-interest in a uniform manifold approximation and projection plot.

Computational Integration of Renal Histology and Urinary Proteomics using Neural Networks

Histological image data and molecular profiles provide context into renal condition. Often, a biopsy is drawn to diagnose or monitor a suspected kidney problem. However, molecular profiles can go beyond a pathologist's ability to see and diagnose. Using AI, we computationally incorporated urinary proteomic profiles with microstructural morphology from renal biopsy to investigate new and existing molecular links to image phenotypes. We studied whole slide images of periodic acid-Schiff stained renal biopsies from 56 DN patients matched with 2,038 proteins measured from each patient's urine. Using Seurat, we identified differentially expressed proteins in patients that developed end-stage renal disease within 2 years of biopsy. Glomeruli, globally sclerotic glomeruli, and tubules were segmented from WSI using our previously published HAIL pipeline. For each glomerulus, 315 handcrafted digital image features were measured, and for tubules, 207 features. We trained fully connected networks to predict urinary protein measurements that were differentially expressed between patients who did/ did not progress to ESRD within 2 years of biopsy. The input to this network was either glomerular or tubular histomorphological features in biopsy. Trained network weights were used as a proxy to rank which morphological features correlated most highly with specific urinary proteins. We identified significant image feature-protein pairs by ranking network weights by magnitude. We also looked at which features on average were most significant in predicting proteins. For both glomeruli and tubules, RGB color values and variance in PAS+ areas (specifically basement membrane for tubules) were, on average, more predictive of molecular profiles than other features. There is a strong connection between molecular profile and image phenotype, which can be elucidated through computational methods. These discovered links can provide insight to disease pathways, and discover new factors contributing to incidence and progression.

A Distributed System Improves Inter-Observer and AI Concordance in Annotating Interstitial Fibrosis and Tubular Atrophy

Histologic examination of interstitial fibrosis and tubular atrophy (IFTA) is critical to determine the extent of irreversible kidney injury in renal disease. The current clinical standard involves pathologist's visual assessment of IFTA, which is prone to inter-observer variability. To address this diagnostic variability, we designed two case studies (CSs), including seven pathologists, using HistomicsTK- a distributed system developed by Kitware Inc. (Clifton Park, NY). Twenty-five whole slide images (WSIs) were classified into a training set of 21 and a validation set of four. The training set was composed of seven unique subsets, each provided to an individual pathologist along with four common WSIs from the validation set. In CS 1, all pathologists individually annotated IFTA in their respective slides. These annotations were then used to train a deep learning algorithm to computationally segment IFTA. In CS 2, manual and computational annotations from CS 1 were first reviewed by the annotators to improve concordance of IFTA annotation. Both the manual and computational annotation processes were then repeated as in CS1. The inter-observer concordance in the validation set was measured by Krippendorff's alpha (KA). The KA for the seven pathologists in CS1 was 0.62 with CI [0.57, 0.67], and after reviewing each other's annotations in CS2, 0.66 with CI [0.60, 0.72]. The respective CS1 and CS2 KA were 0.58 with CI [0.52, 0.64] and 0.63 with CI [0.56, 0.69] when including the deep learner as an eighth annotator. These results suggest that our designed annotation framework refines agreement of spatial annotation of IFTA and demonstrates a human-AI approach to significantly improve the development of computational models.

Amyand's hernia: role of CT for a correct diagnosis

Amyand's hernia consists in the protrusion of the vermiform appendix into an inguinal hernia sac and represents an uncommon condition with a difficult preoperative diagnosis to be recognized with clinical examination and imaging diagnostic tools in order to choose a correct therapeutic approach for the patient. Four types of Amyand's hernias exist. The case of a recurrent type 1 Amyand's hernia is presented. Multi detector computed tomography allowed a correct diagnosis and the subsequent surgical treatment had no complication for the patient. Radiologists and surgeons need to be aware of this pathology and its classification, as well as of the importance of recognizing both the inflamed and normal appendix within the inguinal canal and the abdominal complications. With the availability of multi detector CT scanning, a greater number of type 1 and 2 hernias are able to be preoperatively diagnosed, and type 3 and 4 better characterized in emergency situation, allowing to perform the best surgical treatment and reducing the chances of pathological recurrence.

Specific local immunotherapy in the treatment of hay fever

The authors refer results of a 3-year study carried out on ten patients suffering from hay fever, diagnosed by means of skin tests, specific nasal provocation and serum RAST who underwent specific local immunotherapy consisting of application of an aqueous allergenic extract, the initial level of which was based on threshold values resulting from the nasal provocation test. The two-monthly check ups were based on the evaluation of mucociliary clearance, anterior rhinorheomanometry, specific nasal provocation and the test of Maunsell for blocking antibodies, as well as on the drawing up of a daily symptomatological diary for each single patient. The results were extremely interesting: subsidence of symptoms during the pollinating season, an increase in the number of blocking serum antibodies and of threshold values relative to specific nasal provocation. Conductance and mucociliary clearance, which were both decidedly pathological before beginning the local immunotherapy, slowly returned to the norm. The authors, furthermore, refer that the use of disodium cromoglycate during the first months of specific local immunotherapy which allows them to reach doses 5-7 times greater than those obtainable with the above mentioned form of treatment, offers uncertain advantages as far as local and above all general immunity is concerned and this alone does not justify the use of nasal applications in the treatment of bronchial asthma of allergic origin.

Incidence of allergic rhinitis in children

The authors give an account of the outcome of research done in the Allergo-Immunological Centre of the IInd ENT Division of Rome University, carried out among 210 children who were affected by nasal atopy. Particular stress was placed on the involvement of the nasopharyngo-tubal system. The age of the children ranged from 2-12 years and they underwent: 1) ENT visit; 2) allergy tests; 3) anterior rhinorheomanometry; 4) tubal function tests; 5) mucociliary clearance time; 6) X-ray examination of paranasal sinuses. The results revealed that the most frequent symptom in these children is rhinitis, whatever the allergic sensitization was. The forms of atopy which manifested themselves by chronical allergic patients (D.Pt. and P.O.) were the cause of: 1. asthmatic-type syndromes; 2. early onset of atopic symptoms around 4-7 years of age (9-10 years in the seasonal forms); 3. greater degree of extrinsic rhinitis with edema of the turbinates - the first step towards a polypoid degeneration of such subjects; 4. tubal functional deficit (60% of subjects allergic to P.O. and 50% allergic to D.Pt whereas only 27% are found in the seasonal forms); 5. mucociliary clearance linked directly with the length of disease; involvement of the paranasal sinuses (53/61 patients allergic to D.Pt., 9/28 allergic to P.O., 9/56 allergic to Graminacee). Furthermore the nasal patency was more insufficient in patients affected by the chronical forms of the atopy. In the light of these results the authors advocate focus attention on the significance of an early diagnosis of nasal atopy in children and the need for interdisciplinary collaboration among specialists.

Behaviour on the nasal provocation test in patients affected by conjunctivitis and/or asthma of allergic origin

In the Allergo-Immunological Centre of Rome University we selected 120 patients of both sexes, ranging from 5-65 years of age, affected by asthma and/or conjunctivitis without past or present history of nasal impairment (itching, sneezing, hydrorrhea). As a result of the allergometric tests carried out, the authors divided the samples into three groups: 1) positive reaction to Dermatophagoides Pteronissimus (66.6%); 2) positive reaction to the Graminacee (28.3%); 3) positive reaction to Parietaria officinalis (5.1%). After having undergone the rhinoreomanometric test of nasal provocation, 50% of the patients revealed a positive reaction to the specific allergen, more specifically at 50 PNU/ml 40% of the case were positive, and at 100 PNU/ml 50% were positive. These results are discussed in the light of modern biological knowledge on the mastocytes in normal subjects and in those suffering from allergy.

Immotile cilia syndrome: radial spokes deficiency in a patient with Kartagener's triad

Mucociliary transport and ultrastructure of nasal cilia in a 13 year old boy with Kartagener's triad, were investigated. Mucociliary transport was significantly delayed (greater than 30 minutes). Electron microscopy showed cilia lacking radial spokes, eccentric central tubules, and a dislocation of one the outer doublets. Dynein arms were present. We consider the radial spoke defect as a distinct congenital anomaly which contributes to the pathogenesis of the "immotile cilia syndrome".

Specific local immunotherapy in nasal allergy (preliminary report)

The preliminary results are presented of a study carried out on 23 subjects with nasal allergy to graminacea, 10 of whom were used as a control group to verify the clinical advantages and the possible side effects of a specific local immunotherapy carried out in spray form using progressively increasing doses of allergen in aqueous solution. From a clinical point of view, on the grounds of the score in the clinical diary, we were able to show that during the preseasonal stage only 5 patients treated with the allergen were free of symptoms as were the controls, whereas during the critical phase they were all free of symptoms in contrast to the crisis presented in the control group. The increase in the threshold of nasal provocation and in the serum level of blocking antibodies, revealed in the majority of the subjects affected by pollinosis and treated locally with the allergen, are to be considered fundamental to the clinical advantage obtained. In agreement with what has already been revealed after specific nasal provocation, local treatment with allergens caused in a majority of cases a slight increase in the time of mucociliar clearance, a phenomenon which must be taken carefully into consideration in as much as it could eventually represent an adverse side effect which needs an adequate countermeasure.

[Simple electrostimulator for clinical gustometry]

The authors illustrate the technical characteristics of a low cost, pocket electrogustometer, which works on batteries and offers complete safety both for the patient and for the examiner; this model is able to carry out, both in normal and selected patients, the same functions as can be carried out with more sophisticated and costly equipment, currently available.