SSTU: Swin-Spectral Transformer U-Net for hyperspectral whole slide image reconstruction

Whole Slide Imaging and Hyperspectral Microscopic Imaging provide great quality data with high spatial and spectral resolution for histopathology. Existing Hyperspectral Whole Slide Imaging systems combine the advantages of the techniques above, thus providing rich information for pathological diagnosis. However, it cannot avoid the problems of slow acquisition speed and mass data storage demand. Inspired by the spectral reconstruction task in computer vision and remote sensing, the Swin-Spectral Transformer U-Net (SSTU) has been developed to reconstruct Hyperspectral Whole Slide images (HWSis) from multiple Hyperspectral Microscopic images (HMis) of small Field of View and Whole Slide images (WSis). The Swin-Spectral Transformer (SST) module in SSTU takes full advantage of Transformer in extracting global attention. Firstly, Swin Transformer is exploited in space domain, which overcomes the high computation cost in Vision Transformer structures, while it maintains the spatial features extracted from WSis. Furthermore, Spectral Transformer is exploited to collect the long-range spectral features in HMis. Combined with the multi-scale encoder-bottleneck-decoder structure of U-Net, SSTU network is formed by sequential and symmetric residual connections of SSTs, which reconstructs a selected area of HWSi from coarse to fine. Qualitative and quantitative experiments prove the performance of SSTU in HWSi reconstruction task superior to other state-of-the-art spectral reconstruction methods.

Deep Learning Provides Rapid Screen for Breast Cancer Metastasis with Sentinel Lymph Nodes

OBJECTIVE

Deep learning has been shown to be useful in detecting breast cancer metastases by analyzing whole slide images (WSI) of sentinel lymph nodes; however, it requires extensive analysis of all the lymph node slides. Our deep learning study attempts to provide a rapid screen for metastasis by analyzing only a small set of image patches to detect changes in tumor environment.

METHODS

We designed a convolutional neural network to build a diagnostic model for metastasis detection. We obtained WSIs of Hematoxylin and Eosin-stained slides from 34 cases with equal distribution in positive/negative categories. Two WSIs were selected from each case for a total of 69 WSIs. From each WSI, 40 image patches (100x100 pixels) were obtained to yield 2720 image patches, from which 2160 (79%) were used for training, 240 (9%) for validation, and 320 (12%) for testing. Interobserver variation was also examined among 3 users.

RESULTS

The test results showed excellent diagnostic results: accuracy (91.15%), sensitivity (77.92%), and specificity (92.09%). No significant variation in results was observed among the 3 observers.

CONCLUSION

This preliminary study provided a proof of concept for conducting a rapid screen for metastasis rather than an exhaustive search for tumors in all fields of all sentinel lymph nodes.

Validation of Whole Slide Imaging for primary surgical pathology diagnosis of prostate biopsies

Context

Whole slide imaging (WSI) is an important component of digital pathology which includes digitization of glass slides and their storage as digital images. Implementation of WSI for primary surgical pathology diagnosis is evolving, following various studies which have evaluated the feasibility of WSI technology for primary diagnosis.

Aims, Settings and Design

The present study was a single-center, observational study which included evaluation by three pathologists and aimed at assessing concordance on specialty-specific diagnosis and comparison of time taken for diagnosis on WSI and conventional light microscopy (CLM).

Materials and Methods

Seventy prostate core biopsy slides (reported between January 2016 and December 2016) were scanned using Pannoramic MIDI II scanner, 3DHISTECH, Budapest, Hungary, at 20× and 40×. Sixty slides were used for validation study following training with 10 slides.

Statistical Analysis Used

Intraobserver concordance for diagnosis between the two platforms of evaluation was analyzed using Cohen's κ statistics and intraclass correlation coefficient (ICC); observation time for diagnosis was compared by Wilcoxon signed-rank test.

Results

Interpretation on WSI using 20× and 40× was comparable with no major discordance. A high level of intraobserver agreement was observed between CLM and WSI for all three observers, both for primary diagnosis (κ = 0.9) and Grade group (κ = 0.7-0.8) in cases of prostatic adenocarcinoma. The major discordance rate between CLM and WSI was 3.3%-8.3%, which reflected the expertise of the observers. The time spent for diagnosis using WSI was variable for the three pathologists.

Conclusion

WSI is comparable to CLM and can be safely incorporated for primary histological diagnosis of prostate core biopsies.

Pathology Informatics Education during the COVID-19 Pandemic at Memorial Sloan Kettering Cancer Center (MSKCC)

This review details the development and structure of a four-week rotation in pathology informatics for a resident trainee at Memorial Sloan Kettering Cancer Center (MSKCC) in New York City so that other programs interested in such a rotation can refer to. The role of pathology informatics is exponentially increasing in research and clinical practice. With an ever-expanding role, training in pathology informatics is paramount as pathology training programs and training accreditation bodies recognize the need for pathology informatics in training future pathologists. However, due to its novelty, many training programs are unfamiliar with implementing pathology informatics training. The rotation incorporates educational resources for pathology informatics, guidance in the development, and general topics relevant to pathology informatics training. Informatics topics include anatomic pathology related aspects such as whole slide imaging, laboratory information systems, image analysis, and molecular pathology associated issues such as the bioinformatics pipeline and data processing. Additionally, we highlight how the rotation pivoted to meet the department's informatics needs while still providing an educational experience during the onset of the COVID-19 pandemic. CONCLUSION: As pathology informatics continues to grow and integrate itself into practice, informatics education must also grow to meet the future needs of pathology. As informatics programs develop across institutions, such as the one detailed in this paper, these programs will better equip future pathologists with informatics to approach disease and pathology.

Automated Diagnosis of Lymphoma with Digital Pathology Images Using Deep Learning

Recent studies have shown promising results in using Deep Learning to detect malignancy in whole slide imaging, however, they were limited to just predicting a positive or negative finding for a specific neoplasm. We attempted to use Deep Learning with a convolutional neural network (CNN) algorithm to build a lymphoma diagnostic model for four diagnostic categories: (1) benign lymph node, (2) diffuse large B-cell lymphoma, (3) Burkitt lymphoma, and (4) small lymphocytic lymphoma. Our software was written in Python language. We obtained digital whole-slide images of Hematoxylin and Eosin stained slides of 128 cases including 32 cases for each diagnostic category. Four sets of 5 representative images, 40x40 pixels in dimension, were taken for each case. A total of 2,560 images were obtained from which 1,856 were used for training, 464 for validation, and 240 for testing. For each test set of 5 images, the predicted diagnosis was combined from the prediction of five images. The test results showed excellent diagnostic accuracy at 95% for image-by-image prediction and at 100% for set-by-set prediction. This preliminary study provided a proof of concept for incorporating automated lymphoma diagnostic screen into future pathology work-flow to augment the pathologists' productivity.

A Framework for 3D Vessel Analysis using Whole Slide Images of Liver Tissue Sections

Three-dimensional (3D) high resolution microscopic images have high potential for improving the understanding of both normal and disease processes where structural changes or spatial relationship of disease features are significant. In this paper, we develop a complete framework applicable to 3D pathology analytical imaging, with an application to whole slide images of sequential liver slices for 3D vessel structure analysis. The analysis workflow consists of image registration, segmentation, vessel cross-section association, interpolation, and volumetric rendering. To identify biologically-meaningful correspondence across adjacent slides, we formulate a similarity function for four association cases. The optimal solution is then obtained by constrained Integer Programming. We quantitatively and qualitatively compare our vessel reconstruction results with human annotations. Validation results indicate a satisfactory concordance as measured both by region-based and distance-based metrics. These results demonstrate a promising 3D vessel analysis framework for whole slide images of liver tissue sections.

Semantic analysis of SNOMED CT for a post-coordinated database of histopathology findings

Objective

This research investigated the use of SNOMED CT to represent diagnostic tissue morphologies and notable tissue architectures typically found within a pathologist's microscopic examination report to identify gaps in expressivity of SNOMED CT for use in anatomic pathology.

Methods

24 breast biopsy cases were reviewed by two board certified surgical pathologists who independently described the diagnostically important tissue architectures and diagnostic morphologies observed by microscopic examination. In addition, diagnostic comments and details were extracted from the original diagnostic pathology report. 95 unique clinical statements were extracted from 13 malignant and 11 benign breast needle biopsy cases.

Results

75% of the inventoried diagnostic terms and statements could be represented by valid SNOMED CT expressions. The expressions included one pre-coordinated expression and 73 post-coordinated expressions. No valid SNOMED CT expressions could be identified or developed to unambiguously assert the meaning of 21 statements (ie, 25% of inventoried clinical statements). Evaluation of the findings indicated that SNOMED CT lacked sufficient definitional expressions or the SNOMED CT concept model prohibited use of certain defined concepts needed to describe the numerous, diagnostically important tissue architectures and morphologic changes found within a surgical pathology microscopic examination.

Conclusions

Because information gathered during microscopic histopathology examination provides the basis of pathology diagnoses, additional concept definitions for tissue morphometries and modifications to the SNOMED CT concept model are needed and suggested to represent detailed histopathologic findings in computable fashion for purposes of patient information exchange and research.

Trial registration number

UNMC Institutional Review Board ID# 342-11-EP.

eeDAP: An Evaluation Environment for Digital and Analog Pathology

PURPOSE

The purpose of this work is to present a platform for designing and executing studies that compare pathologists interpreting histopathology of whole slide images (WSI) on a computer display to pathologists interpreting glass slides on an optical microscope.

METHODS

Here we present eeDAP, an evaluation environment for digital and analog pathology. The key element in eeDAP is the registration of the WSI to the glass slide. Registration is accomplished through computer control of the microscope stage and a camera mounted on the microscope that acquires images of the real time microscope view. Registration allows for the evaluation of the same regions of interest (ROIs) in both domains. This can reduce or eliminate disagreements that arise from pathologists interpreting different areas and focuses the comparison on image quality.

RESULTS

We reduced the pathologist interpretation area from an entire glass slide (≈10-30 mm)² to small ROIs <(50 um)². We also made possible the evaluation of individual cells.

CONCLUSIONS

We summarize eeDAP's software and hardware and provide calculations and corresponding images of the microscope field of view and the ROIs extracted from the WSIs. These calculations help provide a sense of eeDAP's functionality and operating principles, while the images provide a sense of the look and feel of studies that can be conducted in the digital and analog domains. The eeDAP software can be downloaded from code.google.com (project: eeDAP) as Matlab source or as a precompiled stand-alone license-free application.