Grayscale representation of infrared microscopy images by extended multiplicative signal correction for registration with histological images

Fast and label-free automated detection of microsatellite status in early colon cancer using artificial intelligence integrated infrared imaging

PURPOSE

Microsatellite instability (MSI) due to mismatch repair (MMR) defects accounts for 15-20% of colon cancers (CC). MSI testing is currently standard of care in CC with immunohistochemistry of the four MMR proteins representing the gold standard. Instead, label-free quantum cascade laser (QCL) based infrared (IR) imaging combined with artificial intelligence (AI) may classify MSI/microsatellite stability (MSS) in unstained tissue sections user-independently and tissue preserving.

METHODS

Paraffin-embedded unstained tissue sections of early CC from patients participating in the multicentre AIO ColoPredict Plus (CPP) 2.0 registry were analysed after dividing into three groups (training, test, and validation). IR images of tissue sections using QCL-IR microscopes were classified by AI (convolutional neural networks [CNN]) using a two-step approach. The first CNN (modified U-Net) detected areas of cancer while the second CNN (VGG-Net) classified MSI/MSS. End-points were area under receiver operating characteristic (AUROC) and area under precision recall curve (AUPRC).

RESULTS

The cancer detection in the first step was based on 629 patients (train n = 273, test n = 138, and validation n = 218). Resulting classification AUROC was 1.0 for the validation dataset. The second step classifying MSI/MSS was performed on 547 patients (train n = 331, test n = 69, and validation n = 147) reaching AUROC and AUPRC of 0.9 and 0.74, respectively, for the validation cohort.

CONCLUSION

Our novel label-free digital pathology approach accurately and rapidly classifies MSI vs. MSS. The tissue sections analysed were not processed leaving the sample unmodified for subsequent analyses. Our approach demonstrates an AI-based decision support tool potentially driving improved patient stratification and precision oncology in the future.

Preclassification of Broadband and Sparse Infrared Data by Multiplicative Signal Correction Approach

Preclassification of raw infrared spectra has often been neglected in scientific literature. Separating spectra of low spectral quality, due to low signal-to-noise ratio, presence of artifacts, and low analyte presence, is crucial for accurate model development. Furthermore, it is very important for sparse data, where it becomes challenging to visually inspect spectra of different natures. Hence, a preclassification approach to separate infrared spectra for sparse data is needed. In this study, we propose a preclassification approach based on Multiplicative Signal Correction (MSC). The MSC approach was applied on human and the bovine knee cartilage broadband Fourier Transform Infrared (FTIR) spectra and on a sparse data subset comprising of only seven wavelengths. The goal of the preclassification was to separate spectra with analyte-rich signals (i.e., cartilage) from spectra with analyte-poor (and high-matrix) signals (i.e., water). The human datasets 1 and 2 contained 814 and 815 spectra, while the bovine dataset contained 396 spectra. A pure water spectrum was used as a reference spectrum in the MSC approach. A threshold for the root mean square error (RMSE) was used to separate cartilage from water spectra for broadband and the sparse spectral data. Additionally, standard noise-to-ratio and principle component analysis were applied on broadband spectra. The fully automated MSC preclassification approach, using water as reference spectrum, performed as well as the manual visual inspection. Moreover, it enabled not only separation of cartilage from water spectra in broadband spectral datasets, but also in sparse datasets where manual visual inspection cannot be applied.

Image fusion of IR and optical microscopy for mapping of biomolecules in tissue

It is shown that a pixel-level image fusion technique can produce images that combine the spatial resolution of optical microscopy images of haematoxylin and eosin (H&E) stained tissue with the chemical information in Fourier transform infrared (FTIR) images. The fused images show minimal distortion and the higher spatial resolution of the H&E images overcomes the diffraction limit on the spatial resolution of the FTIR images. A consideration of the FTIR spectra of nucleic acids and collagen can explain the changes in contrast between non-cancerous oral epithelium and underlying stroma within fused images formed by combining an H&E stain of oral tissue with FTIR images of the tissue obtained at a number of wavenumbers.

Automatic Identification of Paraffin Pixels on FTIR Images Acquired on FFPE Human Samples

The transfer of mid-infrared spectral histopathology to the clinic will be possible provided that its application in clinical practice is simple. Rapid analysis of formalin-fixed paraffin-embedded (FFPE) tissue section is thus a prerequisite. The chemical dewaxing of these samples before image acquisition used by the majority of studies is in contradiction with this principle. Fortunately, the in silico analysis of the images acquired on FFPE samples is possible using extended multiplicative signal correction (EMSC). However, the removal of pure paraffin pixels is essential to perform a relevant classification of tissue spectra. So far, this task was possible only if using manual and subjective histogram analysis. In this article, we thus propose a new automatic and multivariate methodology based on the analysis of optimized combinations of EMSC regression coefficients by validity indices and KMeans clustering to separate paraffin and tissue pixels. The validation of our method is performed using simulated infrared spectral images by measuring the Jaccard index between our partitions and the image model, with values always over 0.90 for diverse baseline complexity and signal-to-noise ratio. These encouraging results were also validated on real images by comparing our method with classical ones and by computing the Jaccard index between our partitions and the KMeans partitions obtained on the infrared image acquired on the same samples but after chemical dewaxing, with values always over 0.84.