Detection of Head and Neck Cancer in Surgical Specimens Using Quantitative Hyperspectral Imaging

Real-Time Intraoperative Decision-Making in Head and Neck Tumor Surgery: A Histopathologically Grounded Hyperspectral Imaging and Deep Learning Approach

BACKGROUND

Accurate and rapid intraoperative tumor margin assessment remains a major challenge in surgical oncology. Current gold-standard methods, such as frozen section histology, are time-consuming, operator-dependent, and prone to misclassification, which limits their clinical utility.

OBJECTIVE

To develop and evaluate a novel hyperspectral imaging (HSI) workflow that integrates deep learning with three-dimensional (3D) tumor modeling for real-time, label-free tumor margin delineation in head and neck squamous cell carcinoma (HNSCC).

METHODS

Freshly resected HNSCC samples were snap-frozen and imaged ex vivo from multiple perspectives using a standardized HSI protocol, resulting in a 3D model derived from HSI. Each sample was serially sectioned, stained, and annotated by pathologists to create high-resolution 3D histological reconstructions. The volumetric histological models were co-registered with the HSI data (n = 712 Datacubes), enabling voxel-wise projection of tumor segmentation maps from the HSI-derived 3D model onto the corresponding histological ground truth. Three deep learning models were trained and validated on these datasets to differentiate tumor from non-tumor regions with high spatial precision.

RESULTS

This work demonstrates strong potential for the proposed HSI system, with an overall classification accuracy of 0.98 and a tumor sensitivity of 0.93, underscoring the system's ability to reliably detect tumor regions and showing high concordance with histopathological findings.

CONCLUSION

The integration of HSI with deep learning and 3D tumor modeling offers a promising approach for precise, real-time intraoperative tumor margin assessment in HNSCC. This novel workflow has the potential to improve surgical precision and patient outcomes by providing rapid, label-free tissue differentiation.

Toward real-time margin assessment in breast-conserving surgery with hyperspectral imaging

Margin assessment in breast-conserving surgery (BSC) remains a critical challenge, with 20-25% of cases resulting in inadequate tumor resection, increasing the risk of local recurrence and the need for additional treatment. In this study, we evaluate the diagnostic performance of hyperspectral imaging (HSI) as a non-invasive technique for assessing resection margins in ex vivo lumpectomy specimens. A dataset of over 200 lumpectomy specimens was collected using two hyperspectral cameras, and a classification algorithm was developed to distinguish between healthy and tumor tissue within margins of 0 and 2 mm. The proposed approach achieved its highest diagnostic performance at a 0 mm margin, with a sensitivity of 92%, specificity of 78%, accuracy of 83%, Matthews correlation coefficient of 68%, and an area under the curve of 89%. The entire resection surface could be imaged and evaluated within 10 minutes, providing a rapid and non-invasive alternative to conventional margin assessment techniques. These findings represent a significant advancement toward real-time intraoperative margin assessment, highlighting the potential of HSI to enhance surgical precision and reduce re-excision rates in BCS.

Improving lung cancer pathological hyperspectral diagnosis through cell-level annotation refinement

Lung cancer remains a major global health challenge, and accurate pathological examination is crucial for early detection. This study aims to enhance hyperspectral pathological image analysis by refining annotations at the cell level and creating a high-quality hyperspectral dataset of lung tumors. We address the challenge of coarse manual annotations in hyperspectral lung cancer datasets, which limit the effectiveness of deep learning models requiring precise labels for training. We propose a semi-automated annotation refinement method that leverages hyperspectral data to enhance pathological diagnosis. Specifically, we employ K-means unsupervised clustering combined with human-guided selection to refine coarse annotations into cell-level masks based on spectral features. Our method is validated using a hyperspectral lung squamous cell carcinoma dataset containing 65 image samples. Experimental results demonstrate that our approach improves pixel-level segmentation accuracy from 77.33% to 92.52% with a lower level of prediction noise. The time required to accurately label each pathological slide is significantly reduced. While pixel-level labeling methods for an entire slide can take over 30 mins, our semi-automated method requires only about 5 mins. To enhance visualization for pathologists, we apply a conservative post-processing strategy for instance segmentation. These results highlight the effectiveness of our method in addressing annotation challenges and improving the accuracy of hyperspectral pathological analysis.

Recent advances of photodiagnosis and treatment for head and neck squamous cell carcinoma

Head and neck squamous cell carcinoma (HNSCC) are the most common type of head and neck tumor that severely threatens human health due to its highly aggressive nature and susceptibility to distant metastasis. The diagnosis of HNSCC currently relies on biopsy and histopathological examination of suspicious lesions. However, the early mucosal changes are subtle and difficult to detect by conventional oral examination. As for treatment, surgery is still the primary treatment modality. Due to the complex anatomy and the lack of intraoperative modalities to accurately determine the incision margins, surgeons are in a dilemma between extensive tumor removal and improving the quality of patient survival. As more knowledge is gained about HNSCC, the increasing recognition of the value of optical imaging has been emphasized. Optical technology offers distinctive possibilities for early preoperative diagnosis, intraoperative real-time visualization of tumor margins, sentinel lymph node biopsies, phototherapy. Fluorescence imaging, narrow-band imaging, Raman spectroscopy, optical coherence tomography, hyperspectral imaging, and photoacoustic imaging have been reported for imaging HNSCC. This article provides a comprehensive overview of the fundamental principles and clinical applications of optical imaging in the diagnosis and treatment of HNSCC, focusing on identifying its strengths and limitations to facilitate advancements in this field.

Perspective on the use of fluorescence molecular imaging for peripheral and deep en face margin assessment

Significance

Current standard practice for margin assessment in solid tumor resection often leads to suboptimal results due to the inability to assess margins completely in a time-efficient manner. On the other hand, for small skin cancers, peripheral and deep en face margin assessment (PDEMA) offers 100% assessment of margins while sparing the utmost amount of normal surrounding tissues. Nonetheless, PDEMA is limited in its use owing to its lengthy tissue processing and imaging time as well as its requirement for high-quality frozen sections and real-time histologic analysis.

Aim

We aim to explore fluorescence molecular imaging (FMI) as a tool for resolving obstacles and integrating PDEMA into the surgeon-to-pathologist workflow for large solid tumors.

Approach

A review of recent pre-clinical and clinical studies using FMI to assess surgical margins was conducted to highlight promising fluorescence imaging technologies utilized in the surgical suite and laboratory.

Results

FMI techniques that provide macroscopic resolution are efficient in time and have a notable ability to identify true negative tissue yet have limited capability in identifying true positive tissues. Moreover, meso- and microscopic FMI methods require additional time to attain a higher resolution but deliver an enhanced sensitivity in detecting true positive tissues. In both cases, experts are still required to learn to interpret the FMI signals, which prohibits a seamless clinical integration.

Conclusions

Our proposed margin assessment platform (MAP) incorporates both macroscopic and, meso- or microscopic imaging with post-processing and machine learning for interpretation, to enable the application of PDEMA into solid tumor surgery. MAP leverages the advantages of each technique and thoroughly tackles the limitations of time and expertise to optimize the efficiency and accuracy of margin assessment and ultimately improve clinical outcomes.

Endoscopic In Vivo Hyperspectral Imaging for Head and Neck Tumor Surgeries Using a Medically Approved CE-Certified Camera with Rapid Visualization During Surgery

Background: In vivo visualization of malignant tumors remains the main challenge during head and neck cancer surgery. This can result in inadequate tumor margin assessment and incomplete tumor resection, adversely affecting patient outcomes. Hyperspectral imaging (HSI) is a promising approach to address this issue. However, its application in surgery has been limited by the lack of medically approved HSI devices compliant with MDR regulations, as well as challenges regarding the integration into the surgical workflow. Methods: In this feasibility study, we employed endoscopic HSI during surgery to visualize the tumor sites of 12 head and neck cancer patients. We optimized the HSI workflow to minimize time required during surgery and to reduce the adaptation period needed for surgeons to adjust to the new workflow. Additionally, we implemented data processing to enable real-time classification and visualization of HSI within the intraoperative setting. HSI evaluation was conducted using principal component analysis and k-means clustering, with this clustering validated through comparison with expert annotations. Results: Our complete HSI workflow requires two to three minutes, with each HSI measurement-including evaluation and visualization-taking less than 10 s, achieving an accuracy of 79%, sensitivity of 72%, and specificity of 84%. Medical personnel became proficient with the HSI system after two surgeries. Conclusions: This study presents an HSI workflow for in vivo tissue differentiation during head and neck cancer surgery, providing accurate and visually accessible results within minimal time. This approach enhances the in vivo evaluation of tumor margins, leading to more clear margins and, consequently, improved patient outcomes.

Computer Vision-Radiomics & Pathognomics

The role of computer vision in extracting radiographic (radiomics) and histopathologic (pathognomics) features is an extension of molecular biomarkers that have been foundational to our understanding across the spectrum of head and neck disorders. Especially within head and neck cancers, machine learning and deep learning applications have yielded advances in the characterization of tumor features, nodal features, and various outcomes. This review aims to overview the landscape of radiomic and pathognomic applications, informing future work to address gaps. Novel methodologies will be needed to potentially engineer ways of integrating multidimensional data inputs to examine disease features to guide prognosis comprehensively and ultimately clinical management.

Trends and Future Directions in Margin Analysis for Head and Neck Cancers

Margin status in head and neck cancer has important prognostic implications. Currently, resection is based on manual palpation and gross visualization followed by intraoperative specimen or tumor bed-based margin analysis using frozen sections. While generally effective, this protocol has several limitations including margin sampling and close and positive margin re-localization. There is a lack of evidence on the association of use of frozen section analysis with improved survival in head and neck cancer. This article reviews novel technologies in head and neck margin analysis such as 3-dimensional scanning, augmented reality, molecular margins, optical imaging, spectroscopy, and artificial intelligence.

Hyperspectral imaging facilitating resect-and-discard strategy through artificial intelligence-assisted diagnosis of colorectal polyps: A pilot study

BACKGROUND AND AIMS

The resect-and-discard strategy for colorectal polyps based on accurate optical diagnosis remains challenges. Our aim was to investigate the feasibility of hyperspectral imaging (HSI) for identifying colorectal polyp properties and diagnosis of colorectal cancer in fresh tissues during colonoscopy.

METHODS

144,900 two dimensional images generated from 161 hyperspectral images of colorectal polyp tissues were prospectively obtained from patients undergoing colonoscopy. A residual neural network model was trained with transfer learning to automatically differentiate colorectal polyps, validated by histopathologic diagnosis. The diagnostic performances of the HSI-AI model and endoscopists were calculated respectively, and the auxiliary efficiency of the model was evaluated after a 2-week interval.

RESULTS

Quantitative HSI revealed histological differences in colorectal polyps. The HSI-AI model showed considerable efficacy in differentiating nonneoplastic polyps, non-advanced adenomas, and advanced neoplasia in vitro, with sensitivities of 96.0%, 94.0%, and 99.0% and specificities of 99.0%, 99.0%, and 96.5%, respectively. With the assistance of the model, the median negative predictive value of neoplastic polyps increased from 50.0% to 88.2% (p = 0.013) in novices.

CONCLUSION

This study demonstrated the feasibility of using HSI as a diagnostic tool to differentiate neoplastic colorectal polyps in vitro and the potential of AI-assisted diagnosis synchronized with colonoscopy. The tool may improve the diagnostic performance of novices and facilitate the application of resect-and-discard strategy to decrease the cost.

Technological Frontiers in Brain Cancer: A Systematic Review and Meta-Analysis of Hyperspectral Imaging in Computer-Aided Diagnosis Systems

Brain cancer is a substantial factor in the mortality associated with cancer, presenting difficulties in the timely identification of the disease. The precision of diagnoses is significantly dependent on the proficiency of radiologists and neurologists. Although there is potential for early detection with computer-aided diagnosis (CAD) algorithms, the majority of current research is hindered by its modest sample sizes. This meta-analysis aims to comprehensively assess the diagnostic test accuracy (DTA) of computer-aided design (CAD) models specifically designed for the detection of brain cancer utilizing hyperspectral (HSI) technology. We employ Quadas-2 criteria to choose seven papers and classify the proposed methodologies according to the artificial intelligence method, cancer type, and publication year. In order to evaluate heterogeneity and diagnostic performance, we utilize Deeks' funnel plot, the forest plot, and accuracy charts. The results of our research suggest that there is no notable variation among the investigations. The CAD techniques that have been examined exhibit a notable level of precision in the automated detection of brain cancer. However, the absence of external validation hinders their potential implementation in real-time clinical settings. This highlights the necessity for additional studies in order to authenticate the CAD models for wider clinical applicability.

Deep neural networks can differentiate thyroid pathologies on infrared hyperspectral images

BACKGROUND AND OBJECTIVE

The thyroid is a gland responsible for producing important body hormones. Several pathologies can affect this gland, such as thyroiditis, hypothyroidism, and thyroid cancer. The visual histological analysis of thyroid specimens is a valuable process that enables pathologists to detect diseases with high efficiency, providing the patient with a better prognosis. Existing computer vision systems developed to aid in the analysis of histological samples have limitations in distinguishing pathologies with similar characteristics or samples containing multiple diseases. To overcome this challenge, hyperspectral images are being studied to represent biological samples based on their molecular interaction with light.

METHODS

In this study, we address the acquisition of infrared absorbance spectra from each voxel of histological specimens. This data is then used for the development of a multiclass fully-connected neural network model that discriminates spectral patterns, enabling the classification of voxels as healthy, cancerous, or goiter.

RESULTS

Through experiments using the k-fold cross-validation protocol, we obtained an average accuracy of 93.66 %, a sensitivity of 93.47 %, and a specificity of 96.93 %. Our results demonstrate the feasibility of using infrared hyperspectral imaging to characterize healthy tissue and thyroid pathologies using absorbance measurements. The proposed deep learning model has the potential to improve diagnostic efficiency and enhance patient outcomes.

Handheld hyperspectral imaging as a tool for the post-mortem interval estimation of human skeletal remains

In forensic medicine, estimating human skeletal remains' post-mortem interval (PMI) can be challenging. Following death, bones undergo a series of chemical and physical transformations due to their interactions with the surrounding environment. Post-mortem changes have been assessed using various methods, but estimating the PMI of skeletal remains could still be improved. We propose a new methodology with handheld hyperspectral imaging (HSI) system based on the first results from 104 human skeletal remains with PMIs ranging between 1 day and 2000 years. To differentiate between forensic and archaeological bone material, the Convolutional Neural Network analyzed 65.000 distinct diagnostic spectra: the classification accuracy was 0.58, 0.62, 0.73, 0.81, and 0.98 for PMIs of 0 week-2 weeks, 2 weeks-6 months, 6 months-1 year, 1 year-10 years, and >100 years, respectively. In conclusion, HSI can be used in forensic medicine to distinguish bone materials >100 years old from those <10 years old with an accuracy of 98%. The model has adequate predictive performance, and handheld HSI could serve as a novel approach to objectively and accurately determine the PMI of human skeletal remains.

Polarized hyperspectral microscopic imaging system for enhancing the visualization of collagen fibers and head and neck squamous cell carcinoma

Significance

Polarized hyperspectral microscopes with the capability of full Stokes vector imaging have potential for many biological and medical applications.

Aim

The aim of this study is to investigate polarized hyperspectral imaging (PHSI) for improving the visualization of collagen fibers, which is an important biomarker related to tumor development, and improving the differentiation of normal and tumor cells on pathologic slides.

Approach

We customized a polarized hyperspectral microscopic imaging system comprising an upright microscope with a motorized stage, two linear polarizers, two liquid crystal variable retarders (LCVRs), and a compact SnapScan hyperspectral camera. The polarizers and LCVRs worked in tandem with the hyperspectral camera to acquire polarized hyperspectral images, which were further used to calculate four Stokes vectors: S 0 , S 1 , S 2 , and S 3 . Synthetic RGB images of the Stokes vectors were generated for the visualization of cellular components in PHSI images. Regions of interest of collagen, normal cells, and tumor cells in the synthetic RGB images were selected, and spectral signatures of the selected components were extracted for comparison. Specifically, we qualitatively and quantitatively investigated the enhanced visualization and spectral characteristics of dense fibers and sparse fibers in normal stroma tissue, fibers accumulated within tumors, and fibers accumulated around tumors.

Results

By employing our customized polarized hyperspectral microscope, we extract the spectral signatures of Stokes vector parameters of collagen as well as of tumor and normal cells. The measurement of Stokes vector parameters increased the image contrast of collagen fibers and cells in the slides.

Conclusions

With the spatial and spectral information from the Stokes vector data cubes (S 0 , S 1 , S 2 , and S 3 ), our PHSI microscope system enhances the visualization of tumor cells and tumor microenvironment components, thus being beneficial for pathology and oncology.

Current role of artificial intelligence in head and neck cancer surgery: a systematic review of literature

Aim

Artificial intelligence (AI) is a new field of science in which computers will provide decisions-supporting tools to help doctors make difficult clinical choices. Recent AI applications in otolaryngology include head and neck oncology, rhinology, neurotology, and laryngology. The aim of this systematic review is to describe the potential uses of AI in head and neck oncology with a special focus on the surgical field.

Methods

The authors performed a systematic review, in accordance with the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines, in the main medical databases, including PubMed, Scopus, and Cochrane Library, considering all original studies published until February 2023 about the role of AI in head and neck cancer surgery. The search strategy included a combination of the following terms: "artificial intelligence" or "machine learning" and "head and neck cancer".

Results

Overall, 303 papers were identified and after duplicate removal (12 papers) and excluding papers not written in English (1 paper) and off-topic (4 papers), papers were assessed for eligibility; finally, only 12 papers were included. Three main fields of clinical interest were identified: the most widely investigated included the role of AI in surgical margins assessment (7 papers); the second most frequently evaluated topic was complications assessment (4 papers); finally, only one paper dealt with the indication of salvage laryngectomy after primary radiotherapy.

Conclusions

The authors report the first systematic review in the literature concerning the role of AI in head and neck cancer surgery. An increasing influx of AI applications to clinical problems in otolaryngology is expected, so specialists should be increasingly prepared to manage the constant changes. It will always remain critical for clinicians to use their skills and knowledge to critically evaluate the additional information provided by AI and make the final decisions on each patient.

Emerging clinical applications in oncology for non-invasive multi- and hyperspectral imaging of cell and tissue autofluorescence

Hyperspectral and multispectral imaging of cell and tissue autofluorescence is an emerging technology in which fluorescence imaging is applied to biological materials across multiple spectral channels. This produces a stack of images where each matched pixel contains information about the sample's spectral properties at that location. This allows precise collection of molecularly specific data from a broad range of native fluorophores. Importantly, complex information, directly reflective of biological status, is collected without staining and tissues can be characterised in situ, without biopsy. For oncology, this can spare the collection of biopsies from sensitive regions and enable accurate tumour mapping. For in vivo tumour analysis, the greatest focus has been on oral cancer, whereas for ex vivo assessment head-and-neck cancers along with colon cancer have been the most studied, followed by oral and eye cancer. This review details the scope and progress of research undertaken towards clinical translation in oncology.

Systematic meta-analysis of computer-aided detection to detect early esophageal cancer using hyperspectral imaging

One of the leading causes of cancer deaths is esophageal cancer (EC) because identifying it in early stage is challenging. Computer-aided diagnosis (CAD) could detect the early stages of EC have been developed in recent years. Therefore, in this study, complete meta-analysis of selected studies that only uses hyperspectral imaging to detect EC is evaluated in terms of their diagnostic test accuracy (DTA). Eight studies are chosen based on the Quadas-2 tool results for systematic DTA analysis, and each of the methods developed in these studies is classified based on the nationality of the data, artificial intelligence, the type of image, the type of cancer detected, and the year of publishing. Deeks' funnel plot, forest plot, and accuracy charts were made. The methods studied in these articles show the automatic diagnosis of EC has a high accuracy, but external validation, which is a prerequisite for real-time clinical applications, is lacking.

Optimal Hyperspectral Band Selection for Tissue Oxygenation Mapping with Generative Adversarial Network

Tissue oxygenation assessment using hyperspectral imaging is an emerging technique for the diagnosis and pre- and post-treatment monitoring of ischemic patients. However, the high spectral resolution of hyperspectral imaging leads to large data sizes and a long imaging time. In this study, we propose a method that utilizes multi-objective evolutionary algorithms to determine the optimal hyperspectral band combination when developing a deep learning model for predicting tissue oxygenation from hyperspectral images. Our results confirm that the deep learning model effectively predicts tissue oxygenation images for various oxygenation states. Moreover, we demonstrate that a high-performance prediction model can be developed using only a small number of spectral bands, indicating the potential for more efficient non-contact tissue oxygenation mapping with the proposed method.Clinical Relevance- The proposed method allows for the non-contact and efficient acquisition of two-dimensional tissue oxygenation information in various oxygenation states.

Survival associated miRNA signature in patients with head and neck carcinomas

Head and neck carcinoma (HNSC) is often diagnosed at advanced stage, incurring poor patient outcome. Despite of advances in chemoradiation and surgery approaches, limited improvements in survival rates of HNSC have been observed over the last decade. Accumulating evidences have demonstrated the importance of microRNAs (miRNAs) in carcinogenesis. In this context, we sought to identify a miRNA signature associated with the survival time in patients with HNSC. This study proposed a survival estimation method called HNSC-Sig that identified a miRNA signature consists of 25 miRNAs associated with the survival in 133 patients with HNSC. HNSC-Sig achieved 10-fold cross validation a mean correlation coefficient and a mean absolute error of 0.85 ± 0.01 and 0.46 ± 0.02 years, respectively, between actual and estimated survival times. The survival analysis revealed that five miRNAs, hsa-miR-3605-3p, hsa-miR-629-3p, hsa-miR-3127-5p, hsa-miR-497-5p, and hsa-miR-374a-5p, were significantly associated with prognosis in patients with HNSC. Comparing the relative expression difference of top 10 prioritized miRNAs, eight miRNAs, hsa-miR-629-3p, hsa-miR-3127-5p, hsa-miR-221-3p, hsa-miR-501-5p, hsa-miR-491-5p, hsa-miR-149-3p, hsa-miR-3934-5p, and hsa-miR-3170, were significantly expressed between cancer and normal groups. In addition, biological relevance, disease association, and target interactions of the miRNA signature were discussed. Our results suggest that identified miRNA signature have potential to serve as biomarker for diagnosis and clinical practice in HNSC.

Tissue Classification of Breast Cancer by Hyperspectral Unmixing

(1) Background: Assessing the resection margins during breast-conserving surgery is an important clinical need to minimize the risk of recurrent breast cancer. However, currently there is no technique that can provide real-time feedback to aid surgeons in the margin assessment. Hyperspectral imaging has the potential to overcome this problem. To classify resection margins with this technique, a tissue discrimination model should be developed, which requires a dataset with accurate ground-truth labels. However, establishing such a dataset for resection specimens is difficult. (2) Methods: In this study, we therefore propose a novel approach based on hyperspectral unmixing to determine which pixels within hyperspectral images should be assigned to the ground-truth labels from histopathology. Subsequently, we use this hyperspectral-unmixing-based approach to develop a tissue discrimination model on the presence of tumor tissue within the resection margins of ex vivo breast lumpectomy specimens. (3) Results: In total, 372 measured locations were included on the lumpectomy resection surface of 189 patients. We achieved a sensitivity of 0.94, specificity of 0.85, accuracy of 0.87, Matthew's correlation coefficient of 0.71, and area under the curve of 0.92. (4) Conclusion: Using this hyperspectral-unmixing-based approach, we demonstrated that the measured locations with hyperspectral imaging on the resection surface of lumpectomy specimens could be classified with excellent performance.

Polarized hyperspectral microscopic imaging for collagen visualization on pathologic slides of head and neck squamous cell carcinoma

We developed a polarized hyperspectral microscope to collect four types of Stokes vector data cubes (S0, S1, S2, and S3) of the pathologic slides with head and neck squamous cell carcinoma (HNSCC). Our system consists of an optical light microscope with a movable stage, two polarizers, two liquid crystal variable retarders (LCVRs), and a SnapScan hyperspectral camera. The polarizers and LCVRs work in tandem with the hyperspectral camera to acquire polarized hyperspectral images. Synthetic pseudo-RGB images are generated from the four Stokes vector data cubes based on a transformation function similar to the spectral response of human eye for the visualization of hyperspectral images. Collagen is the most abundant extracellular matrix (ECM) protein in the human body. A major focus of studying the ECM in tumor microenvironment is the role of collagen in both normal and abnormal function. Collagen tends to accumulate in and around tumors during cancer development and growth. In this study, we acquired images from normal regions containing normal cells and collagen fibers and from tumor regions containing cancerous squamous cells and collagen fibers on HNSCC pathologic slides. The preliminary results demonstrated that our customized polarized hyperspectral microscope is able to improve the visualization of collagen on HNSCC pathologic slides under different situations, including thick fibers of normal stroma, thin fibers of normal stroma, fibers of normal muscle cells, fibers accumulated in tumors, fibers accumulated around tumors. Our preliminary results also demonstrated that the customized polarized hyperspectral microscope is capable of extracting the spectral signatures of collagen based on Stokes vector parameters and can have various applications in pathology and oncology.

Intraoperative Assessment of Tumor Margins in Tissue Sections with Hyperspectral Imaging and Machine Learning

The intraoperative assessment of tumor margins of head and neck cancer is crucial for complete tumor resection and patient outcome. The current standard is to take tumor biopsies during surgery for frozen section analysis by a pathologist after H&E staining. This evaluation is time-consuming, subjective, methodologically limited and underlies a selection bias. Optical methods such as hyperspectral imaging (HSI) are therefore of high interest to overcome these limitations. We aimed to analyze the feasibility and accuracy of an intraoperative HSI assessment on unstained tissue sections taken from seven patients with oral squamous cell carcinoma. Afterwards, the tissue sections were subjected to standard histopathological processing and evaluation. We trained different machine learning models on the HSI data, including a supervised 3D convolutional neural network to perform tumor detection. The results were congruent with the histopathological annotations. Therefore, this approach enables the delineation of tumor margins with artificial HSI-based histopathological information during surgery with high speed and accuracy on par with traditional intraoperative tumor margin assessment (Accuracy: 0.76, Specificity: 0.89, Sensitivity: 0.48). With this, we introduce HSI in combination with ML hyperspectral imaging as a potential new tool for intraoperative tumor margin assessment.

Autofluorescence Image-Guided Endoscopy in the Management of Upper Aerodigestive Tract Tumors

At this juncture, autofluorescence and narrow-band imaging have resurfaced in the medicine arena in parallel with current technology advancement. The emergence of newly developed optical instrumentation in addition to the discovery of new fluorescence biomolecules have contributed to a refined management of diseases and tumors, especially in the management of upper aerodigestive tract tumors. The advancement in multispectral imaging and micro-endoscopy has also escalated the trends further in the setting of the management of this tumor, in order to gain not only the best treatment outcomes but also facilitate early tumor diagnosis. This includes the usage of autofluorescence endoscopy for screening, diagnosis and treatment of this tumor. This is crucial, as microtumoral deposit at the periphery of the gross tumor can be only assessed via an enhanced endoscopy and even more precisely with autofluorescence endoscopic techniques. Overall, with this new technique, optimum management can be achieved for these patients. Hence, the treatment outcomes can be improved and patients are able to attain better prognosis and survival.

Deep Learning in Medical Hyperspectral Images: A Review

With the continuous progress of development, deep learning has made good progress in the analysis and recognition of images, which has also triggered some researchers to explore the area of combining deep learning with hyperspectral medical images and achieve some progress. This paper introduces the principles and techniques of hyperspectral imaging systems, summarizes the common medical hyperspectral imaging systems, and summarizes the progress of some emerging spectral imaging systems through analyzing the literature. In particular, this article introduces the more frequently used medical hyperspectral images and the pre-processing techniques of the spectra, and in other sections, it discusses the main developments of medical hyperspectral combined with deep learning for disease diagnosis. On the basis of the previous review, tne limited factors in the study on the application of deep learning to hyperspectral medical images are outlined, promising research directions are summarized, and the future research prospects are provided for subsequent scholars.

Visible near-infrared hyperspectral imaging and supervised classification for the detection of small intestinal necrosis tissue in vivo

Complete recognition of necrotic areas during small bowel tissue resection remains challenging due to the lack of optimal intraoperative aid identification techniques. This research utilizes hyperspectral imaging techniques to automatically distinguish normal and necrotic areas of small intestinal tissue. Sample data were obtained from the animal model of small intestinal tissue of eight Japanese large-eared white rabbits developed by experienced physicians. A spectral library of normal and necrotic regions of small intestinal tissue was created and processed using six different supervised classification algorithms. The results show that hyperspectral imaging combined with supervised classification algorithms can be a suitable technique to automatically distinguish between normal and necrotic areas of small intestinal tissue. This new technique could aid physicians in objectively identify normal and necrotic areas of small intestinal tissue.

Diagnostic Approaches For COVID-19: Lessons Learned and the Path Forward

Coronavirus disease 2019 (COVID-19) is a transmitted respiratory disease caused by the infection of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Although humankind has experienced several outbreaks of infectious diseases, the COVID-19 pandemic has the highest rate of infection and has had high levels of social and economic repercussions. The current COVID-19 pandemic has highlighted the limitations of existing virological tests, which have failed to be adopted at a rate to properly slow the rapid spread of SARS-CoV-2. Pandemic preparedness has developed as a focus of many governments around the world in the event of a future outbreak. Despite the largely widespread availability of vaccines, the importance of testing has not diminished to monitor the evolution of the virus and the resulting stages of the pandemic. Therefore, developing diagnostic technology that serves as a line of defense has become imperative. In particular, that test should satisfy three criteria to be widely adopted: simplicity, economic feasibility, and accessibility. At the heart of it all, it must enable early diagnosis in the course of infection to reduce spread. However, diagnostic manufacturers need guidance on the optimal characteristics of a virological test to ensure pandemic preparedness and to aid in the effective treatment of viral infections. Nanomaterials are a decisive element in developing COVID-19 diagnostic kits as well as a key contributor to enhance the performance of existing tests. Our objective is to develop a profile of the criteria that should be available in a platform as the target product. In this work, virus detection tests were evaluated from the perspective of the COVID-19 pandemic, and then we generalized the requirements to develop a target product profile for a platform for virus detection.

Review of the Use of Artificial Intelligence in Early Diagnosis and Prevention of Oral Cancer

The global occurrence of oral cancer (OC) has increased in recent years. OC that is diagnosed in its advanced stages results in morbidity and mortality. The use of technology may be beneficial for early detection and diagnosis and thus help the clinician with better patient management. The advent of artificial intelligence (AI) has the potential to improve OC screening. AI can precisely analyze an enormous dataset from various imaging modalities and provide assistance in the field of oncology. This review focused on the applications of AI in the early diagnosis and prevention of OC. A literature search was conducted in the PubMed and Scopus databases using the search terminology “oral cancer” and “artificial intelligence.” Further information regarding the topic was collected by scrutinizing the reference lists of selected articles. Based on the information obtained, this article reviews and discusses the applications and advantages of AI in OC screening, early diagnosis, disease prediction, treatment planning, and prognosis. Limitations and the future scope of AI in OC research are also highlighted.

Construction of a new automatic grading system for jaw bone mineral density level based on deep learning using cone beam computed tomography

To develop and verify an automatic classification method using artificial intelligence deep learning to determine the bone mineral density level of the implant site in oral implant surgery from radiographic data obtained from cone beam computed tomography (CBCT) images. Seventy patients with mandibular dentition defects were scanned using CBCT. These Digital Imaging and Communications in Medicine data were cut into 605 training sets, and then the data were processed with data standardization, and the Hounsfiled Unit (HU) value level was determined as follows: Type 1, 1000–2000; type 2, 700–1000; type 3, 400–700; type 4, 100–400; and type 5, − 200–100. Four trained dental implant physicians manually identified and classified the area of the jaw bone density level in the image using the software LabelMe. Then, with the assistance of the HU value generated by LabelMe, a physician with 20 years of clinical experience confirmed the labeling level. Finally, the HU mean values of various categories marked by dental implant physicians were compared to the mean values detected by the artificial intelligence model to assess the accuracy of artificial intelligence classification. After the model was trained on 605 training sets, the statistical results of the HU mean values of various categories in the dataset detected by the model were almost the same as the HU grading interval on the data annotation. This new classification provides a more detailed solution to guide surgeons to adjust the drilling rate and tool selection during preoperative decision-making and intraoperative hole preparation for oral implant surgery.

Log P analyzation-based discovery of GSH activated biotin-tagged fluorescence probe for selective colorectal cancer imaging

Targeted activatable fluorescent probes could provide an effective approach for colorectal cancer imaging. In this study, F1 was found as an effective targeted activatable fluorescent probe based on log P analysis. In vitro experiments demonstrated that the initial fluorescence of the developed probe F1 was initially well quenched, and the fluorescence increased after the probe interacted with glutathione. Cell imaging results showed that the probe had good cell permeability and selectivity. Remarkably, F1 displayed enhanced tumor tissue fluorescence in MC-38 tumor-bearing mice. Notably, it showed selectivity in imaging clinical specimens of human colorectal cancer tissues. Accordingly, this study shows that log P analysis can facilitate the developing efficient of biotin-tagged activatable probes, and the identified F1 has a good potential in clinical colorectal cancer diagnosis.

Intraoperative imaging in pathology-assisted surgery

The pathological assessment of surgical specimens during surgery can reduce the incidence of positive resection margins, which otherwise can result in additional surgeries or aggressive therapeutic regimens. To improve patient outcomes, intraoperative spectroscopic, fluorescence-based, structural, optoacoustic and radiological imaging techniques are being tested on freshly excised tissue. The specific clinical setting and tumour type largely determine whether endogenous or exogenous contrast is to be detected and whether the tumour specificity of the detected biomarker, image resolution, image-acquisition times or penetration depth are to be prioritized. In this Perspective, we describe current clinical standards for intraoperative tissue analysis and discuss how intraoperative imaging is being implemented. We also discuss potential implementations of intraoperative pathology-assisted surgery for clinical decision-making.

Pterygium and Ocular Surface Squamous Neoplasia: Optical Biopsy Using a Novel Autofluorescence Multispectral Imaging Technique

In this study, differentiation of pterygium vs. ocular surface squamous neoplasia based on multispectral autofluorescence imaging technique was investigated. Fifty (N = 50) patients with histopathological diagnosis of pterygium (PTG) and/or ocular surface squamous neoplasia (OSSN) were recruited. Fixed unstained biopsy specimens were imaged by multispectral microscopy. Tissue autofluorescence images were obtained with a custom-built fluorescent microscope with 59 spectral channels, each with specific excitation and emission wavelength ranges, suitable for the most abundant tissue fluorophores such as elastin, flavins, porphyrin, and lipofuscin. Images were analyzed using a new classification framework called fused-classification, designed to minimize interpatient variability, as an established support vector machine learning method. Normal, PTG, and OSSN regions were automatically detected and delineated, with accuracy evaluated against expert assessment by a specialist in OSSN pathology. Signals from spectral channels yielding signals from elastin, flavins, porphyrin, and lipofuscin were significantly different between regions classified as normal, PTG, and OSSN (p < 0.01). Differential diagnosis of PTG/OSSN and normal tissue had accuracy, sensitivity, and specificity of 88 ± 6%, 84 ± 10% and 91 ± 6%, respectively. Our automated diagnostic method generated maps of the reasonably well circumscribed normal/PTG and OSSN interface. PTG and OSSN margins identified by our automated analysis were in close agreement with the margins found in the H&E sections. Such a map can be rapidly generated on a real time basis and potentially used for intraoperative assessment.

Tumor cell identification and classification in esophageal adenocarcinoma specimens by hyperspectral imaging

Esophageal cancer is the sixth leading cause of cancer-related death worldwide. Histopathological confirmation is a key step in tumor diagnosis. Therefore, simplification in decision-making by discrimination between malignant and non-malignant cells of histological specimens can be provided by combination of new imaging technology and artificial intelligence (AI). In this work, hyperspectral imaging (HSI) data from 95 patients were used to classify three different histopathological features (squamous epithelium cells, esophageal adenocarcinoma (EAC) cells, and tumor stroma cells), based on a multi-layer perceptron with two hidden layers. We achieved an accuracy of 78% for EAC and stroma cells, and 80% for squamous epithelium. HSI combined with machine learning algorithms is a promising and innovative technique, which allows image acquisition beyond Red–Green–Blue (RGB) images. Further method validation and standardization will be necessary, before automated tumor cell identification algorithms can be used in daily clinical practice.

Hyperspectral Imaging for Tissue Classification after Advanced Stage Ovarian Cancer Surgery-A Pilot Study

The most important prognostic factor for the survival of advanced-stage epithelial ovarian cancer (EOC) is the completeness of cytoreductive surgery (CRS). Therefore, an intraoperative technique to detect microscopic tumors would be of great value. The aim of this pilot study is to assess the feasibility of near-infrared hyperspectral imaging (HSI) for EOC detection in ex vivo tissue samples. Images were collected during CRS in 11 patients in the wavelength range of 665−975 nm, and processed by calibration, normalization, and noise filtering. A linear support vector machine (SVM) was employed to classify healthy and tumorous tissue (defined as >50% tumor cells). Classifier performance was evaluated using leave-one-out cross-validation. Images of 26 tissue samples from 10 patients were included, containing 26,446 data points that were matched to histopathology. Tumorous tissue could be classified with an area under the curve of 0.83, a sensitivity of 0.81, a specificity of 0.70, and Matthew’s correlation coefficient of 0.41. This study paves the way to in vivo and intraoperative use of HSI during CRS. Hyperspectral imaging can scan a whole tissue surface in a fast and non-contact way. Our pilot study demonstrates that HSI and SVM learning can be used to discriminate EOC from surrounding tissue.

Automatic detection of head and neck squamous cell carcinoma on pathologic slides using polarized hyperspectral imaging and deep learning

The study is to incorporate polarized hyperspectral imaging (PHSI) with deep learning for automatic detection of head and neck squamous cell carcinoma (SCC) on hematoxylin and eosin (H&E) stained tissue slides. A polarized hyperspectral imaging microscope had been developed in our group. In this paper, we firstly collected the Stokes vector data cubes (S0, S1, S2, and S3) of histologic slides from 17 patients with SCC by the PHSI microscope, under the wavelength range from 467 nm to 750 nm. Secondly, we generated the synthetic RGB images from the original Stokes vector data cubes. Thirdly, we cropped the synthetic RGB images into image patches at the image size of 96×96 pixels, and then set up a ResNet50-based convolutional neural network (CNN) to classify the image patches of the four Stokes vector parameters (S0, S1, S2, and S3) by application of transfer learning. To test the performances of the model, each time we trained the model based on the image patches (S0, S1, S2, and S3) of 16 patients out of 17 patients, and used the trained model to calculate the testing accuracy based on the image patches of the rest 1 patient (S0, S1, S2, and S3). We repeated the process for 6 times and obtained 24 testing accuracies (S0, S1, S2, and S3) from 6 different patients out of the 17 patients. The preliminary results showed that the average testing accuracy (84.2%) on S3 outperformed the average testing accuracy (83.5%) on S0. Furthermore, 4 of 6 testing accuracies of S3 (96.0%, 87.3%, 82.8%, and 86.7%) outperformed the testing accuracies of S0 (93.3%, 85.2%, 80.2%, and 79.0%). The study demonstrated the potential of using polarized hyperspectral imaging and deep learning for automatic detection of head and neck SCC on pathologic slides.

Detecting Aggressive Papillary Thyroid Carcinoma Using Hyperspectral Imaging and Radiomic Features

Hyperspectral imaging (HSI) and radiomics have the potential to improve the accuracy of tumor malignancy prediction and assessment. In this work, we extracted radiomic features of fresh surgical papillary thyroid carcinoma (PTC) specimen that were imaged with HSI. A total of 107 unique radiomic features were extracted. This study includes 72 ex-vivo tissue specimens from 44 patients with pathology-confirmed PTC. With the dilated hyperspectral images, the shape feature of least axis length was able to predict the tumor aggressiveness with a high accuracy. The HSI-based radiomic method may provide a useful tool to aid oncologists in determining tumors with intermediate to high risk and in clinical decision making.

Thyroid Carcinoma Detection on Whole Histologic Slides Using Hyperspectral Imaging and Deep Learning

Hyperspectral imaging (HSI), a non-invasive imaging modality, has been successfully used in many different biological and medical applications. One such application is in the field of oncology, where hyperspectral imaging is being used on histologic samples. This study compares the performances of different image classifiers using different imaging modalities as training data. From a database of 33 fixed tissues from head and neck patients with follicular thyroid carcinoma, we produced three different datasets: an RGB image dataset that was acquired from a whole slide image scanner, a hyperspectral (HS) dataset that was acquired with a compact hyperspectral camera, and an HS-synthesized RGB image dataset. Three separate deep learning classifiers were trained using the three datasets. We show that the deep learning classifier trained on HSI data has an area under the receiver operator characteristic curve (AUC-ROC) of 0.966, higher than that of the classifiers trained on RGB and HSI-synthesized RGB data. This study demonstrates that hyperspectral images improve the performance of cancer classification on whole histologic slides. Hyperspectral imaging and deep learning provide an automatic tool for thyroid cancer detection on whole histologic slides.

Hyperspectral terahertz imaging and optical clearance for cancer classification in breast tumor surgical specimen

Purpose: We investigate the enhancement in terahertz (THz) images of freshly excised breast tumors upon treatment with an optical clearance agent. The hyperspectral imaging and spectral classifications are used to quantitatively demonstrate the image enhancement. Glycerol solution with 60% concentration is applied to excised breast tumor specimens for various time durations to investigate the effectiveness on image enhancement. Approach: THz reflection spectroscopy is utilized to obtain the absorption coefficient and the index of refraction of untreated and glycerol-treated tissues at each frequency up to 3 THz. Two classifiers, spectral angular mapping (SAM) based on several kernels and Euclidean minimum distance (EMD) are implemented to evaluate the effectiveness of the treatment. The testing raw data is obtained from five breast cancer specimens: two untreated specimens and three specimens treated with glycerol solution for 20, 40, or 60 min. All tumors used in the testing data have healthy tissues adjacent to cancerous ones consistent with the challenge faced in lumpectomy surgeries. Results: The glycerol-treated tissues showed a decrease in the absorption coefficients compared with untreated tissues, especially as the period of treatment increased. Although the sensitivity metric of the classifier presented higher values in the untreated tissues compared with the treated ones, the specificity and accuracy metrics demonstrated higher values for the treated tissues compared with the untreated ones. Conclusions: The biocompatible glycerol solution is a potential optical clearance agent in THz imaging while keeping the histopathology imaging intact. The SAM technique provided a good classification of cancerous tissues despite the small amount of cancer in the training data (only 7%). The SAM exponential kernel and EMD presented classification accuracy of ∼ 80 % to 85% compared with linear and polynomial kernels that provided accuracy ranging from 70% to 80%. Overall, glycerol treatment provides a potential improvement in cancer classification in freshly excised breast tumors.

Comparison of Whiskbroom and Pushbroom darkfield elastic light scattering spectroscopic imaging for head and neck cancer identification in a mouse model

The early detection of head and neck cancer is a prolonged challenging task. It requires a precise and accurate identification of tissue alterations as well as a distinct discrimination of cancerous from healthy tissue areas. A novel approach for this purpose uses microspectroscopic techniques with special focus on hyperspectral imaging (HSI) methods. Our proof-of-principle study presents the implementation and application of darkfield elastic light scattering spectroscopy (DF ELSS) as a non-destructive, high-resolution, and fast imaging modality to distinguish lingual healthy from altered tissue regions in a mouse model. The main aspect of our study deals with the comparison of two varying HSI detection principles, which are a point-by-point and line scanning imaging, and whether one might be more appropriate in differentiating several tissue types. Statistical models are formed by deploying a principal component analysis (PCA) with the Bayesian discriminant analysis (DA) on the elastic light scattering (ELS) spectra. Overall accuracy, sensitivity, and precision values of 98% are achieved for both models whereas the overall specificity results in 99%. An additional classification of model-unknown ELS spectra is performed. The predictions are verified with histopathological evaluations of identical HE-stained tissue areas to prove the model's capability of tissue distinction. In the context of our proof-of-principle study, we assess the Pushbroom PCA-DA model to be more suitable for tissue type differentiations and thus tissue classification. In addition to the HE-examination in head and neck cancer diagnosis, the usage of HSI-based statistical models might be conceivable in a daily clinical routine.

Explainable liver tumor delineation in surgical specimens using hyperspectral imaging and deep learning

Surgical removal is the primary treatment for liver cancer, but frequent recurrence caused by residual malignant tissue remains an important challenge, as recurrence leads to high mortality. It is unreliable to distinguish tumors from normal tissues merely under visual inspection. Hyperspectral imaging (HSI) has been proved to be a promising technology for intra-operative use by capturing the spatial and spectral information of tissue in a fast, non-contact and label-free manner. In this work, we investigated the feasibility of HSI for liver tumor delineation on surgical specimens using a multi-task U-Net framework. Measurements are performed on 19 patients and a dataset of 36 specimens was collected with corresponding pathological results serving as the ground truth. The developed framework can achieve an overall sensitivity of 94.48% and a specificity of 87.22%, outperforming the baseline SVM method by a large margin. In particular, we propose to add explanations on the well-trained model from the spatial and spectral dimensions to show the contribution of pixels and spectral channels explicitly. On that basis, a novel saliency-weighted channel selection method is further proposed to select a small subset of 5 spectral channels which provide essentially as much information as using all 224 channels. According to the dominant channels, the absorption difference of hemoglobin and bile content in the normal and malignant tissues seems to be promising markers that could be further exploited.

Artificial Intelligence-based methods in head and neck cancer diagnosis: an overview

BACKGROUND

This paper reviews recent literature employing Artificial Intelligence/Machine Learning (AI/ML) methods for diagnostic evaluation of head and neck cancers (HNC) using automated image analysis.

METHODS

Electronic database searches using MEDLINE via OVID, EMBASE and Google Scholar were conducted to retrieve articles using AI/ML for diagnostic evaluation of HNC (2009-2020). No restrictions were placed on the AI/ML method or imaging modality used.

RESULTS

In total, 32 articles were identified. HNC sites included oral cavity (n = 16), nasopharynx (n = 3), oropharynx (n = 3), larynx (n = 2), salivary glands (n = 2), sinonasal (n = 1) and in five studies multiple sites were studied. Imaging modalities included histological (n = 9), radiological (n = 8), hyperspectral (n = 6), endoscopic/clinical (n = 5), infrared thermal (n = 1) and optical (n = 1). Clinicopathologic/genomic data were used in two studies. Traditional ML methods were employed in 22 studies (69%), deep learning (DL) in eight studies (25%) and a combination of these methods in two studies (6%).

CONCLUSIONS

There is an increasing volume of studies exploring the role of AI/ML to aid HNC detection using a range of imaging modalities. These methods can achieve high degrees of accuracy that can exceed the abilities of human judgement in making data predictions. Large-scale multi-centric prospective studies are required to aid deployment into clinical practice.

A review of the application of machine learning in molecular imaging

Molecular imaging (MI) is a science that uses imaging methods to reflect the changes of molecular level in living state and conduct qualitative and quantitative studies on its biological behaviors in imaging. Optical molecular imaging (OMI) and nuclear medical imaging are two key research fields of MI. OMI technology refers to the optical information generated by the imaging target (such as tumors) due to drug intervention and other reasons. By collecting the optical information, researchers can track the motion trajectory of the imaging target at the molecular level. Owing to its high specificity and sensitivity, OMI has been widely used in preclinical research and clinical surgery. Nuclear medical imaging mainly detects ionizing radiation emitted by radioactive substances. It can provide molecular information for early diagnosis, effective treatment and basic research of diseases, which has become one of the frontiers and hot topics in the field of medicine in the world today. Both OMI and nuclear medical imaging technology require a lot of data processing and analysis. In recent years, artificial intelligence technology, especially neural network-based machine learning (ML) technology, has been widely used in MI because of its powerful data processing capability. It provides a feasible strategy to deal with large and complex data for the requirement of MI. In this review, we will focus on the applications of ML methods in OMI and nuclear medical imaging.

Tongue Tumor Detection in Hyperspectral Images Using Deep Learning Semantic Segmentation

OBJECTIVE

The utilization of hyperspectral imaging (HSI) in real-time tumor segmentation during a surgery have recently received much attention, but it remains a very challenging task.

METHODS

In this work, we propose semantic segmentation methods, and compare them with other relevant deep learning algorithms for tongue tumor segmentation. To the best of our knowledge, this is the first work using deep learning semantic segmentation for tumor detection in HSI data using channel selection, and accounting for more spatial tissue context, and global comparison between the prediction map, and the annotation per sample. Results, and Conclusion: On a clinical data set with tongue squamous cell carcinoma, our best method obtains very strong results of average dice coefficient, and area under the ROC-curve of [Formula: see text], and [Formula: see text], respectively on the original spatial image size. The results show that a very good performance can be achieved even with a limited amount of data. We demonstrate that important information regarding tumor decision is encoded in various channels, but some channel selection, and filtering is beneficial over the full spectra. Moreover, we use both visual (VIS), and near-infrared (NIR) spectrum, rather than commonly used only VIS spectrum; although VIS spectrum is generally of higher significance, we demonstrate NIR spectrum is crucial for tumor capturing in some cases.

SIGNIFICANCE

The HSI technology augmented with accurate deep learning algorithms has a huge potential to be a promising alternative to digital pathology or a doctors' supportive tool in real-time surgeries.

Feedforward Artificial Neural Network-Based Colorectal Cancer Detection Using Hyperspectral Imaging: A Step towards Automatic Optical Biopsy

Simple Summary

Detection of colorectal carcinoma is performed visually by investigators and is confirmed pathologically. With hyperspectral imaging, an expanded spectral range of optical information is now available for analysis. The acquired recordings were analyzed with a neural network, and it was possible to differentiate tumor from healthy mucosa in colorectal carcinoma by automatic classification with high reliability. Classification and visualization were performed based on a four-layer perceptron neural network. Based on a neural network, the classification of CA or AD resulted in a sensitivity of 86% and a specificity of 95%, by means of leave-one-patient-out cross-validation. Additionally, significant differences in terms of perfusion parameters (e.g., oxygen saturation) related to tumor staging and neoadjuvant therapy were observed. This is a step towards optical biopsy.

Abstract

Currently, colorectal cancer (CRC) is mainly identified via a visual assessment during colonoscopy, increasingly used artificial intelligence algorithms, or surgery. Subsequently, CRC is confirmed through a histopathological examination by a pathologist. Hyperspectral imaging (HSI), a non-invasive optical imaging technology, has shown promising results in the medical field. In the current study, we combined HSI with several artificial intelligence algorithms to discriminate CRC. Between July 2019 and May 2020, 54 consecutive patients undergoing colorectal resections for CRC were included. The tumor was imaged from the mucosal side with a hyperspectral camera. The image annotations were classified into three groups (cancer, CA; adenomatous margin around the central tumor, AD; and healthy mucosa, HM). Classification and visualization were performed based on a four-layer perceptron neural network. Based on a neural network, the classification of CA or AD resulted in a sensitivity of 86% and a specificity of 95%, by means of leave-one-patient-out cross-validation. Additionally, significant differences in terms of perfusion parameters (e.g., oxygen saturation) related to tumor staging and neoadjuvant therapy were observed. Hyperspectral imaging combined with automatic classification can be used to differentiate between CRC and healthy mucosa. Additionally, the biological changes induced by chemotherapy to the tissue are detectable with HSI.

Methylene Blue and Proflavine as Intraarterial Marker for Functional Perforazome-Comparative Study

Methylene blue (MB) is both a dye and a medicine known and used for a long time including as lymphatic tracer in melanoma and breast cancer for revealing sentinel lymph nodes. Proflavine (PRO) is an acriflavine dye, used as bacteriostatic disinfectant against many gram-positive bacteria that was also successfully applied to evaluate morphopathological changes in tissues. This study was performed on a group of twenty-eight Wistar rats and had as its main objective the in vivo evaluation of the use of MB and PRO as perforator tracers. The two dyes proved to be effective functional perforasome tracers with medium inflammatory infiltrate in the skin of the island perforator flap which heals perfectly at 14 days with complete absence of the inflammatory reaction. At the same injected amount, PRO seems to determine a greater inflammatory reaction compared with MB, but in smaller concentration, the inflammatory response is absent in the case of PRO. In conclusion, both substances tested within this in vivo study are good functional perforasome tracers, but PRO has the advantage of the absence of inflammatory reaction when using lower concentrations, while preserving unalerted its efficiency as tracer.

Hyperspectral imaging as a diagnostic tool to differentiate between amalgam tattoos and other dark pigmented intraoral lesions

The goal of this project is to identify any in-depth benefits and drawbacks in the diagnosis of amalgam tattoos and other pigmented intraoral lesions using hyperspectral imagery collected from amalgam tattoos, benign, and malignant melanocytic neoplasms. Software solutions capable of classifying pigmented lesions of the skin already exist, but conventional red, green and blue images may be reaching an upper limit in their performance. Emerging technologies, such as hyperspectral imaging (HSI) utilize more than a hundred, continuous data channels, while also collecting data in the infrared. A total of 18 paraffin-embedded human tissue specimens of dark pigmented intraoral lesions (including the lip) were analyzed using visible and near-infrared (VIS-NIR) hyperspectral imagery obtained from HE-stained histopathological slides. Transmittance data were collected between 450 and 900 nm using a snapshot camera mounted to a microscope with a halogen light source. VIS-NIR spectra collected from different specimens, such as melanocytic cells and other tissues (eg, epithelium), produced distinct and diagnostic spectra that were used to identify these materials in several regions of interest, making it possible to distinguish between intraoral amalgam tattoos (intramucosal metallic foreign bodies) and melanocytic lesions of the intraoral mucosa and the lip (each with P < .01 using the independent t test). HSI is presented as a diagnostic tool for the rapidly growing field of digital pathology. In this preliminary study, amalgam tattoos were reliably differentiated from melanocytic lesions of the oral cavity and the lip.

Automatic detection of head and neck squamous cell carcinoma on pathologic slides using polarized hyperspectral imaging and machine learning

The aim of this study is to incorporate polarized hyperspectral imaging (PHSI) with machine learning for automatic detection of head and neck squamous cell carcinoma (SCC) on hematoxylin and eosin (H&E) stained tissue slides. A polarized hyperspectral imaging microscope had been developed in our group. In this paper, we imaged 20 H&E stained tissue slides from 10 patients with SCC of the larynx by the PHSI microscope. Several machine learning algorithms, including support vector machine (SVM), random forest, Gaussian naive Bayes, and logistic regression, were applied to the collected image data for the automatic detection of SCC on the H&E stained tissue slides. The performance of these methods was compared among the collected PHSI data, the pseudo-RGB images generated from the PHSI data, and the PHSI data after applying the principal component analysis (PCA) transformation. The results suggest that SVM is a superior classifier for the classification task based on the PHSI data cubes compared to the other three classifiers. The incorporate of four Stokes vector parameters improved the classification accuracy. Finally, the PCA transformed image data did not improve the accuracy as it might lose some important information from the original PHSI data. The preliminary results show that polarized hyperspectral imaging can have many potential applications in digital pathology.

Pixel-level Tumor Margin Assessment of Surgical Specimen with Hyperspectral Imaging and Deep Learning Classification

Surgery is a major treatment method for squamous cell carcinoma (SCC). During surgery, insufficient tumor margin may lead to local recurrence of cancer. Hyperspectral imaging (HSI) is a promising optical imaging technique for in vivo cancer detection and tumor margin assessment. In this study, a fully convolutional network (FCN) was implemented for tumor classification and margin assessment on hyperspectral images of SCC. The FCN was trained and validated with hyperspectral images of 25 ex vivo SCC surgical specimens from 20 different patients. The network was evaluated per patient and achieved pixel-level tissue classification with an average area under the curve (AUC) of 0.88, as well as 0.83 accuracy, 0.84 sensitivity, and 0.70 specificity across all the 20 patients. The 95% Hausdorff distance of assessed tumor margin in 17 patients was less than 2 mm, and the classification time of each tissue specimen took less than 10 seconds. The proposed methods can potentially facilitate intraoperative tumor margin assessment and improve surgical outcomes.

Comprehensive review of surgical microscopes: technology development and medical applications

SIGNIFICANCE

Surgical microscopes provide adjustable magnification, bright illumination, and clear visualization of the surgical field and have been increasingly used in operating rooms. State-of-the-art surgical microscopes are integrated with various imaging modalities, such as optical coherence tomography (OCT), fluorescence imaging, and augmented reality (AR) for image-guided surgery.

AIM

This comprehensive review is based on the literature of over 500 papers that cover the technology development and applications of surgical microscopy over the past century. The aim of this review is threefold: (i) providing a comprehensive technical overview of surgical microscopes, (ii) providing critical references for microscope selection and system development, and (iii) providing an overview of various medical applications.

APPROACH

More than 500 references were collected and reviewed. A timeline of important milestones during the evolution of surgical microscope is provided in this study. An in-depth technical overview of the optical system, mechanical system, illumination, visualization, and integration with advanced imaging modalities is provided. Various medical applications of surgical microscopes in neurosurgery and spine surgery, ophthalmic surgery, ear-nose-throat (ENT) surgery, endodontics, and plastic and reconstructive surgery are described.

RESULTS

Surgical microscopy has been significantly advanced in the technical aspects of high-end optics, bright and shadow-free illumination, stable and flexible mechanical design, and versatile visualization. New imaging modalities, such as hyperspectral imaging, OCT, fluorescence imaging, photoacoustic microscopy, and laser speckle contrast imaging, are being integrated with surgical microscopes. Advanced visualization and AR are being added to surgical microscopes as new features that are changing clinical practices in the operating room.

CONCLUSIONS

The combination of new imaging technologies and surgical microscopy will enable surgeons to perform challenging procedures and improve surgical outcomes. With advanced visualization and improved ergonomics, the surgical microscope has become a powerful tool in neurosurgery, spinal, ENT, ophthalmic, plastic and reconstructive surgeries.

Hyperspectral Imaging for Glioblastoma Surgery: Improving Tumor Identification Using a Deep Spectral-Spatial Approach

The primary treatment for malignant brain tumors is surgical resection. While gross total resection improves the prognosis, a supratotal resection may result in neurological deficits. On the other hand, accurate intraoperative identification of the tumor boundaries may be very difficult, resulting in subtotal resections. Histological examination of biopsies can be used repeatedly to help achieve gross total resection but this is not practically feasible due to the turn-around time of the tissue analysis. Therefore, intraoperative techniques to recognize tissue types are investigated to expedite the clinical workflow for tumor resection and improve outcome by aiding in the identification and removal of the malignant lesion. Hyperspectral imaging (HSI) is an optical imaging technique with the power of extracting additional information from the imaged tissue. Because HSI images cannot be visually assessed by human observers, we instead exploit artificial intelligence techniques and leverage a Convolutional Neural Network (CNN) to investigate the potential of HSI in twelve in vivo specimens. The proposed framework consists of a 3D-2D hybrid CNN-based approach to create a joint extraction of spectral and spatial information from hyperspectral images. A comparison study was conducted exploiting a 2D CNN, a 1D DNN and two conventional classification methods (SVM, and the SVM classifier combined with the 3D-2D hybrid CNN) to validate the proposed network. An overall accuracy of 80% was found when tumor, healthy tissue and blood vessels were classified, clearly outperforming the state-of-the-art approaches. These results can serve as a basis for brain tumor classification using HSI, and may open future avenues for image-guided neurosurgical applications.

Hyperspectral imaging in wound care: A systematic review

Multispectral and hyperspectral imaging (HSI) are emerging imaging techniques with the potential to transform the way patients with wounds are cared for, but it is not clear whether current systems are capable of delivering real-time tissue characterisation and treatment guidance. We conducted a systematic review of HSI systems that have been assessed in patients, published over the past 32 years. We analysed 140 studies, including 10 different HSI systems. Current in vivo HSI systems generate a tissue oxygenation map. Tissue oxygenation measurements may help to predict those patients at risk of wound formation or delayed healing. No safety concerns were reported in any studies. A small number of studies have demonstrated the capabilities of in vivo label-free HSI, but further work is needed to fully integrate it into the current clinical workflow for different wound aetiologies. As an emerging imaging modality for medical applications, HSI offers great potential for non-invasive disease diagnosis and guidance when treating patients with both acute and chronic wounds.