Tongue Tumor Detection in Hyperspectral Images Using Deep Learning Semantic Segmentation

Quantification of Protein Secondary Structures from Discrete Frequency Infrared Images Using Machine Learning

Discrete frequency infrared (IR) imaging is an exciting experimental technique that has shown promise in various applications in biomedical science. This technique often involves acquiring IR absorptive images at specific frequencies of interest that enable pathologically relevant chemical contrast. However, certain applications, such as tracking the spatial variations in protein secondary structure of tissue specimens, necessary for the characterization of neurodegenerative diseases, require deeper analysis of spectral data. In such cases, the conventional analytical approach involves band fitting the hyperspectral data to extract the relative populations of different structures through their fitted areas under the curve (AUC). While Gaussian spectral fitting for one spectrum is viable, expanding that to an image with millions of pixels, as often applicable for tissue specimens, becomes a computationally expensive process. Alternatives like principal component analysis (PCA) are less structurally interpretable and incompatible with sparsely sampled data. Furthermore, this detracts from the key advantages of discrete frequency imaging by necessitating the acquisition of more finely sampled spectral data that is optimal for curve fitting, resulting in significantly longer data acquisition times, larger datasets, and additional computational overhead. In this work, we demonstrate that a simple two-step regressive neural network model can be utilized to mitigate these challenges and employ discrete frequency imaging for retrieving the results from band fitting without significant loss of fidelity. Our model reduces the data acquisition time nearly six-fold by requiring only seven wavenumbers to accurately interpolate spectral information at a higher resolution and subsequently using the upscaled spectra to accurately predict the component AUCs, which is more than 3000 times faster than spectral fitting. Our approach thus drastically cuts down the data acquisition and analysis time and predicts key differences in protein structure that can be vital towards broadening potential applications of discrete frequency imaging.

Toward real-time margin assessment in breast-conserving surgery with hyperspectral imaging

Margin assessment in breast-conserving surgery (BSC) remains a critical challenge, with 20-25% of cases resulting in inadequate tumor resection, increasing the risk of local recurrence and the need for additional treatment. In this study, we evaluate the diagnostic performance of hyperspectral imaging (HSI) as a non-invasive technique for assessing resection margins in ex vivo lumpectomy specimens. A dataset of over 200 lumpectomy specimens was collected using two hyperspectral cameras, and a classification algorithm was developed to distinguish between healthy and tumor tissue within margins of 0 and 2 mm. The proposed approach achieved its highest diagnostic performance at a 0 mm margin, with a sensitivity of 92%, specificity of 78%, accuracy of 83%, Matthews correlation coefficient of 68%, and an area under the curve of 89%. The entire resection surface could be imaged and evaluated within 10 minutes, providing a rapid and non-invasive alternative to conventional margin assessment techniques. These findings represent a significant advancement toward real-time intraoperative margin assessment, highlighting the potential of HSI to enhance surgical precision and reduce re-excision rates in BCS.

Calibration-Jitter: Augmentation of hyperspectral data for improved surgical scene segmentation

Semantic surgical scene segmentation is crucial for accurately identifying and delineating different tissue types during surgery, enhancing outcomes and reducing complications. Hyperspectral imaging provides detailed information beyond visible color filters, offering an enhanced view of tissue characteristics. Combined with machine learning, it supports critical tumor resection decisions. Traditional augmentations fail to effectively train machine learning models on illumination and sensor sensitivity variations. Learning to handle these variations is crucial to enable models to better generalize, ultimately enhancing their reliability in deployment. In this article, Calibration-Jitter is introduced, a spectral augmentation technique that leverages hyperspectral calibration variations to improve predictive performance. Evaluated on scene segmentation on a neurosurgical dataset, Calibration-Jitter achieved a F1-score of 74.35% with SegFormer, surpassing the previous best of 70.2%. This advancement addresses limitations of traditional augmentations, improving hyperspectral imaging segmentation performance.

Endoscopic In Vivo Hyperspectral Imaging for Head and Neck Tumor Surgeries Using a Medically Approved CE-Certified Camera with Rapid Visualization During Surgery

Background: In vivo visualization of malignant tumors remains the main challenge during head and neck cancer surgery. This can result in inadequate tumor margin assessment and incomplete tumor resection, adversely affecting patient outcomes. Hyperspectral imaging (HSI) is a promising approach to address this issue. However, its application in surgery has been limited by the lack of medically approved HSI devices compliant with MDR regulations, as well as challenges regarding the integration into the surgical workflow. Methods: In this feasibility study, we employed endoscopic HSI during surgery to visualize the tumor sites of 12 head and neck cancer patients. We optimized the HSI workflow to minimize time required during surgery and to reduce the adaptation period needed for surgeons to adjust to the new workflow. Additionally, we implemented data processing to enable real-time classification and visualization of HSI within the intraoperative setting. HSI evaluation was conducted using principal component analysis and k-means clustering, with this clustering validated through comparison with expert annotations. Results: Our complete HSI workflow requires two to three minutes, with each HSI measurement-including evaluation and visualization-taking less than 10 s, achieving an accuracy of 79%, sensitivity of 72%, and specificity of 84%. Medical personnel became proficient with the HSI system after two surgeries. Conclusions: This study presents an HSI workflow for in vivo tissue differentiation during head and neck cancer surgery, providing accurate and visually accessible results within minimal time. This approach enhances the in vivo evaluation of tumor margins, leading to more clear margins and, consequently, improved patient outcomes.

Enhancing Water-Deficient Potato Plant Identification: Assessing Realistic Performance of Attention-Based Deep Neural Networks and Hyperspectral Imaging for Agricultural Applications

Hyperspectral imaging has emerged as a pivotal technology in agricultural research, offering a powerful means to non-invasively monitor stress factors, such as drought, in crops like potato plants. In this context, the integration of attention-based deep learning models presents a promising avenue for enhancing the efficiency of stress detection, by enabling the identification of meaningful spectral channels. This study assesses the performance of deep learning models on two potato plant cultivars exposed to water-deficient conditions. It explores how various sampling strategies and biases impact the classification metrics by using a dual-sensor hyperspectral imaging systems (VNIR -Visible and Near-Infrared and SWIR-Short-Wave Infrared). Moreover, it focuses on pinpointing crucial wavelengths within the concatenated images indicative of water-deficient conditions. The proposed deep learning model yields encouraging results. In the context of binary classification, it achieved an area under the receiver operating characteristic curve (AUC-ROC-Area Under the Receiver Operating Characteristic Curve) of 0.74 (95% CI: 0.70, 0.78) and 0.64 (95% CI: 0.56, 0.69) for the KIS Krka and KIS Savinja varieties, respectively. Moreover, the corresponding F1 scores were 0.67 (95% CI: 0.64, 0.71) and 0.63 (95% CI: 0.56, 0.68). An evaluation of the performance of the datasets with deliberately introduced biases consistently demonstrated superior results in comparison to their non-biased equivalents. Notably, the ROC-AUC values exhibited significant improvements, registering a maximum increase of 10.8% for KIS Krka and 18.9% for KIS Savinja. The wavelengths of greatest significance were observed in the ranges of 475-580 nm, 660-730 nm, 940-970 nm, 1420-1510 nm, 1875-2040 nm, and 2350-2480 nm. These findings suggest that discerning between the two treatments is attainable, despite the absence of prominently manifested symptoms of drought stress in either cultivar through visual observation. The research outcomes carry significant implications for both precision agriculture and potato breeding. In precision agriculture, precise water monitoring enhances resource allocation, irrigation, yield, and loss prevention. Hyperspectral imaging holds potential to expedite drought-tolerant cultivar selection, thereby streamlining breeding for resilient potatoes adaptable to shifting climates.

Hyperspectral imaging and its applications: A review

Hyperspectral imaging has emerged as an effective powerful tool in plentiful military, environmental, and civil applications over the last three decades. The modern remote sensing approaches are adequate for covering huge earth surfaces with phenomenal temporal, spectral, and spatial resolutions. These features make HSI more effective in various applications of remote sensing depending upon the physical estimation of identical material identification and manifold composite surfaces having accomplished spectral resolutions. Recently, HSI has attained immense significance in the research on safety and quality assessment of food, medical analysis, and agriculture applications. This review focuses on HSI fundamentals and its applications like safety and quality assessment of food, medical analysis, agriculture, water resources, plant stress identification, weed & crop discrimination, and flood management. Various investigators have promising solutions for automatic systems depending upon HSI. Future research may use this review as a baseline and future advancement analysis.

Test-time augmentation with synthetic data addresses distribution shifts in spectral imaging

PURPOSE

Surgical scene segmentation is crucial for providing context-aware surgical assistance. Recent studies highlight the significant advantages of hyperspectral imaging (HSI) over traditional RGB data in enhancing segmentation performance. Nevertheless, the current hyperspectral imaging (HSI) datasets remain limited and do not capture the full range of tissue variations encountered clinically.

METHODS

Based on a total of 615 hyperspectral images from a total of 16 pigs, featuring porcine organs in different perfusion states, we carry out an exploration of distribution shifts in spectral imaging caused by perfusion alterations. We further introduce a novel strategy to mitigate such distribution shifts, utilizing synthetic data for test-time augmentation.

RESULTS

The effect of perfusion changes on state-of-the-art (SOA) segmentation networks depended on the organ and the specific perfusion alteration induced. In the case of the kidney, we observed a performance decline of up to 93% when applying a state-of-the-art (SOA) network under ischemic conditions. Our method improved on the state-of-the-art (SOA) by up to 4.6 times.

CONCLUSION

Given its potential wide-ranging relevance to diverse pathologies, our approach may serve as a pivotal tool to enhance neural network generalization within the realm of spectral imaging.

Application of deep learning radiomics in oral squamous cell carcinoma-Extracting more information from medical images using advanced feature analysis

OBJECTIVE

To conduct a systematic review with meta-analyses to assess the recent scientific literature addressing the application of deep learning radiomics in oral squamous cell carcinoma (OSCC).

MATERIALS AND METHODS

Electronic and manual literature retrieval was performed using PubMed, Web of Science, EMbase, Ovid-MEDLINE, and IEEE databases from 2012 to 2023. The ROBINS-I tool was used for quality evaluation; random-effects model was used; and results were reported according to the PRISMA statement.

RESULTS

A total of 26 studies involving 64,731 medical images were included in quantitative synthesis. The meta-analysis showed that, the pooled sensitivity and specificity were 0.88 (95 %CI: 0.87∼0.88) and 0.80 (95 %CI: 0.80∼0.81), respectively. Deeks' asymmetry test revealed there existed slight publication bias (P = 0.03).

CONCLUSIONS

The advances in the application of radiomics combined with learning algorithm in OSCC were reviewed, including diagnosis and differential diagnosis of OSCC, efficacy assessment and prognosis prediction. The demerits of deep learning radiomics at the current stage and its future development direction aimed at medical imaging diagnosis were also summarized and analyzed at the end of the article.

CrossU-Net: Dual-modality cross-attention U-Net for segmentation of precancerous lesions in gastric cancer

Gastric precancerous lesions (GPL) significantly elevate the risk of gastric cancer, and precise diagnosis and timely intervention are critical for patient survival. Due to the elusive pathological features of precancerous lesions, the early detection rate is less than 10%, which hinders lesion localization and diagnosis. In this paper, we provide a GPL pathological dataset and propose a novel method for improving the segmentation accuracy on a limited-scale dataset, namely RGB and Hyperspectral dual-modal pathological image Cross-attention U-Net (CrossU-Net). Specifically, we present a self-supervised pre-training model for hyperspectral images to serve downstream segmentation tasks. Secondly, we design a dual-stream U-Net-based network to extract features from different modal images. To promote information exchange between spatial information in RGB images and spectral information in hyperspectral images, we customize the cross-attention mechanism between the two networks. Furthermore, we use an intermediate agent in this mechanism to improve computational efficiency. Finally, we add a distillation loss to align predicted results for both branches, improving network generalization. Experimental results show that our CrossU-Net achieves accuracy and Dice of 96.53% and 91.62%, respectively, for GPL lesion segmentation, providing a promising spectral research approach for the localization and subsequent quantitative analysis of pathological features in early diagnosis.

Noise learning of instruments for high-contrast, high-resolution and fast hyperspectral microscopy and nanoscopy

The low scattering efficiency of Raman scattering makes it challenging to simultaneously achieve good signal-to-noise ratio (SNR), high imaging speed, and adequate spatial and spectral resolutions. Here, we report a noise learning (NL) approach that estimates the intrinsic noise distribution of each instrument by statistically learning the noise in the pixel-spatial frequency domain. The estimated noise is then removed from the noisy spectra. This enhances the SNR by ca. 10 folds, and suppresses the mean-square error by almost 150 folds. NL allows us to improve the positioning accuracy and spatial resolution and largely eliminates the impact of thermal drift on tip-enhanced Raman spectroscopic nanoimaging. NL is also applicable to enhance SNR in fluorescence and photoluminescence imaging. Our method manages the ground truth spectra and the instrumental noise simultaneously within the training dataset, which bypasses the tedious labelling of huge dataset required in conventional deep learning, potentially shifting deep learning from sample-dependent to instrument-dependent.

Automatic segmentation of inconstant fractured fragments for tibia/fibula from CT images using deep learning

Orthopaedic surgeons need to correctly identify bone fragments using 2D/3D CT images before trauma surgery. Advances in deep learning technology provide good insights into trauma surgery over manual diagnosis. This study demonstrates the application of the DeepLab v3+ -based deep learning model for the automatic segmentation of fragments of the fractured tibia and fibula from CT images and the results of the evaluation of the performance of the automatic segmentation. The deep learning model, which was trained using over 11 million images, showed good performance with a global accuracy of 98.92%, a weighted intersection over the union of 0.9841, and a mean boundary F1 score of 0.8921. Moreover, deep learning performed 5-8 times faster than the experts' recognition performed manually, which is comparatively inefficient, with almost the same significance. This study will play an important role in preoperative surgical planning for trauma surgery with convenience and speed.

Auxiliary Diagnosis of Papillary Thyroid Carcinoma Based on Spectral Phenotype

Thyroid cancer, a common endocrine malignancy, is one of the leading death causes among endocrine tumors. The diagnosis of pathological section analysis suffers from diagnostic delay and cumbersome operating procedures. Therefore, we intend to construct the models based on spectral data that can be potentially used for rapid intraoperative papillary thyroid carcinoma (PTC) diagnosis and characterize PTC characteristics. To alleviate any concerns pathologists may have about using the model, we conducted an analysis of the used bands that can be interpreted pathologically. A spectra acquisition system was first built to acquire spectra of pathological section images from 91 patients. The obtained spectral dataset contains 217 spectra of normal thyroid tissue and 217 spectra of PTC tissue. Clinical data of the corresponding patients were collected for subsequent model interpretability analysis. The experiment has been approved by the Ethics Review Committee of the Wuhu Hospital of East China Normal University. The spectral preprocessing method was used to process the spectra, and the preprocessed signal respectively optimized by the first and secondary informative wavelengths selection was used to develop the PTC detection models. The PTC detection model using mean centering (MC) and multiple scattering correction (MSC) has optimal performance, and the reasons for the good performance were analyzed in combination with the spectral acquisition process and composition of the test slide. For model interpretable analysis, the near-ultraviolet band selected for modeling corresponds to the location of amino acid absorption peak, and this is consistent with the clinical phenomenon of significantly lower amino acid concentrations in PTC patients. Moreover, the absorption peak of hemoglobin selected for modeling is consistent with the low hemoglobin index in PTC patients. In addition, the correlation analysis was performed between the selected wavelengths and the clinical data, and the results show: the reflection intensity of selected wavelengths in normal cells has a moderate correlation with cell arrangement structure, nucleus size and free thyroxine (FT4), and has a strong correlation with triiodothyronine (T3); the reflection intensity of selected bands in PTC cells has a moderate correlation with free triiodothyronine (FT3).

Unsupervised dimensionality reduction of medical hyperspectral imagery in tensor space

BACKGROUND AND OBJECTIVES

Compared with traditional RGB images, medical hyperspectral imagery (HSI) has numerous continuous narrow spectral bands, which can provide rich information for cancer diagnosis. However, the abundant spectral bands also contain a large amount of redundancy information and increase computational complexity. Thus, dimensionality reduction (DR) is essential in HSI analysis. All vector-based DR methods ignore the cubic nature of HSI resulting from vectorization. To overcome the disadvantage of vector-based DR methods, tensor-based techniques have been developed by employing multi-linear algebra.

METHODS

To fully exploit the structure features of medical HSI and enhance computational efficiency, a novel method called unsupervised dimensionality reduction via tensor-based low-rank collaborative graph embedding (TLCGE) is proposed. TLCGE introduces entropy rate superpixel (ERS) segmentation algorithm to generate superpixels. Then, a low-rank collaborative graph weight matrix is constructed on each superpixel, greatly improving the efficiency and robustness of the proposed method. After that, TLCGE reduces dimensions in tensor space to well preserve intrinsic structure of HSI.

RESULTS

The proposed TLCGE is tested on cholangiocarcinoma microscopic hyperspectral data sets. To further demonstrate the effectiveness of the proposed algorithm, other machine learning DR methods are used for comparison. Experimental results on cholangiocarcinoma microscopic hyperspectral data sets validate the effectiveness of the proposed TLCGE.

CONCLUSIONS

The proposed TLCGE is a tensor-based DR method, which can maintain the intrinsic 3-D data structure of medical HSI. By imposing the low-rank and sparse constraints on the objective function, the proposed TLCGE can fully explore the local and global structures within each superpixel. The computational efficiency of the proposed TLCGE is better than other tensor-based DR methods, which can be used as a preprocessing step in real medical HSI classification or segmentation.

Optimal Hyperspectral Band Selection for Tissue Oxygenation Mapping with Generative Adversarial Network

Tissue oxygenation assessment using hyperspectral imaging is an emerging technique for the diagnosis and pre- and post-treatment monitoring of ischemic patients. However, the high spectral resolution of hyperspectral imaging leads to large data sizes and a long imaging time. In this study, we propose a method that utilizes multi-objective evolutionary algorithms to determine the optimal hyperspectral band combination when developing a deep learning model for predicting tissue oxygenation from hyperspectral images. Our results confirm that the deep learning model effectively predicts tissue oxygenation images for various oxygenation states. Moreover, we demonstrate that a high-performance prediction model can be developed using only a small number of spectral bands, indicating the potential for more efficient non-contact tissue oxygenation mapping with the proposed method.Clinical Relevance- The proposed method allows for the non-contact and efficient acquisition of two-dimensional tissue oxygenation information in various oxygenation states.

Spectral imaging enables contrast agent-free real-time ischemia monitoring in laparoscopic surgery

Laparoscopic surgery has evolved as a key technique for cancer diagnosis and therapy. While characterization of the tissue perfusion is crucial in various procedures, such as partial nephrectomy, doing so by means of visual inspection remains highly challenging. We developed a laparoscopic real-time multispectral imaging system featuring a compact and lightweight multispectral camera and the possibility to complement the conventional surgical view of the patient with functional information at a video rate of 25 Hz. To enable contrast agent-free ischemia monitoring during laparoscopic partial nephrectomy, we phrase the problem of ischemia detection as an out-of-distribution detection problem that does not rely on data from any other patient and uses an ensemble of invertible neural networks at its core. An in-human trial demonstrates the feasibility of our approach and highlights the potential of spectral imaging combined with advanced deep learning-based analysis tools for fast, efficient, reliable, and safe functional laparoscopic imaging.

Intraoperative Assessment of Tumor Margins in Tissue Sections with Hyperspectral Imaging and Machine Learning

The intraoperative assessment of tumor margins of head and neck cancer is crucial for complete tumor resection and patient outcome. The current standard is to take tumor biopsies during surgery for frozen section analysis by a pathologist after H&E staining. This evaluation is time-consuming, subjective, methodologically limited and underlies a selection bias. Optical methods such as hyperspectral imaging (HSI) are therefore of high interest to overcome these limitations. We aimed to analyze the feasibility and accuracy of an intraoperative HSI assessment on unstained tissue sections taken from seven patients with oral squamous cell carcinoma. Afterwards, the tissue sections were subjected to standard histopathological processing and evaluation. We trained different machine learning models on the HSI data, including a supervised 3D convolutional neural network to perform tumor detection. The results were congruent with the histopathological annotations. Therefore, this approach enables the delineation of tumor margins with artificial HSI-based histopathological information during surgery with high speed and accuracy on par with traditional intraoperative tumor margin assessment (Accuracy: 0.76, Specificity: 0.89, Sensitivity: 0.48). With this, we introduce HSI in combination with ML hyperspectral imaging as a potential new tool for intraoperative tumor margin assessment.

Deep Learning in Medical Hyperspectral Images: A Review

With the continuous progress of development, deep learning has made good progress in the analysis and recognition of images, which has also triggered some researchers to explore the area of combining deep learning with hyperspectral medical images and achieve some progress. This paper introduces the principles and techniques of hyperspectral imaging systems, summarizes the common medical hyperspectral imaging systems, and summarizes the progress of some emerging spectral imaging systems through analyzing the literature. In particular, this article introduces the more frequently used medical hyperspectral images and the pre-processing techniques of the spectra, and in other sections, it discusses the main developments of medical hyperspectral combined with deep learning for disease diagnosis. On the basis of the previous review, tne limited factors in the study on the application of deep learning to hyperspectral medical images are outlined, promising research directions are summarized, and the future research prospects are provided for subsequent scholars.

Hyperspectral imaging for tumor segmentation on pigmented skin lesions

SIGNIFICANCE

Malignant skin tumors, which include melanoma and nonmelanoma skin cancers, are the most prevalent type of malignant tumor. Gross pathology of pigmented skin lesions (PSL) remains manual, time-consuming, and heavily dependent on the expertise of the medical personnel. Hyperspectral imaging (HSI) can assist in the detection of tumors and evaluate the status of tumor margins by their spectral signatures.

AIM

Tumor segmentation of medical HSI data is a research field. The goal of this study is to propose a framework for HSI-based tumor segmentation of PSL.

APPROACH

An HSI dataset of 28 PSL was prepared. Two frameworks for data preprocessing and tumor segmentation were proposed. Models based on machine learning and deep learning were used at the core of each framework.

RESULTS

Cross-validation performance showed that pixel-wise processing achieves higher segmentation performance, in terms of the Jaccard coefficient. Simultaneous use of spatio-spectral features produced more comprehensive tumor masks. A three-dimensional Xception-based network achieved performance similar to state-of-the-art networks while allowing for more detailed detection of the tumor border.

CONCLUSIONS

Good performance was achieved for melanocytic lesions, but margins were difficult to detect in some cases of basal cell carcinoma. The frameworks proposed in this study could be further improved for robustness against different pathologies and detailed delineation of tissue margins to facilitate computer-assisted diagnosis during gross pathology.

Accuracy of artificial intelligence-assisted detection of Oral Squamous Cell Carcinoma: A systematic review and meta-analysis

Oral Squamous Cell Carcinoma (OSCC) is an aggressive tumor with a poor prognosis. Accurate and timely diagnosis is therefore essential for reducing the burden of advanced disease and improving outcomes. In this meta-analysis, we evaluated the accuracy of artificial intelligence (AI)-assisted technologies in detecting OSCC. We included studies that validated any diagnostic modality that used AI to detect OSCC. A search was performed in six databases: PubMed, Embase, Scopus, Cochrane Library, ProQuest, and Web of Science up to 15 Mar 2022. The Quality Assessment Tool for Diagnostic Accuracy Studies was used to evaluate the included studies' quality, while the Split Component Synthesis method was utilized to quantitatively synthesize the pooled diagnostic efficacy estimates. We considered 16 out of the 566 yielded studies, which included twelve different AI models with a total of 6606 samples. The summary sensitivity, summary specificity, positive and negative likelihood ratios as well as the pooled diagnostic odds ratio were 92.0 % (95 % confidence interval [CI] 86.7-95.4 %), 91.9 % (95 % CI 86.5-95.3 %), 11.4 (95 % CI 6.74-19.2), 0.087 (95 % CI 0.051-0.146) and 132 (95 % CI 62.6-277), respectively. Our findings support the capability of AI-assisted systems to detect OSCC with high accuracy, potentially aiding the histopathological examination in early diagnosis, yet more prospective studies are needed to justify their use in the real population.

Utilizing Deep Machine Learning for Prognostication of Oral Squamous Cell Carcinoma-A Systematic Review

The application of deep machine learning, a subfield of artificial intelligence, has become a growing area of interest in predictive medicine in recent years. The deep machine learning approach has been used to analyze imaging and radiomics and to develop models that have the potential to assist the clinicians to make an informed and guided decision that can assist to improve patient outcomes. Improved prognostication of oral squamous cell carcinoma (OSCC) will greatly benefit the clinical management of oral cancer patients. This review examines the recent development in the field of deep learning for OSCC prognostication. The search was carried out using five different databases-PubMed, Scopus, OvidMedline, Web of Science, and Institute of Electrical and Electronic Engineers (IEEE). The search was carried time from inception until 15 May 2021. There were 34 studies that have used deep machine learning for the prognostication of OSCC. The majority of these studies used a convolutional neural network (CNN). This review showed that a range of novel imaging modalities such as computed tomography (or enhanced computed tomography) images and spectra data have shown significant applicability to improve OSCC outcomes. The average specificity, sensitivity, area under receiving operating characteristics curve [AUC]), and accuracy for studies that used spectra data were 0.97, 0.99, 0.96, and 96.6%, respectively. Conversely, the corresponding average values for these parameters for computed tomography images were 0.84, 0.81, 0.967, and 81.8%, respectively. Ethical concerns such as privacy and confidentiality, data and model bias, peer disagreement, responsibility gap, patient-clinician relationship, and patient autonomy have limited the widespread adoption of these models in daily clinical practices. The accumulated evidence indicates that deep machine learning models have great potential in the prognostication of OSCC. This approach offers a more generic model that requires less data engineering with improved accuracy.

Hyperspectral Imaging for Clinical Applications

Measuring morphological and biochemical features of tissue is crucial for disease diagnosis and surgical guidance, providing clinically significant information related to pathophysiology. Hyperspectral imaging (HSI) techniques obtain both spatial and spectral features of tissue without labeling molecules such as fluorescent dyes, which provides rich information for improved disease diagnosis and treatment. Recent advances in HSI systems have demonstrated its potential for clinical applications, especially in disease diagnosis and image-guided surgery. This review summarizes the basic principle of HSI and optical systems, deep-learning-based image analysis, and clinical applications of HSI to provide insight into this rapidly growing field of research. In addition, the challenges facing the clinical implementation of HSI techniques are discussed.

Deep Learning for Biospectroscopy and Biospectral Imaging: State-of-the-Art and Perspectives

With the advances in instrumentation and sampling techniques, there is an explosive growth of data from molecular and cellular samples. The call to extract more information from the large data sets has greatly challenged the conventional chemometrics method. Deep learning, which utilizes very large data sets for finding hidden features therein and for making accurate predictions for a wide range of applications, has been applied in an unbelievable pace in biospectroscopy and biospectral imaging in the recent 3 years. In this Feature, we first introduce the background and basic knowledge of deep learning. We then focus on the emerging applications of deep learning in the data preprocessing, feature detection, and modeling of the biological samples for spectral analysis and spectroscopic imaging. Finally, we highlight the challenges and limitations in deep learning and the outlook for future directions.

Research and Application of Several Key Techniques in Hyperspectral Image Preprocessing

This paper focuses on image segmentation, image correction and spatial-spectral dimensional denoising of images in hyperspectral image preprocessing to improve the classification accuracy of hyperspectral images. Firstly, the images were filtered and segmented by using spectral angle and principal component analysis, and the segmented results are intersected and then used to mask the hyperspectral images. Hyperspectral images with a excellent segmentation result was obtained. Secondly, the standard reflectance plates with reflectance of 2 and 98% were used as a priori spectral information for image correction of samples with known true spectral information. The mean square error between the corrected and calibrated spectra is less than 0.0001. Comparing with the black-and-white correction method, the classification model constructed based on this method has higher classification accuracy. Finally, the convolution kernel of the one-dimensional Savitzky-Golay (SG) filter was extended into a two-dimensional convolution kernel to perform joint spatial-spectral dimensional filtering (TSG) on the hyperspectral images. The SG filter (m = 7,n = 3) and TSG filter (m = 3,n = 4) were applied to the hyperspectral image of Pavia University and the quality of the hyperspectral image was evaluated. It was found that the TSG filter retained most of the original features while the noise information of the filtered hyperspectral image was less. The hyperspectral images of sample 1-1 and sample 1-2 were processed by the image segmentation and image correction methods proposed in this paper. Then the classification models based on SG filtering and TSG filtering hyperspectral images were constructed, respectively. The results showed that the TSG filter-based model had higher classification accuracy and the classification accuracy is more than 98%.

Hyperspectral Imaging for Glioblastoma Surgery: Improving Tumor Identification Using a Deep Spectral-Spatial Approach

The primary treatment for malignant brain tumors is surgical resection. While gross total resection improves the prognosis, a supratotal resection may result in neurological deficits. On the other hand, accurate intraoperative identification of the tumor boundaries may be very difficult, resulting in subtotal resections. Histological examination of biopsies can be used repeatedly to help achieve gross total resection but this is not practically feasible due to the turn-around time of the tissue analysis. Therefore, intraoperative techniques to recognize tissue types are investigated to expedite the clinical workflow for tumor resection and improve outcome by aiding in the identification and removal of the malignant lesion. Hyperspectral imaging (HSI) is an optical imaging technique with the power of extracting additional information from the imaged tissue. Because HSI images cannot be visually assessed by human observers, we instead exploit artificial intelligence techniques and leverage a Convolutional Neural Network (CNN) to investigate the potential of HSI in twelve in vivo specimens. The proposed framework consists of a 3D-2D hybrid CNN-based approach to create a joint extraction of spectral and spatial information from hyperspectral images. A comparison study was conducted exploiting a 2D CNN, a 1D DNN and two conventional classification methods (SVM, and the SVM classifier combined with the 3D-2D hybrid CNN) to validate the proposed network. An overall accuracy of 80% was found when tumor, healthy tissue and blood vessels were classified, clearly outperforming the state-of-the-art approaches. These results can serve as a basis for brain tumor classification using HSI, and may open future avenues for image-guided neurosurgical applications.