Deep Learning for nasopharyngeal Carcinoma Identification Using Both White Light and Narrow-Band Imaging Endoscopy

Enhancing nasal endoscopy: Classification, detection, and segmentation of anatomic landmarks using a convolutional neural network

KEY POINTS

A convolutional neural network (CNN)-based model can accurately localize and segment turbinates in images obtained during nasal endoscopy (NE). This model represents a starting point for algorithms that comprehensively interpret NE findings.

Computer Vision Foundation Models in Endoscopy: Proof of Concept in Oropharyngeal Cancer

OBJECTIVES

To evaluate the performance of vision transformer-derived image embeddings for distinguishing between normal and neoplastic tissues in the oropharynx and to investigate the potential of computer vision (CV) foundation models in medical imaging.

METHODS

Computational study using endoscopic frames with a focus on the application of a self-supervised vision transformer model (DINOv2) for tissue classification. High-definition endoscopic images were used to extract image patches that were then normalized and processed using the DINOv2 model to obtain embeddings. These embeddings served as input for a standard support vector machine (SVM) to classify the tissues as neoplastic or normal. The model's discriminative performance was validated using an 80-20 train-validation split.

RESULTS

From 38 endoscopic NBI videos, 327 image patches were analyzed. The classification results in the validation cohort demonstrated high accuracy (92%) and precision (89%), with a perfect recall (100%) and an F1-score of 94%. The receiver operating characteristic (ROC) curve yielded an area under the curve (AUC) of 0.96.

CONCLUSION

The use of large vision model-derived embeddings effectively differentiated between neoplastic and normal oropharyngeal tissues. This study supports the feasibility of employing CV foundation models like DINOv2 in the endoscopic evaluation of mucosal lesions, potentially augmenting diagnostic precision in Otorhinolaryngology.

LEVEL OF EVIDENCE

4 Laryngoscope, 2024.

Medical image identification methods: A review

The identification of medical images is an essential task in computer-aided diagnosis, medical image retrieval and mining. Medical image data mainly include electronic health record data and gene information data, etc. Although intelligent imaging provided a good scheme for medical image analysis over traditional methods that rely on the handcrafted features, it remains challenging due to the diversity of imaging modalities and clinical pathologies. Many medical image identification methods provide a good scheme for medical image analysis. The concepts pertinent of methods, such as the machine learning, deep learning, convolutional neural networks, transfer learning, and other image processing technologies for medical image are analyzed and summarized in this paper. We reviewed these recent studies to provide a comprehensive overview of applying these methods in various medical image analysis tasks, such as object detection, image classification, image registration, segmentation, and other tasks. Especially, we emphasized the latest progress and contributions of different methods in medical image analysis, which are summarized base on different application scenarios, including classification, segmentation, detection, and image registration. In addition, the applications of different methods are summarized in different application area, such as pulmonary, brain, digital pathology, brain, skin, lung, renal, breast, neuromyelitis, vertebrae, and musculoskeletal, etc. Critical discussion of open challenges and directions for future research are finally summarized. Especially, excellent algorithms in computer vision, natural language processing, and unmanned driving will be applied to medical image recognition in the future.

The Detection of Nasopharyngeal Carcinomas Using a Neural Network Based on Nasopharyngoscopic Images

OBJECTIVE

To construct and validate a deep convolutional neural network (DCNN)-based artificial intelligence (AI) system for the detection of nasopharyngeal carcinoma (NPC) using archived nasopharyngoscopic images.

METHODS

We retrospectively collected 14107 nasopharyngoscopic images (7108 NPCs and 6999 noncancers) to construct a DCNN model and prepared a validation dataset containing 3501 images (1744 NPCs and 1757 noncancers) from a single center between January 2009 and December 2020. The DCNN model was established using the You Only Look Once (YOLOv5) architecture. Four otolaryngologists were asked to review the images of the validation set to benchmark the DCNN model performance.

RESULTS

The DCNN model analyzed the 3501 images in 69.35 s. For the validation dataset, the precision, recall, accuracy, and F1 score of the DCNN model in the detection of NPCs on white light imaging (WLI) and narrow band imaging (NBI) were 0.845 ± 0.038, 0.942 ± 0.021, 0.920 ± 0.024, and 0.890 ± 0.045, and 0.895 ± 0.045, 0.941 ± 0.018, and 0.975 ± 0.013, 0.918 ± 0.036, respectively. The diagnostic outcome of the DCNN model on WLI and NBI images was significantly higher than that of two junior otolaryngologists (p < 0.05).

CONCLUSION

The DCNN model showed better diagnostic outcomes for NPCs than those of junior otolaryngologists. Therefore, it could assist them in improving their diagnostic level and reducing missed diagnoses.

LEVEL OF EVIDENCE

3 Laryngoscope, 134:127-135, 2024.

Application of narrow band imaging and Lugol's iodine staining in screening for nasopharyngeal carcinoma

BACKGROUND

To investigate the diagnostic value of conventional white light endoscopy (WLE), narrow band imaging (NBI) endoscopy, and Lugol's iodine staining under WLE (endoscopic iodine staining) in the screening and early diagnosis of nasopharyngeal carcinoma.

METHODS

Patients with nasopharyngeal lesions requiring biopsy attending the Department of Otolaryngology Head and Neck Surgery in our hospital between January 2021 and April 2023 were included in this study. Before biopsy, all subjects underwent conventional WLE, NBI endoscopy, and endoscopic iodine staining. On WLE, according to nasopharyngeal lesion morphology and color, patients were diagnosed with nasopharyngeal carcinoma ( +) or chronic hyperplastic nasopharyngitis (-). On NBI endoscopy, according to nasopharyngeal lesion vascular morphology, patients with type V manifestations (nasopharyngeal carcinoma) were categorized as NBI ( +) and patients with type I-IV manifestations (chronic hyperplastic nasopharyngitis) were categorized as NBI (-). Endoscopic iodine staining (1.6% Lugol's iodine solution) was positive ( +) if the mucosal surface was brown with no white patches, or negative (-) if there was no or light brown staining of the mucosal surface. Patients were divided into 2 groups based on histopathological diagnosis: nasopharyngeal carcinoma or chronic hyperplastic nasopharyngitis. Endoscopic diagnoses were compared with histopathological findings. The diagnostic performance of WLE, NBI endoscopy and endoscopic iodine staining for nasopharyngeal carcinoma were determined.

RESULTS

This study included 159 patients. On histopathology, 29 patients were diagnosed with nasopharyngeal carcinoma, and 130 patients were diagnosed with chronic hyperplastic nasopharyngitis. There were no significant differences in the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), accuracy, and area under the receiver operating characteristic (ROC) curve (AUC) of conventional WLE, NBI endoscopy or endoscopic iodine staining for differentiating nasopharyngeal carcinoma and chronic hyperplastic nasopharyngitis. The diagnostic performance of the combination of conventional WLE, NBI endoscopy and endoscopic iodine staining was significantly improved compared to any procedure alone.

CONCLUSIONS

Conventional WLE, NBI endoscopy or endoscopic iodine staining had good diagnostic performance for differentiating nasopharyngeal carcinoma and chronic hyperplastic nasopharyngitis. In particular, NBI endoscopy and endoscopic iodine staining alone or combined had clinical utility for identifying patients with nasopharyngeal lesions that are eligible for a watch-and-wait strategy.

Deep learning for real-time detection of nasopharyngeal carcinoma during nasopharyngeal endoscopy

Nasopharyngeal carcinoma (NPC) is known for high curability during early stage of the disease, and early diagnosis relies on nasopharyngeal endoscopy and subsequent pathological biopsy. To enhance the early diagnosis rate by aiding physicians in the real-time identification of NPC and directing biopsy site selection during endoscopy, we assembled a dataset comprising 2,429 nasopharyngeal endoscopy video frames from 690 patients across three medical centers. With these data, we developed a deep learning-based NPC detection model using the you only look once (YOLO) network. Our model demonstrated high performance, with precision, recall, mean average precision, and F1-score values of 0.977, 0.943, 0.977, and 0.960, respectively, for internal test set and 0.825, 0.743, 0.814, and 0.780 for external test set at 0.5 intersection over union. Remarkably, our model demonstrated a high inference speed (52.9 FPS), surpassing the average frame rate (25.0 FPS) of endoscopy videos, thus making real-time detection in endoscopy feasible.

Artificial Intelligence for Upper Aerodigestive Tract Endoscopy and Laryngoscopy: A Guide for Physicians and State-of-the-Art Review

OBJECTIVE

The endoscopic and laryngoscopic examination is paramount for laryngeal, oropharyngeal, nasopharyngeal, nasal, and oral cavity benign lesions and cancer evaluation. Nevertheless, upper aerodigestive tract (UADT) endoscopy is intrinsically operator-dependent and lacks objective quality standards. At present, there has been an increased interest in artificial intelligence (AI) applications in this area to support physicians during the examination, thus enhancing diagnostic performances. The relative novelty of this research field poses a challenge both for the reviewers and readers as clinicians often lack a specific technical background.

DATA SOURCES

Four bibliographic databases were searched: PubMed, EMBASE, Cochrane, and Google Scholar.

REVIEW METHODS

A structured review of the current literature (up to September 2022) was performed. Search terms related to topics of AI, machine learning (ML), and deep learning (DL) in UADT endoscopy and laryngoscopy were identified and queried by 3 independent reviewers. Citations of selected studies were also evaluated to ensure comprehensiveness.

CONCLUSIONS

Forty-one studies were included in the review. AI and computer vision techniques were used to achieve 3 fundamental tasks in this field: classification, detection, and segmentation. All papers were summarized and reviewed.

IMPLICATIONS FOR PRACTICE

This article comprehensively reviews the latest developments in the application of ML and DL in UADT endoscopy and laryngoscopy, as well as their future clinical implications. The technical basis of AI is also explained, providing guidance for nonexpert readers to allow critical appraisal of the evaluation metrics and the most relevant quality requirements.

Application of Artificial Intelligence to the Diagnosis and Therapy of Nasopharyngeal Carcinoma

Artificial intelligence (AI) is an interdisciplinary field that encompasses a wide range of computer science disciplines, including image recognition, machine learning, human-computer interaction, robotics and so on. Recently, AI, especially deep learning algorithms, has shown excellent performance in the field of image recognition, being able to automatically perform quantitative evaluation of complex medical image features to improve diagnostic accuracy and efficiency. AI has a wider and deeper application in the medical field of diagnosis, treatment and prognosis. Nasopharyngeal carcinoma (NPC) occurs frequently in southern China and Southeast Asian countries and is the most common head and neck cancer in the region. Detecting and treating NPC early is crucial for a good prognosis. This paper describes the basic concepts of AI, including traditional machine learning and deep learning algorithms, and their clinical applications of detecting and assessing NPC lesions, facilitating treatment and predicting prognosis. The main limitations of current AI technologies are briefly described, including interpretability issues, privacy and security and the need for large amounts of annotated data. Finally, we discuss the remaining challenges and the promising future of using AI to diagnose and treat NPC.

Convolutional neural network based anatomical site identification for laryngoscopy quality control: A multicenter study

OBJECTIVES

Video laryngoscopy is an important diagnostic tool for head and neck cancers. The artificial intelligence (AI) system has been shown to monitor blind spots during esophagogastroduodenoscopy. This study aimed to test the performance of AI-driven intelligent laryngoscopy monitoring assistant (ILMA) for landmark anatomical sites identification on laryngoscopic images and videos based on a convolutional neural network (CNN).

MATERIALS AND METHODS

The laryngoscopic images taken from January to December 2018 were retrospectively collected, and ILMA was developed using the CNN model of Inception-ResNet-v2 + Squeeze-and-Excitation Networks (SENet). A total of 16,000 laryngoscopic images were used for training. These were assigned to 20 landmark anatomical sites covering six major head and neck regions. In addition, the performance of ILMA in identifying anatomical sites was validated using 4000 laryngoscopic images and 25 videos provided by five other tertiary hospitals.

RESULTS

ILMA identified the 20 anatomical sites on the laryngoscopic images with a total accuracy of 97.60 %, and the average sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were 100 %, 99.87 %, 97.65 %, and 99.87 %, respectively. In addition, multicenter clinical verification displayed that the accuracy of ILMA in identifying the 20 targeted anatomical sites in 25 laryngoscopic videos from five hospitals was ≥95 %.

CONCLUSION

The proposed CNN-based ILMA model can rapidly and accurately identify the anatomical sites on laryngoscopic images. The model can reflect the coverage of anatomical regions of the head and neck by laryngoscopy, showing application potential in improving the quality of laryngoscopy.

Where do we stand in AI for endoscopic image analysis? Deciphering gaps and future directions

Recent developments in deep learning have enabled data-driven algorithms that can reach human-level performance and beyond. The development and deployment of medical image analysis methods have several challenges, including data heterogeneity due to population diversity and different device manufacturers. In addition, more input from experts is required for a reliable method development process. While the exponential growth in clinical imaging data has enabled deep learning to flourish, data heterogeneity, multi-modality, and rare or inconspicuous disease cases still need to be explored. Endoscopy being highly operator-dependent with grim clinical outcomes in some disease cases, reliable and accurate automated system guidance can improve patient care. Most designed methods must be more generalisable to the unseen target data, patient population variability, and variable disease appearances. The paper reviews recent works on endoscopic image analysis with artificial intelligence (AI) and emphasises the current unmatched needs in this field. Finally, it outlines the future directions for clinically relevant complex AI solutions to improve patient outcomes.

International recommendations for plasma Epstein-Barr virus DNA measurement in nasopharyngeal carcinoma in resource-constrained settings: lessons from the COVID-19 pandemic

The effects of the COVID-19 pandemic continue to constrain health-care staff and resources worldwide, despite the availability of effective vaccines. Aerosol-generating procedures such as endoscopy, a common investigation tool for nasopharyngeal carcinoma, are recognised as a likely cause of SARS-CoV-2 spread in hospitals. Plasma Epstein-Barr virus (EBV) DNA is considered the most accurate biomarker for the routine management of nasopharyngeal carcinoma. A consensus statement on whether plasma EBV DNA can minimise the need for or replace aerosol-generating procedures, imaging methods, and face-to-face consultations in managing nasopharyngeal carcinoma is urgently needed amid the current pandemic and potentially for future highly contagious airborne diseases or natural disasters. We completed a modified Delphi consensus process of three rounds with 33 international experts in otorhinolaryngology or head and neck surgery, radiation oncology, medical oncology, and clinical oncology with vast experience in managing nasopharyngeal carcinoma, representing 51 international professional societies and national clinical trial groups. These consensus recommendations aim to enhance consistency in clinical practice, reduce ambiguity in delivering care, and offer advice for clinicians worldwide who work in endemic and non-endemic regions of nasopharyngeal carcinoma, in the context of COVID-19 and other airborne pandemics, and in future unexpected settings of severe resource constraints and insufficiency of personal protective equipment.

Deep Convolutional Neural Network for Nasopharyngeal Carcinoma Discrimination on MRI by Comparison of Hierarchical and Simple Layered Convolutional Neural Networks

Nasopharyngeal carcinoma (NPC) is one of the most common head and neck cancers. Early diagnosis plays a critical role in the treatment of NPC. To aid diagnosis, deep learning methods can provide interpretable clues for identifying NPC from magnetic resonance images (MRI). To identify the optimal models, we compared the discrimination performance of hierarchical and simple layered convolutional neural networks (CNN). Retrospectively, we collected the MRI images of patients and manually built the tailored NPC image dataset. We examined the performance of the representative CNN models including shallow CNN, ResNet50, ResNet101, and EfficientNet-B7. By fine-tuning, shallow CNN, ResNet50, ResNet101, and EfficientNet-B7 achieved the precision of 72.2%, 94.4%, 92.6%, and 88.4%, displaying the superiority of deep hierarchical neural networks. Among the examined models, ResNet50 with pre-trained weights demonstrated the best classification performance over other types of CNN with accuracy, precision, and an F1-score of 0.93, 0.94, and 0.93, respectively. The fine-tuned ResNet50 achieved the highest prediction performance and can be used as a potential tool for aiding the diagnosis of NPC tumors.