1
|
Gomes RFT, Schmith J, de Figueiredo RM, Freitas SA, Machado GN, Romanini J, Almeida JD, Pereira CT, Rodrigues JDA, Carrard VC. Convolutional neural network misclassification analysis in oral lesions: an error evaluation criterion by image characteristics. Oral Surg Oral Med Oral Pathol Oral Radiol 2024; 137:243-252. [PMID: 38161085 DOI: 10.1016/j.oooo.2023.10.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/01/2023] [Revised: 10/02/2023] [Accepted: 10/04/2023] [Indexed: 01/03/2024]
Abstract
OBJECTIVE This retrospective study analyzed the errors generated by a convolutional neural network (CNN) when performing automated classification of oral lesions according to their clinical characteristics, seeking to identify patterns in systemic errors in the intermediate layers of the CNN. STUDY DESIGN A cross-sectional analysis nested in a previous trial in which automated classification by a CNN model of elementary lesions from clinical images of oral lesions was performed. The resulting CNN classification errors formed the dataset for this study. A total of 116 real outputs were identified that diverged from the estimated outputs, representing 7.6% of the total images analyzed by the CNN. RESULTS The discrepancies between the real and estimated outputs were associated with problems relating to image sharpness, resolution, and focus; human errors; and the impact of data augmentation. CONCLUSIONS From qualitative analysis of errors in the process of automated classification of clinical images, it was possible to confirm the impact of image quality, as well as identify the strong impact of the data augmentation process. Knowledge of the factors that models evaluate to make decisions can increase confidence in the high classification potential of CNNs.
Collapse
Affiliation(s)
- Rita Fabiane Teixeira Gomes
- Department of Oral Pathology, Faculdade de Odontologia-Federal University of Rio Grande do Sul-UFRGS, Porto Alegre, Brazil.
| | - Jean Schmith
- Polytechnic School, University of Vale do Rio dos Sinos-UNISINOS, São Leopoldo, Brazil; Technology in Automation and Electronics Laboratory-TECAE Lab, University of Vale do Rio dos Sinos-UNISINOS, São Leopoldo, Brazil
| | - Rodrigo Marques de Figueiredo
- Polytechnic School, University of Vale do Rio dos Sinos-UNISINOS, São Leopoldo, Brazil; Technology in Automation and Electronics Laboratory-TECAE Lab, University of Vale do Rio dos Sinos-UNISINOS, São Leopoldo, Brazil
| | - Samuel Armbrust Freitas
- Department of Applied Computing, University of Vale do Rio dos Sinos-UNISINOS, São Leopoldo, Brazil
| | | | - Juliana Romanini
- Oral Medicine, Otorhynolaringology Service, Hospital de Clínicas de Porto Alegre (HCPA), Porto Alegre, Rio Grande do Sul, Brazil
| | - Janete Dias Almeida
- Department of Biosciences and Oral Diagnostics, São Paulo State University, Campus São José dos Campos, São Paulo, Brazil
| | | | - Jonas de Almeida Rodrigues
- Department of Surgery and Orthopaedics, Faculdade de Odontologia-Federal University of Rio Grande do Sul-UFRGS, Porto Alegre, Brazil
| | - Vinicius Coelho Carrard
- Department of Oral Pathology, Faculdade de Odontologia-Federal University of Rio Grande do Sul-UFRGS, Porto Alegre, Brazil; TelessaudeRS-UFRGS, Federal University of Rio Grande do Sul, Porto Alegre, Rio Grande do Sul, Brazil; Oral Medicine, Otorhynolaringology Service, Hospital de Clínicas de Porto Alegre (HCPA), Porto Alegre, Rio Grande do Sul, Brazil
| |
Collapse
|
2
|
Song B, KC DR, Yang RY, Li S, Zhang C, Liang R. Classification of Mobile-Based Oral Cancer Images Using the Vision Transformer and the Swin Transformer. Cancers (Basel) 2024; 16:987. [PMID: 38473348 PMCID: PMC10931180 DOI: 10.3390/cancers16050987] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/18/2024] [Revised: 02/23/2024] [Accepted: 02/26/2024] [Indexed: 03/14/2024] Open
Abstract
Oral cancer, a pervasive and rapidly growing malignant disease, poses a significant global health concern. Early and accurate diagnosis is pivotal for improving patient outcomes. Automatic diagnosis methods based on artificial intelligence have shown promising results in the oral cancer field, but the accuracy still needs to be improved for realistic diagnostic scenarios. Vision Transformers (ViT) have outperformed learning CNN models recently in many computer vision benchmark tasks. This study explores the effectiveness of the Vision Transformer and the Swin Transformer, two cutting-edge variants of the transformer architecture, for the mobile-based oral cancer image classification application. The pre-trained Swin transformer model achieved 88.7% accuracy in the binary classification task, outperforming the ViT model by 2.3%, while the conventional convolutional network model VGG19 and ResNet50 achieved 85.2% and 84.5% accuracy. Our experiments demonstrate that these transformer-based architectures outperform traditional convolutional neural networks in terms of oral cancer image classification, and underscore the potential of the ViT and the Swin Transformer in advancing the state of the art in oral cancer image analysis.
Collapse
Affiliation(s)
- Bofan Song
- Wyant College of Optical Sciences, The University of Arizona, Tucson, AZ 85721, USA
| | - Dharma Raj KC
- Computer Science Department, The University of Arizona, Tucson, AZ 85721, USA
| | - Rubin Yuchan Yang
- Computer Science Department, The University of Arizona, Tucson, AZ 85721, USA
| | - Shaobai Li
- Wyant College of Optical Sciences, The University of Arizona, Tucson, AZ 85721, USA
| | - Chicheng Zhang
- Computer Science Department, The University of Arizona, Tucson, AZ 85721, USA
| | - Rongguang Liang
- Wyant College of Optical Sciences, The University of Arizona, Tucson, AZ 85721, USA
| |
Collapse
|
3
|
Hou H, Mitbander R, Tang Y, Azimuddin A, Carns J, Schwarz RA, Richards-Kortum RR. Optical imaging technologies for in vivo cancer detection in low-resource settings. CURRENT OPINION IN BIOMEDICAL ENGINEERING 2023; 28:100495. [PMID: 38406798 PMCID: PMC10883072 DOI: 10.1016/j.cobme.2023.100495] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/27/2024]
Abstract
Cancer continues to affect underserved populations disproportionately. Novel optical imaging technologies, which can provide rapid, non-invasive, and accurate cancer detection at the point of care, have great potential to improve global cancer care. This article reviews the recent technical innovations and clinical translation of low-cost optical imaging technologies, highlighting the advances in both hardware and software, especially the integration of artificial intelligence, to improve in vivo cancer detection in low-resource settings. Additionally, this article provides an overview of existing challenges and future perspectives of adapting optical imaging technologies into clinical practice, which can potentially contribute to novel insights and programs that effectively improve cancer detection in low-resource settings.
Collapse
Affiliation(s)
- Huayu Hou
- Department of Bioengineering, Rice University, Houston, TX 77005, USA
| | - Ruchika Mitbander
- Department of Bioengineering, Rice University, Houston, TX 77005, USA
| | - Yubo Tang
- Department of Bioengineering, Rice University, Houston, TX 77005, USA
| | - Ahad Azimuddin
- School of Medicine, Texas A&M University, Houston, TX 77030, USA
| | - Jennifer Carns
- Department of Bioengineering, Rice University, Houston, TX 77005, USA
| | - Richard A Schwarz
- Department of Bioengineering, Rice University, Houston, TX 77005, USA
| | | |
Collapse
|
4
|
Zhou L, Jiang H, Li G, Ding J, Lv C, Duan M, Wang W, Chen K, Shen N, Huang X. Point-wise spatial network for identifying carcinoma at the upper digestive and respiratory tract. BMC Med Imaging 2023; 23:140. [PMID: 37749498 PMCID: PMC10521533 DOI: 10.1186/s12880-023-01076-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/12/2022] [Accepted: 08/07/2023] [Indexed: 09/27/2023] Open
Abstract
PROBLEM Artificial intelligence has been widely investigated for diagnosis and treatment strategy design, with some models proposed for detecting oral pharyngeal, nasopharyngeal, or laryngeal carcinoma. However, no comprehensive model has been established for these regions. AIM Our hypothesis was that a common pattern in the cancerous appearance of these regions could be recognized and integrated into a single model, thus improving the efficacy of deep learning models. METHODS We utilized a point-wise spatial attention network model to perform semantic segmentation in these regions. RESULTS Our study demonstrated an excellent outcome, with an average mIoU of 86.3%, and an average pixel accuracy of 96.3%. CONCLUSION The research confirmed that the mucosa of oral pharyngeal, nasopharyngeal, and laryngeal regions may share a common appearance, including the appearance of tumors, which can be recognized by a single artificial intelligence model. Therefore, a deep learning model could be constructed to effectively recognize these tumors.
Collapse
Affiliation(s)
- Lei Zhou
- Department of Otorhinolaryngology-Head and Neck Surgery, Zhongshan Hospital Affiliated to Fudan University, Xuhui District, 180 Fenglin Road, , Shanghai, 200032, P. R. China
| | - Huaili Jiang
- Department of Otorhinolaryngology-Head and Neck Surgery, Zhongshan Hospital Affiliated to Fudan University, Xuhui District, 180 Fenglin Road, , Shanghai, 200032, P. R. China
| | - Guangyao Li
- Department of Otorhinolaryngology-Head and Neck Surgery, Zhongshan Hospital Affiliated to Fudan University, Xuhui District, 180 Fenglin Road, , Shanghai, 200032, P. R. China
| | - Jiaye Ding
- Department of Otorhinolaryngology-Head and Neck Surgery, Zhongshan Hospital Affiliated to Fudan University, Xuhui District, 180 Fenglin Road, , Shanghai, 200032, P. R. China
| | - Cuicui Lv
- Department of Otorhinolaryngology-Head and Neck Surgery, Zhongshan Hospital Affiliated to Fudan University, Xuhui District, 180 Fenglin Road, , Shanghai, 200032, P. R. China
| | - Maoli Duan
- Department of Clinical Science, Intervention and Technology, Karolinska Institutet, Stockholm, Sweden
- Department of Otolaryngology Head and Neck Surgery, Karolinska University Hospital, 171 76, Stockholm, Sweden
| | - Wenfeng Wang
- Institute of Artificial Intelligence and Blockchain, Guangzhou University, Guangzhou, 510006, P. R. China
| | - Kongyang Chen
- Institute of Artificial Intelligence and Blockchain, Guangzhou University, Guangzhou, 510006, P. R. China
- Pazhou Lab, Guangzhou, 510330, P. R. China
| | - Na Shen
- Department of Otorhinolaryngology-Head and Neck Surgery, Zhongshan Hospital Affiliated to Fudan University, Xuhui District, 180 Fenglin Road, , Shanghai, 200032, P. R. China.
| | - Xinsheng Huang
- Department of Otorhinolaryngology-Head and Neck Surgery, Zhongshan Hospital Affiliated to Fudan University, Xuhui District, 180 Fenglin Road, , Shanghai, 200032, P. R. China.
| |
Collapse
|
5
|
Gomes RFT, Schuch LF, Martins MD, Honório EF, de Figueiredo RM, Schmith J, Machado GN, Carrard VC. Use of Deep Neural Networks in the Detection and Automated Classification of Lesions Using Clinical Images in Ophthalmology, Dermatology, and Oral Medicine-A Systematic Review. J Digit Imaging 2023; 36:1060-1070. [PMID: 36650299 PMCID: PMC10287602 DOI: 10.1007/s10278-023-00775-3] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2022] [Revised: 01/03/2023] [Accepted: 01/04/2023] [Indexed: 01/19/2023] Open
Abstract
Artificial neural networks (ANN) are artificial intelligence (AI) techniques used in the automated recognition and classification of pathological changes from clinical images in areas such as ophthalmology, dermatology, and oral medicine. The combination of enterprise imaging and AI is gaining notoriety for its potential benefits in healthcare areas such as cardiology, dermatology, ophthalmology, pathology, physiatry, radiation oncology, radiology, and endoscopic. The present study aimed to analyze, through a systematic literature review, the application of performance of ANN and deep learning in the recognition and automated classification of lesions from clinical images, when comparing to the human performance. The PRISMA 2020 approach (Preferred Reporting Items for Systematic Reviews and Meta-analyses) was used by searching four databases of studies that reference the use of IA to define the diagnosis of lesions in ophthalmology, dermatology, and oral medicine areas. A quantitative and qualitative analyses of the articles that met the inclusion criteria were performed. The search yielded the inclusion of 60 studies. It was found that the interest in the topic has increased, especially in the last 3 years. We observed that the performance of IA models is promising, with high accuracy, sensitivity, and specificity, most of them had outcomes equivalent to human comparators. The reproducibility of the performance of models in real-life practice has been reported as a critical point. Study designs and results have been progressively improved. IA resources have the potential to contribute to several areas of health. In the coming years, it is likely to be incorporated into everyday life, contributing to the precision and reducing the time required by the diagnostic process.
Collapse
Affiliation(s)
- Rita Fabiane Teixeira Gomes
- Graduate Program in Dentistry, School of Dentistry, Federal University of Rio Grande Do Sul, Barcelos 2492/503, Bairro Santana, Porto Alegre, RS, CEP 90035-003, Brazil.
| | - Lauren Frenzel Schuch
- Department of Oral Diagnosis, Piracicaba Dental School, University of Campinas, Piracicaba, Brazil
| | - Manoela Domingues Martins
- Graduate Program in Dentistry, School of Dentistry, Federal University of Rio Grande Do Sul, Barcelos 2492/503, Bairro Santana, Porto Alegre, RS, CEP 90035-003, Brazil
- Department of Oral Diagnosis, Piracicaba Dental School, University of Campinas, Piracicaba, Brazil
| | | | - Rodrigo Marques de Figueiredo
- Technology in Automation and Electronics Laboratory - TECAE Lab, University of Vale Do Rio Dos Sinos - UNISINOS, São Leopoldo, Brazil
| | - Jean Schmith
- Technology in Automation and Electronics Laboratory - TECAE Lab, University of Vale Do Rio Dos Sinos - UNISINOS, São Leopoldo, Brazil
| | - Giovanna Nunes Machado
- Technology in Automation and Electronics Laboratory - TECAE Lab, University of Vale Do Rio Dos Sinos - UNISINOS, São Leopoldo, Brazil
| | - Vinicius Coelho Carrard
- Graduate Program in Dentistry, School of Dentistry, Federal University of Rio Grande Do Sul, Barcelos 2492/503, Bairro Santana, Porto Alegre, RS, CEP 90035-003, Brazil
- Department of Epidemiology, School of Medicine, TelessaúdeRS-UFRGS, Federal University of Rio Grande Do Sul, Porto Alegre, RS, Brazil
- Department of Oral Medicine, Otorhinolaryngology Service, Hospital de Clínicas de Porto Alegre (HCPA), Porto Alegre, RS, Brazil
| |
Collapse
|
6
|
Nguyen J, Takesh T, Parsangi N, Song B, Liang R, Wilder-Smith P. Compliance with Specialist Referral for Increased Cancer Risk in Low-Resource Settings: In-Person vs. Telehealth Options. Cancers (Basel) 2023; 15:2775. [PMID: 37345112 DOI: 10.3390/cancers15102775] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/18/2023] [Revised: 05/13/2023] [Accepted: 05/13/2023] [Indexed: 06/23/2023] Open
Abstract
Efforts are underway to improve the accuracy of non-specialist screening for oral cancer (OC) risk, yet better screening will only translate into improved outcomes if at-risk individuals comply with specialist referral. Most individuals from low-resource, minority, and underserved (LRMU) populations fail to complete a specialist referral for OC risk. The goal was to evaluate the impact of a novel approach on specialist referral compliance in individuals with a positive OC risk screening outcome. A total of 60 LRMU subjects who had screened positive for increased OC risk were recruited and given the choice of referral for an in-person (20 subjects) or a telehealth (40 subjects) specialist visit. Referral compliance was tracked weekly over 6 months. Compliance was 30% in the in-person group, and 83% in the telehealth group. Approximately 83-85% of subjects from both groups who had complied with the first specialist referral complied with a second follow-up in-person specialist visit. Overall, 72.5% of subjects who had chosen a remote first specialist visit had entered into the continuum of care by the study end, vs. 25% of individuals in the in-person specialist group. A two-step approach that uses telehealth to overcome barriers may improve specialist referral compliance in LRMU individuals with increased OC risk.
Collapse
Affiliation(s)
- James Nguyen
- Beckman Laser Institute and Medical Clinic, University of California Irvine School of Medicine, Irvine, CA 92612, USA
| | - Thair Takesh
- Beckman Laser Institute and Medical Clinic, University of California Irvine School of Medicine, Irvine, CA 92612, USA
| | - Negah Parsangi
- Beckman Laser Institute and Medical Clinic, University of California Irvine School of Medicine, Irvine, CA 92612, USA
| | - Bofan Song
- College of Optical Sciences, University of Arizona, Tucson, AZ 85721, USA
| | - Rongguang Liang
- College of Optical Sciences, University of Arizona, Tucson, AZ 85721, USA
| | - Petra Wilder-Smith
- Beckman Laser Institute and Medical Clinic, University of California Irvine School of Medicine, Irvine, CA 92612, USA
| |
Collapse
|
7
|
Gomes RFT, Schmith J, de Figueiredo RM, Freitas SA, Machado GN, Romanini J, Carrard VC. Use of Artificial Intelligence in the Classification of Elementary Oral Lesions from Clinical Images. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2023; 20:ijerph20053894. [PMID: 36900902 PMCID: PMC10002140 DOI: 10.3390/ijerph20053894] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/02/2023] [Revised: 02/16/2023] [Accepted: 02/20/2023] [Indexed: 06/01/2023]
Abstract
OBJECTIVES Artificial intelligence has generated a significant impact in the health field. The aim of this study was to perform the training and validation of a convolutional neural network (CNN)-based model to automatically classify six clinical representation categories of oral lesion images. METHOD The CNN model was developed with the objective of automatically classifying the images into six categories of elementary lesions: (1) papule/nodule; (2) macule/spot; (3) vesicle/bullous; (4) erosion; (5) ulcer and (6) plaque. We selected four architectures and using our dataset we decided to test the following architectures: ResNet-50, VGG16, InceptionV3 and Xception. We used the confusion matrix as the main metric for the CNN evaluation and discussion. RESULTS A total of 5069 images of oral mucosa lesions were used. The oral elementary lesions classification reached the best result using an architecture based on InceptionV3. After hyperparameter optimization, we reached more than 71% correct predictions in all six lesion classes. The classification achieved an average accuracy of 95.09% in our dataset. CONCLUSIONS We reported the development of an artificial intelligence model for the automated classification of elementary lesions from oral clinical images, achieving satisfactory performance. Future directions include the study of including trained layers to establish patterns of characteristics that determine benign, potentially malignant and malignant lesions.
Collapse
Affiliation(s)
- Rita Fabiane Teixeira Gomes
- Department of Oral Pathology, Faculdade de Odontologia, Federal University of Rio Grande do Sul (UFRGS), Porto Alegre 90035-003, Brazil
| | - Jean Schmith
- Polytechnic School, University of Vale do Rio dos Sinos—UNISINOS, São Leopoldo 93022-750, Brazil
- Technology in Automation and Electronics Laboratory—TECAE Lab, University of Vale do Rio dos Sinos—UNISINOS, São Leopoldo 93022-750, Brazil
| | - Rodrigo Marques de Figueiredo
- Polytechnic School, University of Vale do Rio dos Sinos—UNISINOS, São Leopoldo 93022-750, Brazil
- Technology in Automation and Electronics Laboratory—TECAE Lab, University of Vale do Rio dos Sinos—UNISINOS, São Leopoldo 93022-750, Brazil
| | - Samuel Armbrust Freitas
- Department of Applied Computing, University of Vale do Rio dos Sinos—UNISINOS, São Leopoldo 93022-750, Brazil
| | - Giovanna Nunes Machado
- Polytechnic School, University of Vale do Rio dos Sinos—UNISINOS, São Leopoldo 93022-750, Brazil
| | - Juliana Romanini
- Oral Medicine, Otorhynolaringology Service, Hospital de Clínicas de Porto Alegre (HCPA), Porto Alegre 90035-003, Brazil
| | - Vinicius Coelho Carrard
- Department of Oral Pathology, Faculdade de Odontologia, Federal University of Rio Grande do Sul (UFRGS), Porto Alegre 90035-003, Brazil
- Oral Medicine, Otorhynolaringology Service, Hospital de Clínicas de Porto Alegre (HCPA), Porto Alegre 90035-003, Brazil
- TelessaudeRS, Federal University of Rio Grande do Sul (UFRGS), Porto Alegre 91501-970, Brazil
| |
Collapse
|
8
|
Song B, Li S, Sunny S, Gurushanth K, Mendonca P, Mukhia N, Patrick S, Peterson T, Gurudath S, Raghavan S, Tsusennaro I, Leivon ST, Kolur T, Shetty V, Bushan V, Ramesh R, Pillai V, Wilder-Smith P, Suresh A, Kuriakose MA, Birur P, Liang R. Exploring uncertainty measures in convolutional neural network for semantic segmentation of oral cancer images. JOURNAL OF BIOMEDICAL OPTICS 2022; 27:115001. [PMID: 36329004 PMCID: PMC9630461 DOI: 10.1117/1.jbo.27.11.115001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/04/2022] [Accepted: 10/13/2022] [Indexed: 06/16/2023]
Abstract
SIGNIFICANCE Oral cancer is one of the most prevalent cancers, especially in middle- and low-income countries such as India. Automatic segmentation of oral cancer images can improve the diagnostic workflow, which is a significant task in oral cancer image analysis. Despite the remarkable success of deep-learning networks in medical segmentation, they rarely provide uncertainty quantification for their output. AIM We aim to estimate uncertainty in a deep-learning approach to semantic segmentation of oral cancer images and to improve the accuracy and reliability of predictions. APPROACH This work introduced a UNet-based Bayesian deep-learning (BDL) model to segment potentially malignant and malignant lesion areas in the oral cavity. The model can quantify uncertainty in predictions. We also developed an efficient model that increased the inference speed, which is almost six times smaller and two times faster (inference speed) than the original UNet. The dataset in this study was collected using our customized screening platform and was annotated by oral oncology specialists. RESULTS The proposed approach achieved good segmentation performance as well as good uncertainty estimation performance. In the experiments, we observed an improvement in pixel accuracy and mean intersection over union by removing uncertain pixels. This result reflects that the model provided less accurate predictions in uncertain areas that may need more attention and further inspection. The experiments also showed that with some performance compromises, the efficient model reduced computation time and model size, which expands the potential for implementation on portable devices used in resource-limited settings. CONCLUSIONS Our study demonstrates the UNet-based BDL model not only can perform potentially malignant and malignant oral lesion segmentation, but also can provide informative pixel-level uncertainty estimation. With this extra uncertainty information, the accuracy and reliability of the model’s prediction can be improved.
Collapse
Affiliation(s)
- Bofan Song
- The University of Arizona, Wyant College of Optical Sciences, Tucson, Arizona, United States
| | - Shaobai Li
- The University of Arizona, Wyant College of Optical Sciences, Tucson, Arizona, United States
| | - Sumsum Sunny
- Mazumdar Shaw Medical Centre, Bangalore, Karnataka, India
| | | | | | - Nirza Mukhia
- KLE Society Institute of Dental Sciences, Bangalore, Karnataka, India
| | | | - Tyler Peterson
- The University of Arizona, Wyant College of Optical Sciences, Tucson, Arizona, United States
| | - Shubha Gurudath
- KLE Society Institute of Dental Sciences, Bangalore, Karnataka, India
| | | | - Imchen Tsusennaro
- Christian Institute of Health Sciences and Research, Dimapur, Nagaland, India
| | - Shirley T. Leivon
- Christian Institute of Health Sciences and Research, Dimapur, Nagaland, India
| | - Trupti Kolur
- Mazumdar Shaw Medical Foundation, Bangalore, Karnataka, India
| | - Vivek Shetty
- Mazumdar Shaw Medical Foundation, Bangalore, Karnataka, India
| | - Vidya Bushan
- Mazumdar Shaw Medical Foundation, Bangalore, Karnataka, India
| | - Rohan Ramesh
- Christian Institute of Health Sciences and Research, Dimapur, Nagaland, India
| | - Vijay Pillai
- Mazumdar Shaw Medical Foundation, Bangalore, Karnataka, India
| | - Petra Wilder-Smith
- University of California, Beckman Laser Institute & Medical Clinic, Irvine, California, United States
| | - Amritha Suresh
- Mazumdar Shaw Medical Centre, Bangalore, Karnataka, India
- Mazumdar Shaw Medical Foundation, Bangalore, Karnataka, India
| | | | - Praveen Birur
- KLE Society Institute of Dental Sciences, Bangalore, Karnataka, India
- Biocon Foundation, Bangalore, Karnataka, India
| | - Rongguang Liang
- The University of Arizona, Wyant College of Optical Sciences, Tucson, Arizona, United States
| |
Collapse
|
9
|
Mobile Health (mHealth) Technology in Early Detection and Diagnosis of Oral Cancer-A Scoping Review of the Current Scenario and Feasibility. JOURNAL OF HEALTHCARE ENGINEERING 2022; 2022:4383303. [PMID: 36312594 PMCID: PMC9605853 DOI: 10.1155/2022/4383303] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/31/2022] [Accepted: 09/30/2022] [Indexed: 11/25/2022]
Abstract
Objective Oral cancer is one of the most common types of cancer with dreadful consequences. But it can be detected early without much expensive equipment. Screening and early detection of oral cancer using Mobile health (mHealth) technology are reported due to the availability of the extensive network of mobile phones across populations. Therefore, we aimed to explore the existing literature regarding mHealth feasibility in the early detection of oral cancer. Materials and Method. An extensive search was conducted to explore the literature on the feasibility of mobile health for early oral cancer. Clinical studies reporting kappa agreement between on-site dentists and offsite health care workers/dentists in the early detection of oral cancer were included in this review. Studies describing the development of a diagnostic device, app development, and qualitative interviews among practitioners trained in using mobile health were also included in this review for a broader perspective on mHealth. Results While most of the studies described various diagnostic accuracies using mHealth for oral cancer early detection, few studies reported the development of mobile applications, novel device designs for mHealth applications, and the feasibility of a few mHealth programs for early oral cancer detection. Community health workers equipped with a mobile phone-based app could identify “abnormal” oral lesions. Overall, many studies reported high sensitivity, specificity, and Kappa value of agreement. Effectiveness, advantages, and barriers in oral cancer screening using mHealth are also described. Conclusion The overall results show that remote diagnosis for early detection of oral cancer using mHealth was found useful in remote settings.
Collapse
|
10
|
Figueroa KC, Song B, Sunny S, Li S, Gurushanth K, Mendonca P, Mukhia N, Patrick S, Gurudath S, Raghavan S, Imchen T, Leivon ST, Kolur T, Shetty V, Bushan V, Ramesh R, Pillai V, Wilder-Smith P, Sigamani A, Suresh A, Kuriakose MA, Birur P, Liang R. Interpretable deep learning approach for oral cancer classification using guided attention inference network. JOURNAL OF BIOMEDICAL OPTICS 2022; 27:JBO-210329GR. [PMID: 35023333 PMCID: PMC8754153 DOI: 10.1117/1.jbo.27.1.015001] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/25/2021] [Accepted: 12/27/2021] [Indexed: 05/05/2023]
Abstract
SIGNIFICANCE Convolutional neural networks (CNNs) show the potential for automated classification of different cancer lesions. However, their lack of interpretability and explainability makes CNNs less than understandable. Furthermore, CNNs may incorrectly concentrate on other areas surrounding the salient object, rather than the network's attention focusing directly on the object to be recognized, as the network has no incentive to focus solely on the correct subjects to be detected. This inhibits the reliability of CNNs, especially for biomedical applications. AIM Develop a deep learning training approach that could provide understandability to its predictions and directly guide the network to concentrate its attention and accurately delineate cancerous regions of the image. APPROACH We utilized Selvaraju et al.'s gradient-weighted class activation mapping to inject interpretability and explainability into CNNs. We adopted a two-stage training process with data augmentation techniques and Li et al.'s guided attention inference network (GAIN) to train images captured using our customized mobile oral screening devices. The GAIN architecture consists of three streams of network training: classification stream, attention mining stream, and bounding box stream. By adopting the GAIN training architecture, we jointly optimized the classification and segmentation accuracy of our CNN by treating these attention maps as reliable priors to develop attention maps with more complete and accurate segmentation. RESULTS The network's attention map will help us to actively understand what the network is focusing on and looking at during its decision-making process. The results also show that the proposed method could guide the trained neural network to highlight and focus its attention on the correct lesion areas in the images when making a decision, rather than focusing its attention on relevant yet incorrect regions. CONCLUSIONS We demonstrate the effectiveness of our approach for more interpretable and reliable oral potentially malignant lesion and malignant lesion classification.
Collapse
Affiliation(s)
- Kevin Chew Figueroa
- The University of Arizona, Wyant College of Optical Sciences, Tucson, Arizona, United States
- Address all correspondence to Kevin C. Figueroa, ; Bofan Song, ; Rongguang Liang,
| | - Bofan Song
- The University of Arizona, Wyant College of Optical Sciences, Tucson, Arizona, United States
- Address all correspondence to Kevin C. Figueroa, ; Bofan Song, ; Rongguang Liang,
| | - Sumsum Sunny
- Mazumdar Shaw Medical Centre, Bangalore, Karnataka, India
| | - Shaobai Li
- The University of Arizona, Wyant College of Optical Sciences, Tucson, Arizona, United States
| | | | | | - Nirza Mukhia
- KLE Society Institute of Dental Sciences, Bangalore, Karnataka, India
| | | | - Shubha Gurudath
- KLE Society Institute of Dental Sciences, Bangalore, Karnataka, India
| | | | - Tsusennaro Imchen
- Christian Institute of Health Sciences and Research, Dimapur, Nagaland, India
| | - Shirley T. Leivon
- Christian Institute of Health Sciences and Research, Dimapur, Nagaland, India
| | - Trupti Kolur
- Mazumdar Shaw Medical Foundation, Bangalore, Karnataka, India
| | - Vivek Shetty
- Mazumdar Shaw Medical Foundation, Bangalore, Karnataka, India
| | - Vidya Bushan
- Mazumdar Shaw Medical Foundation, Bangalore, Karnataka, India
| | - Rohan Ramesh
- Christian Institute of Health Sciences and Research, Dimapur, Nagaland, India
| | - Vijay Pillai
- Mazumdar Shaw Medical Foundation, Bangalore, Karnataka, India
| | - Petra Wilder-Smith
- University of California, Irvine, Beckman Laser Institute & Medical Clinic, Irvine, California, United States
| | - Alben Sigamani
- Mazumdar Shaw Medical Foundation, Bangalore, Karnataka, India
| | - Amritha Suresh
- Mazumdar Shaw Medical Centre, Bangalore, Karnataka, India
- Mazumdar Shaw Medical Foundation, Bangalore, Karnataka, India
| | - Moni Abraham Kuriakose
- Mazumdar Shaw Medical Centre, Bangalore, Karnataka, India
- Mazumdar Shaw Medical Foundation, Bangalore, Karnataka, India
- Cochin Cancer Research Center, Kochi, Kerala, India
| | - Praveen Birur
- KLE Society Institute of Dental Sciences, Bangalore, Karnataka, India
- Biocon Foundation, Bangalore, Karnataka, India
| | - Rongguang Liang
- The University of Arizona, Wyant College of Optical Sciences, Tucson, Arizona, United States
- Address all correspondence to Kevin C. Figueroa, ; Bofan Song, ; Rongguang Liang,
| |
Collapse
|
11
|
Cavalcanti TC, Lew HM, Lee K, Lee SY, Park MK, Hwang JY. Intelligent smartphone-based multimode imaging otoscope for the mobile diagnosis of otitis media. BIOMEDICAL OPTICS EXPRESS 2021; 12:7765-7779. [PMID: 35003865 PMCID: PMC8713661 DOI: 10.1364/boe.441590] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/26/2021] [Revised: 11/10/2021] [Accepted: 11/10/2021] [Indexed: 06/14/2023]
Abstract
Otitis media (OM) is one of the most common ear diseases in children and a common reason for outpatient visits to medical doctors in primary care practices. Adhesive OM (AdOM) is recognized as a sequela of OM with effusion (OME) and often requires surgical intervention. OME and AdOM exhibit similar symptoms, and it is difficult to distinguish between them using a conventional otoscope in a primary care unit. The accuracy of the diagnosis is highly dependent on the experience of the examiner. The development of an advanced otoscope with less variation in diagnostic accuracy by the examiner is crucial for a more accurate diagnosis. Thus, we developed an intelligent smartphone-based multimode imaging otoscope for better diagnosis of OM, even in mobile environments. The system offers spectral and autofluorescence imaging of the tympanic membrane using a smartphone attached to the developed multimode imaging module. Moreover, it is capable of intelligent analysis for distinguishing between normal, OME, and AdOM ears using a machine learning algorithm. Using the developed system, we examined the ears of 69 patients to assess their performance for distinguishing between normal, OME, and AdOM ears. In the classification of ear diseases, the multimode system based on machine learning analysis performed better in terms of accuracy and F1 scores than single RGB image analysis, RGB/fluorescence image analysis, and the analysis of spectral image cubes only, respectively. These results demonstrate that the intelligent multimode diagnostic capability of an otoscope would be beneficial for better diagnosis and management of OM.
Collapse
Affiliation(s)
- Thiago C Cavalcanti
- Department of Information and Communication Engineering, Daegu Gyeongbuk Institute of Science and Technology, Daegu, Republic of Korea
| | - Hah Min Lew
- Department of Information and Communication Engineering, Daegu Gyeongbuk Institute of Science and Technology, Daegu, Republic of Korea
| | - Kyungsu Lee
- Department of Information and Communication Engineering, Daegu Gyeongbuk Institute of Science and Technology, Daegu, Republic of Korea
| | - Sang-Yeon Lee
- Department of Otorhinolaryngology-Head and Neck Surgery, Seoul National University Hospital, Seoul, Republic of Korea
| | - Moo Kyun Park
- Department of Otorhinolaryngology-Head and Neck Surgery, Seoul National University Hospital, Seoul, Republic of Korea
- co-first authors
| | - Jae Youn Hwang
- Department of Information and Communication Engineering, Daegu Gyeongbuk Institute of Science and Technology, Daegu, Republic of Korea
- co-first authors
| |
Collapse
|
12
|
Song B, Sunny S, Li S, Gurushanth K, Mendonca P, Mukhia N, Patrick S, Gurudath S, Raghavan S, Tsusennaro I, Leivon ST, Kolur T, Shetty V, Bushan VR, Ramesh R, Peterson T, Pillai V, Wilder-Smith P, Sigamani A, Suresh A, Kuriakose MA, Birur P, Liang R. Bayesian deep learning for reliable oral cancer image classification. BIOMEDICAL OPTICS EXPRESS 2021; 12:6422-6430. [PMID: 34745746 PMCID: PMC8547976 DOI: 10.1364/boe.432365] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/31/2021] [Revised: 08/29/2021] [Accepted: 09/07/2021] [Indexed: 05/16/2023]
Abstract
In medical imaging, deep learning-based solutions have achieved state-of-the-art performance. However, reliability restricts the integration of deep learning into practical medical workflows since conventional deep learning frameworks cannot quantitatively assess model uncertainty. In this work, we propose to address this shortcoming by utilizing a Bayesian deep network capable of estimating uncertainty to assess oral cancer image classification reliability. We evaluate the model using a large intraoral cheek mucosa image dataset captured using our customized device from high-risk population to show that meaningful uncertainty information can be produced. In addition, our experiments show improved accuracy by uncertainty-informed referral. The accuracy of retained data reaches roughly 90% when referring either 10% of all cases or referring cases whose uncertainty value is greater than 0.3. The performance can be further improved by referring more patients. The experiments show the model is capable of identifying difficult cases needing further inspection.
Collapse
Affiliation(s)
- Bofan Song
- Wyant College of Optical Sciences, The University of Arizona, Tucson, Arizona 85721, USA
| | | | - Shaobai Li
- Wyant College of Optical Sciences, The University of Arizona, Tucson, Arizona 85721, USA
| | | | | | - Nirza Mukhia
- KLE Society Institute of Dental Sciences, Bangalore, India
| | | | | | | | | | - Shirley T Leivon
- Christian Institute of Health Sciences and Research, Dimapur, India
| | - Trupti Kolur
- Mazumdar Shaw Medical Foundation, Bangalore, India
| | - Vivek Shetty
- Mazumdar Shaw Medical Foundation, Bangalore, India
| | | | - Rohan Ramesh
- Christian Institute of Health Sciences and Research, Dimapur, India
| | - Tyler Peterson
- Wyant College of Optical Sciences, The University of Arizona, Tucson, Arizona 85721, USA
| | - Vijay Pillai
- Mazumdar Shaw Medical Foundation, Bangalore, India
| | - Petra Wilder-Smith
- Beckman Laser Institute and Medical Clinic, University of California, Irvine, California 92697, USA
| | | | - Amritha Suresh
- Mazumdar Shaw Medical Centre, Bangalore, India
- Mazumdar Shaw Medical Foundation, Bangalore, India
| | | | - Praveen Birur
- KLE Society Institute of Dental Sciences, Bangalore, India
- Mazumdar Shaw Medical Foundation, Bangalore, India
| | - Rongguang Liang
- Wyant College of Optical Sciences, The University of Arizona, Tucson, Arizona 85721, USA
| |
Collapse
|
13
|
Song B, Li S, Sunny S, Gurushanth K, Mendonca P, Mukhia N, Patrick S, Gurudath S, Raghavan S, Tsusennaro I, Leivon ST, Kolur T, Shetty V, Bushan V, Ramesh R, Peterson T, Pillai V, Wilder-Smith P, Sigamani A, Suresh A, Kuriakose MA, Birur P, Liang R. Classification of imbalanced oral cancer image data from high-risk population. JOURNAL OF BIOMEDICAL OPTICS 2021; 26:JBO-210246R. [PMID: 34689442 PMCID: PMC8536945 DOI: 10.1117/1.jbo.26.10.105001] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 08/03/2021] [Accepted: 09/28/2021] [Indexed: 06/13/2023]
Abstract
SIGNIFICANCE Early detection of oral cancer is vital for high-risk patients, and machine learning-based automatic classification is ideal for disease screening. However, current datasets collected from high-risk populations are unbalanced and often have detrimental effects on the performance of classification. AIM To reduce the class bias caused by data imbalance. APPROACH We collected 3851 polarized white light cheek mucosa images using our customized oral cancer screening device. We use weight balancing, data augmentation, undersampling, focal loss, and ensemble methods to improve the neural network performance of oral cancer image classification with the imbalanced multi-class datasets captured from high-risk populations during oral cancer screening in low-resource settings. RESULTS By applying both data-level and algorithm-level approaches to the deep learning training process, the performance of the minority classes, which were difficult to distinguish at the beginning, has been improved. The accuracy of "premalignancy" class is also increased, which is ideal for screening applications. CONCLUSIONS Experimental results show that the class bias induced by imbalanced oral cancer image datasets could be reduced using both data- and algorithm-level methods. Our study may provide an important basis for helping understand the influence of unbalanced datasets on oral cancer deep learning classifiers and how to mitigate.
Collapse
Affiliation(s)
- Bofan Song
- The University of Arizona, Wyant College of Optical Sciences, Tucson, Arizona, United States
| | - Shaobai Li
- The University of Arizona, Wyant College of Optical Sciences, Tucson, Arizona, United States
| | | | | | | | - Nirza Mukhia
- KLE Society Institute of Dental Sciences, Bangalore, India
| | | | | | | | | | | | - Trupti Kolur
- Mazumdar Shaw Medical Foundation, Bangalore, India
| | - Vivek Shetty
- Mazumdar Shaw Medical Foundation, Bangalore, India
| | - Vidya Bushan
- Mazumdar Shaw Medical Foundation, Bangalore, India
| | - Rohan Ramesh
- Christian Institute of Health Sciences and Research, Dimapur, India
| | - Tyler Peterson
- The University of Arizona, Wyant College of Optical Sciences, Tucson, Arizona, United States
| | - Vijay Pillai
- Mazumdar Shaw Medical Foundation, Bangalore, India
| | - Petra Wilder-Smith
- University of California Beckman Laser Institute and Medical Clinic, Irvine, California, United States
| | | | - Amritha Suresh
- Mazumdar Shaw Medical Centre, Bangalore, India
- Mazumdar Shaw Medical Foundation, Bangalore, India
| | | | - Praveen Birur
- KLE Society Institute of Dental Sciences, Bangalore, India
- Biocon Foundation, Bangalore, India
| | - Rongguang Liang
- The University of Arizona, Wyant College of Optical Sciences, Tucson, Arizona, United States
| |
Collapse
|
14
|
Automated Detection and Classification of Oral Lesions Using Deep Learning to Detect Oral Potentially Malignant Disorders. Cancers (Basel) 2021; 13:cancers13112766. [PMID: 34199471 PMCID: PMC8199603 DOI: 10.3390/cancers13112766] [Citation(s) in RCA: 28] [Impact Index Per Article: 9.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/20/2021] [Revised: 04/24/2021] [Accepted: 04/26/2021] [Indexed: 11/17/2022] Open
Abstract
Simple Summary Oral cancer is the most common type of head and neck cancer worldwide. The detection of oral potentially malignant disorders, which carry a risk of developing into cancer, often provides the best chances for curing the disease and is therefore crucial for improving morbidity and mortality outcomes from oral cancer. In this study, we explored the potential applications of computer vision and deep learning techniques in the oral cancer domain within the scope of photographic images and investigated the prospects of an automated system for identifying oral potentially malignant disorders with a two-stage pipeline. Our preliminary results demonstrate the feasibility of deep learning-based approaches for the automated detection and classification of oral lesions in real time. The proposed model offers great potential as a low-cost and non-invasive tool that can support screening processes and improve the detection of oral potentially malignant disorders. Abstract Oral cancer is the most common type of head and neck cancer worldwide, leading to approximately 177,757 deaths every year. When identified at early stages, oral cancers can achieve survival rates of up to 75–90%. However, the majority of the cases are diagnosed at an advanced stage mainly due to the lack of public awareness about oral cancer signs and the delays in referrals to oral cancer specialists. As early detection and treatment remain to be the most effective measures in improving oral cancer outcomes, the development of vision-based adjunctive technologies that can detect oral potentially malignant disorders (OPMDs), which carry a risk of cancer development, present significant opportunities for the oral cancer screening process. In this study, we explored the potential applications of computer vision techniques in the oral cancer domain within the scope of photographic images and investigated the prospects of an automated system for detecting OPMD. Exploiting the advancements in deep learning, a two-stage model was proposed to detect oral lesions with a detector network and classify the detected region into three categories (benign, OPMD, carcinoma) with a second-stage classifier network. Our preliminary results demonstrate the feasibility of deep learning-based approaches for the automated detection and classification of oral lesions in real time. The proposed model offers great potential as a low-cost and non-invasive tool that can support screening processes and improve detection of OPMD.
Collapse
|
15
|
Song B, Sunny S, Li S, Gurushanth K, Mendonca P, Mukhia N, Patrick S, Gurudath S, Raghavan S, Imchen T, Leivon ST, Kolur T, Shetty V, Bushan V, Ramesh R, Lima N, Pillai V, Wilder-Smith P, Sigamani A, Suresh A, Kuriakose MA, Birur P, Liang R. Mobile-based oral cancer classification for point-of-care screening. JOURNAL OF BIOMEDICAL OPTICS 2021; 26:JBO-210101R. [PMID: 34164967 PMCID: PMC8220969 DOI: 10.1117/1.jbo.26.6.065003] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/07/2021] [Accepted: 06/08/2021] [Indexed: 06/10/2023]
Abstract
SIGNIFICANCE Oral cancer is among the most common cancers globally, especially in low- and middle-income countries. Early detection is the most effective way to reduce the mortality rate. Deep learning-based cancer image classification models usually need to be hosted on a computing server. However, internet connection is unreliable for screening in low-resource settings. AIM To develop a mobile-based dual-mode image classification method and customized Android application for point-of-care oral cancer detection. APPROACH The dataset used in our study was captured among 5025 patients with our customized dual-modality mobile oral screening devices. We trained an efficient network MobileNet with focal loss and converted the model into TensorFlow Lite format. The finalized lite format model is ∼16.3 MB and ideal for smartphone platform operation. We have developed an Android smartphone application in an easy-to-use format that implements the mobile-based dual-modality image classification approach to distinguish oral potentially malignant and malignant images from normal/benign images. RESULTS We investigated the accuracy and running speed on a cost-effective smartphone computing platform. It takes ∼300 ms to process one image pair with the Moto G5 Android smartphone. We tested the proposed method on a standalone dataset and achieved 81% accuracy for distinguishing normal/benign lesions from clinically suspicious lesions, using a gold standard of clinical impression based on the review of images by oral specialists. CONCLUSIONS Our study demonstrates the effectiveness of a mobile-based approach for oral cancer screening in low-resource settings.
Collapse
Affiliation(s)
- Bofan Song
- The University of Arizona, Wyant College of Optical Sciences, Tucson, Arizona, United States
| | - Sumsum Sunny
- Mazumdar Shaw Medical Centre, Bangalore, Karnataka, India
| | - Shaobai Li
- The University of Arizona, Wyant College of Optical Sciences, Tucson, Arizona, United States
| | | | | | - Nirza Mukhia
- KLE Society’s Institute of Dental Sciences, Bangalore, Karnataka, India
| | | | - Shubha Gurudath
- KLE Society’s Institute of Dental Sciences, Bangalore, Karnataka, India
| | | | - Tsusennaro Imchen
- Christian Institute of Health Sciences and Research, Dimapur, Nagaland, India
| | - Shirley T Leivon
- Christian Institute of Health Sciences and Research, Dimapur, Nagaland, India
| | - Trupti Kolur
- Mazumdar Shaw Medical Foundation, Bangalore, Karnataka, India
| | - Vivek Shetty
- Mazumdar Shaw Medical Foundation, Bangalore, Karnataka, India
| | - Vidya Bushan
- Mazumdar Shaw Medical Foundation, Bangalore, Karnataka, India
| | - Rohan Ramesh
- Christian Institute of Health Sciences and Research, Dimapur, Nagaland, India
| | - Natzem Lima
- The University of Arizona, Wyant College of Optical Sciences, Tucson, Arizona, United States
| | - Vijay Pillai
- Mazumdar Shaw Medical Foundation, Bangalore, Karnataka, India
| | - Petra Wilder-Smith
- University of California, Beckman Laser Institute and Medical Clinic, Irvine, California, United States
| | - Alben Sigamani
- Mazumdar Shaw Medical Foundation, Bangalore, Karnataka, India
| | - Amritha Suresh
- Mazumdar Shaw Medical Centre, Bangalore, Karnataka, India
- Mazumdar Shaw Medical Foundation, Bangalore, Karnataka, India
| | - Moni A. Kuriakose
- Mazumdar Shaw Medical Centre, Bangalore, Karnataka, India
- Mazumdar Shaw Medical Foundation, Bangalore, Karnataka, India
- Cochin Cancer Research Center, Kochi, Kerala, India
| | - Praveen Birur
- KLE Society’s Institute of Dental Sciences, Bangalore, Karnataka, India
- Biocon Foundation, Bangalore, Karnataka, India
| | - Rongguang Liang
- The University of Arizona, Wyant College of Optical Sciences, Tucson, Arizona, United States
| |
Collapse
|
16
|
Kaur J, Srivastava R, Borse V. Recent advances in point-of-care diagnostics for oral cancer. Biosens Bioelectron 2021; 178:112995. [PMID: 33515983 DOI: 10.1016/j.bios.2021.112995] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/06/2020] [Revised: 01/07/2021] [Accepted: 01/10/2021] [Indexed: 12/24/2022]
Abstract
Early-stage diagnosis is a crucial step in reducing the mortality rate in oral cancer cases. Point-of-care (POC) devices for oral cancer diagnosis hold great future potential in improving the survival rates as well as the quality of life of oral cancer patients. The conventional oral examination followed by needle biopsy and histopathological analysis have limited diagnostic accuracy. Besides, it involves patient discomfort and is not feasible in resource-limited settings. POC detection of biomarkers and diagnostic adjuncts has emerged as non- or minimally invasive tools for the diagnosis of oral cancer at an early stage. Various biosensors have been developed for the rapid detection of oral cancer biomarkers at the point-of-care. Several optical imaging methods have also been employed as adjuncts to detect alterations in oral tissue indicative of malignancy. This review summarizes the different POC platforms developed for the detection of oral cancer biomarkers, along with various POC imaging and cytological adjuncts that aid in oral cancer diagnosis, especially in low resource settings. Various immunosensors and nucleic acid biosensors developed to detect oral cancer biomarkers are summarized with examples. The different imaging methods used to detect oral tissue malignancy are also discussed herein. Additionally, the currently available commercial devices used as adjuncts in the POC detection of oral cancer are emphasized along with their characteristics. Finally, we discuss the limitations and challenges that persist in translating the developed POC techniques in the clinical settings for oral cancer diagnosis, along with future perspectives.
Collapse
Affiliation(s)
- Jasmeen Kaur
- NanoBios Laboratory, Department of Biosciences and Bioengineering, Indian Institute of Technology Bombay, Powai, Mumbai, 400076, India
| | - Rohit Srivastava
- NanoBios Laboratory, Department of Biosciences and Bioengineering, Indian Institute of Technology Bombay, Powai, Mumbai, 400076, India
| | - Vivek Borse
- NanoBioSens Laboratory, Centre for Nanotechnology, Indian Institute of Technology Guwahati, Guwahati, Assam, 781039, India.
| |
Collapse
|
17
|
Hunt B, Ruiz AJ, Pogue BW. Smartphone-based imaging systems for medical applications: a critical review. JOURNAL OF BIOMEDICAL OPTICS 2021; 26:JBO-200421VR. [PMID: 33860648 PMCID: PMC8047775 DOI: 10.1117/1.jbo.26.4.040902] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/31/2020] [Accepted: 03/29/2021] [Indexed: 05/15/2023]
Abstract
SIGNIFICANCE Smartphones come with an enormous array of functionality and are being more widely utilized with specialized attachments in a range of healthcare applications. A review of key developments and uses, with an assessment of strengths/limitations in various clinical workflows, was completed. AIM Our review studies how smartphone-based imaging (SBI) systems are designed and tested for specialized applications in medicine and healthcare. An evaluation of current research studies is used to provide guidelines for improving the impact of these research advances. APPROACH First, the established and emerging smartphone capabilities that can be leveraged for biomedical imaging are detailed. Then, methods and materials for fabrication of optical, mechanical, and electrical interface components are summarized. Recent systems were categorized into four groups based on their intended application and clinical workflow: ex vivo diagnostic, in vivo diagnostic, monitoring, and treatment guidance. Lastly, strengths and limitations of current SBI systems within these various applications are discussed. RESULTS The native smartphone capabilities for biomedical imaging applications include cameras, touchscreens, networking, computation, 3D sensing, audio, and motion, in addition to commercial wearable peripheral devices. Through user-centered design of custom hardware and software interfaces, these capabilities have the potential to enable portable, easy-to-use, point-of-care biomedical imaging systems. However, due to barriers in programming of custom software and on-board image analysis pipelines, many research prototypes fail to achieve a prospective clinical evaluation as intended. Effective clinical use cases appear to be those in which handheld, noninvasive image guidance is needed and accommodated by the clinical workflow. Handheld systems for in vivo, multispectral, and quantitative fluorescence imaging are a promising development for diagnostic and treatment guidance applications. CONCLUSIONS A holistic assessment of SBI systems must include interpretation of their value for intended clinical settings and how their implementations enable better workflow. A set of six guidelines are proposed to evaluate appropriateness of smartphone utilization in terms of clinical context, completeness, compactness, connectivity, cost, and claims. Ongoing work should prioritize realistic clinical assessments with quantitative and qualitative comparison to non-smartphone systems to clearly demonstrate the value of smartphone-based systems. Improved hardware design to accommodate the rapidly changing smartphone ecosystem, creation of open-source image acquisition and analysis pipelines, and adoption of robust calibration techniques to address phone-to-phone variability are three high priority areas to move SBI research forward.
Collapse
Affiliation(s)
- Brady Hunt
- Dartmouth College, Thayer School of Engineering, Hanover, New Hampshire, United States
- Address all correspondence to Brady Hunt,
| | - Alberto J. Ruiz
- Dartmouth College, Thayer School of Engineering, Hanover, New Hampshire, United States
| | - Brian W. Pogue
- Dartmouth College, Thayer School of Engineering, Hanover, New Hampshire, United States
| |
Collapse
|
18
|
Camalan S, Mahmood H, Binol H, Araújo ALD, Santos-Silva AR, Vargas PA, Lopes MA, Khurram SA, Gurcan MN. Convolutional Neural Network-Based Clinical Predictors of Oral Dysplasia: Class Activation Map Analysis of Deep Learning Results. Cancers (Basel) 2021; 13:cancers13061291. [PMID: 33799466 PMCID: PMC8001078 DOI: 10.3390/cancers13061291] [Citation(s) in RCA: 28] [Impact Index Per Article: 9.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/19/2021] [Revised: 03/08/2021] [Accepted: 03/10/2021] [Indexed: 11/16/2022] Open
Abstract
Oral cancer/oral squamous cell carcinoma is among the top ten most common cancers globally, with over 500,000 new cases and 350,000 associated deaths every year worldwide. There is a critical need for objective, novel technologies that facilitate early, accurate diagnosis. For this purpose, we have developed a method to classify images as "suspicious" and "normal" by performing transfer learning on Inception-ResNet-V2 and generated automated heat maps to highlight the region of the images most likely to be involved in decision making. We have tested the developed method's feasibility on two independent datasets of clinical photographic images of 30 and 24 patients from the UK and Brazil, respectively. Both 10-fold cross-validation and leave-one-patient-out validation methods were performed to test the system, achieving accuracies of 73.6% (±19%) and 90.9% (±12%), F1-scores of 97.9% and 87.2%, and precision values of 95.4% and 99.3% at recall values of 100.0% and 81.1% on these two respective cohorts. This study presents several novel findings and approaches, namely the development and validation of our methods on two datasets collected in different countries showing that using patches instead of the whole lesion image leads to better performance and analyzing which regions of the images are predictive of the classes using class activation map analysis.
Collapse
Affiliation(s)
- Seda Camalan
- Center for Biomedical Informatics, Wake Forest School of Medicine, Winston-Salem, NC 27101, USA; (H.B.); (M.N.G.)
- Correspondence: ; Tel.: +1-(336)-713-7675
| | - Hanya Mahmood
- School of Clinical Dentistry, The University of Sheffield, Sheffield S10 2TA, UK; (H.M.); (S.A.K.)
| | - Hamidullah Binol
- Center for Biomedical Informatics, Wake Forest School of Medicine, Winston-Salem, NC 27101, USA; (H.B.); (M.N.G.)
| | - Anna Luiza Damaceno Araújo
- Oral Diagnosis Department, Semiology and Oral Pathology Areas, Piracicaba Dental School, University of Campinas (UNICAMP), Bairro Areão, Piracicaba 13414-903, São Paulo, Brazil; (A.L.D.A.); (A.R.S.-S.); (P.A.V.); (M.A.L.)
| | - Alan Roger Santos-Silva
- Oral Diagnosis Department, Semiology and Oral Pathology Areas, Piracicaba Dental School, University of Campinas (UNICAMP), Bairro Areão, Piracicaba 13414-903, São Paulo, Brazil; (A.L.D.A.); (A.R.S.-S.); (P.A.V.); (M.A.L.)
| | - Pablo Agustin Vargas
- Oral Diagnosis Department, Semiology and Oral Pathology Areas, Piracicaba Dental School, University of Campinas (UNICAMP), Bairro Areão, Piracicaba 13414-903, São Paulo, Brazil; (A.L.D.A.); (A.R.S.-S.); (P.A.V.); (M.A.L.)
| | - Marcio Ajudarte Lopes
- Oral Diagnosis Department, Semiology and Oral Pathology Areas, Piracicaba Dental School, University of Campinas (UNICAMP), Bairro Areão, Piracicaba 13414-903, São Paulo, Brazil; (A.L.D.A.); (A.R.S.-S.); (P.A.V.); (M.A.L.)
| | - Syed Ali Khurram
- School of Clinical Dentistry, The University of Sheffield, Sheffield S10 2TA, UK; (H.M.); (S.A.K.)
| | - Metin N. Gurcan
- Center for Biomedical Informatics, Wake Forest School of Medicine, Winston-Salem, NC 27101, USA; (H.B.); (M.N.G.)
| |
Collapse
|
19
|
Uthoff RD, Song B, Maarouf M, Shi V, Liang R. Point-of-care, multispectral, smartphone-based dermascopes for dermal lesion screening and erythema monitoring. JOURNAL OF BIOMEDICAL OPTICS 2020; 25:1-21. [PMID: 32578406 PMCID: PMC7309634 DOI: 10.1117/1.jbo.25.6.066004] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/01/2019] [Accepted: 06/08/2020] [Indexed: 05/04/2023]
Abstract
SIGNIFICANCE The rates of melanoma and nonmelanoma skin cancer are rising across the globe. Due to a shortage of board-certified dermatologists, the burden of dermal lesion screening and erythema monitoring has fallen to primary care physicians (PCPs). An adjunctive device for lesion screening and erythema monitoring would be beneficial because PCPs are not typically extensively trained in dermatological care. AIM We aim to examine the feasibility of using a smartphone-camera-based dermascope and a USB-camera-based dermascope utilizing polarized white-light imaging (PWLI) and polarized multispectral imaging (PMSI) to map dermal chromophores and erythema. APPROACH Two dermascopes integrating LED-based PWLI and PMSI with both a smartphone-based camera and a USB-connected camera were developed to capture images of dermal lesions and erythema. Image processing algorithms were implemented to provide chromophore concentrations and redness measures. RESULTS PWLI images were successfully converted to an alternate colorspace for erythema measures, and the spectral bandwidth of the PMSI LED illumination was sufficient for mapping of deoxyhemoglobin, oxyhemoglobin, and melanin chromophores. Both types of dermascopes were able to achieve similar relative concentration results. CONCLUSION Chromophore mapping and erythema monitoring are feasible with PWLI and PMSI using LED illumination and smartphone-based cameras. These systems can provide a simpler, more portable geometry and reduce device costs compared with interference-filter-based or spectrometer-based clinical-grade systems. Future research should include a rigorous clinical trial to collect longitudinal data and a large enough dataset to train and implement a machine learning-based image classifier.
Collapse
Affiliation(s)
- Ross D. Uthoff
- The University of Arizona, James C. Wyant College of Optical Sciences, Tucson, Arizona, United States
| | - Bofan Song
- The University of Arizona, James C. Wyant College of Optical Sciences, Tucson, Arizona, United States
| | - Melody Maarouf
- The University of Arizona, College of Medicine, Department of Medicine, Division of Dermatology, Tucson, Arizona, United States
| | - Vivian Shi
- The University of Arizona, College of Medicine, Department of Medicine, Division of Dermatology, Tucson, Arizona, United States
| | - Rongguang Liang
- The University of Arizona, James C. Wyant College of Optical Sciences, Tucson, Arizona, United States
| |
Collapse
|