1
|
Lambert B, Forbes F, Doyle S, Dehaene H, Dojat M. Trustworthy clinical AI solutions: A unified review of uncertainty quantification in Deep Learning models for medical image analysis. Artif Intell Med 2024; 150:102830. [PMID: 38553168 DOI: 10.1016/j.artmed.2024.102830] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/21/2023] [Revised: 02/28/2024] [Accepted: 03/01/2024] [Indexed: 04/02/2024]
Abstract
The full acceptance of Deep Learning (DL) models in the clinical field is rather low with respect to the quantity of high-performing solutions reported in the literature. End users are particularly reluctant to rely on the opaque predictions of DL models. Uncertainty quantification methods have been proposed in the literature as a potential solution, to reduce the black-box effect of DL models and increase the interpretability and the acceptability of the result by the final user. In this review, we propose an overview of the existing methods to quantify uncertainty associated with DL predictions. We focus on applications to medical image analysis, which present specific challenges due to the high dimensionality of images and their variable quality, as well as constraints associated with real-world clinical routine. Moreover, we discuss the concept of structural uncertainty, a corpus of methods to facilitate the alignment of segmentation uncertainty estimates with clinical attention. We then discuss the evaluation protocols to validate the relevance of uncertainty estimates. Finally, we highlight the open challenges for uncertainty quantification in the medical field.
Collapse
Affiliation(s)
- Benjamin Lambert
- Univ. Grenoble Alpes, Inserm, U1216, Grenoble Institut des Neurosciences, Grenoble, 38000, France; Pixyl Research and Development Laboratory, Grenoble, 38000, France
| | - Florence Forbes
- Univ. Grenoble Alpes, Inria, CNRS, Grenoble INP, LJK, Grenoble, 38000, France
| | - Senan Doyle
- Pixyl Research and Development Laboratory, Grenoble, 38000, France
| | - Harmonie Dehaene
- Pixyl Research and Development Laboratory, Grenoble, 38000, France
| | - Michel Dojat
- Univ. Grenoble Alpes, Inserm, U1216, Grenoble Institut des Neurosciences, Grenoble, 38000, France.
| |
Collapse
|
2
|
Zahari R, Cox J, Obara B. Uncertainty-aware image classification on 3D CT lung. Comput Biol Med 2024; 172:108324. [PMID: 38508053 DOI: 10.1016/j.compbiomed.2024.108324] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/20/2023] [Revised: 03/06/2024] [Accepted: 03/14/2024] [Indexed: 03/22/2024]
Abstract
Early detection is crucial for lung cancer to prolong the patient's survival. Existing model architectures used in such systems have shown promising results. However, they lack reliability and robustness in their predictions and the models are typically evaluated on a single dataset, making them overconfident when a new class is present. With the existence of uncertainty, uncertain images can be referred to medical experts for a second opinion. Thus, we propose an uncertainty-aware framework that includes three phases: data preprocessing and model selection and evaluation, uncertainty quantification (UQ), and uncertainty measurement and data referral for the classification of benign and malignant nodules using 3D CT images. To quantify the uncertainty, we employed three approaches; Monte Carlo Dropout (MCD), Deep Ensemble (DE), and Ensemble Monte Carlo Dropout (EMCD). We evaluated eight different deep learning models consisting of ResNet, DenseNet, and the Inception network family, all of which achieved average F1 scores above 0.832, and the highest average value of 0.845 was obtained using InceptionResNetV2. Furthermore, incorporating the UQ demonstrated significant improvement in the overall model performance. Upon evaluation of the uncertainty estimate, MCD outperforms the other UQ models except for the metric, URecall, where DE and EMCD excel, implying that they are better at identifying incorrect predictions with higher uncertainty levels, which is vital in the medical field. Finally, we show that using a threshold for data referral can greatly improve the performance further, increasing the accuracy up to 0.959.
Collapse
Affiliation(s)
- Rahimi Zahari
- School of Computing, Newcastle University, Newcastle upon Tyne, UK
| | - Julie Cox
- County Durham and Darlington NHS Foundation Trust, County Durham, UK
| | - Boguslaw Obara
- School of Computing, Newcastle University, Newcastle upon Tyne, UK; Biosciences Institute, Newcastle University, Newcastle upon Tyne, UK.
| |
Collapse
|
3
|
Liu Q, Zhang J, Bai Y. Mapping the landscape of artificial intelligence in skin cancer research: a bibliometric analysis. Front Oncol 2023; 13:1222426. [PMID: 37901316 PMCID: PMC10613074 DOI: 10.3389/fonc.2023.1222426] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/14/2023] [Accepted: 09/18/2023] [Indexed: 10/31/2023] Open
Abstract
Objective Artificial intelligence (AI), with its potential to diagnose skin cancer, has the potential to revolutionize future medical and dermatological practices. However, the current knowledge regarding the utilization of AI in skin cancer diagnosis remains somewhat limited, necessitating further research. This study employs visual bibliometric analysis to consolidate and present insights into the evolution and deployment of AI in the context of skin cancer. Through this analysis, we aim to shed light on the research developments, focal areas of interest, and emerging trends within AI and its application to skin cancer diagnosis. Methods On July 14, 2023, articles and reviews about the application of AI in skin cancer, spanning the years from 1900 to 2023, were selected from the Web of Science Core Collection. Co-authorship, co-citation, and co-occurrence analyses of countries, institutions, authors, references, and keywords within this field were conducted using a combination of tools, including CiteSpace V (version 6.2. R3), VOSviewer (version 1.6.18), SCImago, Microsoft Excel 2019, and R 4.2.3. Results A total of 512 papers matching the search terms and inclusion/exclusion criteria were published between 1991 and 2023. The United States leads in publications with 149, followed by India with 61. Germany holds eight positions among the top 10 institutions, while the United States has two. The most prevalent journals cited were Cancer, the European Journal of Cancer, and Sensors. The most frequently cited keywords include "skin cancer", "classification", "artificial intelligence", and "deep learning". Conclusions Research into the application of AI in skin cancer is rapidly expanding, and an increasing number of scholars are dedicating their efforts to this field. With the advancement of AI technology, new opportunities have arisen to enhance the accuracy of skin imaging diagnosis, treatment based on big data, and prognosis prediction. However, at present, the majority of AI research in the field of skin cancer diagnosis is still in the feasibility study stage. It has not yet made significant progress toward practical implementation in clinical settings. To make substantial strides in this field, there is a need to enhance collaboration between countries and institutions. Despite the potential benefits of AI in skin cancer research, numerous challenges remain to be addressed, including developing robust algorithms, resolving data quality issues, and enhancing results interpretability. Consequently, sustained efforts are essential to surmount these obstacles and facilitate the practical application of AI in skin cancer research.
Collapse
Affiliation(s)
- Qianwei Liu
- Graduate School, Beijing University of Chinese Medicine, Beijing, China
| | - Jie Zhang
- Library, Chengdu University of Traditional Chinese Medicine, Chengdu, Sichuan, China
| | - Yanping Bai
- Graduate School, Beijing University of Chinese Medicine, Beijing, China
- Department of Dermatology, China-Japan Friendship Hospital, Beijing, China
| |
Collapse
|
4
|
Mehrtens HA, Kurz A, Bucher TC, Brinker TJ. Benchmarking common uncertainty estimation methods with histopathological images under domain shift and label noise. Med Image Anal 2023; 89:102914. [PMID: 37544085 DOI: 10.1016/j.media.2023.102914] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/22/2022] [Revised: 05/17/2023] [Accepted: 07/25/2023] [Indexed: 08/08/2023]
Abstract
In the past years, deep learning has seen an increase in usage in the domain of histopathological applications. However, while these approaches have shown great potential, in high-risk environments deep learning models need to be able to judge their uncertainty and be able to reject inputs when there is a significant chance of misclassification. In this work, we conduct a rigorous evaluation of the most commonly used uncertainty and robustness methods for the classification of Whole Slide Images, with a focus on the task of selective classification, where the model should reject the classification in situations in which it is uncertain. We conduct our experiments on tile-level under the aspects of domain shift and label noise, as well as on slide-level. In our experiments, we compare Deep Ensembles, Monte-Carlo Dropout, Stochastic Variational Inference, Test-Time Data Augmentation as well as ensembles of the latter approaches. We observe that ensembles of methods generally lead to better uncertainty estimates as well as an increased robustness towards domain shifts and label noise, while contrary to results from classical computer vision benchmarks no systematic gain of the other methods can be shown. Across methods, a rejection of the most uncertain samples reliably leads to a significant increase in classification accuracy on both in-distribution as well as out-of-distribution data. Furthermore, we conduct experiments comparing these methods under varying conditions of label noise. Lastly, we publish our code framework to facilitate further research on uncertainty estimation on histopathological data.
Collapse
Affiliation(s)
- Hendrik A Mehrtens
- Division of Digital Biomarkers for Oncology, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Alexander Kurz
- Division of Digital Biomarkers for Oncology, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Tabea-Clara Bucher
- Division of Digital Biomarkers for Oncology, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Titus J Brinker
- Division of Digital Biomarkers for Oncology, German Cancer Research Center (DKFZ), Heidelberg, Germany.
| |
Collapse
|
5
|
Kuo KM, Talley PC, Chang CS. The accuracy of artificial intelligence used for non-melanoma skin cancer diagnoses: a meta-analysis. BMC Med Inform Decis Mak 2023; 23:138. [PMID: 37501114 PMCID: PMC10375663 DOI: 10.1186/s12911-023-02229-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2022] [Accepted: 07/07/2023] [Indexed: 07/29/2023] Open
Abstract
BACKGROUND With rising incidence of skin cancer and relatively increased mortality rates, an improved diagnosis of such a potentially fatal disease is of vital importance. Although frequently curable, it nevertheless places a considerable burden upon healthcare systems. Among the various types of skin cancers, non-melanoma skin cancer is most prevalent. Despite such prevalence and its associated cost, scant proof concerning the diagnostic accuracy via Artificial Intelligence (AI) for non-melanoma skin cancer exists. This study meta-analyzes the diagnostic test accuracy of AI used to diagnose non-melanoma forms of skin cancer, and it identifies potential covariates that account for heterogeneity between extant studies. METHODS Various electronic databases (Scopus, PubMed, ScienceDirect, SpringerLink, and Dimensions) were examined to discern eligible studies beginning from March 2022. Those AI studies predictive of non-melanoma skin cancer were included. Summary estimates of sensitivity, specificity, and area under receiver operating characteristic curves were used to evaluate diagnostic accuracy. The revised Quality Assessment of Diagnostic Studies served to assess any risk of bias. RESULTS A literature search produced 39 eligible articles for meta-analysis. The summary sensitivity, specificity, and area under receiver operating characteristic curve of AI for diagnosing non-melanoma skin cancer was 0.78, 0.98, & 0.97, respectively. Skin cancer typology, data sources, cross validation, ensemble models, types of techniques, pre-trained models, and image augmentation became significant covariates accounting for heterogeneity in terms of both sensitivity and/or specificity. CONCLUSIONS Meta-analysis results revealed that AI is predictive of non-melanoma with an acceptable performance, but sensitivity may become improved. Further, ensemble models and pre-trained models are employable to improve true positive rating.
Collapse
Affiliation(s)
- Kuang Ming Kuo
- Department of Business Management, National United University, No.1, Miaoli, 360301, Lienda, Taiwan, Republic of China
| | - Paul C Talley
- Department of Applied English, I-Shou University, No. 1, Sec. 1, Syuecheng Rd., Dashu District, 84001, Kaohsiung City, Taiwan, Republic of China
| | - Chao-Sheng Chang
- Department of Occupational Therapy, I-Shou University, No. 1, Yida Rd., Yanchao District, 82445, Kaohsiung City, Taiwan, Republic of China.
- Department of Emergency Medicine, E-Da Hospital, I-Shou University, Kaohsiung, Taiwan, Republic of China.
| |
Collapse
|
6
|
Ahmad N, Shah JH, Khan MA, Baili J, Ansari GJ, Tariq U, Kim YJ, Cha JH. A novel framework of multiclass skin lesion recognition from dermoscopic images using deep learning and explainable AI. Front Oncol 2023; 13:1151257. [PMID: 37346069 PMCID: PMC10281646 DOI: 10.3389/fonc.2023.1151257] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2023] [Accepted: 05/19/2023] [Indexed: 06/23/2023] Open
Abstract
Skin cancer is a serious disease that affects people all over the world. Melanoma is an aggressive form of skin cancer, and early detection can significantly reduce human mortality. In the United States, approximately 97,610 new cases of melanoma will be diagnosed in 2023. However, challenges such as lesion irregularities, low-contrast lesions, intraclass color similarity, redundant features, and imbalanced datasets make improved recognition accuracy using computerized techniques extremely difficult. This work presented a new framework for skin lesion recognition using data augmentation, deep learning, and explainable artificial intelligence. In the proposed framework, data augmentation is performed at the initial step to increase the dataset size, and then two pretrained deep learning models are employed. Both models have been fine-tuned and trained using deep transfer learning. Both models (Xception and ShuffleNet) utilize the global average pooling layer for deep feature extraction. The analysis of this step shows that some important information is missing; therefore, we performed the fusion. After the fusion process, the computational time was increased; therefore, we developed an improved Butterfly Optimization Algorithm. Using this algorithm, only the best features are selected and classified using machine learning classifiers. In addition, a GradCAM-based visualization is performed to analyze the important region in the image. Two publicly available datasets-ISIC2018 and HAM10000-have been utilized and obtained improved accuracy of 99.3% and 91.5%, respectively. Comparing the proposed framework accuracy with state-of-the-art methods reveals improved and less computational time.
Collapse
Affiliation(s)
- Naveed Ahmad
- Department of Computer Science, COMSATS University Islamabad, Wah Cantt, Pakistan
| | - Jamal Hussain Shah
- Department of Computer Science, COMSATS University Islamabad, Wah Cantt, Pakistan
| | - Muhammad Attique Khan
- Department of Computer Science, HITEC University, Taxila, Pakistan
- Department of Informatics, University of Leicester, Leicester, United Kingdom
| | - Jamel Baili
- College of Computer Science, King Khalid University, Abha, Saudi Arabia
| | | | - Usman Tariq
- Department of Management Information Systems, CoBA, Prince Sattam Bin Abdulaziz University, Al-Kharj, Saudi Arabia
| | - Ye Jin Kim
- Department of Computer Science, Hanyang University, Seoul, Republic of Korea
| | - Jae-Hyuk Cha
- Department of Computer Science, Hanyang University, Seoul, Republic of Korea
| |
Collapse
|
7
|
Winkler JK, Blum A, Kommoss K, Enk A, Toberer F, Rosenberger A, Haenssle HA. Assessment of Diagnostic Performance of Dermatologists Cooperating With a Convolutional Neural Network in a Prospective Clinical Study: Human With Machine. JAMA Dermatol 2023; 159:621-627. [PMID: 37133847 PMCID: PMC10157508 DOI: 10.1001/jamadermatol.2023.0905] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/21/2022] [Accepted: 03/05/2023] [Indexed: 05/04/2023]
Abstract
Importance Studies suggest that convolutional neural networks (CNNs) perform equally to trained dermatologists in skin lesion classification tasks. Despite the approval of the first neural networks for clinical use, prospective studies demonstrating benefits of human with machine cooperation are lacking. Objective To assess whether dermatologists benefit from cooperation with a market-approved CNN in classifying melanocytic lesions. Design, Setting, and Participants In this prospective diagnostic 2-center study, dermatologists performed skin cancer screenings using naked-eye examination and dermoscopy. Dermatologists graded suspect melanocytic lesions by the probability of malignancy (range 0-1, threshold for malignancy ≥0.5) and indicated management decisions (no action, follow-up, excision). Next, dermoscopic images of suspect lesions were assessed by a market-approved CNN, Moleanalyzer Pro (FotoFinder Systems). The CNN malignancy scores (range 0-1, threshold for malignancy ≥0.5) were transferred to dermatologists with the request to re-evaluate lesions and revise initial decisions in consideration of CNN results. Reference diagnoses were based on histopathologic examination in 125 (54.8%) lesions or, in the case of nonexcised lesions, on clinical follow-up data and expert consensus. Data were collected from October 2020 to October 2021. Main Outcomes and Measures Primary outcome measures were diagnostic sensitivity and specificity of dermatologists alone and dermatologists cooperating with the CNN. Accuracy and receiver operator characteristic area under the curve (ROC AUC) were considered as additional measures. Results A total of 22 dermatologists detected 228 suspect melanocytic lesions (190 nevi, 38 melanomas) in 188 patients (mean [range] age, 53.4 [19-91] years; 97 [51.6%] male patients). Diagnostic sensitivity and specificity significantly improved when dermatologists additionally integrated CNN results into decision-making (mean sensitivity from 84.2% [95% CI, 69.6%-92.6%] to 100.0% [95% CI, 90.8%-100.0%]; P = .03; mean specificity from 72.1% [95% CI, 65.3%-78.0%] to 83.7% [95% CI, 77.8%-88.3%]; P < .001; mean accuracy from 74.1% [95% CI, 68.1%-79.4%] to 86.4% [95% CI, 81.3%-90.3%]; P < .001; and mean ROC AUC from 0.895 [95% CI, 0.836-0.954] to 0.968 [95% CI, 0.948-0.988]; P = .005). In addition, the CNN alone achieved a comparable sensitivity, higher specificity, and higher diagnostic accuracy compared with dermatologists alone in classifying melanocytic lesions. Moreover, unnecessary excisions of benign nevi were reduced by 19.2%, from 104 (54.7%) of 190 benign nevi to 84 nevi when dermatologists cooperated with the CNN (P < .001). Most lesions were examined by dermatologists with 2 to 5 years (96, 42.1%) or less than 2 years of experience (78, 34.2%); others (54, 23.7%) were evaluated by dermatologists with more than 5 years of experience. Dermatologists with less dermoscopy experience cooperating with the CNN had the most diagnostic improvement compared with more experienced dermatologists. Conclusions and Relevance In this prospective diagnostic study, these findings suggest that dermatologists may improve their performance when they cooperate with the market-approved CNN and that a broader application of this human with machine approach could be beneficial for dermatologists and patients.
Collapse
Affiliation(s)
- Julia K. Winkler
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany
| | - Andreas Blum
- Public, Private and Teaching Practice of Dermatology, Konstanz, Germany
| | - Katharina Kommoss
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany
| | - Alexander Enk
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany
| | - Ferdinand Toberer
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany
| | - Albert Rosenberger
- Institute of Genetic Epidemiology, University Medical Center, Georg-August University of Goettingen, Goettingen, Germany
| | - Holger A. Haenssle
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany
| |
Collapse
|
8
|
Melarkode N, Srinivasan K, Qaisar SM, Plawiak P. AI-Powered Diagnosis of Skin Cancer: A Contemporary Review, Open Challenges and Future Research Directions. Cancers (Basel) 2023; 15. [PMID: 36831525 DOI: 10.3390/cancers15041183] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/03/2022] [Revised: 02/07/2023] [Accepted: 02/08/2023] [Indexed: 02/15/2023] Open
Abstract
Skin cancer continues to remain one of the major healthcare issues across the globe. If diagnosed early, skin cancer can be treated successfully. While early diagnosis is paramount for an effective cure for cancer, the current process requires the involvement of skin cancer specialists, which makes it an expensive procedure and not easily available and affordable in developing countries. This dearth of skin cancer specialists has given rise to the need to develop automated diagnosis systems. In this context, Artificial Intelligence (AI)-based methods have been proposed. These systems can assist in the early detection of skin cancer and can consequently lower its morbidity, and, in turn, alleviate the mortality rate associated with it. Machine learning and deep learning are branches of AI that deal with statistical modeling and inference, which progressively learn from data fed into them to predict desired objectives and characteristics. This survey focuses on Machine Learning and Deep Learning techniques deployed in the field of skin cancer diagnosis, while maintaining a balance between both techniques. A comparison is made to widely used datasets and prevalent review papers, discussing automated skin cancer diagnosis. The study also discusses the insights and lessons yielded by the prior works. The survey culminates with future direction and scope, which will subsequently help in addressing the challenges faced within automated skin cancer diagnosis.
Collapse
|
9
|
Milanés-Hermosilla D, Trujillo-Codorniú R, Lamar-Carbonell S, Sagaró-Zamora R, Tamayo-Pacheco JJ, Villarejo-Mayor JJ, Delisle-Rodriguez D. Robust Motor Imagery Tasks Classification Approach Using Bayesian Neural Network. Sensors (Basel) 2023; 23:703. [PMID: 36679501 PMCID: PMC9862912 DOI: 10.3390/s23020703] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/15/2022] [Revised: 12/30/2022] [Accepted: 01/05/2023] [Indexed: 06/17/2023]
Abstract
The development of Brain-Computer Interfaces based on Motor Imagery (MI) tasks is a relevant research topic worldwide. The design of accurate and reliable BCI systems remains a challenge, mainly in terms of increasing performance and usability. Classifiers based on Bayesian Neural Networks are proposed in this work by using the variational inference, aiming to analyze the uncertainty during the MI prediction. An adaptive threshold scheme is proposed here for MI classification with a reject option, and its performance on both datasets 2a and 2b from BCI Competition IV is compared with other approaches based on thresholds. The results using subject-specific and non-subject-specific training strategies are encouraging. From the uncertainty analysis, considerations for reducing computational cost are proposed for future work.
Collapse
Affiliation(s)
| | - Rafael Trujillo-Codorniú
- Department of Automatic Engineering, University of Oriente, Santiago de Cuba 90500, Cuba
- Electronics, Communications and Computing Services Company for the Nickel Industry, Holguín 80100, Cuba
| | | | - Roberto Sagaró-Zamora
- Department of Mechanical Engineering, University of Oriente, Santiago de Cuba 90500, Cuba
| | | | - John Jairo Villarejo-Mayor
- Department of Electrical and Electronic Engineering, Federal University of Santa Catarina, Florianopolis 88040-900, SC, Brazil
| | - Denis Delisle-Rodriguez
- Postgraduate Program in Neuroengineering, Edmond and Lily Safra International Institute of Neurosciences, Santos Dumont Institute, Macaiba 59280-000, RN, Brazil
| |
Collapse
|
10
|
Nie Y, Sommella P, Carratù M, O’Nils M, Lundgren J. A Deep CNN Transformer Hybrid Model for Skin Lesion Classification of Dermoscopic Images Using Focal Loss. Diagnostics (Basel) 2022; 13:diagnostics13010072. [PMID: 36611363 PMCID: PMC9818899 DOI: 10.3390/diagnostics13010072] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2022] [Revised: 12/08/2022] [Accepted: 12/20/2022] [Indexed: 12/28/2022] Open
Abstract
Skin cancers are the most cancers diagnosed worldwide, with an estimated > 1.5 million new cases in 2020. Use of computer-aided diagnosis (CAD) systems for early detection and classification of skin lesions helps reduce skin cancer mortality rates. Inspired by the success of the transformer network in natural language processing (NLP) and the deep convolutional neural network (DCNN) in computer vision, we propose an end-to-end CNN transformer hybrid model with a focal loss (FL) function to classify skin lesion images. First, the CNN extracts low-level, local feature maps from the dermoscopic images. In the second stage, the vision transformer (ViT) globally models these features, then extracts abstract and high-level semantic information, and finally sends this to the multi-layer perceptron (MLP) head for classification. Based on an evaluation of three different loss functions, the FL-based algorithm is aimed to improve the extreme class imbalance that exists in the International Skin Imaging Collaboration (ISIC) 2018 dataset. The experimental analysis demonstrates that impressive results of skin lesion classification are achieved by employing the hybrid model and FL strategy, which shows significantly high performance and outperforms the existing work.
Collapse
Affiliation(s)
- Yali Nie
- Department of Electronics Design, Mid Sweden University, 85170 Sundsvall, Sweden
| | - Paolo Sommella
- Department of Industrial Engineering, University of Salerno, 84084 Fisciano, SA, Italy
| | - Marco Carratù
- Department of Industrial Engineering, University of Salerno, 84084 Fisciano, SA, Italy
| | - Mattias O’Nils
- Department of Electronics Design, Mid Sweden University, 85170 Sundsvall, Sweden
| | - Jan Lundgren
- Department of Electronics Design, Mid Sweden University, 85170 Sundsvall, Sweden
- Correspondence: ; Tel.: +46-1014-28556
| |
Collapse
|
11
|
Yin W, Huang J, Chen J, Ji Y. A study on skin tumor classification based on dense convolutional networks with fused metadata. Front Oncol 2022; 12:989894. [PMID: 36601473 PMCID: PMC9806866 DOI: 10.3389/fonc.2022.989894] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2022] [Accepted: 11/21/2022] [Indexed: 12/23/2022] Open
Abstract
Skin cancer is the most common cause of death in humans. Statistics show that competent dermatologists have a diagnostic accuracy rate of less than 80%, while inexperienced dermatologists have a diagnostic accuracy rate of less than 60%. The higher rate of misdiagnosis will cause many patients to miss the most effective treatment window, risking the patients' life safety. However, the majority of the current study of neural network-based skin cancer diagnosis remains at the image level without patient clinical data. A deep convolutional network incorporating clinical patient metadata of skin cancer is presented to realize the classification model of skin cancer in order to further increase the accuracy of skin cancer diagnosis. There are three basic steps in the approach. First, the high-level features (edge features, color features, texture features, form features, etc.). Implied by the image were retrieved using the pre-trained DenseNet-169 model on the ImageNet dataset. Second, the MetaNet module is introduced, which uses metadata to control a certain portion of each feature channel in the DenseNet-169 network in order to produce weighted features. The MetaBlock module was added at the same time to improve the features retrieved from photos using metadata, choosing the most pertinent characteristics in accordance with the metadata data. The features of the MetaNet and MetaBlock modules were finally combined to create the MD-Net module, which was then used as input into the classifier to get the classification results for skin cancers. On the PAD-UFES-20 and ISIC 2019 datasets, the suggested methodology was assessed. The DenseNet-169 network model combined with this module, according to experimental data, obtains 81.4% in the balancing accuracy index, and its diagnostic accuracy is up between 8% and 15.6% compared to earlier efforts. Additionally, it solves the problem of actinic keratosis and poorly classified skin fibromas.
Collapse
Affiliation(s)
- Wenjun Yin
- School of Information and Communication, Guilin University Of Electronic Technology, Guilin, China
| | - Jianhua Huang
- School of Information and Communication, Guilin University Of Electronic Technology, Guilin, China,*Correspondence: Jianhua Huang, ; Jianlin Chen,
| | - Jianlin Chen
- Reproductive Endocrinology Clinic, Second Xiangya Hospital of Central South University, Changsha, China,*Correspondence: Jianhua Huang, ; Jianlin Chen,
| | - Yuanfa Ji
- School of Information and Communication, Guilin University Of Electronic Technology, Guilin, China
| |
Collapse
|
12
|
Singh RK, Gorantla R, Allada SGR, Narra P. SkiNet: A deep learning framework for skin lesion diagnosis with uncertainty estimation and explainability. PLoS One 2022; 17:e0276836. [PMID: 36315487 DOI: 10.1371/journal.pone.0276836] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2021] [Accepted: 10/14/2022] [Indexed: 11/05/2022] Open
Abstract
Skin cancer is considered to be the most common human malignancy. Around 5 million new cases of skin cancer are recorded in the United States annually. Early identification and evaluation of skin lesions are of great clinical significance, but the disproportionate dermatologist-patient ratio poses a significant problem in most developing nations. Therefore a novel deep architecture, named as SkiNet, is proposed to provide faster screening solution and assistance to newly trained physicians in the process of clinical diagnosis of skin cancer. The main motive behind SkiNet's design and development is to provide a white box solution, addressing a critical problem of trust and interpretability which is crucial for the wider adoption of Computer-aided diagnosis systems by medical practitioners. The proposed SkiNet is a two-stage pipeline wherein the lesion segmentation is followed by the lesion classification. Monte Carlo dropout and test time augmentation techniques have been employed in the proposed method to estimate epistemic and aleatoric uncertainty. A novel segmentation model named Bayesian MultiResUNet is used to estimate the uncertainty on the predicted segmentation map. Saliency-based methods like XRAI, Grad-CAM and Guided Backprop are explored to provide post-hoc explanations of the deep learning models. The ISIC-2018 dataset is used to perform the experimentation and ablation studies. The results establish the robustness of the proposed model on the traditional benchmarks while addressing the black-box nature of such models to alleviate the skepticism of medical practitioners by incorporating transparency and confidence to the model's prediction.
Collapse
|
13
|
Shetty B, Fernandes R, Rodrigues AP, Chengoden R, Bhattacharya S, Lakshmanna K. Skin lesion classification of dermoscopic images using machine learning and convolutional neural network. Sci Rep 2022; 12:18134. [PMID: 36307467 DOI: 10.1038/s41598-022-22644-9] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/14/2022] [Accepted: 10/18/2022] [Indexed: 12/30/2022] Open
Abstract
Detecting dangerous illnesses connected to the skin organ, particularly malignancy, requires the identification of pigmented skin lesions. Image detection techniques and computer classification capabilities can boost skin cancer detection accuracy. The dataset used for this research work is based on the HAM10000 dataset which consists of 10015 images. The proposed work has chosen a subset of the dataset and performed augmentation. A model with data augmentation tends to learn more distinguishing characteristics and features rather than a model without data augmentation. Involving data augmentation can improve the accuracy of the model. But that model cannot give significant results with the testing data until it is robust. The k-fold cross-validation technique makes the model robust which has been implemented in the proposed work. We have analyzed the classification accuracy of the Machine Learning algorithms and Convolutional Neural Network models. We have concluded that Convolutional Neural Network provides better accuracy compared to other machine learning algorithms implemented in the proposed work. In the proposed system, as the highest, we obtained an accuracy of 95.18% with the CNN model. The proposed work helps early identification of seven classes of skin disease and can be validated and treated appropriately by medical practitioners.
Collapse
|
14
|
Jeong HK, Park C, Henao R, Kheterpal M. Deep Learning in Dermatology: A Systematic Review of Current Approaches, Outcomes, and Limitations. JID Innov 2023; 3:100150. [PMID: 36655135 DOI: 10.1016/j.xjidi.2022.100150] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2022] [Revised: 06/17/2022] [Accepted: 07/15/2022] [Indexed: 01/21/2023] Open
Abstract
Artificial intelligence (AI) has recently made great advances in image classification and malignancy prediction in the field of dermatology. However, understanding the applicability of AI in clinical dermatology practice remains challenging owing to the variability of models, image data, database characteristics, and variable outcome metrics. This systematic review aims to provide a comprehensive overview of dermatology literature using convolutional neural networks. Furthermore, the review summarizes the current landscape of image datasets, transfer learning approaches, challenges, and limitations within current AI literature and current regulatory pathways for approval of models as clinical decision support tools.
Collapse
|
15
|
Kurz A, Hauser K, Mehrtens HA, Krieghoff-Henning E, Hekler A, Kather JN, Fröhling S, von Kalle C, Brinker TJ. Uncertainty Estimation in Medical Image Classification: Systematic Review. JMIR Med Inform 2022; 10:e36427. [PMID: 35916701 PMCID: PMC9382553 DOI: 10.2196/36427] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/14/2022] [Revised: 04/11/2022] [Accepted: 06/04/2022] [Indexed: 01/26/2023] Open
Abstract
Background Deep neural networks are showing impressive results in different medical image classification tasks. However, for real-world applications, there is a need to estimate the network’s uncertainty together with its prediction. Objective In this review, we investigate in what form uncertainty estimation has been applied to the task of medical image classification. We also investigate which metrics are used to describe the effectiveness of the applied uncertainty estimation Methods Google Scholar, PubMed, IEEE Xplore, and ScienceDirect were screened for peer-reviewed studies, published between 2016 and 2021, that deal with uncertainty estimation in medical image classification. The search terms “uncertainty,” “uncertainty estimation,” “network calibration,” and “out-of-distribution detection” were used in combination with the terms “medical images,” “medical image analysis,” and “medical image classification.” Results A total of 22 papers were chosen for detailed analysis through the systematic review process. This paper provides a table for a systematic comparison of the included works with respect to the applied method for estimating the uncertainty. Conclusions The applied methods for estimating uncertainties are diverse, but the sampling-based methods Monte-Carlo Dropout and Deep Ensembles are used most frequently. We concluded that future works can investigate the benefits of uncertainty estimation in collaborative settings of artificial intelligence systems and human experts. International Registered Report Identifier (IRRID) RR2-10.2196/11936
Collapse
Affiliation(s)
- Alexander Kurz
- Digital Biomarkers for Oncology Group, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Katja Hauser
- Digital Biomarkers for Oncology Group, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | | | - Eva Krieghoff-Henning
- Digital Biomarkers for Oncology Group, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Achim Hekler
- Digital Biomarkers for Oncology Group, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | | | - Stefan Fröhling
- Department of Translational Medical Oncology, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Christof von Kalle
- Department of Clinical-Translational Sciences, Berlin Institute of Health (BIH), Berlin, Germany
| | - Titus Josef Brinker
- Digital Biomarkers for Oncology Group, German Cancer Research Center (DKFZ), Heidelberg, Germany
| |
Collapse
|
16
|
Barragán-Montero A, Bibal A, Dastarac MH, Draguet C, Valdés G, Nguyen D, Willems S, Vandewinckele L, Holmström M, Löfman F, Souris K, Sterpin E, Lee JA. Towards a safe and efficient clinical implementation of machine learning in radiation oncology by exploring model interpretability, explainability and data-model dependency. Phys Med Biol 2022; 67:10.1088/1361-6560/ac678a. [PMID: 35421855 PMCID: PMC9870296 DOI: 10.1088/1361-6560/ac678a] [Citation(s) in RCA: 16] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2021] [Accepted: 04/14/2022] [Indexed: 01/26/2023]
Abstract
The interest in machine learning (ML) has grown tremendously in recent years, partly due to the performance leap that occurred with new techniques of deep learning, convolutional neural networks for images, increased computational power, and wider availability of large datasets. Most fields of medicine follow that popular trend and, notably, radiation oncology is one of those that are at the forefront, with already a long tradition in using digital images and fully computerized workflows. ML models are driven by data, and in contrast with many statistical or physical models, they can be very large and complex, with countless generic parameters. This inevitably raises two questions, namely, the tight dependence between the models and the datasets that feed them, and the interpretability of the models, which scales with its complexity. Any problems in the data used to train the model will be later reflected in their performance. This, together with the low interpretability of ML models, makes their implementation into the clinical workflow particularly difficult. Building tools for risk assessment and quality assurance of ML models must involve then two main points: interpretability and data-model dependency. After a joint introduction of both radiation oncology and ML, this paper reviews the main risks and current solutions when applying the latter to workflows in the former. Risks associated with data and models, as well as their interaction, are detailed. Next, the core concepts of interpretability, explainability, and data-model dependency are formally defined and illustrated with examples. Afterwards, a broad discussion goes through key applications of ML in workflows of radiation oncology as well as vendors' perspectives for the clinical implementation of ML.
Collapse
Affiliation(s)
- Ana Barragán-Montero
- Molecular Imaging, Radiation and Oncology (MIRO) Laboratory, Institut de Recherche Expérimentale et Clinique (IREC), UCLouvain, Belgium
| | - Adrien Bibal
- PReCISE, NaDI Institute, Faculty of Computer Science, UNamur and CENTAL, ILC, UCLouvain, Belgium
| | - Margerie Huet Dastarac
- Molecular Imaging, Radiation and Oncology (MIRO) Laboratory, Institut de Recherche Expérimentale et Clinique (IREC), UCLouvain, Belgium
| | - Camille Draguet
- Molecular Imaging, Radiation and Oncology (MIRO) Laboratory, Institut de Recherche Expérimentale et Clinique (IREC), UCLouvain, Belgium
- Department of Oncology, Laboratory of Experimental Radiotherapy, KU Leuven, Belgium
| | - Gilmer Valdés
- Department of Radiation Oncology, Department of Epidemiology and Biostatistics, University of California, San Francisco, United States of America
| | - Dan Nguyen
- Medical Artificial Intelligence and Automation (MAIA) Laboratory, Department of Radiation Oncology, UT Southwestern Medical Center, United States of America
| | - Siri Willems
- ESAT/PSI, KU Leuven Belgium & MIRC, UZ Leuven, Belgium
| | | | | | | | - Kevin Souris
- Molecular Imaging, Radiation and Oncology (MIRO) Laboratory, Institut de Recherche Expérimentale et Clinique (IREC), UCLouvain, Belgium
| | - Edmond Sterpin
- Molecular Imaging, Radiation and Oncology (MIRO) Laboratory, Institut de Recherche Expérimentale et Clinique (IREC), UCLouvain, Belgium
- Department of Oncology, Laboratory of Experimental Radiotherapy, KU Leuven, Belgium
| | - John A Lee
- Molecular Imaging, Radiation and Oncology (MIRO) Laboratory, Institut de Recherche Expérimentale et Clinique (IREC), UCLouvain, Belgium
| |
Collapse
|
17
|
Hoang L, Lee S, Lee E, Kwon K. Multiclass Skin Lesion Classification Using a Novel Lightweight Deep Learning Framework for Smart Healthcare. Applied Sciences 2022; 12:2677. [DOI: 10.3390/app12052677] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/31/2022]
Abstract
Skin lesion classification has recently attracted significant attention. Regularly, physicians take much time to analyze the skin lesions because of the high similarity between these skin lesions. An automated classification system using deep learning can assist physicians in detecting the skin lesion type and enhance the patient’s health. The skin lesion classification has become a hot research area with the evolution of deep learning architecture. In this study, we propose a novel method using a new segmentation approach and wide-ShuffleNet for skin lesion classification. First, we calculate the entropy-based weighting and first-order cumulative moment (EW-FCM) of the skin image. These values are used to separate the lesion from the background. Then, we input the segmentation result into a new deep learning structure wide-ShuffleNet and determine the skin lesion type. We evaluated the proposed method on two large datasets: HAM10000 and ISIC2019. Based on our numerical results, EW-FCM and wide-ShuffleNet achieve more accuracy than state-of-the-art approaches. Additionally, the proposed method is superior lightweight and suitable with a small system like a mobile healthcare system.
Collapse
|
18
|
Yu Z, Nguyen J, Nguyen TD, Kelly J, Mclean C, Bonnington P, Zhang L, Mar V, Ge Z. Early Melanoma Diagnosis With Sequential Dermoscopic Images. IEEE Trans Med Imaging 2022; 41:633-646. [PMID: 34648437 DOI: 10.1109/tmi.2021.3120091] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
Dermatologists often diagnose or rule out early melanoma by evaluating the follow-up dermoscopic images of skin lesions. However, existing algorithms for early melanoma diagnosis are developed using single time-point images of lesions. Ignoring the temporal, morphological changes of lesions can lead to misdiagnosis in borderline cases. In this study, we propose a framework for automated early melanoma diagnosis using sequential dermoscopic images. To this end, we construct our method in three steps. First, we align sequential dermoscopic images of skin lesions using estimated Euclidean transformations, extract the lesion growth region by computing image differences among the consecutive images, and then propose a spatio-temporal network to capture the dermoscopic changes from aligned lesion images and the corresponding difference images. Finally, we develop an early diagnosis module to compute probability scores of malignancy for lesion images over time. We collected 179 serial dermoscopic imaging data from 122 patients to verify our method. Extensive experiments show that the proposed model outperforms other commonly used sequence models. We also compared the diagnostic results of our model with those of seven experienced dermatologists and five registrars. Our model achieved higher diagnostic accuracy than clinicians (63.69% vs. 54.33%, respectively) and provided an earlier diagnosis of melanoma (60.7% vs. 32.7% of melanoma correctly diagnosed on the first follow-up images). These results demonstrate that our model can be used to identify melanocytic lesions that are at high-risk of malignant transformation earlier in the disease process and thereby redefine what is possible in the early detection of melanoma.
Collapse
|
19
|
Senousy Z, Abdelsamea MM, Gaber MM, Abdar M, Acharya UR, Khosravi A, Nahavandi S. MCUa: Multi-level Context and Uncertainty aware Dynamic Deep Ensemble for Breast Cancer Histology Image Classification. IEEE Trans Biomed Eng 2021; 69:818-829. [PMID: 34460359 DOI: 10.1109/tbme.2021.3107446] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
Breast histology image classification is a crucial step in the early diagnosis of breast cancer. In breast pathological diagnosis, Convolutional Neural Networks (CNNs) have demonstrated great success using digitized histology slides. However, tissue classification is still challenging due to the high visual variability of the large-sized digitized samples and the lack of contextual information. In this paper, we propose a novel CNN, called Multi-level Context and Uncertainty aware (MCUa) dynamic deep learning ensemble model. MCUa model consists of several multi-level context-aware models to learn the spatial dependency between image patches in a layer-wise fashion. It exploits the high sensitivity to the multi-level contextual information using an uncertainty quantification component to accomplish a novel dynamic ensemble model. MCUa model has achieved a high accuracy of 98.11% on a breast cancer histology image dataset. Experimental results show the superior effectiveness of the proposed solution compared to the state-of-the-art histology classification models.
Collapse
|
20
|
Abdar M, Samami M, Dehghani Mahmoodabad S, Doan T, Mazoure B, Hashemifesharaki R, Liu L, Khosravi A, Acharya UR, Makarenkov V, Nahavandi S. Uncertainty quantification in skin cancer classification using three-way decision-based Bayesian deep learning. Comput Biol Med 2021; 135:104418. [PMID: 34052016 DOI: 10.1016/j.compbiomed.2021.104418] [Citation(s) in RCA: 68] [Impact Index Per Article: 22.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/03/2021] [Revised: 04/01/2021] [Accepted: 04/17/2021] [Indexed: 12/18/2022]
Abstract
Accurate automated medical image recognition, including classification and segmentation, is one of the most challenging tasks in medical image analysis. Recently, deep learning methods have achieved remarkable success in medical image classification and segmentation, clearly becoming the state-of-the-art methods. However, most of these methods are unable to provide uncertainty quantification (UQ) for their output, often being overconfident, which can lead to disastrous consequences. Bayesian Deep Learning (BDL) methods can be used to quantify uncertainty of traditional deep learning methods, and thus address this issue. We apply three uncertainty quantification methods to deal with uncertainty during skin cancer image classification. They are as follows: Monte Carlo (MC) dropout, Ensemble MC (EMC) dropout and Deep Ensemble (DE). To further resolve the remaining uncertainty after applying the MC, EMC and DE methods, we describe a novel hybrid dynamic BDL model, taking into account uncertainty, based on the Three-Way Decision (TWD) theory. The proposed dynamic model enables us to use different UQ methods and different deep neural networks in distinct classification phases. So, the elements of each phase can be adjusted according to the dataset under consideration. In this study, two best UQ methods (i.e., DE and EMC) are applied in two classification phases (the first and second phases) to analyze two well-known skin cancer datasets, preventing one from making overconfident decisions when it comes to diagnosing the disease. The accuracy and the F1-score of our final solution are, respectively, 88.95% and 89.00% for the first dataset, and 90.96% and 91.00% for the second dataset. Our results suggest that the proposed TWDBDL model can be used effectively at different stages of medical image analysis.
Collapse
Affiliation(s)
- Moloud Abdar
- Institute for Intelligent Systems Research and Innovation (IISRI), Deakin University, Geelong, Australia.
| | - Maryam Samami
- Department of Computer Engineering, Sari Branch, Islamic Azad University, Sari, Iran
| | - Sajjad Dehghani Mahmoodabad
- Department of Artificial Intelligence, Faculty of Computer Engineering, K. N. Toosi University of Technology, Tehran, Iran
| | - Thang Doan
- Department of Computer Science, McGill University / Mila, Montreal, Canada
| | - Bogdan Mazoure
- Department of Computer Science, McGill University / Mila, Montreal, Canada
| | | | - Li Liu
- Center for Machine Vision and Signal Analysis (CMVS), University of Oulu, Finland
| | - Abbas Khosravi
- Institute for Intelligent Systems Research and Innovation (IISRI), Deakin University, Geelong, Australia
| | - U Rajendra Acharya
- School of Engineering, Ngee Ann Polytechnic, Singapore; Department of Biomedical Engineering, Singapore University of Social Sciences, Singapore; Department of Biomedical Informatics and Medical Engineering, Asia University, Taichung, Taiwan
| | - Vladimir Makarenkov
- Department of Computer Science, University of Quebec in Montreal, Montreal, Canada
| | - Saeid Nahavandi
- Institute for Intelligent Systems Research and Innovation (IISRI), Deakin University, Geelong, Australia
| |
Collapse
|
21
|
Dostál O, Procházka A, Vyšata O, Ťupa O, Cejnar P, Vališ M. Recognition of motion patterns using accelerometers for ataxic gait assessment. Neural Comput Appl 2021; 33:2207-15. [DOI: 10.1007/s00521-020-05103-2] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/25/2022]
|
22
|
Mobiny A, Yuan P, Moulik SK, Garg N, Wu CC, Van Nguyen H. DropConnect is effective in modeling uncertainty of Bayesian deep networks. Sci Rep 2021; 11:5458. [PMID: 33750847 PMCID: PMC7943811 DOI: 10.1038/s41598-021-84854-x] [Citation(s) in RCA: 17] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/02/2020] [Accepted: 02/17/2021] [Indexed: 12/26/2022] Open
Abstract
Deep neural networks (DNNs) have achieved state-of-the-art performance in many important domains, including medical diagnosis, security, and autonomous driving. In domains where safety is highly critical, an erroneous decision can result in serious consequences. While a perfect prediction accuracy is not always achievable, recent work on Bayesian deep networks shows that it is possible to know when DNNs are more likely to make mistakes. Knowing what DNNs do not know is desirable to increase the safety of deep learning technology in sensitive applications; Bayesian neural networks attempt to address this challenge. Traditional approaches are computationally intractable and do not scale well to large, complex neural network architectures. In this paper, we develop a theoretical framework to approximate Bayesian inference for DNNs by imposing a Bernoulli distribution on the model weights. This method called Monte Carlo DropConnect (MC-DropConnect) gives us a tool to represent the model uncertainty with little change in the overall model structure or computational cost. We extensively validate the proposed algorithm on multiple network architectures and datasets for classification and semantic segmentation tasks. We also propose new metrics to quantify uncertainty estimates. This enables an objective comparison between MC-DropConnect and prior approaches. Our empirical results demonstrate that the proposed framework yields significant improvement in both prediction accuracy and uncertainty estimation quality compared to the state of the art.
Collapse
Affiliation(s)
- Aryan Mobiny
- Department of Electrical and Computer Engineering, University of Houston, Houston, TX, 77004, USA.
| | - Pengyu Yuan
- Department of Electrical and Computer Engineering, University of Houston, Houston, TX, 77004, USA
| | | | - Naveen Garg
- Department of Diagnostic Radiology, The University of Texas MD Anderson Cancer Center, Houston, TX, 77030, USA
| | - Carol C Wu
- Department of Diagnostic Radiology, The University of Texas MD Anderson Cancer Center, Houston, TX, 77030, USA
| | - Hien Van Nguyen
- Department of Electrical and Computer Engineering, University of Houston, Houston, TX, 77004, USA
| |
Collapse
|
23
|
Cicalese PA, Mobiny A, Shahmoradi Z, Yi X, Mohan C, Van Nguyen H. Kidney Level Lupus Nephritis Classification Using Uncertainty Guided Bayesian Convolutional Neural Networks. IEEE J Biomed Health Inform 2021; 25:315-324. [PMID: 33206612 DOI: 10.1109/jbhi.2020.3039162] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
The kidney biopsy based diagnosis of Lupus Nephritis (LN) is characterized by low inter-observer agreement, with misdiagnosis being associated with increased patient morbidity and mortality. Although various Computer Aided Diagnosis (CAD) systems have been developed for other nephrohistopathological applications, little has been done to accurately classify kidneys based on their kidney level Lupus Glomerulonephritis (LGN) scores. The successful implementation of CAD systems has also been hindered by the diagnosing physician's perceived classifier strengths and weaknesses, which has been shown to have a negative effect on patient outcomes. We propose an Uncertainty-Guided Bayesian Classification (UGBC) scheme that is designed to accurately classify control, class I/II, and class III/IV LGN (3 class) at both the glomerular-level classification task (26,634 segmented glomerulus images) and the kidney-level classification task (87 MRL/lpr mouse kidney sections). Data annotation was performed using a high throughput, bulk labeling scheme that is designed to take advantage of Deep Neural Network's (or DNNs) resistance to label noise. Our augmented UGBC scheme achieved a 94.5% weighted glomerular-level accuracy while achieving a weighted kidney-level accuracy of 96.6%, improving upon the standard Convolutional Neural Network (CNN) architecture by 11.8% and 3.5% respectively.
Collapse
|
24
|
Espinoza JL, Dong LT. Artificial Intelligence Tools for Refining Lung Cancer Screening. J Clin Med 2020; 9:E3860. [PMID: 33261057 DOI: 10.3390/jcm9123860] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/12/2020] [Revised: 11/19/2020] [Accepted: 11/25/2020] [Indexed: 12/19/2022] Open
Abstract
Nearly one-quarter of all cancer deaths worldwide are due to lung cancer, making this disease the leading cause of cancer death among both men and women. The most important determinant of survival in lung cancer is the disease stage at diagnosis, thus developing an effective screening method for early diagnosis has been a long-term goal in lung cancer care. In the last decade, and based on the results of large clinical trials, lung cancer screening programs using low-dose computer tomography (LDCT) in high-risk individuals have been implemented in some clinical settings, however, this method has various limitations, especially a high false-positive rate which eventually results in a number of unnecessary diagnostic and therapeutic interventions among the screened subjects. By using complex algorithms and software, artificial intelligence (AI) is capable to emulate human cognition in the analysis, interpretation, and comprehension of complicated data and currently, it is being successfully applied in various healthcare settings. Taking advantage of the ability of AI to quantify information from images, and its superior capability in recognizing complex patterns in images compared to humans, AI has the potential to aid clinicians in the interpretation of LDCT images obtained in the setting of lung cancer screening. In the last decade, several AI models aimed to improve lung cancer detection have been reported. Some algorithms performed equal or even outperformed experienced radiologists in distinguishing benign from malign lung nodules and some of those models improved diagnostic accuracy and decreased the false-positive rate. Here, we discuss recent publications in which AI algorithms are utilized to assess chest computer tomography (CT) scans imaging obtaining in the setting of lung cancer screening.
Collapse
|
25
|
|
26
|
Adegun A, Viriri S. Deep learning techniques for skin lesion analysis and melanoma cancer detection: a survey of state-of-the-art. Artif Intell Rev 2020. [DOI: 10.1007/s10462-020-09865-y] [Citation(s) in RCA: 40] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/09/2023]
|
27
|
Tschandl P, Rinner C, Apalla Z, Argenziano G, Codella N, Halpern A, Janda M, Lallas A, Longo C, Malvehy J, Paoli J, Puig S, Rosendahl C, Soyer HP, Zalaudek I, Kittler H. Human-computer collaboration for skin cancer recognition. Nat Med 2020; 26:1229-1234. [PMID: 32572267 DOI: 10.1038/s41591-020-0942-0] [Citation(s) in RCA: 257] [Impact Index Per Article: 64.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2019] [Accepted: 05/15/2020] [Indexed: 01/13/2023]
Abstract
The rapid increase in telemedicine coupled with recent advances in diagnostic artificial intelligence (AI) create the imperative to consider the opportunities and risks of inserting AI-based support into new paradigms of care. Here we build on recent achievements in the accuracy of image-based AI for skin cancer diagnosis to address the effects of varied representations of AI-based support across different levels of clinical expertise and multiple clinical workflows. We find that good quality AI-based support of clinical decision-making improves diagnostic accuracy over that of either AI or physicians alone, and that the least experienced clinicians gain the most from AI-based support. We further find that AI-based multiclass probabilities outperformed content-based image retrieval (CBIR) representations of AI in the mobile technology environment, and AI-based support had utility in simulations of second opinions and of telemedicine triage. In addition to demonstrating the potential benefits associated with good quality AI in the hands of non-expert clinicians, we find that faulty AI can mislead the entire spectrum of clinicians, including experts. Lastly, we show that insights derived from AI class-activation maps can inform improvements in human diagnosis. Together, our approach and findings offer a framework for future studies across the spectrum of image-based diagnostics to improve human-computer collaboration in clinical practice.
Collapse
Affiliation(s)
- Philipp Tschandl
- ViDIR Group, Department of Dermatology, Medical University of Vienna, Vienna, Austria
| | - Christoph Rinner
- Center for Medical Statistics, Informatics and Intelligent Systems (CeMSIIS), Medical University of Vienna, Vienna, Austria
| | - Zoe Apalla
- Department of Dermatology, Aristotle University of Thessaloniki, Thessaloniki, Greece
| | | | - Noel Codella
- IBM T. J. Watson Research Center, New York, NY, USA
| | - Allan Halpern
- Dermatology Service, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - Monika Janda
- Centre for Health Services Research, Faculty of Medicine, The University of Queensland, Brisbane, Queensland, Australia
| | - Aimilios Lallas
- Department of Dermatology, Aristotle University of Thessaloniki, Thessaloniki, Greece
| | - Caterina Longo
- Dermatology Unit, University of Modena and Reggio Emilia, Modena, Italy.,Centro Oncologico ad Alta Tecnologia Diagnostica-Dermatologia, Azienda Unità Sanitaria Locale-IRCCS di Reggio Emilia, Reggio Emilia, Italy
| | - Josep Malvehy
- Dermatology Department, Melanoma Unit, Hospital Clínic de Barcelona, IDIBAPS, Universitat de Barcelona, Barcelona, Spain.,Centro de Investigación Biomédica en Red de Enfermedades Raras (CIBER ER), Instituto de Salud Carlos III, Barcelona, Spain
| | - John Paoli
- Department of Dermatology and Venereology, Institute of Clinical Sciences, Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden.,Department of Dermatology and Venereology, Region Västra Götaland, Sahlgrenska University Hospital, Gothenburg, Sweden
| | - Susana Puig
- Dermatology Department, Melanoma Unit, Hospital Clínic de Barcelona, IDIBAPS, Universitat de Barcelona, Barcelona, Spain.,Centro de Investigación Biomédica en Red de Enfermedades Raras (CIBER ER), Instituto de Salud Carlos III, Barcelona, Spain
| | - Cliff Rosendahl
- Faculty of Medicine, The University of Queensland, Brisbane, Queensland, Australia
| | - H Peter Soyer
- Dermatology Research Centre, The University of Queensland Diamantina Institute, The University of Queensland, Brisbane, Queensland, Australia
| | - Iris Zalaudek
- Department of Dermatology, Medical University of Trieste, Trieste, Italy
| | - Harald Kittler
- ViDIR Group, Department of Dermatology, Medical University of Vienna, Vienna, Austria.
| |
Collapse
|
28
|
Mobiny A, Yuan P, Cicalese PA, Van Nguyen H. DECAPS: Detail-Oriented Capsule Networks. Medical Image Computing and Computer Assisted Intervention – MICCAI 2020 2020. [DOI: 10.1007/978-3-030-59710-8_15] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/21/2023]
|