1
|
Miller I, Rosic N, Stapelberg M, Hudson J, Coxon P, Furness J, Walsh J, Climstein M. Performance of Commercial Dermatoscopic Systems That Incorporate Artificial Intelligence for the Identification of Melanoma in General Practice: A Systematic Review. Cancers (Basel) 2024; 16:1443. [PMID: 38611119 PMCID: PMC11011068 DOI: 10.3390/cancers16071443] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/29/2024] [Revised: 04/03/2024] [Accepted: 04/04/2024] [Indexed: 04/14/2024] Open
Abstract
BACKGROUND Cutaneous melanoma remains an increasing global public health burden, particularly in fair-skinned populations. Advancing technologies, particularly artificial intelligence (AI), may provide an additional tool for clinicians to help detect malignancies with a more accurate success rate. This systematic review aimed to report the performance metrics of commercially available convolutional neural networks (CNNs) tasked with detecting MM. METHODS A systematic literature search was performed using CINAHL, Medline, Scopus, ScienceDirect and Web of Science databases. RESULTS A total of 16 articles reporting MM were included in this review. The combined number of melanomas detected was 1160, and non-melanoma lesions were 33,010. The performance of market-approved technology and clinician performance for classifying melanoma was highly heterogeneous, with sensitivity ranging from 16.4 to 100.0%, specificity between 40.0 and 98.3% and accuracy between 44.0 and 92.0%. Less heterogeneity was observed when clinicians worked in unison with AI, with sensitivity ranging between 83.3 and 100.0%, specificity between 83.7 and 87.3%, and accuracy between 86.4 and 86.9%. CONCLUSION Instead of focusing on the performance of AI versus clinicians for classifying melanoma, more consistent performance has been obtained when clinicians' work is supported by AI, facilitating management decisions and improving health outcomes.
Collapse
Affiliation(s)
- Ian Miller
- Aquatic Based Research, Southern Cross University, Bilinga, QLD 4225, Australia; (I.M.); (N.R.)
- Faculty of Health, Southern Cross University, Bilinga, QLD 4225, Australia (P.C.)
- Specialist Suite, John Flynn Hospital, Tugun, QLD 4224, Australia
| | - Nedeljka Rosic
- Aquatic Based Research, Southern Cross University, Bilinga, QLD 4225, Australia; (I.M.); (N.R.)
- Faculty of Health, Southern Cross University, Bilinga, QLD 4225, Australia (P.C.)
| | - Michael Stapelberg
- Aquatic Based Research, Southern Cross University, Bilinga, QLD 4225, Australia; (I.M.); (N.R.)
- Faculty of Health, Southern Cross University, Bilinga, QLD 4225, Australia (P.C.)
- Specialist Suite, John Flynn Hospital, Tugun, QLD 4224, Australia
| | - Jeremy Hudson
- Faculty of Health, Southern Cross University, Bilinga, QLD 4225, Australia (P.C.)
- North Queensland Skin Centre, Townsville, QLD 4810, Australia
| | - Paul Coxon
- Faculty of Health, Southern Cross University, Bilinga, QLD 4225, Australia (P.C.)
- North Queensland Skin Centre, Townsville, QLD 4810, Australia
| | - James Furness
- Water Based Research Unit, Bond University, Robina, QLD 4226, Australia;
| | - Joe Walsh
- Sport Science Institute, Sydney, NSW 2000, Australia;
- AI Consulting Group, Sydney, NSW 2000, Australia
| | - Mike Climstein
- Aquatic Based Research, Southern Cross University, Bilinga, QLD 4225, Australia; (I.M.); (N.R.)
- Faculty of Health, Southern Cross University, Bilinga, QLD 4225, Australia (P.C.)
- Physical Activity, Lifestyle, Ageing and Wellbeing Faculty Research Group, University of Sydney, Sydney, NSW 2050, Australia
| |
Collapse
|
2
|
Wei ML, Tada M, So A, Torres R. Artificial intelligence and skin cancer. Front Med (Lausanne) 2024; 11:1331895. [PMID: 38566925 PMCID: PMC10985205 DOI: 10.3389/fmed.2024.1331895] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2023] [Accepted: 02/26/2024] [Indexed: 04/04/2024] Open
Abstract
Artificial intelligence is poised to rapidly reshape many fields, including that of skin cancer screening and diagnosis, both as a disruptive and assistive technology. Together with the collection and availability of large medical data sets, artificial intelligence will become a powerful tool that can be leveraged by physicians in their diagnoses and treatment plans for patients. This comprehensive review focuses on current progress toward AI applications for patients, primary care providers, dermatologists, and dermatopathologists, explores the diverse applications of image and molecular processing for skin cancer, and highlights AI's potential for patient self-screening and improving diagnostic accuracy for non-dermatologists. We additionally delve into the challenges and barriers to clinical implementation, paths forward for implementation and areas of active research.
Collapse
Affiliation(s)
- Maria L. Wei
- Department of Dermatology, University of California, San Francisco, San Francisco, CA, United States
- Dermatology Service, San Francisco VA Health Care System, San Francisco, CA, United States
| | - Mikio Tada
- Institute for Neurodegenerative Diseases, University of California, San Francisco, San Francisco, CA, United States
| | - Alexandra So
- School of Medicine, University of California, San Francisco, San Francisco, CA, United States
| | - Rodrigo Torres
- Dermatology Service, San Francisco VA Health Care System, San Francisco, CA, United States
| |
Collapse
|
3
|
Hartmann T, Passauer J, Hartmann J, Schmidberger L, Kneilling M, Volc S. Basic principles of artificial intelligence in dermatology explained using melanoma. J Dtsch Dermatol Ges 2024; 22:339-347. [PMID: 38361141 DOI: 10.1111/ddg.15322] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2023] [Accepted: 11/04/2023] [Indexed: 02/17/2024]
Abstract
The use of artificial intelligence (AI) continues to establish itself in the most diverse areas of medicine at an increasingly fast pace. Nevertheless, many healthcare professionals lack the basic technical understanding of how this technology works, which severely limits its application in clinical settings and research. Thus, we would like to discuss the functioning and classification of AI using melanoma as an example in this review to build an understanding of the technology behind AI. For this purpose, elaborate illustrations are used that quickly reveal the technology involved. Previous reviews tend to focus on the potential applications of AI, thereby missing the opportunity to develop a deeper understanding of the subject matter that is so important for clinical application. Malignant melanoma has become a significant burden for healthcare systems. If discovered early, a better prognosis can be expected, which is why skin cancer screening has become increasingly popular and is supported by health insurance. The number of experts remains finite, reducing their availability and leading to longer waiting times. Therefore, innovative ideas need to be implemented to provide the necessary care. Thus, machine learning offers the ability to recognize melanomas from images at a level comparable to experienced dermatologists under optimized conditions.
Collapse
Affiliation(s)
- Tim Hartmann
- Department of Dermatology, University hospital Tübingen, Tübingen, Germany
| | - Johannes Passauer
- Department of Dermatology, University hospital Tübingen, Tübingen, Germany
| | | | - Laura Schmidberger
- Department of Dermatology, University hospital Tübingen, Tübingen, Germany
| | - Manfred Kneilling
- Department of Dermatology, University hospital Tübingen, Tübingen, Germany
- Werner Siemens Imaging Center, Department of Preclinical Imaging and Radiopharmacy, Eberhard Karls University, Tübingen, Germany
- Cluster of Excellence iFIT (EXC 2180) "Image-Guided and Functionally Instructed Tumor Therapies", Eberhard Karls University, Tübingen, Germany
| | - Sebastian Volc
- Department of Dermatology, University hospital Tübingen, Tübingen, Germany
| |
Collapse
|
4
|
Hartmann T, Passauer J, Hartmann J, Schmidberger L, Kneilling M, Volc S. Grundprinzipien der künstlichen Intelligenz in der Dermatologie erklärt am Beispiel des Melanoms. J Dtsch Dermatol Ges 2024; 22:339-349. [PMID: 38450927 DOI: 10.1111/ddg.15322_g] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2023] [Accepted: 11/04/2023] [Indexed: 03/08/2024]
Abstract
ZusammenfassungDer Einsatz von künstlicher Intelligenz (KI) setzt sich in den verschiedensten Bereichen der Medizin immer schneller durch. Dennoch fehlt vielen medizinischen Kollegen das technische Grundverständnis für die Funktionsweise dieser Technologie, was ihre Anwendung in Klinik und Forschung stark einschränkt. Daher möchten wir in dieser Übersichtsarbeit die Funktionsweise und Klassifizierung der KI am Beispiel des Melanoms erörtern, um ein Verständnis für die Technologie hinter der KI zu schaffen. Dazu werden ausführliche Illustrationen verwendet, die die Technologie schnell erklären. Bisherige Übersichten konzentrieren sich eher auf die potenziellen Anwendungen der KI und verpassen die Gelegenheit, ein tieferes Verständnis für die Materie herauszuarbeiten, das für die klinische Anwendung so wichtig ist. Das maligne Melanom ist zu einer erheblichen Belastung für die Gesundheitssysteme geworden. Bei frühzeitiger Entdeckung ist eine bessere Prognose zu erwarten, weshalb das Hautkrebs‐Screening immer populärer und von den Krankenkassen unterstützt wird. Die Zahl der Fachärzte ist jedoch begrenzt, was ihre Verfügbarkeit einschränkt und zu längeren Wartezeiten führt. Daher müssen innovative Ideen umgesetzt werden, um die notwendige Versorgung zu gewährleisten. Das maschinelle Lernen bietet die Möglichkeit, Melanome auf Bildern zu erkennen, und zwar auf einem Niveau, das mit dem von erfahrenen Dermatologen – unter optimierten Bedingungen – vergleichbar ist.
Collapse
Affiliation(s)
- Tim Hartmann
- Hautklinik, Universitätsklinik, Eberhard Karls Universität, Tübingen
| | - Johannes Passauer
- Hautklinik, Universitätsklinik, Eberhard Karls Universität, Tübingen
| | | | | | - Manfred Kneilling
- Hautklinik, Universitätsklinik, Eberhard Karls Universität, Tübingen
- Werner Siemens Imaging Center, Department of Preclinical Imaging and Radiopharmacy, Eberhard Karls University, Tübingen
- Cluster of Excellence iFIT (EXC 2180) "Image-Guided and Functionally Instructed Tumor Therapies", Eberhard Karls Universität, Tübingen
| | - Sebastian Volc
- Hautklinik, Universitätsklinik, Eberhard Karls Universität, Tübingen
| |
Collapse
|
5
|
Chanda T, Hauser K, Hobelsberger S, Bucher TC, Garcia CN, Wies C, Kittler H, Tschandl P, Navarrete-Dechent C, Podlipnik S, Chousakos E, Crnaric I, Majstorovic J, Alhajwan L, Foreman T, Peternel S, Sarap S, Özdemir İ, Barnhill RL, Llamas-Velasco M, Poch G, Korsing S, Sondermann W, Gellrich FF, Heppt MV, Erdmann M, Haferkamp S, Drexler K, Goebeler M, Schilling B, Utikal JS, Ghoreschi K, Fröhling S, Krieghoff-Henning E, Brinker TJ. Dermatologist-like explainable AI enhances trust and confidence in diagnosing melanoma. Nat Commun 2024; 15:524. [PMID: 38225244 PMCID: PMC10789736 DOI: 10.1038/s41467-023-43095-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/24/2023] [Accepted: 10/31/2023] [Indexed: 01/17/2024] Open
Abstract
Artificial intelligence (AI) systems have been shown to help dermatologists diagnose melanoma more accurately, however they lack transparency, hindering user acceptance. Explainable AI (XAI) methods can help to increase transparency, yet often lack precise, domain-specific explanations. Moreover, the impact of XAI methods on dermatologists' decisions has not yet been evaluated. Building upon previous research, we introduce an XAI system that provides precise and domain-specific explanations alongside its differential diagnoses of melanomas and nevi. Through a three-phase study, we assess its impact on dermatologists' diagnostic accuracy, diagnostic confidence, and trust in the XAI-support. Our results show strong alignment between XAI and dermatologist explanations. We also show that dermatologists' confidence in their diagnoses, and their trust in the support system significantly increase with XAI compared to conventional AI. This study highlights dermatologists' willingness to adopt such XAI systems, promoting future use in the clinic.
Collapse
Affiliation(s)
- Tirtha Chanda
- Digital Biomarkers for Oncology Group, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Katja Hauser
- Digital Biomarkers for Oncology Group, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Sarah Hobelsberger
- Department of Dermatology, University Hospital, Technical University Dresden, Dresden, Germany
| | - Tabea-Clara Bucher
- Digital Biomarkers for Oncology Group, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Carina Nogueira Garcia
- Digital Biomarkers for Oncology Group, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Christoph Wies
- Digital Biomarkers for Oncology Group, German Cancer Research Center (DKFZ), Heidelberg, Germany
- Medical Faculty of University Heidelberg, Heidelberg, Germany
| | - Harald Kittler
- Department of Dermatology, Medical University of Vienna, Vienna, Austria
| | - Philipp Tschandl
- Department of Dermatology, Medical University of Vienna, Vienna, Austria
| | - Cristian Navarrete-Dechent
- Department of Dermatology, Escuela de Medicina, Pontificia Universidad Católica de Chile, Santiago, Chile
| | - Sebastian Podlipnik
- Dermatology Department, Hospital Clínic of Barcelona, University of Barcelona, IDIBAPS, Barcelona, Spain
| | - Emmanouil Chousakos
- 1st Department of Pathology, Medical School, National & Kapodistrian University of Athens, Athens, Greece
| | - Iva Crnaric
- Department of Dermatovenereology, Sestre milosrdnice University Hospital Center, Zagreb, Croatia
| | | | - Linda Alhajwan
- Department of Dermatology, Dubai London Clinic, Dubai, United Arab Emirates
| | | | - Sandra Peternel
- Department of Dermatovenereology, Clinical Hospital Center Rijeka, Faculty of Medicine, University of Rijeka, Rijeka, Croatia
| | | | - İrem Özdemir
- Department of Dermatology, Faculty of Medicine, Gazi University, Ankara, Turkey
| | - Raymond L Barnhill
- Department of Translational Research, Institut Curie, Unit of Formation and Research of Medicine University of Paris, Paris, France
| | | | - Gabriela Poch
- Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Department of Dermatology, Venereology and Allergology, Berlin, Germany
| | - Sören Korsing
- Department of Dermatology, University Hospital Essen, University Duisburg-Essen, Essen, Germany
| | - Wiebke Sondermann
- Department of Dermatology, Uniklinikum Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | | | - Markus V Heppt
- Department of Dermatology, Uniklinikum Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Michael Erdmann
- Department of Dermatology, Uniklinikum Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Sebastian Haferkamp
- Department of Dermatology, University Hospital Regensburg, Regensburg, Germany
| | - Konstantin Drexler
- Department of Dermatology, University Hospital Regensburg, Regensburg, Germany
| | - Matthias Goebeler
- Department of Dermatology, Venereology and Allergology, University Hospital Würzburg, Würzburg, Germany
| | - Bastian Schilling
- Department of Dermatology, Venereology and Allergology, University Hospital Würzburg, Würzburg, Germany
| | - Jochen S Utikal
- Department of Dermatology, Venereology and Allergology, University Medical Center Mannheim, Ruprecht-Karl University of Heidelberg, Mannheim, Germany
| | - Kamran Ghoreschi
- Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Department of Dermatology, Venereology and Allergology, Berlin, Germany
| | - Stefan Fröhling
- Division of Translational Medical Oncology, National Center for Tumor Diseases (NCT) Heidelberg and German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Eva Krieghoff-Henning
- Digital Biomarkers for Oncology Group, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Titus J Brinker
- Digital Biomarkers for Oncology Group, German Cancer Research Center (DKFZ), Heidelberg, Germany.
| |
Collapse
|
6
|
Zhu AQ, Wang Q, Shi YL, Ren WW, Cao X, Ren TT, Wang J, Zhang YQ, Sun YK, Chen XW, Lai YX, Ni N, Chen YC, Hu JL, Mou LC, Zhao YJ, Liu YQ, Sun LP, Zhu XX, Xu HX, Guo LH. A deep learning fusion network trained with clinical and high-frequency ultrasound images in the multi-classification of skin diseases in comparison with dermatologists: a prospective and multicenter study. EClinicalMedicine 2024; 67:102391. [PMID: 38274117 PMCID: PMC10808933 DOI: 10.1016/j.eclinm.2023.102391] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/03/2023] [Revised: 12/07/2023] [Accepted: 12/07/2023] [Indexed: 01/27/2024] Open
Abstract
Background Clinical appearance and high-frequency ultrasound (HFUS) are indispensable for diagnosing skin diseases by providing internal and external information. However, their complex combination brings challenges for primary care physicians and dermatologists. Thus, we developed a deep multimodal fusion network (DMFN) model combining analysis of clinical close-up and HFUS images for binary and multiclass classification in skin diseases. Methods Between Jan 10, 2017, and Dec 31, 2020, the DMFN model was trained and validated using 1269 close-ups and 11,852 HFUS images from 1351 skin lesions. The monomodal convolutional neural network (CNN) model was trained and validated with the same close-up images for comparison. Subsequently, we did a prospective and multicenter study in China. Both CNN models were tested prospectively on 422 cases from 4 hospitals and compared with the results from human raters (general practitioners, general dermatologists, and dermatologists specialized in HFUS). The performance of binary classification (benign vs. malignant) and multiclass classification (the specific diagnoses of 17 types of skin diseases) measured by the area under the receiver operating characteristic curve (AUC) were evaluated. This study is registered with www.chictr.org.cn (ChiCTR2300074765). Findings The performance of the DMFN model (AUC, 0.876) was superior to that of the monomodal CNN model (AUC, 0.697) in the binary classification (P = 0.0063), which was also better than that of the general practitioner (AUC, 0.651, P = 0.0025) and general dermatologists (AUC, 0.838; P = 0.0038). By integrating close-up and HFUS images, the DMFN model attained an almost identical performance in comparison to dermatologists (AUC, 0.876 vs. AUC, 0.891; P = 0.0080). For the multiclass classification, the DMFN model (AUC, 0.707) exhibited superior prediction performance compared with general dermatologists (AUC, 0.514; P = 0.0043) and dermatologists specialized in HFUS (AUC, 0.640; P = 0.0083), respectively. Compared to dermatologists specialized in HFUS, the DMFN model showed better or comparable performance in diagnosing 9 of the 17 skin diseases. Interpretation The DMFN model combining analysis of clinical close-up and HFUS images exhibited satisfactory performance in the binary and multiclass classification compared with the dermatologists. It may be a valuable tool for general dermatologists and primary care providers. Funding This work was supported in part by the National Natural Science Foundation of China and the Clinical research project of Shanghai Skin Disease Hospital.
Collapse
Affiliation(s)
- An-Qi Zhu
- Department of Medical Ultrasound, Shanghai Skin Disease Hospital, School of Medicine, Tongji University, Shanghai, China
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, School of Medicine, Tongji University, Shanghai, China
- Department of Ultrasound, Zhongshan Hospital, Institute of Ultrasound in Medicine and Engineering, Fudan University, Shanghai, China
| | - Qiao Wang
- Department of Medical Ultrasound, Shanghai Skin Disease Hospital, School of Medicine, Tongji University, Shanghai, China
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, School of Medicine, Tongji University, Shanghai, China
- Shanghai Engineering Research Center of Ultrasound Diagnosis and Treatment, Shanghai, China
| | - Yi-Lei Shi
- MedAI Technology (Wuxi) Co., Ltd., Wuxi, China
| | - Wei-Wei Ren
- Department of Medical Ultrasound, Shanghai Skin Disease Hospital, School of Medicine, Tongji University, Shanghai, China
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, School of Medicine, Tongji University, Shanghai, China
- Shanghai Engineering Research Center of Ultrasound Diagnosis and Treatment, Shanghai, China
| | - Xu Cao
- MedAI Technology (Wuxi) Co., Ltd., Wuxi, China
| | - Tian-Tian Ren
- Department of Medical Ultrasound, Ma'anshan People's Hospital, Ma'anshan, China
| | - Jing Wang
- Department of Ultrasound, Jiading District Central Hospital Affiliated Shanghai University of Medicine & Health Sciences, Shanghai, China
| | - Ya-Qin Zhang
- Department of Ultrasound, Zhongshan Hospital, Institute of Ultrasound in Medicine and Engineering, Fudan University, Shanghai, China
| | - Yi-Kang Sun
- Department of Ultrasound, Zhongshan Hospital, Institute of Ultrasound in Medicine and Engineering, Fudan University, Shanghai, China
| | - Xue-Wen Chen
- Department of Dermatological Surgery, Shanghai Skin Disease Hospital, School of Medicine, Tongji University, Shanghai, China
| | - Yong-Xian Lai
- Department of Dermatological Surgery, Shanghai Skin Disease Hospital, School of Medicine, Tongji University, Shanghai, China
| | - Na Ni
- Department of Dermatological Surgery, Shanghai Skin Disease Hospital, School of Medicine, Tongji University, Shanghai, China
| | - Yu-Chong Chen
- Department of Dermatological Surgery, Shanghai Skin Disease Hospital, School of Medicine, Tongji University, Shanghai, China
| | | | - Li-Chao Mou
- MedAI Technology (Wuxi) Co., Ltd., Wuxi, China
| | - Yu-Jing Zhao
- Department of Medical Ultrasound, Shanghai Skin Disease Hospital, School of Medicine, Tongji University, Shanghai, China
| | - Ye-Qiang Liu
- Department of Pathology, Shanghai Skin Disease Hospital, School of Medicine, Tongji University, Shanghai, China
| | - Li-Ping Sun
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, School of Medicine, Tongji University, Shanghai, China
- Shanghai Engineering Research Center of Ultrasound Diagnosis and Treatment, Shanghai, China
| | - Xiao-Xiang Zhu
- Chair of Data Science in Earth Observation, Technical University of Munich, Munich, Germany
| | - Hui-Xiong Xu
- Department of Ultrasound, Zhongshan Hospital, Institute of Ultrasound in Medicine and Engineering, Fudan University, Shanghai, China
| | - Le-Hang Guo
- Department of Medical Ultrasound, Shanghai Skin Disease Hospital, School of Medicine, Tongji University, Shanghai, China
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, School of Medicine, Tongji University, Shanghai, China
- Shanghai Engineering Research Center of Ultrasound Diagnosis and Treatment, Shanghai, China
| | - China Alliance of Multi-Center Clinical Study for Ultrasound (Ultra-Chance)
- Department of Medical Ultrasound, Shanghai Skin Disease Hospital, School of Medicine, Tongji University, Shanghai, China
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, School of Medicine, Tongji University, Shanghai, China
- Shanghai Engineering Research Center of Ultrasound Diagnosis and Treatment, Shanghai, China
- MedAI Technology (Wuxi) Co., Ltd., Wuxi, China
- Department of Medical Ultrasound, Ma'anshan People's Hospital, Ma'anshan, China
- Department of Ultrasound, Jiading District Central Hospital Affiliated Shanghai University of Medicine & Health Sciences, Shanghai, China
- Department of Ultrasound, Zhongshan Hospital, Institute of Ultrasound in Medicine and Engineering, Fudan University, Shanghai, China
- Department of Dermatological Surgery, Shanghai Skin Disease Hospital, School of Medicine, Tongji University, Shanghai, China
- Department of Pathology, Shanghai Skin Disease Hospital, School of Medicine, Tongji University, Shanghai, China
- Chair of Data Science in Earth Observation, Technical University of Munich, Munich, Germany
| |
Collapse
|
7
|
Phung M, Muralidharan V, Rotemberg V, Novoa RA, Chiou AS, Sadée CY, Rapaport B, Yekrang K, Bitz J, Gevaert O, Ko JM, Daneshjou R. Best Practices for Clinical Skin Image Acquisition in Translational Artificial Intelligence Research. J Invest Dermatol 2023; 143:1127-1132. [PMID: 37353282 DOI: 10.1016/j.jid.2023.02.035] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/22/2022] [Revised: 01/30/2023] [Accepted: 02/15/2023] [Indexed: 06/25/2023]
Abstract
Recent advances in artificial intelligence research have led to an increase in the development of algorithms for detecting malignancies from clinical and dermoscopic images of skin diseases. These methods are dependent on the collection of training and testing data. There are important considerations when acquiring skin images and data for translational artificial intelligence research. In this paper, we discuss the best practices and challenges for light photography image data collection, covering ethics, image acquisition, labeling, curation, and storage. The purpose of this work is to improve artificial intelligence for malignancy detection by supporting intentional data collection and collaboration between subject matter experts, such as dermatologists and data scientists.
Collapse
Affiliation(s)
- Michelle Phung
- Dermatology Department, Stanford University School of Medicine, Redwood City, California, USA
| | - Vijaytha Muralidharan
- Dermatology Department, Stanford University School of Medicine, Redwood City, California, USA
| | | | - Roberto Andres Novoa
- Dermatology Department, Stanford University School of Medicine, Redwood City, California, USA
| | - Albert Sean Chiou
- Dermatology Department, Stanford University School of Medicine, Redwood City, California, USA
| | - Christoph Y Sadée
- Stanford Center for Biomedical Informatics Research, Stanford University, Stanford, California, USA
| | - Bailie Rapaport
- Dermatology Department, Stanford University School of Medicine, Redwood City, California, USA
| | - Kiana Yekrang
- Dermatology Department, Stanford University School of Medicine, Redwood City, California, USA
| | - Jared Bitz
- Stanford University, Stanford, California, USA
| | - Olivier Gevaert
- Stanford Center for Biomedical Informatics Research, Stanford University, Stanford, California, USA; Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, California, USA
| | - Justin Meng Ko
- Dermatology Department, Stanford University School of Medicine, Redwood City, California, USA
| | - Roxana Daneshjou
- Dermatology Department, Stanford University School of Medicine, Redwood City, California, USA; Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, California, USA.
| |
Collapse
|
8
|
Grossarth S, Mosley D, Madden C, Ike J, Smith I, Huo Y, Wheless L. Recent Advances in Melanoma Diagnosis and Prognosis Using Machine Learning Methods. Curr Oncol Rep 2023; 25:635-645. [PMID: 37000340 PMCID: PMC10339689 DOI: 10.1007/s11912-023-01407-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 03/13/2023] [Indexed: 04/01/2023]
Abstract
PURPOSE OF REVIEW The purpose was to summarize the current role and state of artificial intelligence and machine learning in the diagnosis and management of melanoma. RECENT FINDINGS Deep learning algorithms can identify melanoma from clinical, dermoscopic, and whole slide pathology images with increasing accuracy. Efforts to provide more granular annotation to datasets and to identify new predictors are ongoing. There have been many incremental advances in both melanoma diagnostics and prognostic tools using artificial intelligence and machine learning. Higher quality input data will further improve these models' capabilities.
Collapse
Affiliation(s)
- Sarah Grossarth
- Quillen College of Medicine, East Tennessee State University, Johnson City, TN, USA
| | | | - Christopher Madden
- Department of Dermatology, Vanderbilt University Medicine Center, Nashville, TN, USA
- State University of New York Downstate College of Medicine, Brooklyn, NY, USA
| | - Jacqueline Ike
- Department of Dermatology, Vanderbilt University Medicine Center, Nashville, TN, USA
- Meharry Medical College, Nashville, TN, USA
| | - Isabelle Smith
- Department of Dermatology, Vanderbilt University Medicine Center, Nashville, TN, USA
- Vanderbilt University, Nashville, TN, USA
| | - Yuankai Huo
- Department of Computer Science and Electrical Engineering, Vanderbilt University, Nashville, TN, 37235, USA
| | - Lee Wheless
- Department of Dermatology, Vanderbilt University Medicine Center, Nashville, TN, USA.
- Department of Medicine, Division of Epidemiology, Vanderbilt University Medical Center, Nashville, TN, USA.
- Tennessee Valley Healthcare System VA Medical Center, Nashville, TN, USA.
| |
Collapse
|
9
|
Winkler JK, Blum A, Kommoss K, Enk A, Toberer F, Rosenberger A, Haenssle HA. Assessment of Diagnostic Performance of Dermatologists Cooperating With a Convolutional Neural Network in a Prospective Clinical Study: Human With Machine. JAMA Dermatol 2023; 159:621-627. [PMID: 37133847 PMCID: PMC10157508 DOI: 10.1001/jamadermatol.2023.0905] [Citation(s) in RCA: 15] [Impact Index Per Article: 15.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/21/2022] [Accepted: 03/05/2023] [Indexed: 05/04/2023]
Abstract
Importance Studies suggest that convolutional neural networks (CNNs) perform equally to trained dermatologists in skin lesion classification tasks. Despite the approval of the first neural networks for clinical use, prospective studies demonstrating benefits of human with machine cooperation are lacking. Objective To assess whether dermatologists benefit from cooperation with a market-approved CNN in classifying melanocytic lesions. Design, Setting, and Participants In this prospective diagnostic 2-center study, dermatologists performed skin cancer screenings using naked-eye examination and dermoscopy. Dermatologists graded suspect melanocytic lesions by the probability of malignancy (range 0-1, threshold for malignancy ≥0.5) and indicated management decisions (no action, follow-up, excision). Next, dermoscopic images of suspect lesions were assessed by a market-approved CNN, Moleanalyzer Pro (FotoFinder Systems). The CNN malignancy scores (range 0-1, threshold for malignancy ≥0.5) were transferred to dermatologists with the request to re-evaluate lesions and revise initial decisions in consideration of CNN results. Reference diagnoses were based on histopathologic examination in 125 (54.8%) lesions or, in the case of nonexcised lesions, on clinical follow-up data and expert consensus. Data were collected from October 2020 to October 2021. Main Outcomes and Measures Primary outcome measures were diagnostic sensitivity and specificity of dermatologists alone and dermatologists cooperating with the CNN. Accuracy and receiver operator characteristic area under the curve (ROC AUC) were considered as additional measures. Results A total of 22 dermatologists detected 228 suspect melanocytic lesions (190 nevi, 38 melanomas) in 188 patients (mean [range] age, 53.4 [19-91] years; 97 [51.6%] male patients). Diagnostic sensitivity and specificity significantly improved when dermatologists additionally integrated CNN results into decision-making (mean sensitivity from 84.2% [95% CI, 69.6%-92.6%] to 100.0% [95% CI, 90.8%-100.0%]; P = .03; mean specificity from 72.1% [95% CI, 65.3%-78.0%] to 83.7% [95% CI, 77.8%-88.3%]; P < .001; mean accuracy from 74.1% [95% CI, 68.1%-79.4%] to 86.4% [95% CI, 81.3%-90.3%]; P < .001; and mean ROC AUC from 0.895 [95% CI, 0.836-0.954] to 0.968 [95% CI, 0.948-0.988]; P = .005). In addition, the CNN alone achieved a comparable sensitivity, higher specificity, and higher diagnostic accuracy compared with dermatologists alone in classifying melanocytic lesions. Moreover, unnecessary excisions of benign nevi were reduced by 19.2%, from 104 (54.7%) of 190 benign nevi to 84 nevi when dermatologists cooperated with the CNN (P < .001). Most lesions were examined by dermatologists with 2 to 5 years (96, 42.1%) or less than 2 years of experience (78, 34.2%); others (54, 23.7%) were evaluated by dermatologists with more than 5 years of experience. Dermatologists with less dermoscopy experience cooperating with the CNN had the most diagnostic improvement compared with more experienced dermatologists. Conclusions and Relevance In this prospective diagnostic study, these findings suggest that dermatologists may improve their performance when they cooperate with the market-approved CNN and that a broader application of this human with machine approach could be beneficial for dermatologists and patients.
Collapse
Affiliation(s)
- Julia K. Winkler
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany
| | - Andreas Blum
- Public, Private and Teaching Practice of Dermatology, Konstanz, Germany
| | - Katharina Kommoss
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany
| | - Alexander Enk
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany
| | - Ferdinand Toberer
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany
| | - Albert Rosenberger
- Institute of Genetic Epidemiology, University Medical Center, Georg-August University of Goettingen, Goettingen, Germany
| | - Holger A. Haenssle
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany
| |
Collapse
|
10
|
Li Z, Koban KC, Schenck TL, Giunta RE, Li Q, Sun Y. Artificial Intelligence in Dermatology Image Analysis: Current Developments and Future Trends. J Clin Med 2022; 11:jcm11226826. [PMID: 36431301 PMCID: PMC9693628 DOI: 10.3390/jcm11226826] [Citation(s) in RCA: 27] [Impact Index Per Article: 13.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/24/2022] [Revised: 10/24/2022] [Accepted: 10/28/2022] [Indexed: 11/22/2022] Open
Abstract
BACKGROUND Thanks to the rapid development of computer-based systems and deep-learning-based algorithms, artificial intelligence (AI) has long been integrated into the healthcare field. AI is also particularly helpful in image recognition, surgical assistance and basic research. Due to the unique nature of dermatology, AI-aided dermatological diagnosis based on image recognition has become a modern focus and future trend. Key scientific concepts of review: The use of 3D imaging systems allows clinicians to screen and label skin pigmented lesions and distributed disorders, which can provide an objective assessment and image documentation of lesion sites. Dermatoscopes combined with intelligent software help the dermatologist to easily correlate each close-up image with the corresponding marked lesion in the 3D body map. In addition, AI in the field of prosthetics can assist in the rehabilitation of patients and help to restore limb function after amputation in patients with skin tumors. THE AIM OF THE STUDY For the benefit of patients, dermatologists have an obligation to explore the opportunities, risks and limitations of AI applications. This study focuses on the application of emerging AI in dermatology to aid clinical diagnosis and treatment, analyzes the current state of the field and summarizes its future trends and prospects so as to help dermatologists realize the impact of new technological innovations on traditional practices so that they can embrace and use AI-based medical approaches more quickly.
Collapse
Affiliation(s)
- Zhouxiao Li
- Department of Plastic and Reconstructive Surgery, Shanghai 9th People’s Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai 200023, China
- Division of Hand, Plastic and Aesthetic Surgery, University Hospital, LMU Munich, 80339 Munich, Germany
| | | | - Thilo Ludwig Schenck
- Division of Hand, Plastic and Aesthetic Surgery, University Hospital, LMU Munich, 80339 Munich, Germany
| | - Riccardo Enzo Giunta
- Division of Hand, Plastic and Aesthetic Surgery, University Hospital, LMU Munich, 80339 Munich, Germany
| | - Qingfeng Li
- Department of Plastic and Reconstructive Surgery, Shanghai 9th People’s Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai 200023, China
- Correspondence: (Q.L.); (Y.S.)
| | - Yangbai Sun
- Department of Plastic and Reconstructive Surgery, Shanghai 9th People’s Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai 200023, China
- Correspondence: (Q.L.); (Y.S.)
| |
Collapse
|
11
|
Winkler JK, Haenssle HA. [Artificial intelligence-based classification for the diagnostics of skin cancer]. DERMATOLOGIE (HEIDELBERG, GERMANY) 2022; 73:838-844. [PMID: 36094608 DOI: 10.1007/s00105-022-05058-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 08/22/2022] [Indexed: 06/15/2023]
Abstract
Convolutional neural networks (CNN) achieve a level of performance comparable or even superior to dermatologists in the assessment of pigmented and nonpigmented skin lesions. In the analysis of images by artificial neural networks, images on a pixel level pass through various layers of the network with different graphic filters. Based on excellent study results, a first deep learning network (Moleanalyzer pro, Fotofinder Systems GmBH, Bad Birnbach, Germany) received market approval in Europe. However, such neural networks also reveal relevant limitations, whereby rare entities with insufficient training images are classified less adequately and image artifacts can lead to false diagnoses. Best results can ultimately be achieved in a cooperation of "man with machine". For future skin cancer screening, automated total body mapping is evaluated, which combines total body photography, automated data extraction and assessment of all relevant skin lesions.
Collapse
Affiliation(s)
- Julia K Winkler
- Universitätshautklinik Heidelberg, Im Neuenheimer Feld 440, 69120, Heidelberg, Deutschland.
| | - Holger A Haenssle
- Universitätshautklinik Heidelberg, Im Neuenheimer Feld 440, 69120, Heidelberg, Deutschland
| |
Collapse
|
12
|
Maron RC, Hekler A, Haggenmüller S, von Kalle C, Utikal JS, Müller V, Gaiser M, Meier F, Hobelsberger S, Gellrich FF, Sergon M, Hauschild A, French LE, Heinzerling L, Schlager JG, Ghoreschi K, Schlaak M, Hilke FJ, Poch G, Korsing S, Berking C, Heppt MV, Erdmann M, Haferkamp S, Schadendorf D, Sondermann W, Goebeler M, Schilling B, Kather JN, Fröhling S, Lipka DB, Krieghoff-Henning E, Brinker TJ. Model soups improve performance of dermoscopic skin cancer classifiers. Eur J Cancer 2022; 173:307-316. [PMID: 35973360 DOI: 10.1016/j.ejca.2022.07.002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/09/2022] [Accepted: 07/04/2022] [Indexed: 11/20/2022]
Abstract
BACKGROUND Image-based cancer classifiers suffer from a variety of problems which negatively affect their performance. For example, variation in image brightness or different cameras can already suffice to diminish performance. Ensemble solutions, where multiple model predictions are combined into one, can improve these problems. However, ensembles are computationally intensive and less transparent to practitioners than single model solutions. Constructing model soups, by averaging the weights of multiple models into a single model, could circumvent these limitations while still improving performance. OBJECTIVE To investigate the performance of model soups for a dermoscopic melanoma-nevus skin cancer classification task with respect to (1) generalisation to images from other clinics, (2) robustness against small image changes and (3) calibration such that the confidences correspond closely to the actual predictive uncertainties. METHODS We construct model soups by fine-tuning pre-trained models on seven different image resolutions and subsequently averaging their weights. Performance is evaluated on a multi-source dataset including holdout and external components. RESULTS We find that model soups improve generalisation and calibration on the external component while maintaining performance on the holdout component. For robustness, we observe performance improvements for pertubated test images, while the performance on corrupted test images remains on par. CONCLUSIONS Overall, souping for skin cancer classifiers has a positive effect on generalisation, robustness and calibration. It is easy for practitioners to implement and by combining multiple models into a single model, complexity is reduced. This could be an important factor in achieving clinical applicability, as less complexity generally means more transparency.
Collapse
Affiliation(s)
- Roman C Maron
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Achim Hekler
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Sarah Haggenmüller
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Christof von Kalle
- Department of Clinical-Translational Sciences, Charité University Medicine and Berlin Institute of Health (BIH), Berlin, Germany
| | - Jochen S Utikal
- Department of Dermatology, Venereology and Allergology, University Medical Center Mannheim, Ruprecht-Karl University of Heidelberg, Mannheim, Germany; Skin Cancer Unit, German Cancer Research Center (DKFZ), Heidelberg, Germany; DKFZ Hector Cancer Institute at the University Medical Center Mannheim, Mannheim, Germany
| | - Verena Müller
- Department of Dermatology, Venereology and Allergology, University Medical Center Mannheim, Ruprecht-Karl University of Heidelberg, Mannheim, Germany; Skin Cancer Unit, German Cancer Research Center (DKFZ), Heidelberg, Germany; DKFZ Hector Cancer Institute at the University Medical Center Mannheim, Mannheim, Germany
| | - Maria Gaiser
- Department of Dermatology, Venereology and Allergology, University Medical Center Mannheim, Ruprecht-Karl University of Heidelberg, Mannheim, Germany; Skin Cancer Unit, German Cancer Research Center (DKFZ), Heidelberg, Germany; DKFZ Hector Cancer Institute at the University Medical Center Mannheim, Mannheim, Germany
| | - Friedegund Meier
- Skin Cancer Center at the University Cancer Center and National Center for Tumor Diseases Dresden, Department of Dermatology, University Hospital Carl Gustav Carus, Technische Universität Dresden, Germany
| | - Sarah Hobelsberger
- Skin Cancer Center at the University Cancer Center and National Center for Tumor Diseases Dresden, Department of Dermatology, University Hospital Carl Gustav Carus, Technische Universität Dresden, Germany
| | - Frank F Gellrich
- Skin Cancer Center at the University Cancer Center and National Center for Tumor Diseases Dresden, Department of Dermatology, University Hospital Carl Gustav Carus, Technische Universität Dresden, Germany
| | - Mildred Sergon
- Skin Cancer Center at the University Cancer Center and National Center for Tumor Diseases Dresden, Department of Dermatology, University Hospital Carl Gustav Carus, Technische Universität Dresden, Germany
| | - Axel Hauschild
- Department of Dermatology, University Hospital (UKSH), Kiel, Germany
| | - Lars E French
- Department of Dermatology and Allergy, University Hospital, LMU Munich, Munich, Germany; Dr. Phillip Frost Department of Dermatology and Cutaneous Surgery, University of Miami, Miller School of Medicine, Miami, FL, USA
| | - Lucie Heinzerling
- Department of Dermatology and Allergy, University Hospital, LMU Munich, Munich, Germany
| | - Justin G Schlager
- Department of Dermatology and Allergy, University Hospital, LMU Munich, Munich, Germany
| | - Kamran Ghoreschi
- Department of Dermatology, Venereology and Allergology, Charité - Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Max Schlaak
- Department of Dermatology, Venereology and Allergology, Charité - Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Franz J Hilke
- Department of Dermatology, Venereology and Allergology, Charité - Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Gabriela Poch
- Department of Dermatology, Venereology and Allergology, Charité - Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Sören Korsing
- Department of Dermatology, Venereology and Allergology, Charité - Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Carola Berking
- Department of Dermatology, University Hospital Erlangen, Comprehensive Cancer Center Erlangen - European Metropolitan Region Nürnberg, CCC Alliance WERA, Erlangen, Germany
| | - Markus V Heppt
- Department of Dermatology, University Hospital Erlangen, Comprehensive Cancer Center Erlangen - European Metropolitan Region Nürnberg, CCC Alliance WERA, Erlangen, Germany
| | - Michael Erdmann
- Department of Dermatology, University Hospital Erlangen, Comprehensive Cancer Center Erlangen - European Metropolitan Region Nürnberg, CCC Alliance WERA, Erlangen, Germany
| | - Sebastian Haferkamp
- Department of Dermatology, University Hospital Regensburg, Regensburg, Germany
| | - Dirk Schadendorf
- Department of Dermatology, Venereology and Allergology, University Hospital Essen, Essen, Germany
| | - Wiebke Sondermann
- Department of Dermatology, Venereology and Allergology, University Hospital Essen, Essen, Germany
| | - Matthias Goebeler
- Department of Dermatology, Venereology and Allergology, University Hospital Würzburg, Würzburg, Germany
| | - Bastian Schilling
- Department of Dermatology, Venereology and Allergology, University Hospital Würzburg, Würzburg, Germany
| | - Jakob N Kather
- Department of Medicine III, University Hospital RWTH Aachen, Aachen, Germany
| | - Stefan Fröhling
- Department of Translational Medical Oncology, National Center for Tumor Diseases (NCT) Heidelberg and German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Daniel B Lipka
- Department of Translational Medical Oncology, National Center for Tumor Diseases (NCT) Heidelberg and German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Eva Krieghoff-Henning
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Titus J Brinker
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany.
| |
Collapse
|
13
|
Winkler JK, Tschandl P, Toberer F, Sies K, Fink C, Enk A, Kittler H, Haenssle HA. Monitoring patients at risk for melanoma: May convolutional neural networks replace the strategy of sequential digital dermoscopy? Eur J Cancer 2021; 160:180-188. [PMID: 34840028 DOI: 10.1016/j.ejca.2021.10.030] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/24/2021] [Revised: 10/06/2021] [Accepted: 10/25/2021] [Indexed: 01/11/2023]
Abstract
BACKGROUND Sequential digital dermoscopy (SDD) is applied for early melanoma detection by uncovering dynamic changes of monitored lesions. Convolutional neural networks (CNN) are capable of high diagnostic accuracies similar to trained dermatologists. OBJECTIVES To investigate the capability of CNN to correctly classify melanomas originally diagnosed by mere dynamic changes during SDD. METHODS A retrospective cross-sectional study using image quartets of 59 high-risk patients each containing one melanoma diagnosed by dynamic changes during SDD and three nevi (236 lesions). Two validated CNN classified quartets at baseline or after SDD follow-up at the time of melanoma diagnosis. Moreover, baseline quartets were rated by 26 dermatologists. The main outcome was the number of quartets with correct classifications. RESULTS CNN-1 correctly classified 9 (15.3%) and CNN-2 8 (13.6%) of 59 baseline quartets. In baseline images, CNN-1 attained a sensitivity of 25.4% (16.1%-37.8%) and specificity of 92.7% (87.8%-95.7%), whereas CNN-2 of 28.8% (18.8%-41.4%) and 75.7% (68.9%-81.4%). Expectedly, after SDD follow-up CNN more readily detected melanomas resulting in improved sensitivities (CNN-1: 44.1% [32.2%-56.7%]; CNN-2: 49.2% [36.8%-61.6%]). Dermatologists were told that each baseline quartet contained one melanoma, and on average, correctly classified 24 (22-27) of 59 quartets. Correspondingly, accepting a baseline quartet to be appropriately classified whenever the highest malignancy score was assigned to the melanoma within, CNN-1 and CNN-2 correctly classified 28 (47.5%) and 22 (37.3%) of 59 quartets, respectively. CONCLUSIONS The tested CNN could not replace the strategy of SDD. There is a need for CNN capable of integrating information on dynamic changes into analyses.
Collapse
Affiliation(s)
- Julia K Winkler
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany
| | - Philipp Tschandl
- Department of Dermatology, Medical University of Vienna, Vienna, Austria
| | - Ferdinand Toberer
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany
| | - Katharina Sies
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany
| | - Christine Fink
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany
| | - Alexander Enk
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany
| | - Harald Kittler
- Department of Dermatology, Medical University of Vienna, Vienna, Austria
| | - Holger A Haenssle
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany.
| |
Collapse
|