AI-based smartphone apps for risk assessment of skin cancer need more evaluation and better regulation

Artificial intelligence and skin cancer

Artificial intelligence is poised to rapidly reshape many fields, including that of skin cancer screening and diagnosis, both as a disruptive and assistive technology. Together with the collection and availability of large medical data sets, artificial intelligence will become a powerful tool that can be leveraged by physicians in their diagnoses and treatment plans for patients. This comprehensive review focuses on current progress toward AI applications for patients, primary care providers, dermatologists, and dermatopathologists, explores the diverse applications of image and molecular processing for skin cancer, and highlights AI's potential for patient self-screening and improving diagnostic accuracy for non-dermatologists. We additionally delve into the challenges and barriers to clinical implementation, paths forward for implementation and areas of active research.

Current State of Dermatology Mobile Applications With Artificial Intelligence Features: A Scoping Review

Importance

With advancements in mobile technology and artificial intelligence (AI) methods, there has been a substantial surge in the availability of direct-to-consumer mobile applications (apps) claiming to aid in the assessment and management of diverse skin conditions. Despite widespread patient downloads, these apps exhibit limited evidence supporting their efficacy.

Objective

To identify and characterize current English-language AI dermatology mobile apps available for download, focusing on aspects such as purpose, supporting evidence, regulatory status, clinician input, data privacy measures, and use of image data.

Evidence Review

In this scoping review, both Apple and Android mobile app stores were systematically searched for dermatology-related apps that use AI algorithms. Each app's purpose, target audience, evidence-based claims, algorithm details, data availability, clinician input during development, and data usage privacy policies were evaluated.

Findings

A total of 909 apps were initially identified. Following the removal of 518 duplicates, 391 apps remained. Subsequent review excluded 350 apps due to nonmedical nature, non-English languages, absence of AI features, or unavailability, ultimately leaving 41 apps for detailed analysis. The findings revealed several concerning aspects of the current landscape of AI apps in dermatology. Notably, none of the apps were approved by the US Food and Drug Administration, and only 2 of the apps included disclaimers for the lack of regulatory approval. Overall, the study found that these apps lack supporting evidence, input from clinicians and/or dermatologists, and transparency in algorithm development, data usage, and user privacy.

Conclusions and Relevance

This scoping review determined that although AI dermatology mobile apps hold promise for improving access to care and patient outcomes, in their current state, they may pose harm due to potential risks, lack of consistent validation, and misleading user communication. Addressing challenges in efficacy, safety, and transparency through effective regulation, validation, and standardized evaluation criteria is essential to harness the benefits of these apps while minimizing risks.

Artificial intelligence in the detection of skin cancer: State of the art

The incidence of melanoma is increasing rapidly. This cancer has a good prognosis if detected early. For this reason, various systems of skin lesion image analysis, which support imaging diagnostics of this neoplasm, are developing very dynamically. To detect and recognize neoplastic lesions, such systems use various artificial intelligence (AI) algorithms. This area of computer science applications has recently undergone dynamic development, abounding in several solutions that are effective tools supporting diagnosticians in many medical specialties. In this contribution, a number of applications of different classes of AI algorithms for the detection of this skin melanoma are presented and evaluated. Both classic systems based on the analysis of dermatoscopic images as well as total body systems, enabling the analysis of the patient's whole body to detect moles and pathologic changes, are discussed. These increasingly popular applications that allow the analysis of lesion images using smartphones are also described. The quantitative evaluation of the discussed systems with particular emphasis on the method of validation of the implemented algorithms is presented. The advantages and limitations of AI in the analysis of lesion images are also discussed, and problems requiring a solution for more effective use of AI in dermatology are identified.

Position statement of the EADV Artificial Intelligence (AI) Task Force on AI-assisted smartphone apps and web-based services for skin disease

BACKGROUND

As the use of smartphones continues to surge globally, mobile applications (apps) have become a powerful tool for healthcare engagement. Prominent among these are dermatology apps powered by Artificial Intelligence (AI), which provide immediate diagnostic guidance and educational resources for skin diseases, including skin cancer.

OBJECTIVE

This article, authored by the EADV AI Task Force, seeks to offer insights and recommendations for the present and future deployment of AI-assisted smartphone applications (apps) and web-based services for skin diseases with emphasis on skin cancer detection.

METHODS

An initial position statement was drafted on a comprehensive literature review, which was subsequently refined through two rounds of digital discussions and meticulous feedback by the EADV AI Task Force, ensuring its accuracy, clarity and relevance.

RESULTS

Eight key considerations were identified, including risks associated with inaccuracy and improper user education, a decline in professional skills, the influence of non-medical commercial interests, data security, direct and indirect costs, regulatory approval and the necessity of multidisciplinary implementation. Following these considerations, three main recommendations were formulated: (1) to ensure user trust, app developers should prioritize transparency in data quality, accuracy, intended use, privacy and costs; (2) Apps and web-based services should ensure a uniform user experience for diverse groups of patients; (3) European authorities should adopt a rigorous and consistent regulatory framework for dermatology apps to ensure their safety and accuracy for users.

CONCLUSIONS

The utilisation of AI-assisted smartphone apps and web-based services in diagnosing and treating skin diseases has the potential to greatly benefit patients in their dermatology journeys. By prioritising innovation, fostering collaboration and implementing effective regulations, we can ensure the successful integration of these apps into clinical practice.

Artificial intelligence in mobile health for skin cancer diagnostics at home (AIM HIGH): a pilot feasibility study

Background

Artificial intelligence (AI)-based mobile phone apps (mHealth) have the potential to streamline care for suspicious skin lesions in primary care. This study aims to investigate the conditions and feasibility of a study that incorporates an AI-based app in primary care and evaluates its potential impact.

Methods

We conducted a pilot feasibility study from November 22nd, 2021 to June 9th, 2022 with a mixed-methods design on implementation of an AI-based mHealth app for skin cancer detection in three primary care practices in the Netherlands (Rotterdam, Leiden and Katwijk). The primary outcome was the inclusion and successful participation rate of patients and general practitioners (GPs). Secondary outcomes were the reasons, facilitators and barriers for successful participation and the potential impact in both pathways for future sample size calculations. Patients were offered use of an AI-based mHealth app before consulting their GP. GPs assessed the patients blinded and then unblinded to the app. Qualitative data included observations and audio-diaries from patients and GPs and focus-groups and interviews with GPs and GP assistants.

Findings

Fifty patients were included with a median age of 52 years (IQR 33.5-60.3), 64% were female, and 90% had a light skin type. The average patient inclusion rate was 4-6 per GP practice per month and 84% (n = 42) successfully participated. Similarly, in 90% (n = 45 patients) the GPs also successfully completed the study. GPs never changed their working diagnosis, but did change their treatment plan (n = 5) based on the app's assessments. Notably, 54% of patients with a benign skin lesion and low risk rating, indicated that they would be reassured and cancel their GP visit with these results (p < 0.001).

Interpretation

Our findings suggest that studying implementation of an AI-based mHealth app for detection of skin cancer in the hands of patients or as a diagnostic tool used by GPs in primary care appears feasible. Preliminary results indicate potential to further investigate both intended use settings.

Funding

SkinVision B.V.

Development and Narrow Validation of Computer Vision Approach to Facilitate Assessment of Change in Pigmented Cutaneous Lesions

The documentation of the change in the number and appearance of pigmented cutaneous lesions over time is critical to the early detection of skin cancers and may provide preliminary signals of efficacy in early-phase therapeutic prevention trials for melanoma. Despite substantial progress in computer-aided diagnosis of melanoma, automated methods to assess the evolution of lesions are relatively undeveloped. This report describes the development and narrow validation of mathematical algorithms to register nevi between sequential digital photographs of large areas of skin and to align images for improved detection and quantification of changes. Serial posterior truncal photographs from a pre-existing database were processed and analyzed by the software, and the results were evaluated by a panel of clinicians using a separate Extensible Markup Language‒based application. The software had a high sensitivity for the detection of cutaneous lesions as small as 2 mm. The software registered lesions accurately, with occasional errors at the edges of the images. In one pilot study with 17 patients, the use of the software enabled clinicians to identify new and/or enlarged lesions in 3‒11 additional patients versus the unregistered images. Automated quantification of size change performed similarly to that of human raters. These results support the further development and broader validation of this technique.

Effect of Mobile Phone App-Based Interventions on Quality of Life and Psychological Symptoms Among Adult Cancer Survivors: Systematic Review and Meta-analysis of Randomized Controlled Trials

BACKGROUND

Most patients with cancer experience psychological or physical distress, which can adversely affect their quality of life (QOL). Smartphone app interventions are increasingly being used to improve QOL and psychological outcomes in patients with cancer. However, there is insufficient evidence regarding the effect of this type of intervention, with conflicting results in the literature.

OBJECTIVE

In this systematic review and meta-analysis, we investigated the effectiveness of mobile phone app interventions on QOL and psychological outcomes in adult patients with cancer, with a special focus on intervention duration, type of cancer, intervention theory, treatment strategy, and intervention delivery format.

METHODS

We conducted a literature search of PubMed, Web of Science, the Cochrane Library, Embase, Scopus, China National Knowledge Infrastructure, and WanFang to identify studies involving apps that focused on cancer survivors and QOL or psychological symptoms published from inception to October 30, 2022. We selected only randomized controlled trials that met the inclusion criteria and performed systematic review and meta-analysis. The standardized mean difference (SMD) with a 95% CI was pooled when needed. Sensitivity and subgroup analyses were also conducted.

RESULTS

In total, 30 randomized controlled trials with a total of 5353 participants were included in this meta-analysis. Compared with routine care, app interventions might improve QOL (SMD=0.39, 95% CI 0.27-0.51; P<.001); enhance self-efficacy (SMD=0.15, 95% CI 0.02-0.29; P=.03); and alleviate anxiety (SMD=-0.64, 95% CI -0.73 to -0.56; P<.001), depression (SMD=-0.33, 95% CI -0.58 to -0.08; P=.009), and distress (SMD=-0.34, 95% CI -0.61 to -0.08; P=.01). Short-term (duration of ≤3 months), physician-patient interaction (2-way communication using a smartphone app), and cognitive behavioral therapy interventions might be the most effective for improving QOL and alleviating adverse psychological effects.

CONCLUSIONS

Our study showed that interventions using mobile health apps might improve QOL and self-efficacy as well as alleviate anxiety, depression, and distress in adult cancer survivors. However, these results should be interpreted with caution because of the heterogeneity of the interventions and the study design. More rigorous trials are warranted to confirm the suitable duration and validate the different intervention theories as well as address methodological flaws in previous studies.

TRIAL REGISTRATION

PROSPERO CRD42022370599; https://www.crd.york.ac.uk/PROSPERO/display_record.php?RecordID=370599.

Artificial intelligence in the detection of skin cancer

Recent advances in artificial intelligence (AI) in dermatology have demonstrated the potential to improve the accuracy of skin cancer detection. These capabilities may augment current diagnostic processes and improve the approach to the management of skin cancer. To explain this technology, we discuss fundamental terminology, potential benefits, and limitations of AI, and commercial applications relevant to dermatologists. A clear understanding of the technology may help to reduce physician concerns about AI and promote its use in the clinical setting. Ultimately, the development and validation of AI technologies, their approval by regulatory agencies, and widespread adoption by dermatologists and other clinicians may enhance patient care. Technology-augmented detection of skin cancer has the potential to improve quality of life, reduce health care costs by reducing unnecessary procedures, and promote greater access to high-quality skin assessment. Dermatologists play a critical role in the responsible development and deployment of AI capabilities applied to skin cancer.

Digital skin imaging applications, part II: a comprehensive survey of post-acquisition image utilization features and technology standards

BACKGROUND

Despite the increasing ubiquity and accessibility of teledermatology applications, few studies have comprehensively surveyed their features and technical standards. Importantly, features implemented after the point of capture are often intended to augment image utilization, while technical standards affect interoperability with existing healthcare systems. We aim to comprehensively survey image utilization features and technical characteristics found within publicly discoverable digital skin imaging applications.

MATERIALS AND METHODS

Applications were identified and categorized as described in Part I. Included applications were then further assessed by three independent reviewers for post-imaging content, tools, and functionality. Publicly available information was used to determine the presence or absence of relevant technology standards and/or data characteristics.

RESULTS

A total of 20 post-image acquisition features were identified across three general categories: (1) metadata attachment, (2) functional tools (i.e., those that utilized images or in-app content to perform a user-directed function), and (3) image processing. Over 80% of all applications implemented metadata features, with nearly half having metadata features only. Individual feature occurred and feature richness varied significantly by primary audience (p < 0.0001) and function (p < 0.0001). On average, each application included under three features. Less than half of all applications requested consent for user-uploaded photos and fewer than 10% provided clear data use and privacy policies.

CONCLUSION

Post-imaging functionality in skin imaging applications varies significantly by primary audience and intended function, though nearly all applications implemented metadata labeling. Technical standards are often not implemented or reported consistently. Gaps in the provision of clear consent, data privacy, and data use policies should be urgently addressed.

Over-Detection of Melanoma-Suspect Lesions by a CE-Certified Smartphone App: Performance in Comparison to Dermatologists, 2D and 3D Convolutional Neural Networks in a Prospective Data Set of 1204 Pigmented Skin Lesions Involving Patients' Perception

Simple Summary

Early detection and resection of cutaneous melanoma are crucial for a good prognosis. However, visual distinction of early melanomas from benign nevi remains challenging. New artificial intelligence-based approaches for risk stratification of pigmented skin lesions provide screening methods for laypersons with increasing use of smartphone applications (apps). Our study aims to prospectively investigate the diagnostic accuracy of a CE-certified smartphone app, SkinVision^®, in melanoma recognition. Based on classification into three different risk scores, the app provides a recommendation to consult a dermatologist. In addition, both patients’ and dermatologists’ perspectives towards AI-based mobile health apps were evaluated. We observed that the app classified a significantly higher number of lesions as high-risk than dermatologists, which would have led to a clinically harmful number of unnecessary excisions. The diagnostic performance of the app in dichotomous classification of 1204 pigmented skin lesions (risk classification for nevus vs. melanoma) remained below advertised rates with low sensitivity (41.3–83.3%) and specificity (60.0–82.9%). The confidence in the app was low among both patients and dermatologists, and no patient favored an assessment by the app alone. Although smartphone apps are a potential medium for increasing awareness of melanoma screening in the lay population, they should be evaluated for certification with prospective real-world evidence.

Abstract

The exponential increase in algorithm-based mobile health (mHealth) applications (apps) for melanoma screening is a reaction to a growing market. However, the performance of available apps remains to be investigated. In this prospective study, we investigated the diagnostic accuracy of a class 1 CE-certified smartphone app in melanoma risk stratification and its patient and dermatologist satisfaction. Pigmented skin lesions ≥ 3 mm and any suspicious smaller lesions were assessed by the smartphone app SkinVision^® (SkinVision^® B.V., Amsterdam, the Netherlands, App-Version 6.8.1), 2D FotoFinder ATBM^® master (FotoFinder ATBM^® Systems GmbH, Bad Birnbach, Germany, Version 3.3.1.0), 3D Vectra^® WB360 (Canfield Scientific, Parsippany, NJ, USA, Version 4.7.1) total body photography (TBP) devices, and dermatologists. The high-risk score of the smartphone app was compared with the two gold standards: histological diagnosis, or if not available, the combination of dermatologists’, 2D and 3D risk assessments. A total of 1204 lesions among 114 patients (mean age 59 years; 51% females (55 patients at high-risk for developing a melanoma, 59 melanoma patients)) were included. The smartphone app’s sensitivity, specificity, and area under the receiver operating characteristics (AUROC) varied between 41.3–83.3%, 60.0–82.9%, and 0.62–0.72% according to two study-defined reference standards. Additionally, all patients and dermatologists completed a newly created questionnaire for preference and trust of screening type. The smartphone app was rated as trustworthy by 36% (20/55) of patients at high-risk for melanoma, 49% (29/59) of melanoma patients, and 8.8% (10/114) of dermatologists. Most of the patients rated the 2D TBP imaging (93% (51/55) resp. 88% (52/59)) and the 3D TBP imaging (91% (50/55) resp. 90% (53/59)) as trustworthy. A skin cancer screening by combination of dermatologist and smartphone app was favored by only 1.8% (1/55) resp. 3.4% (2/59) of the patients; no patient preferred an assessment by a smartphone app alone. The diagnostic accuracy in clinical practice was not as reliable as previously advertised and the satisfaction with smartphone apps for melanoma risk stratification was scarce. MHealth apps might be a potential medium to increase awareness for melanoma screening in the lay population, but healthcare professionals and users should be alerted to the potential harm of over-detection and poor performance. In conclusion, we suggest further robust evidence-based evaluation before including market-approved apps in self-examination for public health benefits.

Accuracy of commercially available smartphone applications for the detection of melanoma

Artificial intelligence (AI) has shown promise in the analysis of images for detection of melanoma.¹ The number of available dermatology smartphone applications ("apps") is rapidly growing and there is increasing interest in apps that provide diagnosis or triage of skin lesions.^{2, 3} A 2020 systematic review found that nine studies evaluating six apps had poor study design and high risk of bias.³ To date, no studies have evaluated the accuracy of apps using an independent test set of clinical images comparable to those submitted through smartphones .