A systematic review and meta-analysis of artificial intelligence versus clinicians for skin cancer diagnosis

Scientific research of artificial intelligence (AI) in dermatology has increased exponentially. The objective of this study was to perform a systematic review and meta-analysis to evaluate the performance of AI algorithms for skin cancer classification in comparison to clinicians with different levels of expertise. Based on PRISMA guidelines, 3 electronic databases (PubMed, Embase, and Cochrane Library) were screened for relevant articles up to August 2022. The quality of the studies was assessed using QUADAS-2. A meta-analysis of sensitivity and specificity was performed for the accuracy of AI and clinicians. Fifty-three studies were included in the systematic review, and 19 met the inclusion criteria for the meta-analysis. Considering all studies and all subgroups of clinicians, we found a sensitivity (Sn) and specificity (Sp) of 87.0% and 77.1% for AI algorithms, respectively, and a Sn of 79.78% and Sp of 73.6% for all clinicians (overall); differences were statistically significant for both Sn and Sp. The difference between AI performance (Sn 92.5%, Sp 66.5%) vs. generalists (Sn 64.6%, Sp 72.8%), was greater, when compared with expert clinicians. Performance between AI algorithms (Sn 86.3%, Sp 78.4%) vs expert dermatologists (Sn 84.2%, Sp 74.4%) was clinically comparable. Limitations of AI algorithms in clinical practice should be considered, and future studies should focus on real-world settings, and towards AI-assistance.

International Skin Imaging Collaboration-Designated Diagnoses (ISIC-DX): Consensus terminology for lesion diagnostic labeling

BACKGROUND

A common terminology for diagnosis is critically important for clinical communication, education, research and artificial intelligence. Prevailing lexicons are limited in fully representing skin neoplasms.

OBJECTIVES

To achieve expert consensus on diagnostic terms for skin neoplasms and their hierarchical mapping.

METHODS

Diagnostic terms were extracted from textbooks, publications and extant diagnostic codes. Terms were hierarchically mapped to super-categories (e.g. 'benign') and cellular/tissue-differentiation categories (e.g. 'melanocytic'), and appended with pertinent-modifiers and synonyms. These terms were evaluated using a modified-Delphi consensus approach. Experts from the International-Skin-Imaging-Collaboration (ISIC) were surveyed on agreement with terms and their hierarchical mapping; they could suggest modifying, deleting or adding terms. Consensus threshold was >75% for the initial rounds and >50% for the final round.

RESULTS

Eighteen experts completed all Delphi rounds. Of 379 terms, 356 (94%) reached consensus in round one. Eleven of 226 (5%) benign-category terms, 6/140 (4%) malignant-category terms and 6/13 (46%) indeterminate-category terms did not reach initial agreement. Following three rounds, final consensus consisted of 362 terms mapped to 3 super-categories and 41 cellular/tissue-differentiation categories.

CONCLUSIONS

We have created, agreed upon, and made public a taxonomy for skin neoplasms and their hierarchical mapping. Further study will be needed to evaluate the utility and completeness of the lexicon.

Immunologic Profiling of Immune-Related Cutaneous Adverse Events with Checkpoint Inhibitors Reveals Polarized Actionable Pathways

PURPOSE

Immune-related cutaneous adverse events (ircAEs) occur in ≥50% of patients treated with checkpoint inhibitors (CPI), but mechanisms are poorly understood.

EXPERIMENTAL DESIGN

Phenotyping/biomarker analyses were conducted in 200 patients on CPIs (139 with ircAEs, 61 without, control) to characterize their clinical presentation and immunologic endotypes. Cytokines were evaluated in skin biopsies, skin tape strip (STS) extracts and plasma using real-time PCR and Meso Scale Discovery multiplex cytokine assays.

RESULTS

Eight ircAE phenotypes were identified: pruritus (26%), maculopapular rash (MPR; 21%), eczema (19%), lichenoid (11%), urticaria (8%), psoriasiform (6%), vitiligo (5%), and bullous dermatitis (4%). All phenotypes showed skin lymphocyte and eosinophil infiltrates. Skin biopsy PCR revealed the highest increase in IFN-gamma mRNA in patients with lichenoid (p<0.0001) and psoriasiform dermatitis (p<0.01) as compared to patients without ircAEs, while the highest IL-13 mRNA levels were detected in the eczema (p<0.0001, compared to control). IL-17A mRNA was selectively increased in psoriasiform (p<0.001), lichenoid (p<0.0001), bullous dermatitis (p<0.05) and MPR (p<0.001), compared to control. Distinct cytokine profiles were confirmed in STS and plasma. Analysis determined increased skin/plasma IL-4 cytokine in pruritus, skin IL-13 in eczema, plasma IL-5 and IL-31 in eczema and urticaria, and mixed-cytokine pathways in MPR. Broad inhibition via corticosteroids or type 2-cytokine targeted inhibition resulted in clinical benefit in these ircAEs. In contrast, significant skin upregulation of type 1/type 17 pathways was found in psoriasiform, lichenoid, bullous dermatitis, and type 1 activation in vitiligo.

CONCLUSIONS

Distinct immunologic ircAE endotypes suggest actionable targets for precision medicine-based interventions.

Mixed histiocytic neoplasms: A multicentre series revealing diverse somatic mutations and responses to targeted therapy

Histiocytic neoplasms are diverse clonal haematopoietic disorders, and clinical disease is mediated by tumorous infiltration as well as uncontrolled systemic inflammation. Individual subtypes include Langerhans cell histiocytosis (LCH), Rosai-Dorfman-Destombes disease (RDD) and Erdheim-Chester disease (ECD), and these have been characterized with respect to clinical phenotypes, driver mutations and treatment paradigms. Less is known about patients with mixed histiocytic neoplasms (MXH), that is two or more coexisting disorders. This international collaboration examined patients with biopsy-proven MXH with respect to component disease subtypes, oncogenic driver mutations and responses to conventional (chemotherapeutic or immunosuppressive) versus targeted (BRAF or MEK inhibitor) therapies. Twenty-seven patients were studied with ECD/LCH (19/27), ECD/RDD (6/27), RDD/LCH (1/27) and ECD/RDD/LCH (1/27). Mutations previously undescribed in MXH were identified, including KRAS, MAP2K2, MAPK3, non-V600-BRAF, RAF1 and a BICD2-BRAF fusion. A repeated-measure generalized estimating equation demonstrated that targeted treatment was statistically significantly (1) more likely to result in a complete response (CR), partial response (PR) or stable disease (SD) (odds ratio [OR]: 17.34, 95% CI: 2.19-137.00, p = 0.007), and (2) less likely to result in progression (OR: 0.08, 95% CI: 0.03-0.23, p < 0.0001). Histiocytic neoplasms represent an entity with underappreciated clinical and molecular diversity, poor responsiveness to conventional therapy and exquisite sensitivity to targeted therapy.

Consent and Identifiability for Patient Images in Research, Education, and Image-Based Artificial Intelligence

This survey study reports the perspectives and preferences of US adults regarding use of photographs of their skin in medical research, education, and development of image-based artificial intelligence (AI).

Current State of Dermatology Mobile Applications With Artificial Intelligence Features: A Scoping Review

Importance

With advancements in mobile technology and artificial intelligence (AI) methods, there has been a substantial surge in the availability of direct-to-consumer mobile applications (apps) claiming to aid in the assessment and management of diverse skin conditions. Despite widespread patient downloads, these apps exhibit limited evidence supporting their efficacy.

Objective

To identify and characterize current English-language AI dermatology mobile apps available for download, focusing on aspects such as purpose, supporting evidence, regulatory status, clinician input, data privacy measures, and use of image data.

Evidence Review

In this scoping review, both Apple and Android mobile app stores were systematically searched for dermatology-related apps that use AI algorithms. Each app's purpose, target audience, evidence-based claims, algorithm details, data availability, clinician input during development, and data usage privacy policies were evaluated.

Findings

A total of 909 apps were initially identified. Following the removal of 518 duplicates, 391 apps remained. Subsequent review excluded 350 apps due to nonmedical nature, non-English languages, absence of AI features, or unavailability, ultimately leaving 41 apps for detailed analysis. The findings revealed several concerning aspects of the current landscape of AI apps in dermatology. Notably, none of the apps were approved by the US Food and Drug Administration, and only 2 of the apps included disclaimers for the lack of regulatory approval. Overall, the study found that these apps lack supporting evidence, input from clinicians and/or dermatologists, and transparency in algorithm development, data usage, and user privacy.

Conclusions and Relevance

This scoping review determined that although AI dermatology mobile apps hold promise for improving access to care and patient outcomes, in their current state, they may pose harm due to potential risks, lack of consistent validation, and misleading user communication. Addressing challenges in efficacy, safety, and transparency through effective regulation, validation, and standardized evaluation criteria is essential to harness the benefits of these apps while minimizing risks.

Expert Agreement on the Presence and Spatial Localization of Melanocytic Features in Dermoscopy

Dermoscopy aids in melanoma detection; however, agreement on dermoscopic features, including those of high clinical relevance, remains poor. In this study, we attempted to evaluate agreement among experts on exemplar images not only for the presence of melanocytic-specific features but also for spatial localization. This was a cross-sectional, multicenter, observational study. Dermoscopy images exhibiting at least 1 of 31 melanocytic-specific features were submitted by 25 world experts as exemplars. Using a web-based platform that allows for image markup of specific contrast-defined regions (superpixels), 20 expert readers annotated 248 dermoscopic images in collections of 62 images. Each collection was reviewed by five independent readers. A total of 4,507 feature observations were performed. Good-to-excellent agreement was found for 14 of 31 features (45.2%), with eight achieving excellent agreement (Gwet's AC >0.75) and seven of them being melanoma-specific features. These features were peppering/granularity (0.91), shiny white streaks (0.89), typical pigment network (0.83), blotch irregular (0.82), negative network (0.81), irregular globules (0.78), dotted vessels (0.77), and blue-whitish veil (0.76). By utilizing an exemplar dataset, a good-to-excellent agreement was found for 14 features that have previously been shown useful in discriminating nevi from melanoma. All images are public (www.isic-archive.com) and can be used for education, scientific communication, and machine learning experiments.

Utility of skin tone on pulse oximetry in critically ill patients: a prospective cohort study

Importance

Pulse oximetry, a ubiquitous vital sign in modern medicine, has inequitable accuracy that disproportionately affects Black and Hispanic patients, with associated increases in mortality, organ dysfunction, and oxygen therapy. Although the root cause of these clinical performance discrepancies is believed to be skin tone, previous retrospective studies used self-reported race or ethnicity as a surrogate for skin tone.

Objective

To determine the utility of objectively measured skin tone in explaining pulse oximetry discrepancies.

Design Setting and Participants

Admitted hospital patients at Duke University Hospital were eligible for this prospective cohort study if they had pulse oximetry recorded up to 5 minutes prior to arterial blood gas (ABG) measurements. Skin tone was measured across sixteen body locations using administered visual scales (Fitzpatrick Skin Type, Monk Skin Tone, and Von Luschan), reflectance colorimetry (Delfin SkinColorCatch [L*, individual typology angle {ITA}, Melanin Index {MI}]), and reflectance spectrophotometry (Konica Minolta CM-700D [L*], Variable Spectro 1 [L*]).

Main Outcomes and Measures

Mean directional bias, variability of bias, and accuracy root mean square (ARMS), comparing pulse oximetry and ABG measurements. Linear mixed-effects models were fitted to estimate mean directional bias while accounting for clinical confounders.

Results

128 patients (57 Black, 56 White) with 521 ABG-pulse oximetry pairs were recruited, none with hidden hypoxemia. Skin tone data was prospectively collected using 6 measurement methods, generating 8 measurements. The collected skin tone measurements were shown to yield differences among each other and overlap with self-reported racial groups, suggesting that skin tone could potentially provide information beyond self-reported race. Among the eight skin tone measurements in this study, and compared to self-reported race, the Monk Scale had the best relationship with differences in pulse oximetry bias (point estimate: -2.40%; 95% CI: -4.32%, -0.48%; p=0.01) when comparing patients with lighter and dark skin tones.

Conclusions and relevance

We found clinical performance differences in pulse oximetry, especially in darker skin tones. Additional studies are needed to determine the relative contributions of skin tone measures and other potential factors on pulse oximetry discrepancies.

Position statement of the EADV Artificial Intelligence (AI) Task Force on AI-assisted smartphone apps and web-based services for skin disease

BACKGROUND

As the use of smartphones continues to surge globally, mobile applications (apps) have become a powerful tool for healthcare engagement. Prominent among these are dermatology apps powered by Artificial Intelligence (AI), which provide immediate diagnostic guidance and educational resources for skin diseases, including skin cancer.

OBJECTIVE

This article, authored by the EADV AI Task Force, seeks to offer insights and recommendations for the present and future deployment of AI-assisted smartphone applications (apps) and web-based services for skin diseases with emphasis on skin cancer detection.

METHODS

An initial position statement was drafted on a comprehensive literature review, which was subsequently refined through two rounds of digital discussions and meticulous feedback by the EADV AI Task Force, ensuring its accuracy, clarity and relevance.

RESULTS

Eight key considerations were identified, including risks associated with inaccuracy and improper user education, a decline in professional skills, the influence of non-medical commercial interests, data security, direct and indirect costs, regulatory approval and the necessity of multidisciplinary implementation. Following these considerations, three main recommendations were formulated: (1) to ensure user trust, app developers should prioritize transparency in data quality, accuracy, intended use, privacy and costs; (2) Apps and web-based services should ensure a uniform user experience for diverse groups of patients; (3) European authorities should adopt a rigorous and consistent regulatory framework for dermatology apps to ensure their safety and accuracy for users.

CONCLUSIONS

The utilisation of AI-assisted smartphone apps and web-based services in diagnosing and treating skin diseases has the potential to greatly benefit patients in their dermatology journeys. By prioritising innovation, fostering collaboration and implementing effective regulations, we can ensure the successful integration of these apps into clinical practice.

Outcomes after interruption of targeted therapy in patients with histiocytic neoplasms

Little is known about outcomes following interruption of targeted therapy in adult patients with histiocytic neoplasms. This is an IRB-approved study of patients with histiocytic neoplasms whose BRAF and MEK inhibitors were interrupted after achieving complete or partial response by 18-fluorodeoxyglucose positron emission tomography (FDG-PET). 17/22 (77%) of patients experienced disease relapse following treatment interruption. Achieving a complete response prior to interruption, having a mutation other than BRAFV600E, and receiving MEK inhibition only were each associated with a statistically significant improvement in relapse-free survival. Relapse is common following treatment interruption however some patients may be suitable for limited-duration treatment.

Large language models propagate race-based medicine

Large language models (LLMs) are being integrated into healthcare systems; but these models may recapitulate harmful, race-based medicine. The objective of this study is to assess whether four commercially available large language models (LLMs) propagate harmful, inaccurate, race-based content when responding to eight different scenarios that check for race-based medicine or widespread misconceptions around race. Questions were derived from discussions among four physician experts and prior work on race-based medical misconceptions believed by medical trainees. We assessed four large language models with nine different questions that were interrogated five times each with a total of 45 responses per model. All models had examples of perpetuating race-based medicine in their responses. Models were not always consistent in their responses when asked the same question repeatedly. LLMs are being proposed for use in the healthcare setting, with some models already connecting to electronic health record systems. However, this study shows that based on our findings, these LLMs could potentially cause harm by perpetuating debunked, racist ideas.

Skin Tone Analysis for Representation in Educational Materials (STAR-ED) using machine learning

Images depicting dark skin tones are significantly underrepresented in the educational materials used to teach primary care physicians and dermatologists to recognize skin diseases. This could contribute to disparities in skin disease diagnosis across different racial groups. Previously, domain experts have manually assessed textbooks to estimate the diversity in skin images. Manual assessment does not scale to many educational materials and introduces human errors. To automate this process, we present the Skin Tone Analysis for Representation in EDucational materials (STAR-ED) framework, which assesses skin tone representation in medical education materials using machine learning. Given a document (e.g., a textbook in .pdf), STAR-ED applies content parsing to extract text, images, and table entities in a structured format. Next, it identifies images containing skin, segments the skin-containing portions of those images, and estimates the skin tone using machine learning. STAR-ED was developed using the Fitzpatrick17k dataset. We then externally tested STAR-ED on four commonly used medical textbooks. Results show strong performance in detecting skin images (0.96 ± 0.02 AUROC and 0.90 ± 0.06 F1 score) and classifying skin tones (0.87 ± 0.01 AUROC and 0.91 ± 0.00 F1 score). STAR-ED quantifies the imbalanced representation of skin tones in four medical textbooks: brown and black skin tones (Fitzpatrick V-VI) images constitute only 10.5% of all skin images. We envision this technology as a tool for medical educators, publishers, and practitioners to assess skin tone diversity in their educational materials.

Improving Artificial Intelligence-Based Diagnosis on Pediatric Skin Lesions

Artificial intelligence algorithms to classify melanoma are dependent on their training data, which limits generalizability. The objective of this study was to compare the performance of an artificial intelligence model trained on a standard adult-predominant dermoscopic dataset before and after the addition of additional pediatric training images. The performances were compared using held-out adult and pediatric test sets of images. We trained two models: one (model A) on an adult-predominant dataset (37,662 images from the International Skin Imaging Collaboration) and the other (model A+P) on an additional 1,536 pediatric images. We compared performance between the two models on adult and pediatric held-out test images separately using the area under the receiver operating characteristic curve. We then used Gradient-weighted Class Activation Maps and background skin masking to understand the contributions of the lesion versus background skin to algorithm decision making. Adding images from a pediatric population with different epidemiological and visual patterns to current reference standard datasets improved algorithm performance on pediatric images without diminishing performance on adult images. This suggests a way that dermatologic artificial intelligence models can be made more generalizable. The presence of background skin was important to the pediatric-specific improvement seen between models. Our study highlights the importance of carefully curated and labeled data from diverse inputs to improve the generalizability of AI models for dermatology, in this case applied to dermoscopic images of adult and pediatric lesions to improve melanoma detection.

A reinforcement learning model for AI-based decision support in skin cancer

We investigated whether human preferences hold the potential to improve diagnostic artificial intelligence (AI)-based decision support using skin cancer diagnosis as a use case. We utilized nonuniform rewards and penalties based on expert-generated tables, balancing the benefits and harms of various diagnostic errors, which were applied using reinforcement learning. Compared with supervised learning, the reinforcement learning model improved the sensitivity for melanoma from 61.4% to 79.5% (95% confidence interval (CI): 73.5-85.6%) and for basal cell carcinoma from 79.4% to 87.1% (95% CI: 80.3-93.9%). AI overconfidence was also reduced while simultaneously maintaining accuracy. Reinforcement learning increased the rate of correct diagnoses made by dermatologists by 12.0% (95% CI: 8.8-15.1%) and improved the rate of optimal management decisions from 57.4% to 65.3% (95% CI: 61.7-68.9%). We further demonstrated that the reward-adjusted reinforcement learning model and a threshold-based model outperformed naïve supervised learning in various clinical scenarios. Our findings suggest the potential for incorporating human preferences into image-based diagnostic algorithms.

Best Practices for Clinical Skin Image Acquisition in Translational Artificial Intelligence Research

Recent advances in artificial intelligence research have led to an increase in the development of algorithms for detecting malignancies from clinical and dermoscopic images of skin diseases. These methods are dependent on the collection of training and testing data. There are important considerations when acquiring skin images and data for translational artificial intelligence research. In this paper, we discuss the best practices and challenges for light photography image data collection, covering ethics, image acquisition, labeling, curation, and storage. The purpose of this work is to improve artificial intelligence for malignancy detection by supporting intentional data collection and collaboration between subject matter experts, such as dermatologists and data scientists.

Use of Asynchronous Interprofessional e-Consults in Cancer Patients During the COVID-19 Pandemic: Current and Future Role

The COVID-19 pandemic created a unique challenge to health care systems, requiring rapid implementation of telemedicine services to provide continued care to patients while preserving personal protective equipment and decreasing the risk of disease transmission. Herein, we describe how our institution, an urban cancer center, utilized provider-to-provider telemedicine consultations (interprofessional e-consults) to provide subspecialty access to care to vulnerable patients in the epicenter of a global pandemic.

Agreement Between Experts and an Untrained Crowd for Identifying Dermoscopic Features Using a Gamified App: Reader Feasibility Study

BACKGROUND

Dermoscopy is commonly used for the evaluation of pigmented lesions, but agreement between experts for identification of dermoscopic structures is known to be relatively poor. Expert labeling of medical data is a bottleneck in the development of machine learning (ML) tools, and crowdsourcing has been demonstrated as a cost- and time-efficient method for the annotation of medical images.

OBJECTIVE

The aim of this study is to demonstrate that crowdsourcing can be used to label basic dermoscopic structures from images of pigmented lesions with similar reliability to a group of experts.

METHODS

First, we obtained labels of 248 images of melanocytic lesions with 31 dermoscopic "subfeatures" labeled by 20 dermoscopy experts. These were then collapsed into 6 dermoscopic "superfeatures" based on structural similarity, due to low interrater reliability (IRR): dots, globules, lines, network structures, regression structures, and vessels. These images were then used as the gold standard for the crowd study. The commercial platform DiagnosUs was used to obtain annotations from a nonexpert crowd for the presence or absence of the 6 superfeatures in each of the 248 images. We replicated this methodology with a group of 7 dermatologists to allow direct comparison with the nonexpert crowd. The Cohen κ value was used to measure agreement across raters.

RESULTS

In total, we obtained 139,731 ratings of the 6 dermoscopic superfeatures from the crowd. There was relatively lower agreement for the identification of dots and globules (the median κ values were 0.526 and 0.395, respectively), whereas network structures and vessels showed the highest agreement (the median κ values were 0.581 and 0.798, respectively). This pattern was also seen among the expert raters, who had median κ values of 0.483 and 0.517 for dots and globules, respectively, and 0.758 and 0.790 for network structures and vessels. The median κ values between nonexperts and thresholded average-expert readers were 0.709 for dots, 0.719 for globules, 0.714 for lines, 0.838 for network structures, 0.818 for regression structures, and 0.728 for vessels.

CONCLUSIONS

This study confirmed that IRR for different dermoscopic features varied among a group of experts; a similar pattern was observed in a nonexpert crowd. There was good or excellent agreement for each of the 6 superfeatures between the crowd and the experts, highlighting the similar reliability of the crowd for labeling dermoscopic images. This confirms the feasibility and dependability of using crowdsourcing as a scalable solution to annotate large sets of dermoscopic images, with several potential clinical and educational applications, including the development of novel, explainable ML tools.

Pancreatic cancer: Cutaneous metastases, clinical descriptors and outcomes

BACKGROUND

Cutaneous metastases in pancreatic cancer (PC) are rare. Herein, we evaluate the clinical, genomic, and other descriptors of patients with PC and cutaneous metastases.

METHODS

Institutional databases were queried, and clinical history, demographics, PC cutaneous metastasis details, and overall survival (OS) from cutaneous metastasis diagnosis were abstracted. OS was estimated using Kaplan-Meier methods.

RESULTS

Forty patients were identified, and median age (Q1-Q3, IQR) of PC diagnosis was 66.0 (59.3-72.3, 12.9) years. Most patients had Stage IV disease at diagnosis (n = 26, 65%). The most common location of the primary tumor was the tail of the pancreas (n = 17, 43%). The most common cutaneous metastasis site was the abdomen (n = 31, 78%), with umbilical lesions occurring in 74% (n = 23) of abdominal lesions. The median OS (95% CI) was 11.4 months (7.0, 20.4). Twenty-three patients had umbilical metastases (58%), and 17 patients had non-umbilical metastases (43%). The median OS (95% CI) was 13.7 (7.0, 28.7) months in patients with umbilical metastases and 8.9 (4.1, Not reached) months in patients with non-umbilical metastases (p = 0.1). Sixteen of 40 (40%) patients underwent somatic testing, and findings were consistent with known profiles. Germline testing in 12 (30%) patients identified pathogenic variants in patients: CHEK2, BRCA1, and ATM.

CONCLUSION

Cutaneous metastases from PC most frequently arise from a pancreas tail primary site and most frequently occur in the umbilicus. Cutaneous metastases may generally be categorized as umbilical or non-umbilical metastases.

The long-term evolution of melanocytic nevi among high-risk adults

BACKGROUND

There is little understanding regarding the long-term natural history of melanocytic nevi among adults.

OBJECTIVE

The objective of the study was to describe the long-term natural history of individual nevi located on the torso of high-risk patients.

METHODS

All patients attending Memorial Sloan Kettering Cancer Center (MSKCC) who underwent two total body photography (TBP) sessions 15+ years apart were included ('retrospective' group). To account for a potential selection bias, we also included consecutive patients who had TBP 15+ years ago and consented to undergo follow-up TBP ('prospective' group). We compared baseline and follow-up torso images on the TBPs and evaluated the number of total, new and disappearing nevi; number of seborrheic keratoses and actinic keratoses; each nevus' diameter at both time points; each nevus' colour change; the presence of clinical atypia; and when dermoscopy was available, the dermoscopic features at each time point.

RESULTS

One hundred six patients were included in the study. Although the average age of the patients was 40 at baseline TBP, most patients developed new nevi between imaging sessions (median 16.4 years) with an average of 2.6 (SD = 4.8) nevi per participant. The average number of disappearing nevi was 0.3 (SD = 0.6). In addition, 62/106 (58%) patients had an absolute increase, and 9/106 (8%) patients had an absolute decrease in their total nevus count. Roughly half (49%: 1416/2890) of the nevi that could be evaluated at both time points increased in diameter by at least 25%. Only 6% (159/2890) of nevi shrunk in diameter by at least 25%. Patients with a history of melanoma had a higher rate of disappearing nevi, and their nevi were more likely to grow. Most nevi demonstrated no significant dermoscopic changes.

CONCLUSIONS

High-risk patients acquire new nevi throughout life with very few nevi disappearing over time. Contrary to prior reports, most nevi in adults increase in diameter, while few nevi shrink.

Tertiary lymphoid structures accompanied by fibrillary matrix morphology impact anti-tumor immunity in basal cell carcinomas

Tertiary lymphoid structures (TLS) are specialized lymphoid formations that serve as local repertoire of T- and B-cells at sites of chronic inflammation, autoimmunity, and cancer. While presence of TLS has been associated with improved response to immune checkpoint blockade therapies and overall outcomes in several cancers, its prognostic value in basal cell carcinoma (BCC) has not been investigated. Herein, we determined the prognostic impact of TLS by relating its prevalence and maturation with outcome measures of anti-tumor immunity, namely tumor infiltrating lymphocytes (TILs) and tumor killing. In 30 distinct BCCs, we show the presence of TLS was significantly enriched in tumors harboring a nodular component and more mature primary TLS was associated with TIL counts. Moreover, assessment of the fibrillary matrix surrounding tumors showed discrete morphologies significantly associated with higher TIL counts, critically accounting for heterogeneity in TIL count distribution within TLS maturation stages. Specifically, increased length of fibers and lacunarity of the matrix with concomitant reduction in density and alignment of fibers were present surrounding tumors displaying high TIL counts. Given the interest in inducing TLS formation as a therapeutic intervention as well as its documented prognostic value, elucidating potential impediments to the ability of TLS in driving anti-tumor immunity within the tumor microenvironment warrants further investigation. These results begin to address and highlight the need to integrate stromal features which may present a hindrance to TLS formation and/or effective function as a mediator of immunotherapy response.

Disparities in dermatology AI performance on a diverse, curated clinical image set

An estimated 3 billion people lack access to dermatological care globally. Artificial intelligence (AI) may aid in triaging skin diseases and identifying malignancies. However, most AI models have not been assessed on images of diverse skin tones or uncommon diseases. Thus, we created the Diverse Dermatology Images (DDI) dataset-the first publicly available, expertly curated, and pathologically confirmed image dataset with diverse skin tones. We show that state-of-the-art dermatology AI models exhibit substantial limitations on the DDI dataset, particularly on dark skin tones and uncommon diseases. We find that dermatologists, who often label AI datasets, also perform worse on images of dark skin tones and uncommon diseases. Fine-tuning AI models on the DDI images closes the performance gap between light and dark skin tones. These findings identify important weaknesses and biases in dermatology AI that should be addressed for reliable application to diverse patients and diseases.

Digital skin imaging applications, part I: Assessment of image acquisition technique features

BACKGROUND

The rapid adoption of digital skin imaging applications has increased the utilization of smartphone-acquired images in dermatology. While this has enormous potential for scaling the assessment of concerning skin lesions, the insufficient quality of many consumer/patient-taken images can undermine clinical accuracy and potentially harm patients due to lack of diagnostic interpretability. We aim to characterize the current state of digital skin imaging applications and comprehensively assess how image acquisition features address image quality.

MATERIALS AND METHODS

Publicly discoverable mobile, web, and desktop-based skin imaging applications, identified through keyword searches in mobile app stores, Google Search queries, previous teledermatology studies, and expert recommendations were independently assessed by three reviewers. Applications were categorized by primary audience (consumer-facing, nonhospital-based practice, or enterprise/health system), function (education, store-and-forward teledermatology, live-interactive teledermatology, electronic medical record adjunct/clinical imaging storage, or clinical triage), in-app connection to a healthcare provider (yes or no), and user type (patient, provider, or both).

RESULTS

Just over half (57%) of 191 included skin imaging applications had at least one of 14 image acquisition technique features. Those that were consumer-facing, intended for educational use, and designed for both patient and physician users had significantly greater feature richness (p < 0.05). The most common feature was the inclusion of text-based imaging tips, followed by the requirement to submit multiple images and body area matching.

CONCLUSION

Very few skin imaging applications included more than one image acquisition technique feature. Feature richness varied significantly by audience, function, and user categories. Users of digital dermatology tools should consider which applications have standardized features that improve image quality.

Clinical characteristics and treatment outcomes in patients with histiocytic neoplasms harboring class 3 MAP2K1 mutations, including treatment with the ERK inhibitor ulixertinib

e19081

Background: Histiocytic neoplasms (HN) are clonal myeloid disorders with diverse clinical phenotypes. HN nearly invariably harbor mutations of the mitogen activated protein kinase (MAPK) pathway, including the BRAFV600E mutation in HN subtypes that are responsive to BRAF inhibition. More recently characterized, the second most frequently mutated gene driving HN is MAP2K1, with broad responsiveness to MEK inhibition reported. The most common MAP2K1 variant observed in our cohort (n=300 patients) is the exon 3 p.E102_I103 in-frame deletion, among the Class 3 MAP2K1 mutants predicted to be resistant to allosteric MEK inhibition. We present clinical and treatment characteristics of HN patients with Class 3 MAP2K1 mutations. Methods: Patients with HN and exon 3 p.E102_I103del or similar mutations identified by tumor sequencing were included. Sites of disease were captured. First- and later-line treatments were categorized as observation, chemotherapy (vinblastine, cytarabine, cladribine, methotrexate), immune modulation (anakinra or interferon), MEK inhibition (trametinib or cobimetinib), or ERK inhibition (ulixertinib). Clinical and radiologic responses were captured as partial response (PR), complete response (CR), or progressive disease (PD). PD includes relapse following PR or CR. Results: 16 patients were identified. 8 (50%) were female, and median age at HN diagnosis was 31 (range 22-58). 10 patients had Langerhans cell histiocytosis (LCH), 4 had Erdheim-Chester disease (ECD), 2 had mixed histiocytosis. Sites of HN were bone (16; 100%), lymph node (8; 50%), brain (8; 50%), skin/subcutaneous (4; 25%), retroperitoneum (3; 19%), cardiovascular (3; 19%), abdomen (2; 13%), reproductive (1; 6%) and other sites (5; 31%). Mutations identified were MAP2K1 p.E102_103del (13; 81%), p.L101_103delinsF (1; 6%), p.P105_I107delinsL (1; 6%), and p.I103_A106del (1; 6%). 2 (13%) patients had spontaneous regression of disease and were observed; 3 (19%) patients had CR to first-line chemotherapy. 3 (19%) patients have had CR/PR to first-line MEK inhibition. 8 (50%) patients had PD following chemotherapy and/or immune modulation; of those, 1 was lost to follow-up, 4 had CR/PR to MEK inhibition; however, 3 had PD despite MEK inhibition. These three patients and one treatment-naïve patient were treated with an oral ERK1/2 inhibitor, ulixertinib, on prospective protocols. 3 of 4 had a clinical or radiologic PR (1) or CR (2). Conclusions: Histiocytic neoplasms with Class 3 MAP2K1 mutations represent a diverse spectrum of disease characterized by frequent bone, nodal and neurologic involvement, by frequent resistance to chemotherapy. This entity is resistant to MEK inhibition in some patients, a phenomenon previously undocumented, and responsive to ERK inhibition, which may be a promising therapeutic approach to HN.

Features of Skin Cancer in Black Individuals: A Single-Institution Retrospective Cohort Study

Introduction

Minimal knowledge exists regarding skin cancers in Black individuals, which may adversely affect patient care.

Objectives

To describe clinical features and risk factors of skin cancers in Black individuals.

Methods

Retrospective study of Black individuals diagnosed with skin cancer between January 2000 and January 2020 at our institution.

Results

38,589 patients were diagnosed with skin cancer, of which 165 were Black individuals. One-hundred-thirteen of these Black individuals were diagnosed with melanoma, 35 with squamous cell carcinoma (SCC), and 17 with basal cell carcinoma (BCC). Most melanomas (80.0%, n = 90) were of the acral subtype; 75% (6 of 8 cases with dermoscopic images) displayed a parallel ridge pattern (PRP). The surrounding uninvolved background skin was visible in 7 cases, all demonstrating a PRP. This disappeared adjacent to most of the melanoma lesions (n = 4, 57.1%). creating a peripheral hypopigmented “halo”. The nonmelanoma skin cancers were pigmented and had similar dermoscopic features as reported in predominantly White populations. Most SCCs (n = 5, 71.4%) had a hypopigmented “halo” and most BCCs (n = 10, 55.6%) had an accentuated reticular network adjacent to the lesions.

Conclusions

Skin cancers are pigmented in Black individuals. In both acral melanomas and SCCs, we noted a peripheral rim of hypopigmentation between the lesions and the surrounding uninvolved background skin, while BCCs had accentuation of the background pigmentation adjacent to the lesions. Most acral melanomas displayed a PRP, which was also seen in surrounding uninvolved background skin.

Pancreatic cancer: Cutaneous metastases, clinical descriptors and outcomes

615

Background: The occurrence of cutaneous metastasis from pancreatic cancer (PC) is rare, and the exact incidence is unknown. The literature to date is primarily limited to isolated case reports. Herein, we evaluate the clinical, genomic, and other descriptors of patients with PC and cutaneous metastases. Methods: Institutional databases were queried using search terms “pancreas cancer” and “cutaneous mets”. Clinical history, demographics, PC cutaneous metastasis details, and survival outcomes were abstracted. Results were described using descriptive statistics, and overall survival (OS) from the diagnosis of cutaneous metastasis was estimated using Kaplan-Meier methods. Results: Of 140 on initial search, 40 patients met inclusion criteria of PC and cutaneous metastases and were analyzed. The median age (Q1-Q3, IQR) of pancreatic cancer diagnosis was 66.0 (59.3-72.3, 12.9) years. Most common histologic subtype was adenocarcinoma (n= 39, 98%), and one patient had a neuroendocrine malignancy. Most patients had stage IV disease at diagnosis (n=26, 65%). The most common location of the primary tumor was tail of the pancreas (n=17, 43%). Forty-eight percent (n= 19) had cutaneous metastasis at/within one month of cancer diagnosis. Most patients received chemotherapy (n=37, 93%), with 14 patients (35%) patients also receiving local therapy in the form of local excision or radiation. The most common cutaneous metastasis site was the abdomen (n=40, 66%), with umbilical lesions occurring in 58% (n=23) of abdominal lesions. The median interval (Q1-Q3, IQR) between diagnosis of pancreatic cancer and development of cutaneous metastasis was 1.4 (0-14.5, 14.5) months. The median OS (95% CI) from cutaneous metastasis diagnosis was 11 months (7.0, 20). Table details the observed differences between umbilical vs. non-umbilical metastases. Sixteen of 40 (40%) patients underwent somatic testing. The most frequently mutated genes were KRAS (n= 16, 100%), TP53 (n=7, 44%), CDKN2Ap14ARF (n=5, 31%), CDKN2Ap16INK4A (n=5, 31%), and CDKN2B (n=3, 19%). Germline testing was undertaken in 12 (30%) patients, and pathogenic variants were observed in 3: CHEK2 (n=1, 8%), BRCA1 (n=1, 8%), and ATM (n= 1, 8%). Summary of cutaneous metastasis characteristics. Conclusions: Cutaneous metastases from PC are rare and can be present at the time of diagnosis of stage IV disease, occurring most frequently in the umbilicus. Cutaneous metastases can be classified into umbilical and non-umbilical metastases, which may be due to a different biology.[Table: see text]

Checklist for Evaluation of Image-Based Artificial Intelligence Reports in Dermatology: CLEAR Derm Consensus Guidelines From the International Skin Imaging Collaboration Artificial Intelligence Working Group

IMPORTANCE

The use of artificial intelligence (AI) is accelerating in all aspects of medicine and has the potential to transform clinical care and dermatology workflows. However, to develop image-based algorithms for dermatology applications, comprehensive criteria establishing development and performance evaluation standards are required to ensure product fairness, reliability, and safety.

OBJECTIVE

To consolidate limited existing literature with expert opinion to guide developers and reviewers of dermatology AI.

EVIDENCE REVIEW

In this consensus statement, the 19 members of the International Skin Imaging Collaboration AI working group volunteered to provide a consensus statement. A systematic PubMed search was performed of English-language articles published between December 1, 2008, and August 24, 2021, for "artificial intelligence" and "reporting guidelines," as well as other pertinent studies identified by the expert panel. Factors that were viewed as critical to AI development and performance evaluation were included and underwent 2 rounds of electronic discussion to achieve consensus.

FINDINGS

A checklist of items was developed that outlines best practices of image-based AI development and assessment in dermatology.

CONCLUSIONS AND RELEVANCE

Clinically effective AI needs to be fair, reliable, and safe; this checklist of best practices will help both developers and reviewers achieve this goal.

Accuracy of commercially available smartphone applications for the detection of melanoma

Artificial intelligence (AI) has shown promise in the analysis of images for detection of melanoma.¹ The number of available dermatology smartphone applications ("apps") is rapidly growing and there is increasing interest in apps that provide diagnosis or triage of skin lesions.^{2, 3} A 2020 systematic review found that nine studies evaluating six apps had poor study design and high risk of bias.³ To date, no studies have evaluated the accuracy of apps using an independent test set of clinical images comparable to those submitted through smartphones .

Lack of Transparency and Potential Bias in Artificial Intelligence Data Sets and Algorithms: A Scoping Review

Importance

Clinical artificial intelligence (AI) algorithms have the potential to improve clinical care, but fair, generalizable algorithms depend on the clinical data on which they are trained and tested.

Objective

To assess whether data sets used for training diagnostic AI algorithms addressing skin disease are adequately described and to identify potential sources of bias in these data sets.

Data Sources

In this scoping review, PubMed was used to search for peer-reviewed research articles published between January 1, 2015, and November 1, 2020, with the following paired search terms: deep learning and dermatology, artificial intelligence and dermatology, deep learning and dermatologist, and artificial intelligence and dermatologist.

Study Selection

Studies that developed or tested an existing deep learning algorithm for triage, diagnosis, or monitoring using clinical or dermoscopic images of skin disease were selected, and the articles were independently reviewed by 2 investigators to verify that they met selection criteria.

Consensus Process

Data set audit criteria were determined by consensus of all authors after reviewing existing literature to highlight data set transparency and sources of bias.

Results

A total of 70 unique studies were included. Among these studies, 1 065 291 images were used to develop or test AI algorithms, of which only 257 372 (24.2%) were publicly available. Only 14 studies (20.0%) included descriptions of patient ethnicity or race in at least 1 data set used. Only 7 studies (10.0%) included any information about skin tone in at least 1 data set used. Thirty-six of the 56 studies developing new AI algorithms for cutaneous malignant neoplasms (64.3%) met the gold standard criteria for disease labeling. Public data sets were cited more often than private data sets, suggesting that public data sets contribute more to new development and benchmarks.

Conclusions and Relevance

This scoping review identified 3 issues in data sets that are used to develop and test clinical AI algorithms for skin disease that should be addressed before clinical translation: (1) sparsity of data set characterization and lack of transparency, (2) nonstandard and unverified disease labels, and (3) inability to fully assess patient diversity used for algorithm development and testing.

Suggested methodology for longitudinal evaluation of nevi based on clinical images

BACKGROUND

Melanoma screening includes the assessment of changes in melanocytic lesions using images. However, previous studies of normal nevus temporal changes showed variable results and the optimal method for evaluating these changes remains unclear. Our aim was to evaluate the reproducibility of (a) nevus count done at a single time point (method I) versus two time points (method II); and (b) manual and automated nevus diameter measurements.

MATERIALS AND METHODS

In a first experiment, participants used either a single time point or a two time point annotation method to evaluate the total number and size of nevi on the back of an atypical mole syndrome patient. A Monte Carlo simulation was used to calculate the variance observed. In a second experiment, manual measurements of nevi on 2D images were compared to an automated measurement on 3D images. Percent difference in the paired manual and automated measurements was calculated.

RESULTS

Mean nevus count was 137 in method I and 115.5 in method II. The standard deviation was greater in method I (38.80) than in method II (4.65) (p = 0.0025). Manual diameter measurements had intraclass correlation coefficient of 0.88. The observed mean percent difference between manual and automated diameter measurements was 1.5%. Lightly pigmented and laterally located nevi had a higher percent difference.

CONCLUSIONS

Comparison of nevi from two different time points is more consistent than nevus count performed separately at each time point. In addition, except for selected cases, automated measurements of nevus diameter on 3D images can be used as a time-saving reproducible substitute for manual measurement on 2D images.

Multispecialty Enterprise Imaging Workgroup Consensus on Interactive Multimedia Reporting Current State and Road to the Future: HIMSS-SIIM Collaborative White Paper

Diagnostic and evidential static image, video clip, and sound multimedia are captured during routine clinical care in cardiology, dermatology, ophthalmology, pathology, physiatry, radiation oncology, radiology, endoscopic procedural specialties, and other medical disciplines. Providers typically describe the multimedia findings in contemporaneous electronic health record clinical notes or associate a textual interpretative report. Visual communication aids commonly used to connect, synthesize, and supplement multimedia and descriptive text outside medicine remain technically challenging to integrate into patient care. Such beneficial interactive elements may include hyperlinks between text, multimedia elements, alphanumeric and geometric annotations, tables, graphs, timelines, diagrams, anatomic maps, and hyperlinks to external educational references that patients or provider consumers may find valuable. This HIMSS-SIIM Enterprise Imaging Community workgroup white paper outlines the current and desired clinical future state of interactive multimedia reporting (IMR). The workgroup adopted a consensus definition of IMR as “interactive medical documentation that combines clinical images, videos, sound, imaging metadata, and/or image annotations with text, typographic emphases, tables, graphs, event timelines, anatomic maps, hyperlinks, and/or educational resources to optimize communication between medical professionals, and between medical professionals and their patients.” This white paper also serves as a precursor for future efforts toward solving technical issues impeding routine interactive multimedia report creation and ingestion into electronic health records.

Quantifying the clinical severity of immune-related cutaneous adverse events in clinical trial patients: A prospective study using 3D-total body photography

e13548

Background: Accurate and comprehensive assessment of dermatologic adverse events (AEs) in clinical trials is challenging, given the heterogeneity of appearance and perception of these AEs. For dermatologic AEs, Common Terminology Criteria for Adverse Events (CTCAE) grading of clinical severity primarily relies on the clinician’s reporting of body surface area involved (BSA%) estimated by visual assessment combined with clinician’s interpretation of psychosocial impact, which can vary among raters. Although patient reported outcome (PRO-CTCAE) and QoL (SKINDEX-16) measures have been incorporated to improve accuracy and reliability of symptomatic AE evaluations, the subjective nature of dermatologic CTCAE grading remains. Clinical photography is routinely used to aid in visual comparison but cannot be incorporated to standardized measures due to inconsistencies in lighting, distance from the camera, and position. This prospective study aims to validate the accuracy and utility of affected BSA% using 3D-total body photography (TBP) and quantitative imaging analytics for standardized, objective assessment of immune-related cutaneous adverse events (ircAE) in clinical trials and/or prospective studies. Methods: Polarized and non-polarized TBPs (Canfield Vectra WB360) were acquired on two dermatology clinic visits 2-6 weeks apart (n = 8, to date). CTCAE, PRO-CTCAE, SKINDEX-16 were evaluated by one investigator for both visits. Image analysis including BSA% calculation was conducted using Vectra measurement software by an investigator blinded to clinical grades. Means and ranges for change in CTCAE, BSA%, and SKINDEX-16 were calculated. Results: To date, 29 patients with ircAE have been enrolled with 8 completing both visits. A greater improvement in affected BSA% (-1.9, -13.0, -20.1) and SKINDEX-16 (-8,-11.9,-19.5) were associated with a greater degree of clinical improvement measured by CTCAE (0 to -3). Widest range in the degree of change in BSA% (-24.4, -6.8) and SKINDEX-16 (-31,19) was observed in the group with intermediate change in CTCAE (-2 to -1, n = 5). Conclusions: Estimating affected BSA% via visual assessment is subject to human error and rely on memory or comparison with unstandardized photos to detect clinical improvement of dermatologic conditions. This poses a challenge particularly for cases with mild or moderate clinical improvement, demonstrated by the wide range in the degree of change for BSA% and SKINDEX-16. With consistent lighting, position, and objective measurements, 3D-TBP quantitative image analysis shows promise in reproducible, standardized monitoring and quantification of ircAE clinical severity. Furthermore, the quantitative nature of TBP measurement suggests its potential utility for correlation with underlying immunophenotyping correlative studies and clinical trials.