Consistency of convolutional neural networks in dermoscopic melanoma recognition: A prospective real-world study about the pitfalls of augmented intelligence

BACKGROUND

Deep-learning convolutional neural networks (CNNs) have outperformed even experienced dermatologists in dermoscopic melanoma detection under controlled conditions. It remains unexplored how real-world dermoscopic image transformations affect CNN robustness.

OBJECTIVES

To investigate the consistency of melanoma risk assessment by two commercially available CNNs to help formulate recommendations for current clinical use.

METHODS

A comparative cohort study was conducted from January to July 2022 at the Department of Dermatology, University Hospital Basel. Five dermoscopic images of 116 different lesions on the torso of 66 patients were captured consecutively by the same operator without deliberate rotation. Classification was performed by two CNNs (CNN-1/CNN-2). Lesions were divided into four subgroups based on their initial risk scoring and clinical dignity assessment. Reliability was assessed by variation and intraclass correlation coefficients. Excisions were performed for melanoma suspicion or two consecutively elevated CNN risk scores, and benign lesions were confirmed by expert consensus (n = 3).

RESULTS

117 repeated image series of 116 melanocytic lesions (2 melanomas, 16 dysplastic naevi, 29 naevi, 1 solar lentigo, 1 suspicious and 67 benign) were classified. CNN-1 demonstrated superior measurement repeatability for clinically benign lesions with an initial malignant risk score (mean variation coefficient (mvc): CNN-1: 49.5(±34.3)%; CNN-2: 71.4(±22.5)%; p = 0.03), while CNN-2 outperformed for clinically benign lesions with benign scoring (mvc: CNN-1: 49.7(±22.7)%; CNN-2: 23.8(±29.3)%; p = 0.002). Both systems exhibited lowest score consistency for lesions with an initial malignant risk score and benign assessment. In this context, averaging three initial risk scores achieved highest sensitivity of dignity assessment (CNN-1: 94%; CNN-2: 89%). Intraclass correlation coefficients indicated 'moderate'-to-'good' reliability for both systems (CNN-1: 0.80, 95% CI:0.71-0.87, p < 0.001; CNN-2: 0.67, 95% CI:0.55-0.77, p < 0.001).

CONCLUSIONS

Potential user-induced image changes can significantly influence CNN classification. For clinical application, we recommend using the average of three initial risk scores. Furthermore, we advocate for CNN robustness optimization by cross-validation with repeated image sets.

TRIAL REGISTRATION

ClinicalTrials.gov (NCT04605822).

A comparison of alternative polymer excipients and processing methods for making solid dispersions of a poorly water soluble drug

Solid dispersions were prepared with the extremely poorly water soluble drug, probucol and the water soluble polymers, polyvinyl pyrrolidone (PVP), polyacrylic acid (PAA) or polyethylene oxide (PEO) and blends of these polymers. The solid dispersions were prepared either by the solvent evaporation method, or by compression moulding into films. The materials were characterised by a combination of thermal analysis and FT-Raman spectroscopy. The physical state of the drug was observed to be dependent on the carrier, thus the PVP solid dispersions contained amorphous probucol, whilst the PAA and PEO systems contained the crystalline polymorph II. The method of production was not found to greatly influence the state of the drug in the solid dispersion. The greatest extent of release into solution was observed for the binary blend of drug and PEO, and the blending of polymers was not found to have any advantageous effects in this study.

Nosocomial transmission of hepatitis B virus infection through multiple-dose vials

The source of acute hepatitis B virus (HBV) infection in two women (55 and 72 years old) was investigated. They displayed no risk factors for acquiring HBV infection, other than treatment with local anaesthetic injections some months previously. The HBV strains were sequenced and showed distinct homology to strains seen in Swedish intravenous drug users (IVDU). Prior to these patients' acute infection, an outbreak of HBV had occurred among IVDU in the same county. Analysis of the HBV strains from six of these IVDUs showed their core promoter, precore and pre-S sequences (679 nucleotides) to be identical to those from the two patients. Cross-contamination between samples was excluded and the most likely source of infection was thought to be multiple-dose vials of local anaesthetic that had been contaminated with the HBV strain circulating among the IVDU population in the community. We believe that multiple-dose vials have no place in modern healthcare and recommend sequence homology analysis as an alternative or additional way to trace a source of HBV infection.