AI outperformed every dermatologist in dermoscopic melanoma diagnosis, using an optimized deep-CNN architecture with custom mini-batch logic and loss function

A scoping review of robustness concepts for machine learning in healthcare

While machine learning (ML)-based solutions-often referred to as artificial intelligence (AI) solutions-have demonstrated comparable or superior performance to human experts across various healthcare applications, their vulnerability to perturbations and stability to variations due to new environments-essentially, their robustness-remains ambiguous and often overlooked. In this review, we aimed to identify the types of robustness addressed in the literature for ML models in healthcare. A total of 274 eligible records were retrieved from PubMed, Web of Science, IEEE Xplore, and additional sources. Eight general concepts of robustness emerged. Furthermore, an analysis of those concepts across types of data and types of predictive models revealed that the concepts were differently addressed. Our findings offer valuable insights for stakeholders seeking to understand and navigate the robustness of machine learning models during their development, validation, and deployment in healthcare settings, where interpretation of robustness may vary.

Deep Learning With Optical Coherence Tomography for Melanoma Identification and Risk Prediction

Malignant melanoma is the most severe skin cancer with a rising incidence rate. Several noninvasive image techniques and computer-aided diagnosis systems have been developed to help find melanoma in its early stages. However, most previous research utilized dermoscopic images to build a diagnosis model, and only a few used prospective datasets. This study develops and evaluates a convolutional neural network (CNN) for melanoma identification and risk prediction using optical coherence tomography (OCT) imaging of mice skin. Longitudinal tests are performed on four animal models: melanoma mice, dysplastic nevus mice, and their respective controls. The CNN classifies melanoma and healthy tissues with high sensitivity (0.99) and specificity (0.98) and also assigns a risk score to each image based on the probability of melanoma presence, which may facilitate early diagnosis and management of melanoma in clinical settings.

Artificial intelligence in medicine and the negative outcome penalty paradox

Artificial intelligence (AI) holds considerable promise for transforming clinical diagnostics. While much has been written both about public attitudes toward the use of AI tools in medicine and about uncertainty regarding legal liability that may be delaying its adoption, the interface of these two issues has so far drawn less attention. However, understanding this interface is essential to determining how jury behaviour is likely to influence adoption of AI by physicians. One distinctive concern identified in this paper is a 'negative outcome penalty paradox' (NOPP) in which physicians risk being penalised by juries in cases with negative outcomes, whether they overrule AI determinations or accept them. The paper notes three reasons why AI in medicine is uniquely susceptible to the NOPP and urges serious further consideration of this complex dilemma.

Artificial Intelligence for Diagnosis in Otologic Patients: Is It Ready to Be Your Doctor?

OBJECTIVE

Investigate the precision of language-model artificial intelligence (AI) in diagnosing conditions by contrasting its predictions with diagnoses made by board-certified otologic/neurotologic surgeons using patient-described symptoms.

STUDY DESIGN

Prospective cohort study.

SETTING

Tertiary care center.

PATIENTS

One hundred adults participated in the study. These included new patients or established patients returning with new symptoms. Individuals were excluded if they could not provide a written description of their symptoms.

INTERVENTIONS

Summaries of the patient's symptoms were supplied to three publicly available AI platforms: Chat GPT 4.0, Google Bard, and WebMD "Symptom Checker."

MAIN OUTCOME MEASURES

This study evaluates the accuracy of three distinct AI platforms in diagnosing otologic conditions by comparing AI results with the diagnosis determined by a neurotologist with the same information provided to the AI platforms and again after a complete history and physical examination.

RESULTS

The study includes 100 patients (52 men and 48 women; average age of 59.2 yr). Fleiss' kappa between AI and the physician is -0.103 (p < 0.01). The chi-squared test between AI and the physician is χ2 = 12.95 (df = 2; p < 0.001). Fleiss' kappa between AI models is 0.409. Diagnostic accuracies are 22.45, 12.24, and 5.10% for ChatGPT 4.0, Google Bard, and WebMD, respectively.

CONCLUSIONS

Contemporary language-model AI platforms can generate extensive differential diagnoses with limited data input. However, doctors can refine these diagnoses through focused history-taking, physical examinations, and clinical experience-skills that current AI platforms lack.

Digital health technologies: Compounding the existing ethical challenges of the 'right' not to know

INTRODUCTION

Doctors hold a prima facie duty to respect the autonomy of their patients. This manifests as the patient's 'right' not to know when patients wish to remain unaware of medical information regarding their health, and poses ethical challenges for good medical practice. This paper explores how the emergence of digital health technologies might impact upon the patient's 'right' not to know.

METHOD

The capabilities of digital health technologies are surveyed and ethical implications of their effects on the 'right' not to know are explored.

FINDINGS

Digital health technologies are increasingly collecting, processing and presenting medical data as clinically useful information, which simultaneously presents large opportunities for improved health outcomes while compounding the existing ethical challenges generated by the patient's 'right' not to know.

CONCLUSION

These digital tools should be designed to include functionality that mitigates these ethical challenges, and allows the preservation of their user's autonomy with regard to the medical information they wish to learn and not learn about.

Deep Learning Techniques for the Dermoscopic Differential Diagnosis of Benign/Malignant Melanocytic Skin Lesions: From the Past to the Present

There has been growing scientific interest in the research field of deep learning techniques applied to skin cancer diagnosis in the last decade. Though encouraging data have been globally reported, several discrepancies have been observed in terms of study methodology, result presentations and validation in clinical settings. The present review aimed to screen the scientific literature on the application of DL techniques to dermoscopic melanoma/nevi differential diagnosis and extrapolate those original studies adequately by reporting on a DL model, comparing them among clinicians and/or another DL architecture. The second aim was to examine those studies together according to a standard set of statistical measures, and the third was to provide dermatologists with a comprehensive explanation and definition of the most used artificial intelligence (AI) terms to better/further understand the scientific literature on this topic and, in parallel, to be updated on the newest applications in the medical dermatologic field, along with a historical perspective. After screening nearly 2000 records, a subset of 54 was selected. Comparing the 20 studies reporting on convolutional neural network (CNN)/deep convolutional neural network (DCNN) models, we have a scenario of highly performant DL algorithms, especially in terms of low false positive results, with average values of accuracy (83.99%), sensitivity (77.74%), and specificity (80.61%). Looking at the comparison with diagnoses by clinicians (13 studies), the main difference relies on the specificity values, with a +15.63% increase for the CNN/DCNN models (average specificity of 84.87%) compared to humans (average specificity of 64.24%) with a 14,85% gap in average accuracy; the sensitivity values were comparable (79.77% for DL and 79.78% for humans). To obtain higher diagnostic accuracy and feasibility in clinical practice, rather than in experimental retrospective settings, future DL models should be based on a large dataset integrating dermoscopic images with relevant clinical and anamnestic data that is prospectively tested and adequately compared with physicians.

Fusion between an Algorithm Based on the Characterization of Melanocytic Lesions' Asymmetry with an Ensemble of Convolutional Neural Networks for Melanoma Detection

Melanoma is still a major health problem worldwide. Early diagnosis is the first step toward reducing its mortality, but it remains a challenge even for experienced dermatologists. Although computer-aided systems have been developed to help diagnosis, the lack of insight into their predictions is still a significant limitation toward acceptance by the medical community. To tackle this issue, we designed handcrafted expert features representing color asymmetry within the lesions, which are parts of the approach used by dermatologists in their daily practice. These features are given to an artificial neural network classifying between nevi and melanoma. We compare our results with an ensemble of 7 state-of-the-art convolutional neural networks and merge the 2 approaches by computing the average prediction. Our experiments are done on a subset of the International Skin Imaging Collaboration 2019 dataset (6296 nevi, 1361 melanomas). The artificial neural network based on asymmetry achieved an area under the curve of 0.873, sensitivity of 90%, and specificity of 67%; the convolutional neural network approach achieved an area under the curve of 0.938, sensitivity of 91%, and specificity of 82%; and the fusion of both approaches achieved an area under the curve of 0.942, sensitivity of 92%, and specificity of 82%. Merging the knowledge of dermatologists with convolutional neural networks showed high performance for melanoma detection, encouraging collaboration between computer science and medical fields.

Dermoscopy Differential Diagnosis Explorer (D3X) Ontology to Aggregate and Link Dermoscopic Patterns to Differential Diagnoses: Development and Usability Study

BACKGROUND

Dermoscopy is a growing field that uses microscopy to allow dermatologists and primary care physicians to identify skin lesions. For a given skin lesion, a wide variety of differential diagnoses exist, which may be challenging for inexperienced users to name and understand.

OBJECTIVE

In this study, we describe the creation of the dermoscopy differential diagnosis explorer (D3X), an ontology linking dermoscopic patterns to differential diagnoses.

METHODS

Existing ontologies that were incorporated into D3X include the elements of visuals ontology and dermoscopy elements of visuals ontology, which connect visual features to dermoscopic patterns. A list of differential diagnoses for each pattern was generated from the literature and in consultation with domain experts. Open-source images were incorporated from DermNet, Dermoscopedia, and open-access research papers.

RESULTS

D3X was encoded in the OWL 2 web ontology language and includes 3041 logical axioms, 1519 classes, 103 object properties, and 20 data properties. We compared D3X with publicly available ontologies in the dermatology domain using a semiotic theory-driven metric to measure the innate qualities of D3X with others. The results indicate that D3X is adequately comparable with other ontologies of the dermatology domain.

CONCLUSIONS

The D3X ontology is a resource that can link and integrate dermoscopic differential diagnoses and supplementary information with existing ontology-based resources. Future directions include developing a web application based on D3X for dermoscopy education and clinical practice.

Malignant Melanoma: An Overview, New Perspectives, and Vitamin D Signaling

Melanoma, originating through malignant transformation of melanin-producing melanocytes, is a formidable malignancy, characterized by local invasiveness, recurrence, early metastasis, resistance to therapy, and a high mortality rate. This review discusses etiologic and risk factors for melanoma, diagnostic and prognostic tools, including recent advances in molecular biology, omics, and bioinformatics, and provides an overview of its therapy. Since the incidence of melanoma is rising and mortality remains unacceptably high, we discuss its inherent properties, including melanogenesis, that make this disease resilient to treatment and propose to use AI to solve the above complex and multidimensional problems. We provide an overview on vitamin D and its anticancerogenic properties, and report recent advances in this field that can provide solutions for the prevention and/or therapy of melanoma. Experimental papers and clinicopathological studies on the role of vitamin D status and signaling pathways initiated by its active metabolites in melanoma prognosis and therapy are reviewed. We conclude that vitamin D signaling, defined by specific nuclear receptors and selective activation by specific vitamin D hydroxyderivatives, can provide a benefit for new or existing therapeutic approaches. We propose to target vitamin D signaling with the use of computational biology and AI tools to provide a solution to the melanoma problem.

The Sociodemographic Biases in Machine Learning Algorithms: A Biomedical Informatics Perspective

Artificial intelligence models represented in machine learning algorithms are promising tools for risk assessment used to guide clinical and other health care decisions. Machine learning algorithms, however, may house biases that propagate stereotypes, inequities, and discrimination that contribute to socioeconomic health care disparities. The biases include those related to some sociodemographic characteristics such as race, ethnicity, gender, age, insurance, and socioeconomic status from the use of erroneous electronic health record data. Additionally, there is concern that training data and algorithmic biases in large language models pose potential drawbacks. These biases affect the lives and livelihoods of a significant percentage of the population in the United States and globally. The social and economic consequences of the associated backlash cannot be underestimated. Here, we outline some of the sociodemographic, training data, and algorithmic biases that undermine sound health care risk assessment and medical decision-making that should be addressed in the health care system. We present a perspective and overview of these biases by gender, race, ethnicity, age, historically marginalized communities, algorithmic bias, biased evaluations, implicit bias, selection/sampling bias, socioeconomic status biases, biased data distributions, cultural biases and insurance status bias, conformation bias, information bias and anchoring biases and make recommendations to improve large language model training data, including de-biasing techniques such as counterfactual role-reversed sentences during knowledge distillation, fine-tuning, prefix attachment at training time, the use of toxicity classifiers, retrieval augmented generation and algorithmic modification to mitigate the biases moving forward.

A systematic review and meta-analysis of artificial intelligence versus clinicians for skin cancer diagnosis

Scientific research of artificial intelligence (AI) in dermatology has increased exponentially. The objective of this study was to perform a systematic review and meta-analysis to evaluate the performance of AI algorithms for skin cancer classification in comparison to clinicians with different levels of expertise. Based on PRISMA guidelines, 3 electronic databases (PubMed, Embase, and Cochrane Library) were screened for relevant articles up to August 2022. The quality of the studies was assessed using QUADAS-2. A meta-analysis of sensitivity and specificity was performed for the accuracy of AI and clinicians. Fifty-three studies were included in the systematic review, and 19 met the inclusion criteria for the meta-analysis. Considering all studies and all subgroups of clinicians, we found a sensitivity (Sn) and specificity (Sp) of 87.0% and 77.1% for AI algorithms, respectively, and a Sn of 79.78% and Sp of 73.6% for all clinicians (overall); differences were statistically significant for both Sn and Sp. The difference between AI performance (Sn 92.5%, Sp 66.5%) vs. generalists (Sn 64.6%, Sp 72.8%), was greater, when compared with expert clinicians. Performance between AI algorithms (Sn 86.3%, Sp 78.4%) vs expert dermatologists (Sn 84.2%, Sp 74.4%) was clinically comparable. Limitations of AI algorithms in clinical practice should be considered, and future studies should focus on real-world settings, and towards AI-assistance.

Examining labelling guidelines for AI-based software as a medical device: A review and analysis of dermatology mobile applications in Australia

In recent years, there has been a surge in the development of AI-based Software as a Medical Device (SaMD), particularly in visual specialties such as dermatology. In Australia, the Therapeutic Goods Administration (TGA) regulates AI-based SaMD to ensure its safe use. Proper labelling of these devices is crucial to ensure that healthcare professionals and the general public understand how to use them and interpret results accurately. However, guidelines for labelling AI-based SaMD in dermatology are lacking, which may result in products failing to provide essential information about algorithm development and performance metrics. This review examines existing labelling guidelines for AI-based SaMD across visual medical specialties, with a specific focus on dermatology. Common recommendations for labelling are identified and applied to currently available dermatology AI-based SaMD mobile applications to determine usage of these labels. Of the 21 AI-based SaMD mobile applications identified, none fully comply with common labelling recommendations. Results highlight the need for standardized labelling guidelines. Ensuring transparency and accessibility of information is essential for the safe integration of AI into health care and preventing potential risks associated with inaccurate clinical decisions.

Transformer-based framework for multi-class segmentation of skin cancer from histopathology images

Introduction

Non-melanoma skin cancer comprising Basal cell carcinoma (BCC), Squamous cell carcinoma (SCC), and Intraepidermal carcinoma (IEC) has the highest incidence rate among skin cancers. Intelligent decision support systems may address the issue of the limited number of subject experts and help in mitigating the parity of health services between urban centers and remote areas.

Method

In this research, we propose a transformer-based model for the segmentation of histopathology images not only into inflammation and cancers such as BCC, SCC, and IEC but also to identify skin tissues and boundaries that are important in decision-making. Accurate segmentation of these tissue types will eventually lead to accurate detection and classification of non-melanoma skin cancer. The segmentation according to tissue types and their visual representation before classification enhances the trust of pathologists and doctors being relatable to how most pathologists approach this problem. The visualization of the confidence of the model in its prediction through uncertainty maps is also what distinguishes this study from most deep learning methods.

Results

The evaluation of proposed system is carried out using publicly available dataset. The application of our proposed segmentation system demonstrated good performance with an F1 score of 0.908, mean intersection over union (mIoU) of 0.653, and average accuracy of 83.1%, advocating that the system can be used as a decision support system successfully and has the potential of subsequently maturing into a fully automated system.

Discussion

This study is an attempt to automate the segmentation of the most occurring non-melanoma skin cancer using a transformer-based deep learning technique applied to histopathology skin images. Highly accurate segmentation and visual representation of histopathology images according to tissue types by the proposed system implies that the system can be used for skin-related routine pathology tasks including cancer and other anomaly detection, their classification, and measurement of surgical margins in the case of cancer cases.

Computer vision model for the detection of canine pododermatitis and neoplasia of the paw

BACKGROUND

Artificial intelligence (AI) has been used successfully in human dermatology. AI utilises convolutional neural networks (CNN) to accomplish tasks such as image classification, object detection and segmentation, facilitating early diagnosis. Computer vision (CV), a field of AI, has shown great results in detecting signs of human skin diseases. Canine paw skin diseases are a common problem in general veterinary practice, and computer vision tools could facilitate the detection and monitoring of disease processes. Currently, no such tool is available in veterinary dermatology.

ANIMALS

Digital images of paws from healthy dogs and paws with pododermatitis or neoplasia were used.

OBJECTIVES

We tested the novel object detection model Pawgnosis, a Tiny YOLOv4 image analysis model deployed on a microcomputer with a camera for the rapid detection of canine pododermatitis and neoplasia.

MATERIALS AND METHODS

The prediction performance metrics used to evaluate the models included mean average precision (mAP), precision, recall, average precision (AP) for accuracy and frames per second (FPS) for speed.

RESULTS

A large dataset labelled by a single individual (Dataset A) used to train a Tiny YOLOv4 model provided the best results with a mean mAP of 0.95, precision of 0.86, recall of 0.93 and 20 FPS.

CONCLUSIONS AND CLINICAL RELEVANCE

This novel object detection model has the potential for application in the field of veterinary dermatology.

AI-driven decision support systems and epistemic reliance: a qualitative study on obstetricians' and midwives' perspectives on integrating AI-driven CTG into clinical decision making

BACKGROUND

Given that AI-driven decision support systems (AI-DSS) are intended to assist in medical decision making, it is essential that clinicians are willing to incorporate AI-DSS into their practice. This study takes as a case study the use of AI-driven cardiotography (CTG), a type of AI-DSS, in the context of intrapartum care. Focusing on the perspectives of obstetricians and midwives regarding the ethical and trust-related issues of incorporating AI-driven tools in their practice, this paper explores the conditions that AI-driven CTG must fulfill for clinicians to feel justified in incorporating this assistive technology into their decision-making processes regarding interventions in labor.

METHODS

This study is based on semi-structured interviews conducted online with eight obstetricians and five midwives based in England. Participants were asked about their current decision-making processes about when to intervene in labor, how AI-driven CTG might enhance or disrupt this process, and what it would take for them to trust this kind of technology. Interviews were transcribed verbatim and analyzed with thematic analysis. NVivo software was used to organize thematic codes that recurred in interviews to identify the issues that mattered most to participants. Topics and themes that were repeated across interviews were identified to form the basis of the analysis and conclusions of this paper.

RESULTS

There were four major themes that emerged from our interviews with obstetricians and midwives regarding the conditions that AI-driven CTG must fulfill: (1) the importance of accurate and efficient risk assessments; (2) the capacity for personalization and individualized medicine; (3) the lack of significance regarding the type of institution that develops technology; and (4) the need for transparency in the development process.

CONCLUSIONS

Accuracy, efficiency, personalization abilities, transparency, and clear evidence that it can improve outcomes are conditions that clinicians deem necessary for AI-DSS to meet in order to be considered reliable and therefore worthy of being incorporated into the decision-making process. Importantly, healthcare professionals considered themselves as the epistemic authorities in the clinical context and the bearers of responsibility for delivering appropriate care. Therefore, what mattered to them was being able to evaluate the reliability of AI-DSS on their own terms, and have confidence in implementing them in their practice.

Artificial Intelligence Algorithms for Benign vs. Malignant Dermoscopic Skin Lesion Image Classification

In recent decades, the incidence of melanoma has grown rapidly. Hence, early diagnosis is crucial to improving clinical outcomes. Here, we propose and compare a classical image analysis-based machine learning method with a deep learning one to automatically classify benign vs. malignant dermoscopic skin lesion images. The same dataset of 25,122 publicly available dermoscopic images was used to train both models, while a disjointed test set of 200 images was used for the evaluation phase. The training dataset was randomly divided into 10 datasets of 19,932 images to obtain an equal distribution between the two classes. By testing both models on the disjoint set, the deep learning-based method returned accuracy of 85.4 ± 3.2% and specificity of 75.5 ± 7.6%, while the machine learning one showed accuracy and specificity of 73.8 ± 1.1% and 44.5 ± 4.7%, respectively. Although both approaches performed well in the validation phase, the convolutional neural network outperformed the ensemble boosted tree classifier on the disjoint test set, showing better generalization ability. The integration of new melanoma detection algorithms with digital dermoscopic devices could enable a faster screening of the population, improve patient management, and achieve better survival rates.

General Overview of Artificial Intelligence for Interstitial Cystitis in Urology

Our understanding of interstitial cystitis/bladder pain syndrome (IC/BPS) has evolved over time. The diagnosis of IC/BPS is primarily based on symptoms such as urgency, frequency, and bladder or pelvic pain. While the exact causes of IC/BPS remain unclear, it is thought to involve several factors, including abnormalities in the bladder's urothelium, mast cell degranulation within the bladder, inflammation of the bladder, and altered innervation of the bladder. Treatment options include patient education, dietary and lifestyle modifications, medications, intravesical therapy, and surgical interventions. This review article provides insights into IC/BPS, including aspects of treatment, prognosis prediction, and emerging therapeutic options. Additionally, it explores the application of deep learning for diagnosing major diseases associated with IC/BPS.

Application possibilities of artificial intelligence in facial vascularized composite allotransplantation-a narrative review

Facial vascularized composite allotransplantation (FVCA) is an emerging field of reconstructive surgery that represents a dogmatic shift in the surgical treatment of patients with severe facial disfigurements. While conventional reconstructive strategies were previously considered the goldstandard for patients with devastating facial trauma, FVCA has demonstrated promising short- and long-term outcomes. Yet, there remain several obstacles that complicate the integration of FVCA procedures into the standard workflow for facial trauma patients. Artificial intelligence (AI) has been shown to provide targeted and resource-effective solutions for persisting clinical challenges in various specialties. However, there is a paucity of studies elucidating the combination of FVCA and AI to overcome such hurdles. Here, we delineate the application possibilities of AI in the field of FVCA and discuss the use of AI technology for FVCA outcome simulation, diagnosis and prediction of rejection episodes, and malignancy screening. This line of research may serve as a fundament for future studies linking these two revolutionary biotechnologies.

Refactoring and performance analysis of the main CNN architectures: using false negative rate minimization to solve the clinical images melanoma detection problem

BACKGROUND

Melanoma is one of the deadliest tumors in the world. Early detection is critical for first-line therapy in this tumor pathology and it remains challenging due to the need for histological analysis to ensure correctness in diagnosis. Therefore, multiple computer-aided diagnosis (CAD) systems working on melanoma images were proposed to mitigate the need of a biopsy. However, although the high global accuracy is declared in literature results, the CAD systems for the health fields must focus on the lowest false negative rate (FNR) possible to qualify as a diagnosis support system. The final goal must be to avoid classification type 2 errors to prevent life-threatening situations. Another goal could be to create an easy-to-use system for both physicians and patients.

RESULTS

To achieve the minimization of type 2 error, we performed a wide exploratory analysis of the principal convolutional neural network (CNN) architectures published for the multiple image classification problem; we adapted these networks to the melanoma clinical image binary classification problem (MCIBCP). We collected and analyzed performance data to identify the best CNN architecture, in terms of FNR, usable for solving the MCIBCP problem. Then, to provide a starting point for an easy-to-use CAD system, we used a clinical image dataset (MED-NODE) because clinical images are easier to access: they can be taken by a smartphone or other hand-size devices. Despite the lower resolution than dermoscopic images, the results in the literature would suggest that it would be possible to achieve high classification performance by using clinical images. In this work, we used MED-NODE, which consists of 170 clinical images (70 images of melanoma and 100 images of naevi). We optimized the following CNNs for the MCIBCP problem: Alexnet, DenseNet, GoogleNet Inception V3, GoogleNet, MobileNet, ShuffleNet, SqueezeNet, and VGG16.

CONCLUSIONS

The results suggest that a CNN built on the VGG or AlexNet structure can ensure the lowest FNR (0.07) and (0.13), respectively. In both cases, discrete global performance is ensured: 73% (accuracy), 82% (sensitivity) and 59% (specificity) for VGG; 89% (accuracy), 87% (sensitivity) and 90% (specificity) for AlexNet.

MDF-Net for abnormality detection by fusing X-rays with clinical data

This study investigates the effects of including patients' clinical information on the performance of deep learning (DL) classifiers for disease location in chest X-ray images. Although current classifiers achieve high performance using chest X-ray images alone, consultations with practicing radiologists indicate that clinical data is highly informative and essential for interpreting medical images and making proper diagnoses. In this work, we propose a novel architecture consisting of two fusion methods that enable the model to simultaneously process patients' clinical data (structured data) and chest X-rays (image data). Since these data modalities are in different dimensional spaces, we propose a spatial arrangement strategy, spatialization, to facilitate the multimodal learning process in a Mask R-CNN model. We performed an extensive experimental evaluation using MIMIC-Eye, a dataset comprising different modalities: MIMIC-CXR (chest X-ray images), MIMIC IV-ED (patients' clinical data), and REFLACX (annotations of disease locations in chest X-rays). Results show that incorporating patients' clinical data in a DL model together with the proposed fusion methods improves the disease localization in chest X-rays by 12% in terms of Average Precision compared to a standard Mask R-CNN using chest X-rays alone. Further ablation studies also emphasize the importance of multimodal DL architectures and the incorporation of patients' clinical data in disease localization. In the interest of fostering scientific reproducibility, the architecture proposed within this investigation has been made publicly accessible( https://github.com/ChihchengHsieh/multimodal-abnormalities-detection ).

Analysis of Artificial Intelligence-Based Approaches Applied to Non-Invasive Imaging for Early Detection of Melanoma: A Systematic Review

BACKGROUND

Melanoma, the deadliest form of skin cancer, poses a significant public health challenge worldwide. Early detection is crucial for improved patient outcomes. Non-invasive skin imaging techniques allow for improved diagnostic accuracy; however, their use is often limited due to the need for skilled practitioners trained to interpret images in a standardized fashion. Recent innovations in artificial intelligence (AI)-based techniques for skin lesion image interpretation show potential for the use of AI in the early detection of melanoma.

OBJECTIVE

The aim of this study was to evaluate the current state of AI-based techniques used in combination with non-invasive diagnostic imaging modalities including reflectance confocal microscopy (RCM), optical coherence tomography (OCT), and dermoscopy. We also aimed to determine whether the application of AI-based techniques can lead to improved diagnostic accuracy of melanoma.

METHODS

A systematic search was conducted via the Medline/PubMed, Cochrane, and Embase databases for eligible publications between 2018 and 2022. Screening methods adhered to the 2020 version of the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. Included studies utilized AI-based algorithms for melanoma detection and directly addressed the review objectives.

RESULTS

We retrieved 40 papers amongst the three databases. All studies directly comparing the performance of AI-based techniques with dermatologists reported the superior or equivalent performance of AI-based techniques in improving the detection of melanoma. In studies directly comparing algorithm performance on dermoscopy images to dermatologists, AI-based algorithms achieved a higher ROC (>80%) in the detection of melanoma. In these comparative studies using dermoscopic images, the mean algorithm sensitivity was 83.01% and the mean algorithm specificity was 85.58%. Studies evaluating machine learning in conjunction with OCT boasted accuracy of 95%, while studies evaluating RCM reported a mean accuracy rate of 82.72%.

CONCLUSIONS

Our results demonstrate the robust potential of AI-based techniques to improve diagnostic accuracy and patient outcomes through the early identification of melanoma. Further studies are needed to assess the generalizability of these AI-based techniques across different populations and skin types, improve standardization in image processing, and further compare the performance of AI-based techniques with board-certified dermatologists to evaluate clinical applicability.

Molecular Classifiers in Skin Cancers: Challenges and Promises

Skin cancers are common and heterogenous malignancies affecting up to two in three Australians before age 70. Despite recent developments in diagnosis and therapeutic strategies, the mortality rate and costs associated with managing patients with skin cancers remain high. The lack of well-defined clinical and histopathological features makes their diagnosis and classification difficult in some cases and the prognostication difficult in most skin cancers. Recent advancements in large-scale "omics" studies, including genomics, transcriptomics, proteomics, metabolomics and imaging-omics, have provided invaluable information about the molecular and visual landscape of skin cancers. On many occasions, it has refined tumor classification and has improved prognostication and therapeutic stratification, leading to improved patient outcomes. Therefore, this paper reviews the recent advancements in omics approaches and appraises their limitations and potential for better classification and stratification of skin cancers.

Implementation of a dermatoscopy curriculum during residency at Augsburg University Hospital in Germany

BACKGROUND AND OBJECTIVES

To date, there is no structured program for dermatoscopy training during residency in Germany. Whether and how much dermatoscopy training is acquired is left to the initiative of each resident, although dermatoscopy is one of the core competencies of dermatological training and daily practice. The aim of the study was to establish a structured dermatoscopy curriculum during residency at the University Hospital Augsburg.

PATIENTS AND METHODS

An online platform with dermatoscopy modules was created, accessible regardless of time and place. Practical skills were acquired under the personal guidance of a dermatoscopy expert. Participants were tested on their level of knowledge before and after completing the modules. Test scores on management decisions and correct dermatoscopic diagnosis were analyzed.

RESULTS

Results of 28 participants showed improvements in management decisions from pre- to posttest (74.0% vs. 89.4%) and in dermatoscopic accuracy (65.0% vs. 85.6%). Pre- vs. posttest differences in test score (7.05/10 vs. 8.94/10 points) and correct diagnosis were significant (p < 0.001).

CONCLUSIONS

The dermatoscopy curriculum increases the number of correct management decisions and dermatoscopy diagnoses. This will result in more skin cancers being detected, and fewer benign lesions being excised. The curriculum can be offered to other dermatology training centers and medical professionals.

Implementierung eines Dermatoskopie-Curriculums in der Facharztausbildung am Universitätsklinikum Augsburg

Hintergrund und Ziele

Bislang gibt es in Deutschland kein strukturiertes Programm für die Dermatoskopieausbildung während der Facharztausbildung. Es bleibt der Initiative des einzelnen Assistenzarztes überlassen, ob und in welchem Umfang er sich in der Dermatoskopie weiterbildet, obwohl die Dermatoskopie zu den Kernkompetenzen der dermatologischen Ausbildung und der täglichen Praxis gehört. Ziel der Studie war die Etablierung eines strukturierten Dermatoskopie‐Curriculums während der dermatologischen Facharztausbildung am Universitätsklinikum Augsburg.

Patienten und Methodik

Es wurde eine Online‐Plattform mit Dermatoskopie‐Modulen geschaffen, auf die von überall und jederzeit zugegriffen werden kann. Praktische Fertigkeiten wurden unter individueller Anleitung eines Dermatoskopie‐Experten erworben. Die Teilnehmer wurden vor und nach Abschluss der Module auf ihren Wissensstand getestet. Die Testergebnisse zum therapeutischen Management und zur korrekten dermatoskopischen Diagnose wurden analysiert.

Ergebnisse

Die Ergebnisse der 28 Teilnehmer verbesserten sich vom Eingangs‐ zum Abschlusstest bei der Managemententscheidung (74,0% vs. 89,4%) und bei der dermatoskopischen Genauigkeit (65,0% vs. 85,6%). Die Unterschiede zwischen Eingangs‐ und Abschlusstest bei der Gesamtpunktzahl (7,05/10 vs. 8,94/10 Punkte) und bei der richtigen Diagnose waren signifikant (p < 0,001).

Schlussfolgerungen

Das Dermatoskopie‐Curriculum verbessert die Managemententscheidungen und die dermatoskopische Diagnostik der Teilnehmer. Das wird dazu führen, dass mehr Hautkrebsfälle erkannt werden und weniger gutartige Läsionen reseziert werden müssen. Das Curriculum kann anderen dermatologischen Ausbildungszentren und Gesundheitsberufen angeboten werden.

Investigation and benchmarking of U-Nets on prostate segmentation tasks

In healthcare, a growing number of physicians and support staff are striving to facilitate personalized radiotherapy regimens for patients with prostate cancer. This is because individual patient biology is unique, and employing a single approach for all is inefficient. A crucial step for customizing radiotherapy planning and gaining fundamental information about the disease, is the identification and delineation of targeted structures. However, accurate biomedical image segmentation is time-consuming, requires considerable experience and is prone to observer variability. In the past decade, the use of deep learning models has significantly increased in the field of medical image segmentation. At present, a vast number of anatomical structures can be demarcated on a clinician's level with deep learning models. These models would not only unload work, but they can offer unbiased characterization of the disease. The main architectures used in segmentation are the U-Net and its variants, that exhibit outstanding performances. However, reproducing results or directly comparing methods is often limited by closed source of data and the large heterogeneity among medical images. With this in mind, our intention is to provide a reliable source for assessing deep learning models. As an example, we chose the challenging task of delineating the prostate gland in multi-modal images. First, this paper provides a comprehensive review of current state-of-the-art convolutional neural networks for 3D prostate segmentation. Second, utilizing public and in-house CT and MR datasets of varying properties, we created a framework for an objective comparison of automatic prostate segmentation algorithms. The framework was used for rigorous evaluations of the models, highlighting their strengths and weaknesses.

Recent Advances in Melanoma Diagnosis and Prognosis Using Machine Learning Methods

PURPOSE OF REVIEW

The purpose was to summarize the current role and state of artificial intelligence and machine learning in the diagnosis and management of melanoma.

RECENT FINDINGS

Deep learning algorithms can identify melanoma from clinical, dermoscopic, and whole slide pathology images with increasing accuracy. Efforts to provide more granular annotation to datasets and to identify new predictors are ongoing. There have been many incremental advances in both melanoma diagnostics and prognostic tools using artificial intelligence and machine learning. Higher quality input data will further improve these models' capabilities.

Intravesical Bladder Treatment and Deep Learning Applications to Improve Irritative Voiding Symptoms Caused by Interstitial Cystitis: A Literature Review

Our comprehension of interstitial cystitis/painful bladder syndrome (IC/PBS) has evolved over time. The term painful bladder syndrome, preferred by the International Continence Society, is characterized as "a syndrome marked by suprapubic pain during bladder filling, alongside increased daytime and nighttime frequency, in the absence of any proven urinary infection or other pathology." The diagnosis of IC/PBS primarily relies on symptoms of urgency/frequency and bladder/pelvic pain. The exact pathogenesis of IC/PBS remains a mystery, but it is postulated to be multifactorial. Theories range from bladder urothelial abnormalities, mast cell degranulation in the bladder, bladder inflammation, to altered bladder innervation. Therapeutic strategies encompass patient education, dietary and lifestyle modifications, medication, intravesical therapy, and surgical intervention. This article delves into the diagnosis, treatment, and prognosis prediction of IC/PBS, presenting the latest research findings, artificial intelligence technology applications in diagnosing major diseases in IC/PBS, and emerging treatment alternatives.

Melanoma Clinical Decision Support System: An Artificial Intelligence-Based Tool to Diagnose and Predict Disease Outcome in Early-Stage Melanoma Patients

This study set out to assess the performance of an artificial intelligence (AI) algorithm based on clinical data and dermatoscopic imaging for the early diagnosis of melanoma, and its capacity to define the metastatic progression of melanoma through serological and histopathological biomarkers, enabling dermatologists to make more informed decisions about patient management. Integrated analysis of demographic data, images of the skin lesions, and serum and histopathological markers were analyzed in a group of 196 patients with melanoma. The interleukins (ILs) IL-4, IL-6, IL-10, and IL-17A as well as IFNγ (interferon), GM-CSF (granulocyte and macrophage colony-stimulating factor), TGFβ (transforming growth factor), and the protein DCD (dermcidin) were quantified in the serum of melanoma patients at the time of diagnosis, and the expression of the RKIP, PIRIN, BCL2, BCL3, MITF, and ANXA5 proteins was detected by immunohistochemistry (IHC) in melanoma biopsies. An AI algorithm was used to improve the early diagnosis of melanoma and to predict the risk of metastasis and of disease-free survival. Two models were obtained to predict metastasis (including "all patients" or only patients "at early stages of melanoma"), and a series of attributes were seen to predict the progression of metastasis: Breslow thickness, infiltrating BCL-2 expressing lymphocytes, and IL-4 and IL-6 serum levels. Importantly, a decrease in serum GM-CSF seems to be a marker of poor prognosis in patients with early-stage melanomas.

Determining the clinical applicability of machine learning models through assessment of reporting across skin phototypes and rarer skin cancer types: A systematic review

Machine learning (ML) models for skin cancer recognition may have variable performance across different skin phototypes and skin cancer types. Overall performance metrics alone are insufficient to detect poor subgroup performance. We aimed (1) to assess whether studies of ML models reported results separately for different skin phototypes and rarer skin cancers, and (2) to graphically represent the skin cancer training datasets used by current ML models. In this systematic review, we searched PubMed, Embase and CENTRAL. We included all studies in medical journals assessing an ML technique for skin cancer diagnosis that used clinical or dermoscopic images from 1 January 2012 to 22 September 2021. No language restrictions were applied. We considered rarer skin cancers to be skin cancers other than pigmented melanoma, basal cell carcinoma and squamous cell carcinoma. We identified 114 studies for inclusion. Rarer skin cancers were included by 8/114 studies (7.0%), and results for a rarer skin cancer were reported separately in 1/114 studies (0.9%). Performance was reported across all skin phototypes in 1/114 studies (0.9%), but performance was uncertain in skin phototypes I and VI from minimal representation of the skin phototypes in the test dataset (9/3756 and 1/3756, respectively). For training datasets, although public datasets were most frequently used, with the most widely used being the International Skin Imaging Collaboration (ISIC) archive (65/114 studies, 57.0%), the largest datasets were private. Our review identified that most ML models did not report performance separately for rarer skin cancers and different skin phototypes. A degree of variability in ML model performance across subgroups is expected, but the current lack of transparency is not justifiable and risks models being used inappropriately in populations in whom accuracy is low.

Deep Learning Models for Cystoscopic Recognition of Hunner Lesion in Interstitial Cystitis

Background

Accurate cystoscopic recognition of Hunner lesions (HLs) is indispensable for better treatment prognosis in managing patients with Hunner-type interstitial cystitis (HIC), but frequently challenging due to its varying appearance.

Objective

To develop a deep learning (DL) system for cystoscopic recognition of a HL using artificial intelligence (AI).

Design setting and participants

A total of 626 cystoscopic images collected from January 8, 2019 to December 24, 2020, consisting of 360 images of HLs from 41 patients with HIC and 266 images of flat reddish mucosal lesions resembling HLs from 41 control patients including those with bladder cancer and other chronic cystitis, were used to create a dataset with an 8:2 ratio of training images and test images for transfer learning and external validation, respectively. AI-based five DL models were constructed, using a pretrained convolutional neural network model that was retrained to output 1 for a HL and 0 for control. A five-fold cross-validation method was applied for internal validation.

Outcome measurements and statistical analysis

True- and false-positive rates were plotted as a receiver operating curve when the threshold changed from 0 to 1. Accuracy, sensitivity, and specificity were evaluated at a threshold of 0.5. Diagnostic performance of the models was compared with that of urologists as a reader study.

Results and limitations

The mean area under the curve of the models reached 0.919, with mean sensitivity of 81.9% and specificity of 85.2% in the test dataset. In the reader study, the mean accuracy, sensitivity, and specificity were, respectively, 83.0%, 80.4%, and 85.6% for the models, and 62.4%, 79.6%, and 45.2% for expert urologists. Limitations include the diagnostic nature of a HL as warranted assertibility.

Conclusions

We constructed the first DL system that recognizes HLs with accuracy exceeding that of humans. This AI-driven system assists physicians with proper cystoscopic recognition of a HL.

Patient summary

In this diagnostic study, we developed a deep learning system for cystoscopic recognition of Hunner lesions in patients with interstitial cystitis. The mean area under the curve of the constructed system reached 0.919 with mean sensitivity of 81.9% and specificity of 85.2%, demonstrating diagnostic accuracy exceeding that of human expert urologists in detecting Hunner lesions. This deep learning system assists physicians with proper diagnosis of a Hunner lesion.

A Deep Learning Approach for Diabetic Foot Ulcer Classification and Recognition

Diabetic foot ulcer (DFU) is one of the major complications of diabetes and results in the amputation of lower limb if not treated timely and properly. Despite the traditional clinical approaches used in DFU classification, automatic methods based on a deep learning framework show promising results. In this paper, we present several end-to-end CNN-based deep learning architectures, i.e., AlexNet, VGG16/19, GoogLeNet, ResNet50.101, MobileNet, SqueezeNet, and DenseNet, for infection and ischemia categorization using the benchmark dataset DFU2020. We fine-tune the weight to overcome a lack of data and reduce the computational cost. Affine transform techniques are used for the augmentation of input data. The results indicate that the ResNet50 achieves the highest accuracy of 99.49% and 84.76% for Ischaemia and infection, respectively.

Artificial intelligence in the detection of skin cancer

Recent advances in artificial intelligence (AI) in dermatology have demonstrated the potential to improve the accuracy of skin cancer detection. These capabilities may augment current diagnostic processes and improve the approach to the management of skin cancer. To explain this technology, we discuss fundamental terminology, potential benefits, and limitations of AI, and commercial applications relevant to dermatologists. A clear understanding of the technology may help to reduce physician concerns about AI and promote its use in the clinical setting. Ultimately, the development and validation of AI technologies, their approval by regulatory agencies, and widespread adoption by dermatologists and other clinicians may enhance patient care. Technology-augmented detection of skin cancer has the potential to improve quality of life, reduce health care costs by reducing unnecessary procedures, and promote greater access to high-quality skin assessment. Dermatologists play a critical role in the responsible development and deployment of AI capabilities applied to skin cancer.

Computer Aided Diagnosis of Melanoma Using Deep Neural Networks and Game Theory: Application on Dermoscopic Images of Skin Lesions

Early detection of melanoma remains a daily challenge due to the increasing number of cases and the lack of dermatologists. Thus, AI-assisted diagnosis is considered as a possible solution for this issue. Despite the great advances brought by deep learning and especially convolutional neural networks (CNNs), computer-aided diagnosis (CAD) systems are still not used in clinical practice. This may be explained by the dermatologist's fear of being misled by a false negative and the assimilation of CNNs to a "black box", making their decision process difficult to understand by a non-expert. Decision theory, especially game theory, is a potential solution as it focuses on identifying the best decision option that maximizes the decision-maker's expected utility. This study presents a new framework for automated melanoma diagnosis. Pursuing the goal of improving the performance of existing systems, our approach also attempts to bring more transparency in the decision process. The proposed framework includes a multi-class CNN and six binary CNNs assimilated to players. The players' strategies is to first cluster the pigmented lesions (melanoma, nevus, and benign keratosis), using the introduced method of evaluating the confidence of the predictions, into confidence level (confident, medium, uncertain). Then, a subset of players has the strategy to refine the diagnosis for difficult lesions with medium and uncertain prediction. We used EfficientNetB5 as the backbone of our networks and evaluated our approach on the public ISIC dataset consisting of 8917 lesions: melanoma (1113), nevi (6705) and benign keratosis (1099). The proposed framework achieved an area under the receiver operating curve (AUROC) of 0.93 for melanoma, 0.96 for nevus and 0.97 for benign keratosis. Furthermore, our approach outperformed existing methods in this task, improving the balanced accuracy (BACC) of the best compared method from 77% to 86%. These results suggest that our framework provides an effective and explainable decision-making strategy. This approach could help dermatologists in their clinical practice for patients with atypical and difficult-to-diagnose pigmented lesions. We also believe that our system could serve as a didactic tool for less experienced dermatologists.

Prediction of micro-hardness in thread rolling of St37 by convolutional neural networks and transfer learning

This study introduces a non-destructive method by applying convolutional neural networks (CNN) to predict the micro-hardness of the thread-rolled steel. Material microstructure images were collected for our research, and micro-hardness tests were conducted to label the extracted microstructure images. In recent years, researchers have used machine learning (ML) and deep learning (DL) models to predict material properties for forming, machining, additive manufacturing, and other processes. However, they encountered industrial limitations primarily because of the absence of historical information on new and unknown materials, which are necessary to predict material properties by DL models. These problems can be solved by employing CNN models. In our work, we used a CNN model with two convolutional layers and visual geometry group (VGG19) as transfer learning (TL). We predicted four classes of micro-hardness of the St37 rolled threads. The prediction results of the micro-hardness test images by our proposed CNN model and pre-trained VGG19 model are comparable. Our proposed model has produced the same precision and recall scores as VGG19 for class B and class C hardness. VGG19 performed slightly better than our model for precision in class A and recall in class D. We observed that the training time of our proposed model using the CPU (central processing unit) was approximately nine times faster than the VGG19 model. Our proposed CNN and VGG19 have direct applications in advanced manufacturing (AM). They can automatically predict the micro-hardness in the thread rolling of St37. Our proposed model requires less memory and computational power and can be deployed more efficiently than the VGG19 model.

SUPPLEMENTARY INFORMATION

The online version contains supplementary material available at 10.1007/s00170-022-10355-4.

Efficacy of Digestive Endoscope Based on Artificial Intelligence System in Diagnosing Early Esophageal Carcinoma

OBJECTIVE

To explore the efficacy of digestive endoscopy (DEN) based on artificial intelligence (AI) system in diagnosing early esophageal carcinoma.

METHODS

The clinical data of 300 patients with suspected esophageal carcinoma treated in our hospital from January 2018 to January 2020 were retrospectively analyzed; among them, 198 were diagnosed with esophageal carcinoma after pathological examination, and 102 had benign esophageal lesion. An AI system based on convolutional neural network (CNN) was adopted to assess the DEN images of patients with early esophageal carcinoma. A total of 200 patients (148 with early esophageal carcinoma and 52 with benign esophageal lesion) were selected as the learning group for the Inception V3 image classification system to learn; and the rest 100 patients (50 with early esophageal carcinoma and 50 with benign esophageal lesion) were included in the diagnosis group for the Inception V3 system to assist the narrow-band imaging (NBI) with diagnosis. The diagnosis results from Inception V3-assisted NBI were compared with those from imaging physicians, and the diagnostic efficacy diagram was drawn.

RESULTS

The diagnosis rate of AI-NBI was significantly faster than that of physician diagnosis (0.02 ± 0.01 vs. 5.65 ± 0.32 s (mean rate of two physicians), P < 0.001); between AI-NBI diagnosis and physician diagnosis, no statistical differences in sensitivity (90.0% vs. 92.0%), specificity (92.0% vs. 94.0%), and accuracy (91.0% vs. 93.0%) were observed (P > 0.05); and according to the ROC curves, AUC (95% CI) of AI-NBI diagnosis = 0.910 (0.845-0.975), and AUC (95% CI) of physician diagnosis = 0.930 (0.872-0.988).

CONCLUSION

CNN-based AI system can assist NBI in screening early esophageal carcinoma, which has a good application prospect in the clinical diagnosis of early esophageal carcinoma.

The Spectrum of Spitz Melanocytic Lesions: From Morphologic Diagnosis to Molecular Classification

Spitz tumors represent a distinct subtype of melanocytic lesions with characteristic histopathologic features, some of which are overlapping with melanoma. More common in the pediatric and younger population, they can be clinically suspected by recognizing specific patterns on dermatoscopic examination, and several subtypes have been described. We now classify these lesions into benign Spitz nevi, intermediate lesions identified as “atypical Spitz tumors” (or Spitz melanocytoma) and malignant Spitz melanoma. More recently a large body of work has uncovered the molecular underpinning of Spitz tumors, including mutations in the HRAS gene and several gene fusions involving several protein kinases. Here we present an overarching view of our current knowledge and understanding of Spitz tumors, detailing clinical, histopathological and molecular features characteristic of these lesions.