AI-based detection of erythema migrans and disambiguation against other skin lesions

Expert opinion elicitation for assisting deep learning based Lyme disease classifier with patient data

BACKGROUND

Diagnosing erythema migrans (EM) skin lesion, the most common early symptom of Lyme disease, using deep learning techniques can be effective to prevent long-term complications. Existing works on deep learning based EM recognition only utilizes lesion image due to the lack of a dataset of Lyme disease related images with associated patient data. Doctors rely on patient information about the background of the skin lesion to confirm their diagnosis. To assist deep learning model with a probability score calculated from patient data, this study elicited opinions from fifteen expert doctors. To the best of our knowledge, this is the first expert elicitation work to calculate Lyme disease probability from patient data.

METHODS

For the elicitation process, a questionnaire with questions and possible answers related to EM was prepared. Doctors provided relative weights to different answers to the questions. We converted doctors' evaluations to probability scores using Gaussian mixture based density estimation. We exploited formal concept analysis and decision tree for elicited model validation and explanation. We also proposed an algorithm for combining independent probability estimates from multiple modalities, such as merging the EM probability score from a deep learning image classifier with the elicited score from patient data.

RESULTS

We successfully elicited opinions from fifteen expert doctors to create a model for obtaining EM probability scores from patient data.

CONCLUSIONS

The elicited probability score and the proposed algorithm can be utilized to make image based deep learning Lyme disease pre-scanners robust. The proposed elicitation and validation process is easy for doctors to follow and can help address related medical diagnosis problems where it is challenging to collect patient data.

Medical Gaslighting and Lyme Disease: The Patient Experience

Even though there are approximately half a million new cases of Lyme disease in the US annually, according to the CDC, it is often undiagnosed or misdiagnosed, which can result in a chronic, multisystemic condition. Lyme disease is a recognized public health threat and is a designated "notifiable disease". As such, Lyme disease is mandated to be reported by the CDC. Despite this, both acute and chronic Lyme disease (CLD) have been relegated to the category of "contested illnesses", which can lead to medical gaslighting. By analyzing results from an online survey of respondents with Lyme disease (n = 986), we elucidate the lived experiences of people who have been pushed to the margins of the medical system by having their symptoms attributed to mental illness, anxiety, stress, and aging. Further, respondents have had their blood tests and erythema migrans (EM) rashes discounted and were told that CLD simply does not exist. As a result, a series of fruitless consultations often result in the delay of a correct diagnosis, which has deleterious consequences. This is the first study that addresses an extensive range of gaslighting techniques experienced by this patient population.

Lyme rashes disease classification using deep feature fusion technique

Automatic classification of Lyme disease rashes on the skin helps clinicians and dermatologists' probe and investigate Lyme skin rashes effectively. This paper proposes a new in-depth features fusion system to classify Lyme disease rashes. The proposed method consists of two main steps. First, three different deep learning models, Densenet201, InceptionV3, and Exception, were trained independently to extract the deep features from the erythema migrans (EM) images. Second, a deep feature fusion mechanism (meta classifier) is developed to integrate the deep features before the final classification output layer. The meta classifier is a basic deep convolutional neural network trained on original images and features extracted from base level three deep learning models. In the feature fusion mechanism, the last three layers of base models are dropped out and connected to the meta classifier. The proposed deep feature fusion method significantly improved the classification process, where the classification accuracy was 98.97%, which is particularly impressive than the other state-of-the-art models.

Lyme borreliosis diagnosis: state of the art of improvements and innovations

With almost 700 000 estimated cases each year in the United States and Europe, Lyme borreliosis (LB), also called Lyme disease, is the most common tick-borne illness in the world. Transmitted by ticks of the genus Ixodes and caused by bacteria Borrelia burgdorferi sensu lato, LB occurs with various symptoms, such as erythema migrans, which is characteristic, whereas others involve blurred clinical features such as fatigue, headaches, arthralgia, and myalgia. The diagnosis of Lyme borreliosis, based on a standard two-tiered serology, is the subject of many debates and controversies, since it relies on an indirect approach which suffers from a low sensitivity depending on the stage of the disease. Above all, early detection of the disease raises some issues. Inappropriate diagnosis of Lyme borreliosis leads to therapeutic wandering, inducing potential chronic infection with a strong antibody response that fails to clear the infection. Early and proper detection of Lyme disease is essential to propose an adequate treatment to patients and avoid the persistence of the pathogen. This review presents the available tests, with an emphasis on the improvements of the current diagnosis, the innovative methods and ideas which, ultimately, will allow more precise detection of LB.

The Impact of Telemedicine in the Diagnosis of Erythema Migrans during the COVID Pandemic: A Comparison with In-Person Diagnosis in the Pre-COVID Era

Background: Erythema migrans (EM) is the hallmark manifestation of the Lyme borreliosis (LB), and therefore its presence and recognition are sufficient to make a diagnosis and to start proper antibiotic treatment to attempt to eradicate the infection. Methods: In this study we compared the clinical data of 439 patients who presented an EM either according to the diagnostic modality through physical assessment or through telemedicine. Conclusions: Our data clearly show that telemedicine for EM diagnosis is useful as it enables prompt administration of appropriate antibiotic therapy, which is critical to avoid complications, especially for neurologic and articular entities. Therefore, telemedicine is a tool that could be adopted for the diagnosis of Lyme disease both by specialized centers but also by general practitioners.

An fMRI Sequence Representation Learning Framework for Attention Deficit Hyperactivity Disorder Classification

For attention deficit hyperactivity disorder (ADHD), a common neurological disease, accurate identification is the basis for treatment. In this paper, a novel end-to-end representation learning framework for ADHD classification of functional magnetic resonance imaging (fMRI) sequences is proposed. With such a framework, the complexity of the sequence representation learning neural network decreases, the overfitting problem of deep learning for small samples cases is solved effectively, and superior classification performance is achieved. Specifically, a data conversion module was designed to convert a two-dimensional sequence into a three-dimensional image, which expands the modeling area and greatly reduces the computational complexity. The transfer learning method was utilized to freeze or fine-tune the parameters of the pre-trained neural network to reduce the risk of overfitting in the cases with small samples. Hierarchical feature extraction can be performed automatically by combining the sequence representation learning modules with a weighted cross-entropy loss. Experiments were conducted both with individual imaging sites and combining them, and the results showed that the classification average accuracies with the proposed framework were 73.73% and 72.02%, respectively, which are much higher than those of the existing methods.

Exploring convolutional neural networks with transfer learning for diagnosing Lyme disease from skin lesion images

BACKGROUND AND OBJECTIVE

Lyme disease which is one of the most common infectious vector-borne diseases manifests itself in most cases with erythema migrans (EM) skin lesions. Recent studies show that convolutional neural networks (CNNs) perform well to identify skin lesions from images. Lightweight CNN based pre-scanner applications for resource-constrained mobile devices can help users with early diagnosis of Lyme disease and prevent the transition to a severe late form thanks to appropriate antibiotic therapy. Also, resource-intensive CNN based robust computer applications can assist non-expert practitioners with an accurate diagnosis. The main objective of this study is to extensively analyze the effectiveness of CNNs for diagnosing Lyme disease from images and to find out the best CNN architectures considering resource constraints.

METHODS

First, we created an EM dataset with the help of expert dermatologists from Clermont-Ferrand University Hospital Center of France. Second, we benchmarked this dataset for twenty-three CNN architectures customized from VGG, ResNet, DenseNet, MobileNet, Xception, NASNet, and EfficientNet architectures in terms of predictive performance, computational complexity, and statistical significance. Third, to improve the performance of the CNNs, we used custom transfer learning from ImageNet pre-trained models as well as pre-trained the CNNs with the skin lesion dataset HAM10000. Fourth, for model explainability, we utilized Gradient-weighted Class Activation Mapping to visualize the regions of input that are significant to the CNNs for making predictions. Fifth, we provided guidelines for model selection based on predictive performance and computational complexity.

RESULTS

Customized ResNet50 architecture gave the best classification accuracy of 84.42% ±1.36, AUC of 0.9189±0.0115, precision of 83.1%±2.49, sensitivity of 87.93%±1.47, and specificity of 80.65%±3.59. A lightweight model customized from EfficientNetB0 also performed well with an accuracy of 83.13%±1.2, AUC of 0.9094±0.0129, precision of 82.83%±1.75, sensitivity of 85.21% ±3.91, and specificity of 80.89%±2.95. All the trained models are publicly available at https://dappem.limos.fr/download.html, which can be used by others for transfer learning and building pre-scanners for Lyme disease.

CONCLUSION

Our study confirmed the effectiveness of even some lightweight CNNs for building Lyme disease pre-scanner mobile applications to assist people with an initial self-assessment and referring them to expert dermatologist for further diagnosis.

TARA: Training and Representation Alteration for AI Fairness and Domain Generalization

We propose a novel method for enforcing AI fairness with respect to protected or sensitive factors. This method uses a dual strategy performing training and representation alteration (TARA) for the mitigation of prominent causes of AI bias. It includes the use of representation learning alteration via adversarial independence to suppress the bias-inducing dependence of the data representation from protected factors and training set alteration via intelligent augmentation to address bias-causing data imbalance by using generative models that allow the fine control of sensitive factors related to underrepresented populations via domain adaptation and latent space manipulation. When testing our methods on image analytics, experiments demonstrate that TARA significantly or fully debiases baseline models while outperforming competing debiasing methods that have the same amount of information-for example, with (% overall accuracy, % accuracy gap) = (78.8, 0.5) versus the baseline method's score of (71.8, 10.5) for Eye-PACS, and (73.7, 11.8) versus (69.1, 21.7) for CelebA. Furthermore, recognizing certain limitations in current metrics used for assessing debiasing performance, we propose novel conjunctive debiasing metrics. Our experiments also demonstrate the ability of these novel metrics in assessing the Pareto efficiency of the proposed methods.

Identification of public submitted tick images: A neural network approach

Ticks and tick-borne diseases represent a growing public health threat in North America and Europe. The number of ticks, their geographical distribution, and the incidence of tick-borne diseases, like Lyme disease, are all on the rise. Accurate, real-time tick-image identification through a smartphone app or similar platform could help mitigate this threat by informing users of the risks associated with encountered ticks and by providing researchers and public health agencies with additional data on tick activity and geographic range. Here we outline the requirements for such a system, present a model that meets those requirements, and discuss remaining challenges and frontiers in automated tick identification. We compiled a user-generated dataset of more than 12,000 images of the three most common tick species found on humans in the U.S.: Amblyomma americanum, Dermacentor variabilis, and Ixodes scapularis. We used image augmentation to further increase the size of our dataset to more than 90,000 images. Here we report the development and validation of a convolutional neural network which we call "TickIDNet," that scores an 87.8% identification accuracy across all three species, outperforming the accuracy of identifications done by a member of the general public or healthcare professionals. However, the model fails to match the performance of experts with formal entomological training. We find that image quality, particularly the size of the tick in the image (measured in pixels), plays a significant role in the network's ability to correctly identify an image: images where the tick is small are less likely to be correctly identified because of the small object detection problem in deep learning. TickIDNet's performance can be increased by using confidence thresholds to introduce an "unsure" class and building image submission pipelines that encourage better quality photos. Our findings suggest that deep learning represents a promising frontier for tick identification that should be further explored and deployed as part of the toolkit for addressing the public health consequences of tick-borne diseases.

The mathematics of erythema: Development of machine learning models for artificial intelligence assisted measurement and severity scoring of radiation induced dermatitis

Although significant advancements in computer-aided diagnostics using artificial intelligence (AI) have been made, to date, no viable method for radiation-induced skin reaction (RISR) analysis and classification is available. The objective of this single-center study was to develop machine learning and deep learning approaches using deep convolutional neural networks (CNNs) for automatic classification of RISRs according to the Common Terminology Criteria for Adverse Events (CTCAE) grading system. Scarletred^Ⓡ Vision, a novel and state-of-the-art digital skin imaging method capable of remote monitoring and objective assessment of acute RISRs was used to convert 2D digital skin images using the CIELAB color space and conduct SEV* measurements. A set of different machine learning and deep convolutional neural network-based algorithms has been explored for the automatic classification of RISRs. A total of 2263 distinct images from 209 patients were analyzed for training and testing the machine learning and CNN algorithms. For a 2-class problem of healthy skin (grade 0) versus erythema (grade ≥ 1), all machine learning models produced an accuracy of above 70%, and the sensitivity and specificity of erythema recognition were 67-72% and 72-83%, respectively. The CNN produced a test accuracy of 74%, sensitivity of 66%, and specificity of 83% for predicting healthy and erythema cases. For the severity grade prediction of a 3-class problem (grade 0 versus 1 versus 2), the overall test accuracy was 60-67%, and the sensitivities were 56-82%, 35-59%, and 65-72%, respectively. For estimating the severity grade of each class, the CNN obtained an accuracy of 73%, 66%, and 82%, respectively. Ensemble learning combines several individual predictions to obtain a better generalization performance. Furthermore, we exploited ensemble learning by deploying a CNN model as a meta-learner. The ensemble CNN based on bagging and majority voting shows an accuracy, sensitivity and specificity of 87%, 90%, and 82% for a 2-class problem, respectively. For a 3-class problem, the ensemble CNN shows an overall accuracy of 66%, while for each grade (0, 1, and 2) accuracies were 76%, 69%, and 87%, sensitivities were 70%, 57%, and 71%, and specificities were 78%, 75%, and 95%, respectively. This study is the first to focus on erythema in radiation-dermatitis and produces benchmark results using machine learning models. The outcome of this study validates that the proposed system can act as a pre-screening and decision support tool for oncologists or patients to provide fast, reliable, and efficient assessment of erythema grading.

Recent Progress in Lyme Disease and Remaining Challenges

Lyme disease (also known as Lyme borreliosis) is the most common vector-borne disease in the United States with an estimated 476,000 cases per year. While historically, the long-term impact of Lyme disease on patients has been controversial, mounting evidence supports the idea that a substantial number of patients experience persistent symptoms following treatment. The research community has largely lacked the necessary funding to properly advance the scientific and clinical understanding of the disease, or to develop and evaluate innovative approaches for prevention, diagnosis, and treatment. Given the many outstanding questions raised into the diagnosis, clinical presentation and treatment of Lyme disease, and the underlying molecular mechanisms that trigger persistent disease, there is an urgent need for more support. This review article summarizes progress over the past 5 years in our understanding of Lyme and tick-borne diseases in the United States and highlights remaining challenges.

Adopting low-shot deep learning for the detection of conjunctival melanoma using ocular surface images

BACKGROUND AND OBJECTIVE

The purpose of the present study was to investigate low-shot deep learning models applied to conjunctival melanoma detection using a small dataset with ocular surface images.

METHODS

A dataset was composed of anonymized images of four classes; conjunctival melanoma (136), nevus or melanosis (93), pterygium (75), and normal conjunctiva (94). Before training involving conventional deep learning models, two generative adversarial networks (GANs) were constructed to augment the training dataset for low-shot learning. The collected data were randomly divided into training (70%), validation (10%), and test (20%) datasets. Moreover, 3D melanoma phantoms were designed to build an external validation set using a smartphone. The GoogleNet, InceptionV3, NASNet, ResNet50, and MobileNetV2 architectures were trained through transfer learning and validated using the test and external validation datasets.

RESULTS

The deep learning model demonstrated a significant improvement in the classification accuracy of conjunctival lesions using synthetic images generated by the GAN models. MobileNetV2 with GAN-based augmentation displayed the highest accuracy of 87.5% in the four-class classification and 97.2% in the binary classification for the detection of conjunctival melanoma. It showed an accuracy of 94.0% using 3D melanoma phantom images captured using a smartphone camera.

CONCLUSIONS

The present study described a low-shot deep learning model that can detect conjunctival melanomas using ocular surface images. To the best of our knowledge, this study is the first to develop a deep learning model to detect conjunctival melanoma using a digital imaging device such as smartphone camera.