MSHF: A Multi-Source Heterogeneous Fundus (MSHF) Dataset for Image Quality Assessment

Open ultrawidefield fundus image dataset with disease diagnosis and clinical image quality assessment

Ultrawidefield fundus (UWF) images have a wide imaging range (200° of the retinal region), which offers the opportunity to show more information for ophthalmic diseases. Image quality assessment (IQA) is a prerequisite for applying UWF and is crucial for developing artificial intelligence-driven diagnosis and screening systems. Most image quality systems have been applied to the assessments of natural images, but whether these systems are suitable for evaluating the UWF image quality remains debatable. Additionally, existing IQA datasets only provide photographs of diabetic retinopathy (DR) patients and quality evaluation results applicable for natural image, neglecting patients' clinical information. To address these issues, we established a real-world clinical practice ultra-widefield fundus images dataset, with 700 high-resolution UWF images and corresponding clinical information from six common fundus diseases and healthy volunteers. The image quality is annotated by three ophthalmologists based on the field of view, illumination, artifact, contrast, and overall quality. This dataset illustrates the distribution of UWF image quality across diseases in clinical practice, offering a foundation for developing effective IQA systems.

Enhancing the ophthalmic AI assessment with a fundus image quality classifier using local and global attention mechanisms

Background

The assessment of image quality (IQA) plays a pivotal role in the realm of image-based computer-aided diagnosis techniques, with fundus imaging standing as the primary method for the screening and diagnosis of ophthalmic diseases. Conventional studies on fundus IQA tend to rely on simplistic datasets for evaluation, predominantly focusing on either local or global information, rather than a synthesis of both. Moreover, the interpretability of these studies often lacks compelling evidence. In order to address these issues, this study introduces the Local and Global Attention Aggregated Deep Neural Network (LGAANet), an innovative approach that integrates both local and global information for enhanced analysis.

Methods

The LGAANet was developed and validated using a Multi-Source Heterogeneous Fundus (MSHF) database, encompassing a diverse collection of images. This dataset includes 802 color fundus photography (CFP) images (302 from portable cameras), and 500 ultrawide-field (UWF) images from 904 patients with diabetic retinopathy (DR) and glaucoma, as well as healthy individuals. The assessment of image quality was meticulously carried out by a trio of ophthalmologists, leveraging the human visual system as a benchmark. Furthermore, the model employs attention mechanisms and saliency maps to bolster its interpretability.

Results

In testing with the CFP dataset, LGAANet demonstrated remarkable accuracy in three critical dimensions of image quality (illumination, clarity and contrast based on the characteristics of human visual system, and indicates the potential aspects to improve the image quality), recording scores of 0.947, 0.924, and 0.947, respectively. Similarly, when applied to the UWF dataset, the model achieved accuracies of 0.889, 0.913, and 0.923, respectively. These results underscore the efficacy of LGAANet in distinguishing between varying degrees of image quality with high precision.

Conclusion

To our knowledge, LGAANet represents the inaugural algorithm trained on an MSHF dataset specifically for fundus IQA, marking a significant milestone in the advancement of computer-aided diagnosis in ophthalmology. This research significantly contributes to the field, offering a novel methodology for the assessment and interpretation of fundus images in the detection and diagnosis of ocular diseases.

Multimodal Eye Imaging, Retina Characteristics, and Psychological Assessment Dataset

The eyes provide insights into psychology, potentially offering a distinctive perspective for psychological health profiles. However, there exist a notable deficiency in datasets that simultaneously encompass eye features and psychological assessments. To address this gap, our study presents a dataset that included Fundus Photography, Psychological Assessment, Retina Characteristics, and Multimodal Imaging (FPRM). FPRM dataset comprise fundus images at different wavelengths (548 nm and 605 nm), image of oxygen saturation for the retina and 8 specific retinal vessels, videos of retinal blood flow and pupillary light reflex, along with 61 items of multimodal quantitative measurement from 384 participants. Additionally, it features psychological assessments across five dimensions (geriatric depression, generalized anxiety disorder, insomnia, activities of daily living, and deterioration), accompanied by fundus photographs and 6 items of retina characteristics from 1683 participants. FPRM dataset is the first to integrate multimodal ophthalmic data and psychological assessments, not only advancing the development of machine learning applications but also facilitating in-depth research into the relationship between eye health and psychological health profiles.

A fundus image dataset for intelligent retinopathy of prematurity system

Image-based artificial intelligence (AI) systems stand as the major modality for evaluating ophthalmic conditions. However, most of the currently available AI systems are designed for experimental research using single-central datasets. Most of them fell short of application in real-world clinical settings. In this study, we collected a dataset of 1,099 fundus images in both normal and pathologic eyes from 483 premature infants for intelligent retinopathy of prematurity (ROP) system development and validation. Dataset diversity was visualized with a spatial scatter plot. Image classification was conducted by three annotators. To the best of our knowledge, this is one of the largest fundus datasets on ROP, and we believe it is conducive to the real-world application of AI systems.

Parrot optimizer: Algorithm and applications to medical problems

Stochastic optimization methods have gained significant prominence as effective techniques in contemporary research, addressing complex optimization challenges efficiently. This paper introduces the Parrot Optimizer (PO), an efficient optimization method inspired by key behaviors observed in trained Pyrrhura Molinae parrots. The study features qualitative analysis and comprehensive experiments to showcase the distinct characteristics of the Parrot Optimizer in handling various optimization problems. Performance evaluation involves benchmarking the proposed PO on 35 functions, encompassing classical cases and problems from the IEEE CEC 2022 test sets, and comparing it with eight popular algorithms. The results vividly highlight the competitive advantages of the PO in terms of its exploratory and exploitative traits. Furthermore, parameter sensitivity experiments explore the adaptability of the proposed PO under varying configurations. The developed PO demonstrates effectiveness and superiority when applied to engineering design problems. To further extend the assessment to real-world applications, we included the application of PO to disease diagnosis and medical image segmentation problems, which are highly relevant and significant in the medical field. In conclusion, the findings substantiate that the PO is a promising and competitive algorithm, surpassing some existing algorithms in the literature. The supplementary files and open source codes of the proposed Parrot Optimizer (PO) is available at https://aliasgharheidari.com/PO.html and https://github.com/junbolian/PO.

Elderly and visually impaired indoor activity monitoring based on Wi-Fi and Deep Hybrid convolutional neural network

A drop in physical activity and a deterioration in the capacity to undertake daily life activities are both connected with ageing and have negative effects on physical and mental health. An Elderly and Visually Impaired Human Activity Monitoring (EV-HAM) system that keeps tabs on a person's routine and steps in if a change in behaviour or a crisis might greatly help an elderly person or a visually impaired. These individuals may find greater freedom with the help of an EVHAM system. As the backbone of human-centric applications like actively supported living and in-home monitoring for the elderly and visually impaired, an EVHAM system is essential. Big data-driven product design is flourishing in this age of 5G and the IoT. Recent advancements in processing power and software architectures have also contributed to the emergence and development of artificial intelligence (AI). In this context, the digital twin has emerged as a state-of-the-art technology that bridges the gap between the real and virtual worlds by evaluating data from several sensors using artificial intelligence algorithms. Although promising findings have been reported by Wi-Fi-based human activity identification techniques so far, their effectiveness is vulnerable to environmental variations. Using the environment-independent fingerprints generated from the Wi-Fi channel state information (CSI), we introduce Wi-Sense. This human activity identification system employs a Deep Hybrid convolutional neural network (DHCNN). The proposed system begins by collecting the CSI with a regular Wi-Fi Network Interface Controller. Wi-Sense uses the CSI ratio technique to lessen the effect of noise and the phase offset. The t- Distributed Stochastic Neighbor Embedding (t-SNE) is used to eliminate unnecessary data further. The data dimension is decreased, and the negative effects on the environment are eliminated in this process. The resulting spectrogram of the processed data exposes the activity's micro-Doppler fingerprints as a function of both time and location. These spectrograms are put to use in the training of a DHCNN. Based on our findings, EVHAM can accurately identify these actions 99% of the time.

Biosensor-Based Multimodal Deep Human Locomotion Decoding via Internet of Healthcare Things

Multiple Internet of Healthcare Things (IoHT)-based devices have been utilized as sensing methodologies for human locomotion decoding to aid in applications related to e-healthcare. Different measurement conditions affect the daily routine monitoring, including the sensor type, wearing style, data retrieval method, and processing model. Currently, several models are present in this domain that include a variety of techniques for pre-processing, descriptor extraction, and reduction, along with the classification of data captured from multiple sensors. However, such models consisting of multiple subject-based data using different techniques may degrade the accuracy rate of locomotion decoding. Therefore, this study proposes a deep neural network model that not only applies the state-of-the-art Quaternion-based filtration technique for motion and ambient data along with background subtraction and skeleton modeling for video-based data, but also learns important descriptors from novel graph-based representations and Gaussian Markov random-field mechanisms. Due to the non-linear nature of data, these descriptors are further utilized to extract the codebook via the Gaussian mixture regression model. Furthermore, the codebook is provided to the recurrent neural network to classify the activities for the locomotion-decoding system. We show the validity of the proposed model across two publicly available data sampling strategies, namely, the HWU-USP and LARa datasets. The proposed model is significantly improved over previous systems, as it achieved 82.22% and 82.50% for the HWU-USP and LARa datasets, respectively. The proposed IoHT-based locomotion-decoding model is useful for unobtrusive human activity recognition over extended periods in e-healthcare facilities.

Trends in using deep learning algorithms in biomedical prediction systems

In the domain of using DL-based methods in medical and healthcare prediction systems, the utilization of state-of-the-art deep learning (DL) methodologies assumes paramount significance. DL has attained remarkable achievements across diverse domains, rendering its efficacy particularly noteworthy in this context. The integration of DL with health and medical prediction systems enables real-time analysis of vast and intricate datasets, yielding insights that significantly enhance healthcare outcomes and operational efficiency in the industry. This comprehensive literature review systematically investigates the latest DL solutions for the challenges encountered in medical healthcare, with a specific emphasis on DL applications in the medical domain. By categorizing cutting-edge DL approaches into distinct categories, including convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), long short-term memory (LSTM) models, support vector machine (SVM), and hybrid models, this study delves into their underlying principles, merits, limitations, methodologies, simulation environments, and datasets. Notably, the majority of the scrutinized articles were published in 2022, underscoring the contemporaneous nature of the research. Moreover, this review accentuates the forefront advancements in DL techniques and their practical applications within the realm of medical prediction systems, while simultaneously addressing the challenges that hinder the widespread implementation of DL in image segmentation within the medical healthcare domains. These discerned insights serve as compelling impetuses for future studies aimed at the progressive advancement of using DL-based methods in medical and health prediction systems. The evaluation metrics employed across the reviewed articles encompass a broad spectrum of features, encompassing accuracy, precision, specificity, F-score, adoptability, adaptability, and scalability.

Medical image analysis using deep learning algorithms

In the field of medical image analysis within deep learning (DL), the importance of employing advanced DL techniques cannot be overstated. DL has achieved impressive results in various areas, making it particularly noteworthy for medical image analysis in healthcare. The integration of DL with medical image analysis enables real-time analysis of vast and intricate datasets, yielding insights that significantly enhance healthcare outcomes and operational efficiency in the industry. This extensive review of existing literature conducts a thorough examination of the most recent deep learning (DL) approaches designed to address the difficulties faced in medical healthcare, particularly focusing on the use of deep learning algorithms in medical image analysis. Falling all the investigated papers into five different categories in terms of their techniques, we have assessed them according to some critical parameters. Through a systematic categorization of state-of-the-art DL techniques, such as Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Generative Adversarial Networks (GANs), Long Short-term Memory (LSTM) models, and hybrid models, this study explores their underlying principles, advantages, limitations, methodologies, simulation environments, and datasets. Based on our results, Python was the most frequent programming language used for implementing the proposed methods in the investigated papers. Notably, the majority of the scrutinized papers were published in 2021, underscoring the contemporaneous nature of the research. Moreover, this review accentuates the forefront advancements in DL techniques and their practical applications within the realm of medical image analysis, while simultaneously addressing the challenges that hinder the widespread implementation of DL in image analysis within the medical healthcare domains. These discerned insights serve as compelling impetuses for future studies aimed at the progressive advancement of image analysis in medical healthcare research. The evaluation metrics employed across the reviewed articles encompass a broad spectrum of features, encompassing accuracy, sensitivity, specificity, F-score, robustness, computational complexity, and generalizability.

Facial wrinkle segmentation using weighted deep supervision and semi-automatic labeling

Facial wrinkles are important indicators of human aging. Recently, a method using deep learning and a semi-automatic labeling was proposed to segment facial wrinkles, which showed much better performance than conventional image-processing-based methods. However, the difficulty of wrinkle segmentation remains challenging due to the thinness of wrinkles and their small proportion in the entire image. Therefore, performance improvement in wrinkle segmentation is still necessary. To address this issue, we propose a novel loss function that takes into account the thickness of wrinkles based on the semi-automatic labeling approach. First, considering the different spatial dimensions of the decoder in the U-Net architecture, we generated weighted wrinkle maps from ground truth. These weighted wrinkle maps were used to calculate the training losses more accurately than the existing deep supervision approach. This new loss computation approach is defined as weighted deep supervision in our study. The proposed method was evaluated using an image dataset obtained from a professional skin analysis device and labeled using semi-automatic labeling. In our experiment, the proposed weighted deep supervision showed higher Jaccard Similarity Index (JSI) performance for wrinkle segmentation compared to conventional deep supervision and traditional image processing methods. Additionally, we conducted experiments on the labeling using a semi-automatic labeling approach, which had not been explored in previous research, and compared it with human labeling. The semi-automatic labeling technology showed more consistent wrinkle labels than human-made labels. Furthermore, to assess the scalability of the proposed method to other domains, we applied it to retinal vessel segmentation. The results demonstrated superior performance of the proposed method compared to existing retinal vessel segmentation approaches. In conclusion, the proposed method offers high performance and can be easily applied to various biomedical domains and U-Net-based architectures. Therefore, the proposed approach will be beneficial for various biomedical imaging approaches. To facilitate this, we have made the source code of the proposed method publicly available at: https://github.com/resemin/WeightedDeepSupervision.

Deploying efficient net batch normalizations (BNs) for grading diabetic retinopathy severity levels from fundus images

Diabetic retinopathy (DR) is one of the main causes of blindness in people around the world. Early diagnosis and treatment of DR can be accomplished by organizing large regular screening programs. Still, it is difficult to spot diabetic retinopathy timely because the situation might not indicate signs in the primary stages of the disease. Due to a drastic increase in diabetic patients, there is an urgent need for efficient diabetic retinopathy detecting systems. Auto-encoders, sparse coding, and limited Boltzmann machines were used as a few past deep learning (DL) techniques and features for the classification of DR. Convolutional Neural Networks (CNN) have been identified as a promising solution for detecting and classifying DR. We employ the deep learning capabilities of efficient net batch normalization (BNs) pre-trained models to automatically acquire discriminative features from fundus images. However, we successfully achieved F1 scores above 80% on all efficient net BNs in the EYE-PACS dataset (calculated F1 score for DeepDRiD another dataset) and the results are better than previous studies. In this paper, we improved the accuracy and F1 score of the efficient net BNs pre-trained models on the EYE-PACS dataset by applying a Gaussian Smooth filter and data augmentation transforms. Using our proposed technique, we have achieved F1 scores of 84% and 87% for EYE-PACS and DeepDRiD.

Enhancing Skin Cancer Detection and Classification in Dermoscopic Images through Concatenated MobileNetV2 and Xception Models

One of the most promising research initiatives in the healthcare field is focused on the rising incidence of skin cancer worldwide and improving early discovery methods for the disease. The most significant factor in the fatalities caused by skin cancer is the late identification of the disease. The likelihood of human survival may be significantly improved by performing an early diagnosis followed by appropriate therapy. It is not a simple process to extract the elements from the photographs of the tumors that may be used for the prospective identification of skin cancer. Several deep learning models are widely used to extract efficient features for a skin cancer diagnosis; nevertheless, the literature demonstrates that there is still room for additional improvements in various performance metrics. This study proposes a hybrid deep convolutional neural network architecture for identifying skin cancer by adding two main heuristics. These include Xception and MobileNetV2 models. Data augmentation was introduced to balance the dataset, and the transfer learning technique was utilized to resolve the challenges of the absence of labeled datasets. It has been detected that the suggested method of employing Xception in conjunction with MobileNetV2 attains the most excellent performance, particularly concerning the dataset that was evaluated: specifically, it produced 97.56% accuracy, 97.00% area under the curve, 100% sensitivity, 93.33% precision, 96.55% F1 score, and 0.0370 false favorable rates. This research has implications for clinical practice and public health, offering a valuable tool for dermatologists and healthcare professionals in their fight against skin cancer.