Why ResNet Works? Residuals Generalize

A Language-Guided Progressive Fusion Network with semantic density alignment for Medical Visual Question Answering

Medical Visual Question Answering (Med-VQA) is a critical multimodal task with the potential to address the scarcity and imbalance of medical resources. However, most existing studies overlook the limitations of the inconsistency in information density between medical images and text, as well as the long-tail distribution in datasets, which continue to make Med-VQA an open challenge. To overcome these issues, this study proposes a Language-Guided Progressive Fusion Network (LGPFN) with three key modules: Question-Guided Progressive Multimodal Fusion (QPMF), Language-Gate Mechanism (LGM), and Triple Semantic Feature Alignment (TriSFA). QPMF progressively guides the fusion of visual and textual features using both global and local question representations. LGM, a linguistic rule-based module, distinguishes between Closed-Ended (CE) and Open-Ended (OE) samples, directing the fused features to the appropriate classifiers. Finally, TriSFA captures the rich semantic information of OE answers and mine the underlying associations among fused features, predicted answers, and ground truths, aligning them in a ternary semantic feature space. The proposed LGPFN framework outperforms existing state-of-the-art models, achieving the best overall accuracies of 80.39%, 84.07%, 75.74%, and 70.60% on the VQA-RAD, SLAKE, PathVQA, and VQA-Med 2019 datasets, respectively. These results demonstrate the effectiveness and generalizability of the proposed model, underscoring its potential as a medical Artificial Intelligent (AI) agent that could benefit universal health coverage.

Small parallel residual convolutional neural network and traffic congestion detection

In the development process of modern cities, traffic congestion has become an increasingly severe challenge. Image-based traffic congestion detection can help traffic managers grasp the traffic status in real time and help urban residents avoid congested areas, which is of great significance. Based on the advantages and disadvantage of residual networks, this paper introduces residual units as the basic part of the model. In order to increase the model capacity, a parallel mechanism is introduced. At the same time, in order to reduce the time complexity and space complexity of the algorithm, this paper reduces the scale of large convolutional neural network models and proposes a small parallel residual convolutional neural network (SPRCNN) as an image classification model and applied it to traffic congestion detection. This paper conducts experiments on the Traffic net and CCTRIB datasets, and conducts comparative experiments and spatiotemporal complexity analysis. The results show that the method proposed in this paper is superior to existing large pre-trained models.

Inverse Design of High-Performance Thermoelectric Materials via a Generative Model Combined with Experimental Verification

The emergence of inverse design approaches leveraging generative models offers a promising avenue for thermoelectric material design. However, these models heavily depend on diverse training data, and current thermoelectric data sets are limited, primarily encompassing group IV-VI materials operating within moderate temperature ranges. This constraint poses a significant challenge in the pursuit of materials with high thermoelectric figure of merit (zT) through generative modeling. Our study introduces an inverse design model tailored for the constrained thermoelectric materials data set. By augmenting the data with 2000 entries from the experimental literature and incorporating a generative model featuring a diversity loss function and residual network (ResNet) architecture to enhance complexity, our approach has been trained to systematically generate high-zT thermoelectric materials across various temperature ranges. Under predefined high-zT criteria, our deep generative model successfully predicted 100 doped materials with zT values exceeding 1.0. Furthermore, this research analyzes density of states (DOS) plots for the generated materials, identifying 25 unreported previously potential thermoelectric candidates in the material database. Notably, we experimentally validated the synthesis of Mg3.1Sb0.5Bi1.497Te0.003, a representative thermoelectric material from the Mg3(Sb, Bi)2 family suitable for room temperature applications. This validation underscores the efficacy of our model in exploring and discovering novel thermoelectric materials.

Automated Cerebrovascular Segmentation and Visualization of Intracranial Time-of-Flight Magnetic Resonance Angiography Based on Deep Learning

Time-of-flight magnetic resonance angiography (TOF-MRA) is a non-contrast technique used to visualize neurovascular. However, manual reconstruction of the volume render (VR) by radiologists is time-consuming and labor-intensive. Deep learning-based (DL-based) vessel segmentation technology may provide intelligent automation workflow. To evaluate the image quality of DL vessel segmentation for automatically acquiring intracranial arteries in TOF-MRA. A total of 394 TOF-MRA scans were selected, which included cerebral vascular health, aneurysms, or stenoses. Both our proposed method and two state-of-the-art DL methods are evaluated on external datasets for generalization ability. For qualitative assessment, two experienced clinical radiologists evaluated the image quality of cerebrovascular diagnostic and visualization (scoring 0-5 as unacceptable to excellent) obtained by manual VR reconstruction or automatic convolutional neural network (CNN) segmentation. The proposed CNN outperforms the other two DL-based methods in clinical scoring on external datasets, and its visualization was evaluated by readers as having the appearance of the radiologists' manual reconstructions. Scoring of proposed CNN and VR of intracranial arteries demonstrated good to excellent agreement with no significant differences (median, 5.0 and 5.0, P ≥ 12) at healthy-type scans. All proposed CNN image quality were considered to have adequate diagnostic quality (median scores > 2). Quantitative analysis demonstrated a superior dice similarity coefficient of cerebrovascular overlap (training sets and validation sets; 0.947 and 0.927). Automatic cerebrovascular segmentation using DL is feasible and the image quality in terms of vessel integrity, collateral circulation and lesion morphology is comparable to expert manual VR without significant differences.

Comparative analysis of deep learning architectures for thyroid eye disease detection using facial photographs

PURPOSE

To compare two artificial intelligence (AI) models, residual neural networks ResNet-50 and ResNet-101, for screening thyroid eye disease (TED) using frontal face photographs, and to test these models under clinical conditions.

METHODS

A total of 1601 face photographs were obtained. These photographs were preprocessed by cropping to a region centered around the eyes. For the deep learning process, photographs from 643 TED patients and 643 healthy individuals were used for training the ResNet models. Additionally, 81 photographs of TED patients and 74 of normal subjects were used as the validation dataset. Finally, 80 TED cases and 80 healthy subjects comprised the test dataset. For application tests under clinical conditions, data from 25 TED patients and 25 healthy individuals were utilized to evaluate the non-inferiority of the AI models, with general ophthalmologists and fellowships as the control group.

RESULTS

In the test set verification of the ResNet-50 AI model, the area under the receiver operating characteristic (ROC) curve (AUC), accuracy, sensitivity, and specificity were 0.94, 0.88, 0.64, and 0.92, respectively. For the ResNet-101 AI model, these metrics were 0.93, 0.84, 0.76, and 0.92, respectively. In the application tests under clinical conditions, to evaluate the non-inferiority of the ResNet-50 AI model, the AUC, accuracy, sensitivity, and specificity were 0.82, 0.82, 0.88, and 0.76, respectively. For the ResNet-101 AI model, these metrics were 0.91, 0.84, 0.92, and 0.76, respectively, with no statistically significant differences between the two models for any of the metrics (all p-values > 0.05).

CONCLUSIONS

Face image-based TED screening using ResNet-50 and ResNet-101 AI models shows acceptable accuracy, sensitivity, and specificity for distinguishing TED from healthy subjects.

Diagnostic value of deep learning of multimodal imaging of thyroid for TI-RADS category 3-5 classification

BACKGROUND

Thyroid nodules classified within the Thyroid Imaging Reporting and Data Systems (TI-RADS) category 3-5 are typically regarded as having varying degrees of malignancy risk, with the risk increasing from TI-RADS 3 to TI-RADS 5. While some of these nodules may undergo fine-needle aspiration (FNA) biopsy to assess their nature, this procedure carries a risk of false negatives and inherent complications. To avoid the need for unnecessary biopsy examination, we explored a method for distinguishing the benign and malignant characteristics of thyroid TI-RADS 3-5 nodules based on deep-learning ultrasound images combined with computed tomography (CT).

METHODS

Thyroid nodules, assessed as American College of Radiology (ACR) TI-RADS category 3-5 through conventional ultrasound, all of which had postoperative pathology results, were examined using both conventional ultrasound and CT before operation. We investigated the effectiveness of deep-learning models based on ultrasound alone, CT alone, and a combination of both imaging modalities using the following metrics: Area Under Curve (AUC), sensitivity, accuracy, and positive predictive value (PPV). Additionally, we compared the diagnostic efficacy of the combined methods with manual readings of ultrasound and CT.

RESULTS

A total of 768 thyroid nodules falling within TI-RADS categories 3-5 were identified across 768 patients. The dataset comprised 499 malignant and 269 benign cases. For the automatic identification of thyroid TI-RADS category 3-5 nodules, deep learning combined with ultrasound and CT demonstrated a significantly higher AUC (0.930; 95% CI: 0.892, 0.969) compared to the application of ultrasound alone AUC (0.901; 95% CI: 0.856, 0.947) or CT alone AUC (0.776; 95% CI: 0.713, 0.840). Additionally, the AUC of combined modalities surpassed that of radiologists'assessments using ultrasound alone AUCmean (0.725；95% CI:0.677, 0.773), CT alone AUCmean (0.617; 95% CI:0.564, 0.669). Deep learning method combined with ultrasound and CT imaging of thyroid can allow more accurate and precise classification of nodules within TI-RADS categories 3-5.

Impact of deep learning on pediatric elbow fracture detection: a systematic review and meta-analysis

OBJECTIVES

Pediatric elbow fractures are a common injury among children. Recent advancements in artificial intelligence (AI), particularly deep learning (DL), have shown promise in diagnosing these fractures. This study systematically evaluated the performance of DL models in detecting pediatric elbow fractures.

MATERIALS AND METHODS

A comprehensive search was conducted in PubMed (Medline), EMBASE, and IEEE Xplore for studies published up to October 20, 2023. Studies employing DL models for detecting elbow fractures in patients aged 0 to 16 years were included. Key performance metrics, including sensitivity, specificity, and area under the curve (AUC), were extracted. The study was registered in PROSPERO (ID: CRD42023470558).

RESULTS

The search identified 22 studies, of which six met the inclusion criteria for the meta-analysis. The pooled sensitivity of DL models for pediatric elbow fracture detection was 0.93 (95% CI: 0.91-0.96). Specificity values ranged from 0.84 to 0.92 across studies, with a pooled estimate of 0.89 (95% CI: 0.85-0.92). The AUC ranged from 0.91 to 0.99, with a pooled estimate of 0.95 (95% CI: 0.93-0.97). Further analysis highlighted the impact of preprocessing techniques and the choice of model backbone architecture on performance.

CONCLUSION

DL models demonstrate exceptional accuracy in detecting pediatric elbow fractures. For optimal performance, we recommend leveraging backbone architectures like ResNet, combined with manual preprocessing supervised by radiology and orthopedic experts.

Few-shot learning for non-vitrified ice segmentation

This study introduces Ice Finder, a novel tool for quantifying crystalline ice in cryo-electron tomography, addressing a critical gap in existing methodologies. We present the first application of the meta-learning paradigm to this field, demonstrating that diverse tomographic tasks across datasets can be unified under a single meta-learning framework. By leveraging few-shot learning, our approach enhances domain generalization and adaptability to domain shifts, enabling rapid adaptation to new datasets with minimal examples. Ice Finder's performance is evaluated on a comprehensive set of in situ datasets from EMPIAR, showcasing its ease of use, fast processing capabilities, and millisecond inference times.

Attentive Learning Facilitates Generalization of Neural Networks

This article studies the generalization of neural networks (NNs) by examining how a network changes when trained on a training sample with or without out-of-distribution (OoD) examples. If the network's predictions are less influenced by fitting OoD examples, then the network learns attentively from the clean training set. A new notion, dataset-distraction stability, is proposed to measure the influence. Extensive CIFAR-10/100 experiments on the different VGG, ResNet, WideResNet, ViT architectures, and optimizers show a negative correlation between the dataset-distraction stability and generalizability. With the distraction stability, we decompose the learning process on the training set into multiple learning processes on the subsets of drawn from simpler distributions, i.e., distributions of smaller intrinsic dimensions (IDs), and furthermore, a tighter generalization bound is derived. Through attentive learning, miraculous generalization in deep learning can be explained and novel algorithms can also be designed.

Classification of diabetic retinopathy algorithm based on a novel dual-path multi-module model

Diabetic retinopathy is a chronic disease of the eye that is precipitated via diabetes. As the disease progresses, the blood vessels in the retina are issue to modifications such as dilation, leakage, and new blood vessel formation. Early detection and treatment of the lesions are vital for the prevention and reduction of imaginative and prescient loss. A new dual-path multi-module network algorithm for diabetic retinopathy classification is proposed in this paper, aiming to accurately classify the diabetic retinopathy stage to facilitate early diagnosis and intervention. To obtain the purpose of fact augmentation, the algorithm first enhances retinal lesion features using color correcting and multi-scale fusion algorithms. It then optimizes the local records via a multi-path multiplexing structure with convolutional kernels of exclusive sizes. Finally, a multi-feature fusion module is used to improve the accuracy of the diabetic retinopathy classification model. Two public datasets and a real hospital dataset are used to validate the algorithm. The accuracy is 98.9%, 99.3%, and 98.3%, respectively. The experimental results not only confirm the advancement and practicability of the algorithm in the field of automatic DR diagnosis, but also foretell its broad application prospects in clinical settings, which is expected to provide strong technical support for the early screening and treatment of diabetic retinopathy.

Analysis of Cardiac Arrhythmias Based on ResNet-ICBAM-2DCNN Dual-Channel Feature Fusion

Cardiovascular disease (CVD) poses a significant challenge to global health, with cardiac arrhythmia representing one of its most prevalent manifestations. The timely and precise classification of arrhythmias is critical for the effective management of CVD. This study introduces an innovative approach to enhancing arrhythmia classification accuracy through advanced Electrocardiogram (ECG) signal processing. We propose a dual-channel feature fusion strategy designed to enhance the precision and objectivity of ECG analysis. Initially, we apply an Improved Complete Ensemble Empirical Mode Decomposition with Adaptive Noise (ICEEMDAN) and enhanced wavelet thresholding for robust noise reduction. Subsequently, in the primary channel, region of interest features are emphasized using a ResNet-ICBAM network model for feature extraction. In parallel, the secondary channel transforms 1D ECG signals into Gram angular difference field (GADF), Markov transition field (MTF), and recurrence plot (RP) representations, which are then subjected to two-dimensional convolutional neural network (2D-CNN) feature extraction. Post-extraction, the features from both channels are fused and classified. When evaluated on the MIT-BIH database, our method achieves a classification accuracy of 97.80%. Compared to other methods, our approach of two-channel feature fusion has a significant improvement in overall performance by adding a 2D convolutional network. This methodology represents a substantial advancement in ECG signal processing, offering significant potential for clinical applications and improving patient care efficiency and accuracy.

Advancing semantic segmentation: Enhanced UNet algorithm with attention mechanism and deformable convolution

This paper presents a novel method for improving semantic segmentation performance in computer vision tasks. Our approach utilizes an enhanced UNet architecture that leverages an improved ResNet50 backbone. We replace the last layer of ResNet50 with deformable convolution to enhance feature representation. Additionally, we incorporate an attention mechanism, specifically ECA-ASPP (Attention Spatial Pyramid Pooling), in the encoding path of UNet to capture multi-scale contextual information effectively. In the decoding path of UNet, we explore the use of attention mechanisms after concatenating low-level features with high-level features. Specifically, we investigate two types of attention mechanisms: ECA (Efficient Channel Attention) and LKA (Large Kernel Attention). Our experiments demonstrate that incorporating attention after concatenation improves segmentation accuracy. Furthermore, we compare the performance of ECA and LKA modules in the decoder path. The results indicate that the LKA module outperforms the ECA module. This finding highlights the importance of exploring different attention mechanisms and their impact on segmentation performance. To evaluate the effectiveness of the proposed method, we conduct experiments on benchmark datasets, including Stanford and Cityscapes, as well as the newly introduced WildPASS and DensPASS datasets. Based on our experiments, the proposed method achieved state-of-the-art results including mIoU 85.79 and 82.25 for the Stanford dataset, and the Cityscapes dataset, respectively. The results demonstrate that our proposed method performs well on these datasets, achieving state-of-the-art results with high segmentation accuracy.

Leveraging U-Net and selective feature extraction for land cover classification using remote sensing imagery

In this study, we explore an enhancement to the U-Net architecture by integrating SK-ResNeXt as the encoder for Land Cover Classification (LCC) tasks using Multispectral Imaging (MSI). SK-ResNeXt introduces cardinality and adaptive kernel sizes, allowing U-Net to better capture multi-scale features and adjust more effectively to variations in spatial resolution, thereby enhancing the model's ability to segment complex land cover types. We evaluate this approach using the Five-Billion-Pixels dataset, composed of 150 large-scale RGB-NIR images and over 5 billion labeled pixels across 24 categories. The approach achieves notable improvements over the baseline U-Net, with gains of 5.312% in Overall Accuracy (OA) and 8.906% in mean Intersection over Union (mIoU) when using the RGB configuration. With the RG-NIR configuration, these improvements increase to 6.928% in OA and 6.938% in mIoU, while the RGB-NIR configuration yields gains of 5.854% in OA and 7.794% in mIoU. Furthermore, the approach not only outperforms other well-established models such as DeepLabV3, DeepLabV3+, Ma-Net, SegFormer, and PSPNet, particularly with the RGB-NIR configuration, but also surpasses recent state-of-the-art methods. Visual tests confirmed this superiority, showing that the studied approach achieves notable improvements in certain classes, such as lakes, rivers, industrial areas, residential areas, and vegetation, where the other architectures struggled to achieve accurate segmentation. These results demonstrate the potential and capability of the explored approach to effectively handle MSI and enhance LCC results.

A Robust Initialization of Residual Blocks for Effective ResNet Training Without Batch Normalization

Batch normalization is an essential component of all state-of-the-art neural networks architectures. However, since it introduces many practical issues, much recent research has been devoted to designing normalization-free architectures. In this brief, we show that weights initialization is key to train ResNet-like normalization-free networks. In particular, we propose a slight modification to the summation operation of a block output to the skip-connection branch, so that the whole network is correctly initialized. We show that this modified architecture achieves competitive results on CIFAR-10, CIFAR-100 and ImageNet without further regularization nor algorithmic modifications.

IMPACT: interpretable microbial phenotype analysis via microbial characteristic traits

MOTIVATION

The human gut microbiome, consisting of trillions of bacteria, significantly impacts health and disease. High-throughput profiling through the advancement of modern technology provides the potential to enhance our understanding of the link between the microbiome and complex disease outcomes. However, there remains an open challenge where current microbiome models lack interpretability of microbial features, limiting a deeper understanding of the role of the gut microbiome in disease. To address this, we present a framework that combines a feature engineering step to transform tabular abundance data to image format using functional microbial annotation databases, with a residual spatial attention transformer block architecture for phenotype classification.

RESULTS

Our model, IMPACT, delivers improved predictive accuracy performance across multiclass classification compared to similar methods. More importantly, our approach provides interpretable feature importance through image classification saliency methods. This enables the extraction of taxa markers (features) associated with a disease outcome and also their associated functional microbial traits and metabolites.

AVAILABILITY AND IMPLEMENTATION

IMPACT is available at https://github.com/SydneyBioX/IMPACT. We providedirect installation of IMPACT via pip.

Deep learning-assisted detection of psychoactive water pollutants using behavioral profiling of zebrafish embryos

Water pollution poses a significant risk to the environment and human health, necessitating the development of innovative detection methods. In this study, a series of representative psychoactive compounds were selected as model pollutants, and a new approach combining zebrafish embryo behavioral phenotyping with deep learning was used to identify and classify water pollutants. Zebrafish embryos were exposed to 17 psychoactive compounds at environmentally relevant concentrations (1 and 10 μg/L), and their locomotor behavior was recorded at 5 and 6 days post-fertilization (dpf). We constructed six distinct zebrafish locomotor track datasets encompassing various observation times and developmental stages and evaluated multiple deep learning models on these datasets. The results demonstrated that the ResNet101 model performed optimally on the 1-min track dataset at 6 dpf, achieving an accuracy of 65.35 %. Interpretability analyses revealed that the model effectively focused on the relevant locomotor track features for classification. These findings suggest that the integration of zebrafish embryo behavioral analysis with deep learning can serve as an environmentally friendly and economical method for detecting water pollutants. This approach offers a new perspective for water quality monitoring and has the potential to assist existing chemical analytical techniques in detection, thereby advancing environmental toxicology research and water pollution control efforts.

MRI-based whole-brain elastography and volumetric measurements to predict brain age

Brain age, as a correlate of an individual's chronological age obtained from structural and functional neuroimaging data, enables assessing developmental or neurodegenerative pathology relative to the overall population. Accurately inferring brain age from brain magnetic resonance imaging (MRI) data requires imaging methods sensitive to tissue health and sophisticated statistical models to identify the underlying age-related brain changes. Magnetic resonance elastography (MRE) is a specialized MRI technique which has emerged as a reliable, non-invasive method to measure the brain's mechanical properties, such as the viscoelastic shear stiffness and damping ratio. These mechanical properties have been shown to change across the life span, reflect neurodegenerative diseases, and are associated with individual differences in cognitive function. Here, we aim to develop a machine learning framework to accurately predict a healthy individual's chronological age from maps of brain mechanical properties. This framework can later be applied to understand neurostructural deviations from normal in individuals with neurodevelopmental or neurodegenerative conditions. Using 3D convolutional networks as deep learning models and more traditional statistical models, we relate chronological age as a function of multiple modalities of whole-brain measurements: stiffness, damping ratio, and volume. Evaluations on held-out subjects show that combining stiffness and volume in a multimodal approach achieves the most accurate predictions. Interpretation of the different models highlights important regions that are distinct between the modalities. The results demonstrate the complementary value of MRE measurements in brain age models, which, in future studies, could improve model sensitivity to brain integrity differences in individuals with neuropathology.

Application of social media communication for museum based on the deep mediatization and artificial intelligence

Based on deep mediatization theory and artificial intelligence (AI) technology, this study explores the effective improvement of museums' social media communication by applying Convolutional Neural Network (CNN) technology. Firstly, the social media content from four different museums is collected, a dataset containing tens of thousands of images is constructed, and a CNN-based model is designed for automatic identification and classification of image content. The model is trained and tested through a series of experiments, evaluating its performance in enhancing museums' social media communication. Experimental results indicate that the CNN model significantly enhances user participation, access rates, retention rates, and sharing rates of content. Specifically, user participation increased from 15 to 25%, reflecting a 66.7% rise. Content coverage increased from 20 to 35%, showing a 75% increase. User retention rate rose from 10 to 20%, indicating a 100% increase. Content sharing rate increased from 5 to 15%, reflecting a 200% rise. Additionally, the study discusses the model's performance across various museum types, batch sizes, and learning rate settings, verifying its robustness and wide applicability.

Retrosynthetic analysis via deep learning to improve pilomatricoma diagnoses

BACKGROUND

Pilomatricoma, a benign childhood skin tumor, presents diagnostic challenges due to its manifestation variations and requires surgical excision upon histological confirmation of its characteristic cellular features. Recent artificial intelligence (AI) advancements in pathology promise enhanced diagnostic accuracy and treatment approaches for this neoplasm.

METHODS

We employed a multiscale transfer learning model, initiating the training process at high resolutions and adapting to broader scales. For evaluation purposes, we applied metrics such as accuracy, precision, recall, the F1 score, and the area under the receiver operating characteristic curve (AUROC) to measure the performance of the model, with the statistical significance of the results assessed via two-sided P tests. Our novel approach also included a retrosynthetic saliency mapping technique to achieve enhanced lesion visualization in whole-slide images (WSIs), supporting pathologists' diagnostic processes.

RESULTS

Our model effectively navigated the challenges of global-scale classification, achieving a high validation accuracy of up to 0.973 despite some initial fluctuations. This method displayed excellent accuracy in terms of identifying basaloid and ghost cells, especially at lower scales, with slight variability in its ghost cell accuracy and more noticeable changes in the 'Other' category at higher scales. The consistent performance attained for basaloid cells was clear across all scales, whereas areas for improvement were identified in the 'Other' category. The model also excelled at generating detailed and interpretable saliency maps for lesion visualization purposes, thereby enhancing its value in digital pathology diagnostics.

CONCLUSION

Our pilomatricoma study demonstrates the efficacy of a deep learning-based histopathological diagnosis model, as validated by its high performance across various scales, and it is enhanced by an innovative retrosynthetic approach for saliency mapping.

Automatic quality assessment of knee radiographs using knowledge graphs and convolutional neural networks

BACKGROUND

X-ray radiography is a widely used imaging technique worldwide, and its image quality directly affects diagnostic accuracy. Therefore, X-ray image quality control (QC) is essential. However, subjectively assessing image quality is inefficient and inconsistent, especially when large amounts of image data are being evaluated. Thus, subjective assessment cannot meet current QC needs.

PURPOSE

To meet current QC needs and improve the efficiency of image quality assessment, a complete set of quality assessment criteria must be established and implemented using artificial intelligence (AI) technology. Therefore, we proposed a multi-criteria AI system for automatically assessing the image quality of knee radiographs.

METHODS

A knee radiograph QC knowledge graph containing 16 "acquisition technique" labels representing 16 image quality defects and five "clarity" labels representing five grades of clarity were developed. Ten radiographic technologists conducted three rounds of QC based on this graph. The single-person QC results were denoted as QC1 and QC2, and the multi-person QC results were denoted as QC3. Each technologist labeled each image only once. The ResNet model structure was then used to simultaneously perform classification (detection of image quality defects) and regression (output of a clarity score) tasks to construct an image QC system. The QC3 results, comprising 4324 anteroposterior and lateral knee radiographs, were used for model training (70% of the images), validation (10%), and testing (20%). The 865 test set data were used to evaluate the effectiveness of the AI model, and an AI QC result, QC4, was automatically generated by the model after training. Finally, using a double-blind method, the senior QC expert reviewed the final QC results of the test set with reference to the results QC3 and QC4 and used them as a reference standard to evaluate the performance of the model. The precision and mean absolute error (MAE) were used to evaluate the quality of all the labels in relation to the reference standard.

RESULTS

For the 16 "acquisition technique" features, QC4 exhibited the highest weighted average precision (98.42% ± 0.81%), followed by QC3 (91.39% ± 1.35%), QC2 (87.84% ± 1.68%), and QC1 (87.35% ± 1.71%). For the image clarity features, the MAEs between QC1, QC2, QC3, and QC4 and the reference standard were 0.508 ± 0.021, 0.475 ± 0.019, 0.237 ± 0.016, and 0.303 ± 0.018, respectively.

CONCLUSIONS

The experimental results show that our automated quality assessment system performed well in classifying the acquisition technique used for knee radiographs. The image clarity quality evaluation accuracy of the model must be further improved but is generally close to that of radiographic technologists. Intelligent QC methods using knowledge graphs and convolutional neural networks have the potential for clinical applications.

An Integrated Multi-omics prediction model for stroke recurrence based on Lnet transformer layer and dynamic weighting mechanism

BACKGROUND AND OBJECTIVE

Stroke is a disease with high mortality and disability. Importantly, the fatality rate demonstrates a significant increase among patients afflicted by recurrent strokes compared to those experiencing their initial stroke episode. Currently, the existing research encounters three primary challenges. The first is the lack of a reliable, multi-omics image dataset related to stroke recurrence. The second is how to establish a high-performance feature extraction model and eliminate noise from continuous magnetic resonance imaging (MRI) data. The third is how to integration multi-omics data and dynamically weighted for different omics data.

METHODS

We systematically compiled MRI and conventional detection data from a cohort comprising 737 stroke patients and established PSTSZC, a multi-omics dataset for predicting stroke recurrence. We introduced the first-ever Integrated Multi-omics Prediction Model for Stroke Recurrence, MPSR, which is based on ResNet, Lnet-transformer, LSTM and dynamically weighted DNN. The MPSR model comprises two principal modules, the Feature Extraction Module, and the Integrated Multi-Omics Prediction Module. In the Feature Extraction module, we proposed a novel Lnet regularization layer, which effectively addresses noise issues in MRI data. In the Integrated Multi-omics Prediction Module, we propose a dynamic weighted mechanism based on evaluators, which mitigates the noise impact brought about by low-performance omics.

RESULTS

We compared seven single-omics models and six state-of-the-art multi-omics stroke recurrence models. The experimental results demonstrate that the MPSR model exhibited superior performance. The accuracy, AUROC, specificity, and sensitivity of the MPSR model can reach 0.96, 0.97, 1, and 0.94, respectively, which is higher than the results of contrast model.

CONCLUSION

MPSR is the first available high-performance multi-omics prediction model for stroke recurrence. We assert that the MPSR model holds the potential to function as a valuable tool in assisting clinicians in accurately diagnosing individuals with a predisposition to stroke recurrence.

A novel deep learning model based on transformer and cross modality attention for classification of sleep stages

The classification of sleep stages is crucial for gaining insights into an individual's sleep patterns and identifying potential health issues. Employing several important physiological channels in different views, each providing a distinct perspective on sleep patterns, can have a great impact on the efficiency of the classification models. In the context of neural networks and deep learning models, transformers are very effective, especially when dealing with time series data, and have shown remarkable compatibility with sequential data analysis as physiological channels. On the other hand, cross-modality attention by integrating information from multiple views of the data enables to capture relationships among different modalities, allowing models to selectively focus on relevant information from each modality. In this paper, we introduce a novel deep-learning model based on transformer encoder-decoder and cross-modal attention for sleep stage classification. The proposed model processes information from various physiological channels with different modalities using the Sleep Heart Health Study Dataset (SHHS) data and leverages transformer encoders for feature extraction and cross-modal attention for effective integration to feed into the transformer decoder. The combination of these elements increased the accuracy of the model up to 91.33% in classifying five classes of sleep stages. Empirical evaluations demonstrated the model's superior performance compared to standalone approaches and other state-of-the-art techniques, showcasing the potential of combining transformer and cross-modal attention for improved sleep stage classification.

SALR: Sharpness-Aware Learning Rate Scheduler for Improved Generalization

In an effort to improve generalization in deep learning and automate the process of learning rate scheduling, we propose SALR: a sharpness-aware learning rate update technique designed to recover flat minimizers. Our method dynamically updates the learning rate of gradient-based optimizers based on the local sharpness of the loss function. This allows optimizers to automatically increase learning rates at sharp valleys to increase the chance of escaping them. We demonstrate the effectiveness of SALR when adopted by various algorithms over a broad range of networks. Our experiments indicate that SALR improves generalization, converges faster, and drives solutions to significantly flatter regions.

Differentiation of tuberculous and brucellar spondylitis using conventional MRI-based deep learning algorithms

PURPOSE

To investigate the feasibility of deep learning (DL) based on conventional MRI to differentiate tuberculous spondylitis (TS) from brucellar spondylitis (BS).

METHODS

A total of 383 patients with TS (n = 182) or BS (n = 201) were enrolled from April 2013 to May 2023 and randomly divided into training (n = 307) and validation (n = 76) sets. Sagittal T1WI, T2WI, and fat-suppressed (FS) T2WI images were used to construct single-sequence DL models and combined models based on VGG19, VGG16, ResNet18, and DenseNet121 network. The area under the receiver operating characteristic curve (AUC) was used to assess the classification performance. The AUC of DL models was compared with that of two radiologists with different levels of experience.

RESULTS

The AUCs based on VGG19, ResNet18, VGG16, and DenseNet121 ranged from 0.885 to 0.973, 0.873 to 0.944, 0.882 to 0.929, and 0.801 to 0.933, respectively, and VGG19 models performed better. The diagnostic efficiency of combined models outperformed single-sequence DL models. The combined model of T1WI, T2WI, and FS T2WI based on VGG19 achieved optimal performance, with an AUC of 0.973. In addition, the performance of all combined models based on T1WI, T2WI, and FS T2WI was better than that of two radiologists (P<0.05).

CONCLUSION

The DL models have potential guiding value in the diagnosis of TS and BS based on conventional MRI and provide a certain reference for clinical work.

Unlocking Predictive Capability and Enhancing Sensing Performances of Plasmonic Hydrogen Sensors via Phase Space Reconstruction and Convolutional Neural Networks

This study innovates plasmonic hydrogen sensors (PHSs) by applying phase space reconstruction (PSR) and convolutional neural networks (CNNs), overcoming previous predictive and sensing limitations. Utilizing a low-cost and efficient colloidal lithography technique, palladium nanocap arrays are created and their spectral signals are transformed into images using PSR and then trained using CNNs for predicting the hydrogen level. The model achieves accurate predictions with average accuracies of 0.95 for pure hydrogen and 0.97 for mixed gases. Performance improvements observed are a reduction in response time by up to 3.7 times (average 2.1 times) across pressures, SNR increased by up to 9.3 times (average 3.9 times) across pressures, and LOD decreased from 16 Pa to an extrapolated 3 Pa, a 5.3-fold improvement. A practical application of remote hydrogen sensing without electronics in hydrogen environments is actualized and achieves a 0.98 average test accuracy. This methodology reimagines PHS capabilities, facilitating advancements in hydrogen monitoring technologies and intelligent spectrum-based sensing.

Assessing the Influence of B-US, CDFI, SE, and Patient Age on Predicting Molecular Subtypes in Breast Lesions Using Deep Learning Algorithms

OBJECTIVES

Our study aims to investigate the impact of B-mode ultrasound (B-US) imaging, color Doppler flow imaging (CDFI), strain elastography (SE), and patient age on the prediction of molecular subtypes in breast lesions.

METHODS

Totally 2272 multimodal ultrasound imaging was collected from 198 patients. The ResNet-18 network was employed to predict four molecular subtypes from B-US imaging, CDFI, and SE of patients with different ages. All the images were split into training and testing datasets by the ratio of 80%:20%. The predictive performance on testing dataset was evaluated through 5 metrics including mean accuracy, precision, recall, F1-scores, and confusion matrix.

RESULTS

Based on B-US imaging, the test mean accuracy is 74.50%, the precision is 74.84%, the recall is 72.48%, and the F1-scores is 0.73. By combining B-US imaging with CDFI, the results were increased to 85.41%, 85.03%, 85.05%, and 0.84, respectively. With the integration of B-US imaging and SE, the results were changed to 75.64%, 74.69%, 73.86%, and 0.74, respectively. Using images from patients under 40 years old, the results were 90.48%, 90.88%, 88.47%, and 0.89. When images from patients who are above 40 years old, they were changed to 81.96%, 83.12%, 80.5%, and 0.81, respectively.

CONCLUSION

Multimodal ultrasound imaging can be used to accurately predict the molecular subtypes of breast lesions. In addition to B-US imaging, CDFI rather than SE contribute further to improve predictive performance. The predictive performance is notably better for patients under 40 years old compared with those who are 40 years old and above.

A Parkinson's Auxiliary Diagnosis Algorithm Based on a Hyperparameter Optimization Method of Deep Learning

Parkinson's disease is a common mental disease in the world, especially in the middle-aged and elderly groups. Today, clinical diagnosis is the main diagnostic method of Parkinson's disease, but the diagnosis results are not ideal, especially in the early stage of the disease. In this paper, a Parkinson's auxiliary diagnosis algorithm based on a hyperparameter optimization method of deep learning is proposed for the Parkinson's diagnosis. The diagnosis system uses ResNet50 to achieve feature extraction and Parkinson's classification, mainly including speech signal processing part, algorithm improvement part based on Artificial Bee Colony algorithm (ABC) and optimizing the hyperparameters of ResNet50 part. The improved algorithm is called Gbest Dimension Artificial Bee Colony algorithm (GDABC), proposing "Range pruning strategy" which aims at narrowing the scope of search and "Dimension adjustment strategy" which is to adjust gbest dimension by dimension. The accuracy of the diagnosis system in the verification set of Mobile Device Voice Recordings at King's College London (MDVR-CKL) dataset can reach more than 96%. Compared with current Parkinson's sound diagnosis methods and other optimization algorithms, our auxiliary diagnosis system shows better classification performance on the dataset within limited time and resources.

Improving traffic accident severity prediction using MobileNet transfer learning model and SHAP XAI technique

Traffic accidents remain a leading cause of fatalities, injuries, and significant disruptions on highways. Comprehending the contributing factors to these occurrences is paramount in enhancing safety on road networks. Recent studies have demonstrated the utility of predictive modeling in gaining insights into the factors that precipitate accidents. However, there has been a dearth of focus on explaining the inner workings of complex machine learning and deep learning models and the manner in which various features influence accident prediction models. As a result, there is a risk that these models may be seen as black boxes, and their findings may not be fully trusted by stakeholders. The main objective of this study is to create predictive models using various transfer learning techniques and to provide insights into the most impactful factors using Shapley values. To predict the severity of injuries in accidents, Multilayer Perceptron (MLP), Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), Residual Networks (ResNet), EfficientNetB4, InceptionV3, Extreme Inception (Xception), and MobileNet are employed. Among the models, the MobileNet showed the highest results with 98.17% accuracy. Additionally, by understanding how different features affect accident prediction models, researchers can gain a deeper understanding of the factors that contribute to accidents and develop more effective interventions to prevent them.

ResD-Net: A model for rapid prediction of antioxidant activity in gentian root using FT-IR spectroscopy

Gentian, an herb resource known for its antioxidant properties, has garnered significant attention. However, existing methods are time-consuming and destructive for assessing the antioxidant activity in gentian root samples. In this study, we propose a method for swiftly predicting the antioxidant activity of gentian root using FT-IR spectroscopy combined with chemometrics. We employed machine learning and deep learning models to establish the relationship between FT-IR spectra and DPPH free radical scavenging activity. The results of model fitting reveal that the deep learning model outperforms the machine learning model. The model's performance was enhanced by incorporating the Double-Net and residual connection strategy. The enhanced model, named ResD-Net, excels in feature extraction and also avoids gradient vanishing. The ResD-Net model achieves an R2 of 0.933, an RMSE of 0.02, and an RPD of 3.856. These results support the accuracy and applicability of this method for rapidly predicting antioxidant activity in gentian root samples.

Deep Learning Detection and Segmentation of Facet Joints in Ultrasound Images Based on Convolutional Neural Networks and Enhanced Data Annotation

The facet joint injection is the most common procedure used to release lower back pain. In this paper, we proposed a deep learning method for detecting and segmenting facet joints in ultrasound images based on convolutional neural networks (CNNs) and enhanced data annotation. In the enhanced data annotation, a facet joint was considered as the first target and the ventral complex as the second target to improve the capability of CNNs in recognizing the facet joint. A total of 300 cases of patients undergoing pain treatment were included. The ultrasound images were captured and labeled by two professional anesthesiologists, and then augmented to train a deep learning model based on the Mask Region-based CNN (Mask R-CNN). The performance of the deep learning model was evaluated using the average precision (AP) on the testing sets. The data augmentation and data annotation methods were found to improve the AP. The AP50 for facet joint detection and segmentation was 90.4% and 85.0%, respectively, demonstrating the satisfying performance of the deep learning model. We presented a deep learning method for facet joint detection and segmentation in ultrasound images based on enhanced data annotation and the Mask R-CNN. The feasibility and potential of deep learning techniques in facet joint ultrasound image analysis have been demonstrated.

CorNet: Autonomous feature learning in raw Corvis ST data for keratoconus diagnosis via residual CNN approach

PURPOSE

To ascertain whether the integration of raw Corvis ST data with an end-to-end CNN can enhance the diagnosis of keratoconus (KC).

METHOD

The Corvis ST is a non-contact device for in vivo measurement of corneal biomechanics. The CorNet was trained and validated on a dataset consisting of 1786 Corvis ST raw data from 1112 normal eyes and 674 KC eyes. Each raw data consists of the anterior and posterior corneal surface elevation during air-puff induced dynamic deformation. The architecture of CorNet utilizes four ResNet-inspired convolutional structures that employ 1 × 1 convolution in identity mapping. Gradient-weighted Class Activation Mapping (Grad-CAM) was adopted to visualize the attention allocation to diagnostic areas. Discriminative performance was assessed using metrics including the AUC of ROC curve, sensitivity, specificity, precision, accuracy, and F1 score.

RESULTS

CorNet demonstrated outstanding performance in distinguishing KC from normal eyes, achieving an AUC of 0.971 (sensitivity: 92.49%, specificity: 91.54%) in the validation set, outperforming the best existing Corvis ST parameters, namely the Corvis Biomechanical Index (CBI) with an AUC of 0.947, and its updated version for Chinese populations (cCBI) with an AUC of 0.963. Though the ROC curve analysis showed no significant difference between CorNet and cCBI (p = 0.295), it indicated a notable difference between CorNet and CBI (p = 0.011). The Grad-CAM visualizations highlighted the significance of corneal deformation data during the loading phase rather than the unloading phase for KC diagnosis.

CONCLUSION

This study proposed an end-to-end CNN approach utilizing raw biomechanical data by Corvis ST for KC detection, showing effectiveness comparable to or surpassing existing parameters provided by Corvis ST. The CorNet, autonomously learning comprehensive temporal and spatial features, demonstrated a promising performance for advancing KC diagnosis in ophthalmology.

A Deep-Learning Model for Predicting the Efficacy of Non-vascularized Fibular Grafting Using Digital Radiography

RATIONALE AND OBJECTIVES

To develop a fully automated deep-learning (DL) model using digital radiography (DR) with relatively high accuracy for predicting the efficacy of non-vascularized fibular grafting (NVFG) and identifying suitable patients for this procedure.

MATERIALS AND METHODS

A retrospective analysis was conducted on osteonecrosis of femoral head patients who underwent NVFG between June 2009 and June 2021. All patients underwent standard preoperative anteroposterior (AP) and frog-lateral (FL) DR. Subsequently, the radiographs were pre-processed and labeled based on the follow-up results. The dataset was randomly divided into training and testing datasets. The DL-based prediction model was developed in the training dataset and its diagnostic performance was evaluated using the testing dataset.

RESULTS

A total of 339 patients with 432 hips were included in this study, with a hip preservation success rate of 71.52% as of June 2023. The hips were randomly divided into a training dataset (n = 324) and a testing dataset (n = 108). The ensemble model in predicting the efficacy of NVFG, reaching an accuracy of 78.9%, a precision of 78.7%, a recall of 96.0%, a F1-score of 86.5%, and an area under the curve (AUC) of 0.780. FL views (AUC, 0.71) exhibited better performance compared to AP views (AUC, 0.66).

CONCLUSION

The proposed DL model using DR enables automatic and efficient prediction of NVFG efficacy without additional clinical and financial burden. It can be seamlessly integrated into various clinical scenarios, serving as a practical tool to identify suitable patients for NVFG.

Deep learning-based facial analysis for predicting difficult videolaryngoscopy: a feasibility study

While videolaryngoscopy has resulted in better overall success rates of tracheal intubation, airway assessment is still an important prerequisite for safe airway management. This study aimed to create an artificial intelligence model to identify difficult videolaryngoscopy using a neural network. Baseline characteristics, medical history, bedside examination and seven facial images were included as predictor variables. ResNet-18 was introduced to recognise images and extract features. Different machine learning algorithms were utilised to develop predictive models. A videolaryngoscopy view of Cormack-Lehane grade of 1 or 2 was classified as 'non-difficult', while grade 3 or 4 was classified as 'difficult'. A total of 5849 patients were included, of whom 5335 had non-difficult and 514 had difficult videolaryngoscopy. The facial model (only including facial images) using the Light Gradient Boosting Machine algorithm showed the highest area under the curve (95%CI) of 0.779 (0.733-0.825) with a sensitivity (95%CI) of 0.757 (0.650-0.845) and specificity (95%CI) of 0.721 (0.626-0.794) in the test set. Compared with bedside examination and multivariate scores (El-Ganzouri and Wilson), the facial model had significantly higher predictive performance (p < 0.001). Artificial intelligence-based facial analysis is a feasible technique for predicting difficulty during videolaryngoscopy, and the model developed using neural networks has higher predictive performance than traditional methods.

High-precision Drosophila heart segmentation and dynamic cardiac parameter measurement for optogenetics-OCT-based cardiac function research

Drosophila model has been widely used to study cardiac functions, especially combined with optogenetics and optical coherence tomography (OCT) that can continuously acquire mass cross-sectional images of the Drosophila heart in vivo over time. It's urgent to quickly and accurately obtain dynamic Drosophila cardiac parameters such as heartbeat rate for cardiac function quantitative analysis through these mass cross-sectional images of the Drosophila heart. Here we present a deep-learning method that integrates U-Net and generative adversarial network architectures while incorporating residually connected convolutions for high-precision OCT image segmentation of Drosophila heart and dynamic cardiac parameter measurements for optogenetics-OCT-based cardiac function research. We compared our proposed network with the previous approaches and our segmentation results achieved the accuracy of intersection over union and Dice similarity coefficient higher than 98%, which can be used to better quantify dynamic heart parameters and improve the efficiency of Drosophila-model-based cardiac research via the optogenetics-OCT-based platform.

Identification of Atrial Fibrillation With Single-Lead Mobile ECG During Normal Sinus Rhythm Using Deep Learning

BACKGROUND

The acquisition of single-lead electrocardiogram (ECG) from mobile devices offers a more practical approach to arrhythmia detection. Using artificial intelligence for atrial fibrillation (AF) identification enhances screening efficiency. However, the potential of single-lead ECG for AF identification during normal sinus rhythm (NSR) remains under-explored. This study introduces a method to identify AF using single-lead mobile ECG during NSR.

METHODS

We employed three deep learning models: recurrent neural network (RNN), long short-term memory (LSTM), and residual neural networks (ResNet50). From a dataset comprising 13,509 ECGs from 6,719 patients, 10,287 NSR ECGs from 5,170 patients were selected. Single-lead mobile ECGs underwent noise filtering and segmentation into 10-second intervals. A random under-sampling was applied to reduce bias from data imbalance. The final analysis involved 31,767 ECG segments, including 15,157 labeled as masked AF and 16,610 as Healthy.

RESULTS

ResNet50 outperformed the other models, achieving a recall of 79.3%, precision of 65.8%, F1-score of 71.9%, accuracy of 70.5%, and an area under the receiver operating characteristic curve (AUC) of 0.79 in identifying AF from NSR ECGs. Comparative performance scores for RNN and LSTM were 0.75 and 0.74, respectively. In an external validation set, ResNet50 attained an F1-score of 64.1%, recall of 68.9%, precision of 60.0%, accuracy of 63.4%, and AUC of 0.68.

CONCLUSION

The deep learning model using single-lead mobile ECG during NSR effectively identified AF at risk in future. However, further research is needed to enhance the performance of deep learning models for clinical application.

Automatic prediction of obstructive sleep apnea event using deep learning algorithm based on ECG and thoracic movement signals

BACKGROUND

Obstructive sleep apnea (OSA) is a sleeping disorder that can cause multiple complications.

AIMS/OBJECTIVE

Our aim is to build an automatic deep learning model for OSA event detection using combined signals from the electrocardiogram (ECG) and thoracic movement signals.

MATERIALS AND METHODS

We retrospectively obtained 420 cases of PSG data and extracted the signals of ECG, as well as the thoracic movement signal. A deep learning algorithm named ResNeSt34 was used to construct the model using ECG with or without thoracic movement signal. The model performance was assessed by parameters such as accuracy, precision, recall, F1-score, receiver operating characteristic (ROC), and area under the ROC curve (AUC).

RESULTS

The model using combined signals of ECG and thoracic movement signal performed much better than the model using ECG alone. The former had accuracy, precision, recall, F1-score, and AUC values of 89.0%, 88.8%, 89.0%, 88.2%, and 92.9%, respectively, while the latter had values of 84.1%, 83.1%, 84.1%, 83.3%, and 82.8%, respectively.

CONCLUSIONS AND SIGNIFICANCE

The automatic OSA event detection model using combined signals of ECG and thoracic movement signal with the ResNeSt34 algorithm is reliable and can be used for OSA screening.

CT-based radiomics nomogram for overall survival prediction in patients with cervical cancer treated with concurrent chemoradiotherapy

Background and purpose

To establish and validate a hybrid radiomics model to predict overall survival in cervical cancer patients receiving concurrent chemoradiotherapy (CCRT).

Methods

We retrospectively collected 367 cervical cancer patients receiving chemoradiotherapy from the First Affiliated Hospital of Soochow University in China and divided them into a training set and a test set in a ratio of 7:3. Handcrafted and deep learning (DL)-based radiomics features were extracted from the contrast-enhanced computed tomography (CT), and the two types of radiomics signatures were calculated based on the features selected using the least absolute shrinkage and selection operator (LASSO) Cox regression. A hybrid radiomics nomogram was constructed by integrating independent clinical risk factors, handcrafted radiomics signature, and DL-based radiomics signature in the training set and was validated in the test set.

Results

The hybrid radiomics nomogram exhibited favorable performance in predicting overall survival, with areas under the receiver operating characteristic curve (AUCs) for 1, 3, and 5 years in the training set of 0.833, 0.777, and 0.871, respectively, and in the test set of 0.811, 0.713, and 0.730, respectively. Furthermore, the hybrid radiomics nomogram outperformed the single clinical model, handcrafted radiomics signature, and DL-based radiomics signature in both the training (C-index: 0.793) and test sets (C-index: 0.721). The calibration curves and decision curve analysis (DCA) indicated that our hybrid nomogram had good calibration and clinical benefits. Finally, our hybrid nomogram demonstrated value in stratifying patients into high- and low-risk groups (cutoff value: 5.6).

Conclusion

A high-performance hybrid radiomics model based on pre-radiotherapy CT was established, presenting strengths in risk stratification.

AAFL: automatic association feature learning for gene signature identification of cancer subtypes in single-cell RNA-seq data

Single-cell RNA-sequencing (scRNA-seq) technologies have enabled the study of human cancers in individual cells, which explores the cellular heterogeneity and the genotypic status of tumors. Gene signature identification plays an important role in the precise classification of cancer subtypes. However, most existing gene selection methods only select the same informative genes for each subtype. In this study, we propose a novel gene selection method, automatic association feature learning (AAFL), which automatically identifies different gene signatures for different cell subpopulations (cancer subtypes) at the same time. The proposed AAFL method combines the residual network with the low-rank network, which selects genes that are most associated with the corresponding cell subpopulations. Moreover, the differential expression genes are acquired before gene selection to filter the redundant genes. We apply the proposed feature learning method to the real cancer scRNA-seq data sets (melanoma) to identify cancer subtypes and detect gene signatures of identified cancer subtypes. The experimental results demonstrate that the proposed method can automatically identify different gene signatures for identified cancer subtypes. Gene ontology enrichment analysis shows that the identified gene signatures of different subtypes reveal the key biological processes and pathways. These gene signatures are expected to bring important implications for understanding cellular heterogeneity and the complex ecosystem of tumors.

A Self-Supervised Model Based on CutPaste-Mix for Ductile Cast Iron Pipe Surface Defect Classification

Online surface inspection systems have gradually found applications in industrial settings. However, the manual effort required to sift through a vast amount of data to identify defect images remains costly. This study delves into a self-supervised binary classification algorithm for addressing the task of defect image classification within ductile cast iron pipe (DCIP) images. Leveraging the CutPaste-Mix data augmentation strategy, we combine defect-free data with enhanced data to input into a deep convolutional neural network. Through Gaussian Density Estimation, we compute anomaly scores to achieve the classification of abnormal regions. Our approach has been implemented in real-world scenarios, involving equipment installation, data collection, and experimentation. The results demonstrate the robust performance of our method, in both the DCIP image dataset and practical field application, achieving an impressive 99.5 AUC (Area Under Curve). This presents a cost-effective means of providing data support for subsequent DCIP surface inspection model training.

Exploring EEG characteristics of multi-level mental stress based on human-machine system

Objective.The understanding of cognitive states is important for the development of human-machine systems (HMSs), and one of the fundamental but challenging issues is the understanding and assessment of the operator's mental stress state in real task scenarios.Approach.In this paper, a virtual unmanned vehicle (UAV) driving task with multi-challenge-level was created to explore the operator's mental stress, and the human brain activity during the task was tracked in real time via electroencephalography (EEG). A mental stress analysis dataset for the virtual UAV task was then developed and used to explore the neural activation patterns associated with mental stress activity. Finally, a multiple attention-based convolutional neural network (MACN) was constructed for automatic stress assessment using the extracted stress-sensitive neural activation features.Main Results.The statistical results of EEG power spectral density (PSD) showed that frontal theta-PSD decreased with increasing task difficulty, and central beta-PSD increased with increasing task difficulty, indicating that neural patterns showed different trends under different levels of mental stress. The performance of the proposed MACN was evaluated based on the dimensional model, and results showed that average three-class classification accuracies of 89.49%/89.88% were respectively achieved for arousal/valence.Significance.The results of this paper suggest that objective assessment of mental stress in a HMS based on a virtual UAV scenario is feasible, and the proposed method provides a promising solution for cognitive computing and applications in human-machine tasks.

Special Traffic Event Detection: Framework, Dataset Generation, and Deep Neural Network Perspectives

Identifying early special traffic events is crucial for efficient traffic control management. If there are a sufficient number of vehicles equipped with automatic event detection and report gadgets, this enables a more rapid response to special events, including road debris, unexpected pedestrians, accidents, and malfunctioning vehicles. To address the needs of such a system and service, we propose a framework for an in-vehicle module-based special traffic event and emergency detection and safe driving monitoring service, which utilizes the modified ResNet classification algorithm to improve the efficiency of traffic management on highways. Due to the fact that this type of classification problem has scarcely been proposed, we have adapted various classification algorithms and corresponding datasets specifically designed for detecting special traffic events. By utilizing datasets containing data on road debris and malfunctioning or crashed vehicles obtained from Korean highways, we demonstrate the feasibility of our algorithms. Our main contributions encompass a thorough adaptation of various deep-learning algorithms and class definitions aimed at detecting actual emergencies on highways. We have also developed a dataset and detection algorithm specifically tailored for this task. Furthermore, our final end-to-end algorithm showcases a notable 9.2% improvement in performance compared to the object accident detection-based algorithm.

A Soft-Reference Breast Ultrasound Image Quality Assessment Method That Considers the Local Lesion Area

The quality of breast ultrasound images has a significant impact on the accuracy of disease diagnosis. Existing image quality assessment (IQA) methods usually use pixel-level feature statistical methods or end-to-end deep learning methods, which focus on the global image quality but ignore the image quality of the lesion region. However, in clinical practice, doctors' evaluation of ultrasound image quality relies more on the local area of the lesion, which determines the diagnostic value of ultrasound images. In this study, a global-local integrated IQA framework for breast ultrasound images was proposed to learn doctors' clinical evaluation standards. In this study, 1285 breast ultrasound images were collected and scored by experienced doctors. After being classified as either images with lesions or images without lesions, they were evaluated using soft-reference IQA or bilinear CNN IQA, respectively. Experiments showed that for ultrasound images with lesions, our proposed soft-reference IQA achieved PLCC 0.8418 with doctors' annotation, while the existing end-to-end deep learning method that did not consider the local lesion features only achieved PLCC 0.6606. Due to the accuracy improvement for the images with lesions, our proposed global-local integrated IQA framework had better performance in the IQA task than the existing end-to-end deep learning method, with PLCC improving from 0.8306 to 0.8851.

A deep learning model using hyperspectral image for EUS-FNA cytology diagnosis in pancreatic ductal adenocarcinoma

BACKGROUND AND AIMS

Endoscopic ultrasonography-guided fine-needle aspiration/biopsy (EUS-FNA/B) is considered to be a first-line procedure for the pathological diagnosis of pancreatic cancer owing to its high accuracy and low complication rate. The number of new cases of pancreatic ductal adenocarcinoma (PDAC) is increasing, and its accurate pathological diagnosis poses a challenge for cytopathologists. Our aim was to develop a hyperspectral imaging (HSI)-based convolution neural network (CNN) algorithm to aid in the diagnosis of pancreatic EUS-FNA cytology specimens.

METHODS

HSI images were captured of pancreatic EUS-FNA cytological specimens from benign pancreatic tissues (n = 33) and PDAC (n = 39) prepared using a liquid-based cytology method. A CNN was established to test the diagnostic performance, and Attribution Guided Factorization Visualization (AGF-Visualization) was used to visualize the regions of important classification features identified by the model.

RESULTS

A total of 1913 HSI images were obtained. Our ResNet18-SimSiam model achieved an accuracy of 0.9204, sensitivity of 0.9310 and specificity of 0.9123 (area under the curve of 0.9625) when trained on HSI images for the differentiation of PDAC cytological specimens from benign pancreatic cells. AGF-Visualization confirmed that the diagnoses were based on the features of tumor cell nuclei.

CONCLUSIONS

An HSI-based model was developed to diagnose cytological PDAC specimens obtained using EUS-guided sampling. Under the supervision of experienced cytopathologists, we performed multi-staged consecutive in-depth learning of the model. Its superior diagnostic performance could be of value for cytologists when diagnosing PDAC.

Video-Based Human Activity Recognition Using Deep Learning Approaches

Due to its capacity to gather vast, high-level data about human activity from wearable or stationary sensors, human activity recognition substantially impacts people's day-to-day lives. Multiple people and things may be seen acting in the video, dispersed throughout the frame in various places. Because of this, modeling the interactions between many entities in spatial dimensions is necessary for visual reasoning in the action recognition task. The main aim of this paper is to evaluate and map the current scenario of human actions in red, green, and blue videos, based on deep learning models. A residual network (ResNet) and a vision transformer architecture (ViT) with a semi-supervised learning approach are evaluated. The DINO (self-DIstillation with NO labels) is used to enhance the potential of the ResNet and ViT. The evaluated benchmark is the human motion database (HMDB51), which tries to better capture the richness and complexity of human actions. The obtained results for video classification with the proposed ViT are promising based on performance metrics and results from the recent literature. The results obtained using a bi-dimensional ViT with long short-term memory demonstrated great performance in human action recognition when applied to the HMDB51 dataset. The mentioned architecture presented 96.7 ± 0.35% and 41.0 ± 0.27% in terms of accuracy (mean ± standard deviation values) in the train and test phases of the HMDB51 dataset, respectively.

Accurate Differentiation of Spinal Tuberculosis and Spinal Metastases Using MR-Based Deep Learning Algorithms

Purpose

To explore the application of deep learning (DL) methods based on T2 sagittal MR images for discriminating between spinal tuberculosis (STB) and spinal metastases (SM).

Patients and Methods

A total of 121 patients with histologically confirmed STB and SM across four institutions were retrospectively analyzed. Data from two institutions were used for developing deep learning models and internal validation, while the remaining institutions' data were used for external testing. Utilizing MVITV2, EfficientNet-B3, ResNet101, and ResNet34 as backbone networks, we developed four distinct DL models and evaluated their diagnostic performance based on metrics such as accuracy (ACC), area under the receiver operating characteristic curve (AUC), F1 score, and confusion matrix. Furthermore, the external test images were blindly evaluated by two spine surgeons with different levels of experience. We also used Gradient-Class Activation Maps to visualize the high-dimensional features of different DL models.

Results

For the internal validation set, MVITV2 outperformed other models with an accuracy of 98.7%, F1 score of 98.6%, and AUC of 0.98. Other models followed in this order: EfficientNet-B3 (ACC: 96.1%, F1 score: 95.9%, AUC: 0.99), ResNet101 (ACC: 85.5%, F1 score: 84.8%, AUC: 0.90), and ResNet34 (ACC: 81.6%, F1 score: 80.7%, AUC: 0.85). For the external test set, MVITV2 again performed excellently with an accuracy of 91.9%, F1 score of 91.5%, and an AUC of 0.95. EfficientNet-B3 came second (ACC: 85.9, F1 score: 91.5%, AUC: 0.91), followed by ResNet101 (ACC:80.8, F1 score: 80.0%, AUC: 0.87) and ResNet34 (ACC: 78.8, F1 score: 77.9%, AUC: 0.86). Additionally, the diagnostic accuracy of the less experienced spine surgeon was 73.7%, while that of the more experienced surgeon was 88.9%.

Conclusion

Deep learning based on T2WI sagittal images can help discriminate between STB and SM, and can achieve a level of diagnostic performance comparable with that produced by experienced spine surgeons.

Deep-Learning Model of ResNet Combined with CBAM for Malignant-Benign Pulmonary Nodules Classification on Computed Tomography Images

Background and Objectives: Lung cancer remains a leading cause of cancer mortality worldwide. Accurately classifying benign pulmonary nodules and malignant ones is crucial for early diagnosis and improved patient outcomes. The purpose of this study is to explore the deep-learning model of ResNet combined with a convolutional block attention module (CBAM) for the differentiation between benign and malignant lung cancer, based on computed tomography (CT) images, morphological features, and clinical information. Methods and materials: In this study, 8241 CT slices containing pulmonary nodules were retrospectively included. A random sample comprising 20% (n = 1647) of the images was used as the test set, and the remaining data were used as the training set. ResNet combined CBAM (ResNet-CBAM) was used to establish classifiers on the basis of images, morphological features, and clinical information. Nonsubsampled dual-tree complex contourlet transform (NSDTCT) combined with SVM classifier (NSDTCT-SVM) was used as a comparative model. Results: The AUC and the accuracy of the CBAM-ResNet model were 0.940 and 0.867, respectively, in test set when there were only images as inputs. By combining the morphological features and clinical information, CBAM-ResNet shows better performance (AUC: 0.957, accuracy: 0.898). In comparison, a radiomic analysis using NSDTCT-SVM achieved AUC and accuracy values of 0.807 and 0.779, respectively. Conclusions: Our findings demonstrate that deep-learning models, combined with additional information, can enhance the classification performance of pulmonary nodules. This model can assist clinicians in accurately diagnosing pulmonary nodules in clinical practice.

Diagnosis of COVID-19 Using Chest X-ray Images and Disease Symptoms Based on Stacking Ensemble Deep Learning

The COVID-19 virus is one of the most devastating illnesses humanity has ever faced. COVID-19 is an infection that is hard to diagnose until it has caused lung damage or blood clots. As a result, it is one of the most insidious diseases due to the lack of knowledge of its symptoms. Artificial intelligence (AI) technologies are being investigated for the early detection of COVID-19 using symptoms and chest X-ray images. Therefore, this work proposes stacking ensemble models using two types of COVID-19 datasets, symptoms and chest X-ray scans, to identify COVID-19. The first proposed model is a stacking ensemble model that is merged from the outputs of pre-trained models in the stacking: multi-layer perceptron (MLP), recurrent neural network (RNN), long short-term memory (LSTM), and gated recurrent unit (GRU). Stacking trains and evaluates the meta-learner as a support vector machine (SVM) to predict the final decision. Two datasets of COVID-19 symptoms are used to compare the first proposed model with MLP, RNN, LSTM, and GRU models. The second proposed model is a stacking ensemble model that is merged from the outputs of pre-trained DL models in the stacking: VGG16, InceptionV3, Resnet50, and DenseNet121; it uses stacking to train and evaluate the meta-learner (SVM) to identify the final prediction. Two datasets of COVID-19 chest X-ray images are used to compare the second proposed model with other DL models. The result has shown that the proposed models achieve the highest performance compared to other models for each dataset.

Effect of Contrast Level and Image Format on a Deep Learning Algorithm for the Detection of Pneumothorax with Chest Radiography

Under the black-box nature in the deep learning model, it is uncertain how the change in contrast level and format affects the performance. We aimed to investigate the effect of contrast level and image format on the effectiveness of deep learning for diagnosing pneumothorax on chest radiographs. We collected 3316 images (1016 pneumothorax and 2300 normal images), and all images were set to the standard contrast level (100%) and stored in the Digital Imaging and Communication in Medicine and Joint Photographic Experts Group (JPEG) formats. Data were randomly separated into 80% of training and 20% of test sets, and the contrast of images in the test set was changed to 5 levels (50%, 75%, 100%, 125%, and 150%). We trained the model to detect pneumothorax using ResNet-50 with 100% level images and tested with 5-level images in the two formats. While comparing the overall performance between each contrast level in the two formats, the area under the receiver-operating characteristic curve (AUC) was significantly different (all p < 0.001) except between 125 and 150% in JPEG format (p = 0.382). When comparing the two formats at same contrast levels, AUC was significantly different (all p < 0.001) except 50% and 100% (p = 0.079 and p = 0.082, respectively). The contrast level and format of medical images could influence the performance of the deep learning model. It is required to train with various contrast levels and formats of image, and further image processing for improvement and maintenance of the performance.

4mCBERT: A computing tool for the identification of DNA N4-methylcytosine sites by sequence- and chemical-derived information based on ensemble learning strategies

N4-methylcytosine (4mC) is an important DNA chemical modification pattern which is a new methylation modification discovered in recent years and plays critical roles in gene expression regulation, defense against invading genetic elements, genomic imprinting, and so on. Identifying 4mC site from DNA sequence segment contributes to discovering more novel modification patterns. In this paper, we present a model called 4mCBERT that encodes DNA sequence segments by sequence characteristics including one-hot, electron-ion interaction pseudopotential, nucleotide chemical property, word2vec and chemical information containing physicochemical properties (PCP), chemical bidirectional encoder representations from transformers (chemical BERT) and employs ensemble learning framework to develop a prediction model. PCP and chemical BERT features are firstly constructed and applied to predict 4mC sites and show positive contributions to identifying 4mC. For the Matthew's Correlation Coefficient, 4mCBERT significantly outperformed other state-of-the-art models on six independent benchmark datasets including A. thaliana, C. elegans, D. melanogaster, E. coli, G. Pickering, and G. subterraneous by 4.32 % to 24.39 %, 2.52 % to 31.65 %, 2 % to 16.49 %, 6.63 % to 35.15, 8.59 % to 61.85 %, and 8.45 % to 34.45 %. Moreover, 4mCBERT is designed to allow users to predict 4mC sites and retrain 4mC prediction models. In brief, 4mCBERT shows higher performance on six benchmark datasets by incorporating sequence- and chemical-driven information and is available at http://cczubio.top/4mCBERT and https://github.com/abcair/4mCBERT.

Deep residual network enabled smart hyperspectral image analysis and its application to monitoring moisture, size distribution and contents of four bioactive compounds of granules in the fluid-bed granulation process of Guanxinning tablets

Bed collapse is a serious problem in a fluid-bed granulation process of traditional Chinese medicine. Moisture content and size distribution are regarded as two pivotal influencing factors. Herein, a smart hyperspectral image analysis methodology was established via deep residual network (ResNet) algorithm, which was then applied to monitoring moisture content, size distribution and contents of four bioactive compounds of granules in the fluid-bed granulation process of Guanxinning tablets. First, a hyperspectral imaging camera was utilized to acquire hyperspectral images of 132 real granule samples in the spectral region of 389-1020 nm. Second, the moisture content and size distribution of the granules were measured with a laser particle sizer and a fast moisture analyzer, respectively. Moreover, the contents of danshensu, ferulic acid, rosmarinic acid and salvianolic acid B of the granules were determined by using high-performance liquid chromatography-diode array detection. Third, ResNet quantitative calibration models were built, which consisted of convolutional layer, maxpooling layer, four convolutional blocks with residual learning function and two fully connected layers. As a result, the R_c² values for the moisture content, granule sizes and contents of four bioactive compounds are determined to be 0.957, 0.986, 0.936, 0.959, 0.937, 0.938, 0.956, 0.889, 0.914 and 0.928, whereas the R_p² values are calculated as 0.940, 0.969, 0.904, 0.930, 0.925, 0.928, 0.896, 0.849, 0.844, and 0.905, respectively. The predicted values matched well with the measured values. These findings indicated that ResNet algorithm driven hyperspectral image analysis is feasible for monitoring both the physical and chemical properties of Guanxinning tablets at the same time.